Real-time voice translation system

Abstract

A real-time speech translation system includes multiple electronic devices, a server, a speech-to-text model, and a translation model. One of the electronic devices serves as a speaker device, while the others serve as multiple reader devices. The server provides data transmission between the speaker device and the reader devices via the WebSocket communication protocol. The speech-to-text model receives a speech language setting and speech data output by the speaker device and converts them into text data. The speaker device transmits the speech language setting and the text data to the reader devices. The reader devices output the speech language setting, the text data, and a subtitle language setting to the translation model. The translation model translates the subtitle data into subtitle data corresponding to the subtitle language setting and then outputs it to the reader devices for displaying the subtitle.

Claims

1. A real-time voice translation system, comprising: a plurality of electronic devices, each electronic device including an audio input module for receiving a voice input and outputting voice data, an operation interface for outputting a voice language setting or a subtitle language setting, and a display module for displaying a subtitle corresponding to the subtitle language setting, wherein one electronic device serves as a speaker device, and the other electronic devices serve as a plurality of reader devices, the operation interface of the speaker device is used to output the voice language setting, and the operation interface of each reader device is used to output the subtitle language setting; a server connected to the electronic devices via a network and providing data transmission between the speaker device and the reader devices via the WebSocket communication protocol; A speech-to-text model connects to the speaker device via a network, receives the speech language setting and speech data output by the speaker device, converts the speech data into text data corresponding to the speech language setting, and then outputs the text data to the speaker device. The speaker device transmits the speech language setting and text data to the reader devices via a server. A translation model connects to the reader devices via a network to receive multiple translation request messages output by at least some of the reader devices. Each translation request message contains information about the speech language setting, the text data, and the subtitle language setting. The translation model translates the text data into subtitle data corresponding to the subtitle language setting and then outputs the subtitle data to the corresponding reader device. The display module of the reader device displays the subtitle data according to the subtitle data.

2. The real-time voice translation system as described in claim 1, wherein, When the language setting of the audio and the language setting of the subtitle are the same, each reader device displays the subtitle based on the text data. When the language setting of the audio and the language setting of the subtitle are different, the device outputs the translation request information and displays the subtitle based on the subtitle data output by the translation model.

3. The real-time voice translation system as described in claim 1, wherein, The server connects the speech-to-text model and the translation model via a network. The speaker device connects to the speech-to-text model via the server, and the reader devices connect to the translation model via the server.

4. The real-time voice translation system as described in claim 3, wherein, The speaker device outputs the language setting and the voice data, and transmits them to the speech-to-text model via the server. The speech-to-text model outputs the text data, and transmits it to the speaker device via the server.

5. The real-time voice translation system as described in claim 3, wherein, After each reader device outputs the translation request information, it transmits it to the translation model via the server. The translation model outputs the subtitle data and transmits it to the corresponding reader device via the server.

6. The real-time voice translation system as described in claim 1, wherein, The speaker device can be switched to the reader device, and any of the reader devices can be switched to the speaker device.

7. The real-time voice translation system as described in claim 1, wherein, The speaker device also transmits the voice data to the reader devices via the server, and each electronic device further includes an audio output module for outputting a voice output based on the voice data.

8. The real-time voice translation system as described in claim 1, wherein, The subtitle has a dual-column stacked display of a real-time subtitle bar and a history subtitle bar. The real-time subtitle bar is used to display the subtitle data generated in real time, and the history subtitle bar is used to display the subtitle data of the previous sentence.

9. The real-time voice translation system as described in claim 1, wherein, The translation model includes a translation cache module and multiple translation engines. The translation cache module is used to temporarily store the text data and the corresponding subtitle language settings. When the reader device outputs the translation request information, the translation model first compares the text data in the translation cache module. If there is no matching data, one of the translation engines provides the translation. If one of the translation engines cannot provide the translation, it automatically switches to another translation engine to provide the translation.

10. The real-time speech translation system as described in claim 1, wherein, Each of the electronic devices also includes a question-and-answer module. The question-and-answer module of the speaker device is used to output a question-and-answer activation command to the server. The server outputs a corresponding command to the reader devices, causing the question-and-answer module of the reader devices to enter a question-asking state, allowing any reader device's question-and-answer module to receive audience question data. The server receives the audience question data from the reader device and broadcasts the audience question data to the speaker device and other reader devices.

11. The real-time speech translation system as described in claim 1, wherein, The speech-to-text model includes a sentence buffer module, which is used to determine the silence time of the speech data. If the silence time does not exceed a preset delay time, the text corresponding to the speech data is continuously merged until the silence time exceeds the preset delay time, and the merged text is output as a sentence as the text data.

12. The real-time speech translation system as described in claim 1, wherein, Each of the electronic devices also includes a sensitive word filtering module, which receives the text data or the subtitle data, replaces the text data or the subtitle data that matches a preset sensitive word library with specific symbols, and then outputs the replaced text data or the subtitle data to the display module to be displayed as the subtitle.

Images