Speech translation system, Bluetooth earphone and translation method

Abstract

The invention discloses a voice translation system, a Bluetooth earphone and a translation method, the system comprises an earphone, an application program APP and a cloud server, the earphone is provided with a Bluetooth connection module, a voice pickup module, a voice mixing module, a voice coding module, a voice decoding module and a voice playing module; the APP is provided with a voice decoding and separating module, a network uploading module, a voice coding and transmitting module and a user interface display module; the cloud server is provided with a translation processing module and a multilingual support module. Through interconnection of the earphone, the application program APP and the cloud server, the functions of conversation real-time translation, simultaneous interpretation, recording and character transcription are provided, multi-language translation in the global range is supported, and the system is suitable for various scenes such as business, academic and daily communication. The system is compatible with a mainstream operating system, covers wide languages, and can meet diversified requirements of different user groups.

Description

Technical Field

[0001] The present invention relates to the field of speech translation technology, and in particular to a speech translation system, a Bluetooth headset and a translation method. Background Art

[0002] In the current headset market, the intelligent functions of headsets are gradually enriched, providing users with a more convenient audio experience. However, despite the continuous advancement of headset technology, existing headsets still have some key limitations in providing real-time language translation and call content processing.

[0003] First, the lack of call translation function: Although existing headsets can support call functions after connecting to mobile phones, they do not have the ability to perform real-time translation during calls. This limits the application of headsets in scenarios that require instant language conversion, such as communicating with people of different language backgrounds during business communications or travel.

[0004] Second, the limitation of multi-language support: Headphones on the market usually only support limited language options, which makes them incapable of multi-language communication. When users face different language environments, they often need additional translation tools or services, which increases the complexity and cost of communication.

[0005] Third, the inflexibility of language switching: When the translation settings need to be adjusted according to the other party's language, the fixed translation mode of existing headphones cannot meet the needs of users. This inflexibility is particularly obvious in multilingual communication, which limits the convenience of headphones in practical applications.

[0006] Fourth, lack of call content transcription: Existing headsets cannot achieve real-time voice transcription during a call, which makes it difficult for users to review and organize information after the call. This function is particularly important in situations where the content of the call needs to be recorded, such as business meetings or academic discussions.

[0007] In view of the shortcomings of existing headsets in real-time translation, multi-language support, language switching flexibility, and call content transcription, the present invention proposes an innovative headset translation system. The system aims to provide a solution that can translate in real time, support multiple languages, allow flexible language switching, and achieve real-time transcription of call content, thereby improving the practicality and user experience of headsets in multiple language environments. Summary of the invention

[0008] In view of this, the present invention aims at the deficiencies in the prior art, and its main purpose is to provide a voice translation system, a Bluetooth headset and a translation method, which provide real-time translation of calls, simultaneous interpretation, recording and transcription functions through the interconnection of headsets, application APP and cloud servers, support multi-language translation worldwide, and are suitable for various scenarios such as business, academic and daily communication. It is compatible with mainstream operating systems, covers a wide range of languages, and meets the diverse needs of different user groups. In addition, the system also supports multi-language translation during live broadcasts, which promotes cross-cultural communication and information sharing.

[0009] To achieve the above purpose, the present invention adopts the following technical solutions:

[0010] A speech translation system comprising

[0011] Headphones with:

[0012] A Bluetooth connection module, used to establish a Bluetooth connection with a mobile device;

[0013] Voice pickup module, used to pick up user voice through a built-in high-sensitivity microphone;

[0014] A voice mixing module is used to mix the voice information received by the microphone with the downlink voice information of the mobile device to generate mixed voice information;

[0015] The voice coding module performs OPUS coding or MSBC coding on the mixed voice information and then sends it uplink to the application APP;

[0016] The voice decoding module is used to perform OPUS decoding or MSBC decoding processing on the received downlink voice information that has been OPUS encoded or MSBC encoded;

[0017] Voice playback module, used to play the decoded voice;

[0018] The application APP has:

[0019] The voice decoding and separation module is used to decode the mixed voice information of OPUS or MSBC encoding sent by the headset and separate it into voice information of independent channels;

[0020] A network upload module is used to upload the separated voice information to the cloud server;

[0021] The voice coding and transmission module is used to encode the voice information translated by the cloud server and send it back to the headset through the mobile device network;

[0022] User interface display module, used to provide a user operation interface, including language selection, translation result display and system settings;

[0023] Cloud servers with:

[0024] The translation processing module is used to perform real-time translation, transcription, and text-to-speech processing on the voice information uploaded by the application APP;

[0025] The multilingual support module is used to ensure that the system supports multiple different languages.

[0026] As a preferred solution, the cloud server further includes a database for storing and retrieving translation data and a machine learning module for optimizing the translation algorithm.

[0027] As a preferred solution, the network uploading module uploads the separated voice information to the cloud server via a mobile data network or Wi-Fi.

[0028] As a preferred solution, the application APP decodes the encoded mixed voice information sent by the headset and separates it into voice information of two independent channels.

[0029] As a preferred solution, the Bluetooth connection module is a multi-mode Bluetooth with classic Bluetooth, BLE protocol and SPP protocol.

[0030] As a preferred solution, the translation processing module uses TTS technology to generate an audio file in pcm format to convert text into speech.

[0031] As a preferred solution, it is suitable for Apple iOS, Android and Hongmeng systems.

[0032] As a preferred solution, the multi-language support module supports languages ​​of various countries in the world.

[0033] A translation method applied to the speech translation system comprises the following steps:

[0034] S1. Establish a wireless connection with the mobile device through the Bluetooth connection module of the headset;

[0035] S2, picking up the user's voice through the voice pickup module of the earphone;

[0036] S3, mixing the picked-up voice with the downlink voice information of the mobile device through the voice mixing module of the earphone; encoding the mixed voice through the voice encoding module;

[0037] S4, in the application APP, the mixed voice is decoded by the voice decoding and separation module and separated into voice information of independent channels;

[0038] S5. Upload the separated voice information to the cloud server through the network upload module of the application APP;

[0039] S6. The uploaded voice information is translated, transcribed and converted into text-to-speech by a translation processing module in the cloud server;

[0040] S7, displaying the translated text information on the application APP;

[0041] S8, encoding the voice information translated by the cloud server through the voice encoding and transmission module of the application APP and sending it back to the headset through the Bluetooth connection module;

[0042] S9, after the voice decoding module of the earphone decodes the voice, it plays the translated downlink voice of the mobile device through the voice playing module; at the same time, the voice pickup module picks up the translated voice and encodes it through the voice encoding module, and sends it to the mobile device call module, which transmits it to the remote mobile device.

[0043] A Bluetooth headset, comprising

[0044] A Bluetooth connection module, used to establish a Bluetooth connection with a mobile device;

[0045] Voice pickup module, used to pick up user voice through a built-in high-sensitivity microphone;

[0046] A voice mixing module is used to mix the voice information received by the microphone with the downlink voice information of the mobile device to generate mixed voice information;

[0047] The voice coding module performs OPUS coding or MSBC coding on the mixed voice information and then sends it uplink to the application APP;

[0048] The voice decoding module is used to perform OPUS decoding or MSBC decoding processing on the received downlink voice information that has been OPUS encoded or MSBC encoded;

[0049] The voice playback module is used to play the decoded voice.

[0050] Compared with the prior art, the present invention has obvious advantages and beneficial effects. Specifically, it can be seen from the above technical solution that by interconnecting the headset, the application APP and the operation server to form a translation system, the translation system has the following advantages:

[0051] First, the real-time translation function of calls. After the headset is connected to the smartphone via Bluetooth, it can realize the real-time translation function of calls during network calls. This function allows users to receive the translation results of the other party's voice in real time during international calls, breaking the language barrier and greatly improving communication efficiency. Whether it is business negotiations, academic seminars or daily communication, users can enjoy seamless language support.

[0052] Second, the simultaneous interpretation function, after pairing the Bluetooth headset with the mobile phone, users can use the simultaneous interpretation function to achieve instant translation of voice information when making calls around the world. This technology can synchronously capture the speaker's voice and quickly translate it into the language set by the user, providing a near real-time translation experience, which is suitable for a variety of scenarios such as multinational meetings, remote work and international travel.

[0053] Third, recording and transcribing functions. In addition to real-time translation and simultaneous interpretation, the headset also has a recording function that can record the content of the call. Combined with the voice information transcribing function, users can convert call recordings into text records for subsequent review and analysis. This function is extremely valuable for meeting records, legal consultations, or any occasions that require recording and review.

[0054] Fourth, the multi-language translation function of the headset covers all countries and regions around the world, and supports multi-language translation during live broadcasts, including voice and text. No matter where the user is, they can get real-time language support during live broadcasts, and realize cross-cultural communication and information sharing. This is particularly important for fields such as international live broadcasts, distance education, and global news reporting.

[0055] Fifth, system compatibility and language support. The call translation and transcription functions of this headset are not only applicable to domestic and international cloud models, but also compatible with Apple iOS, Android and Hongmeng systems. At the same time, it supports all languages ​​supported by the cloud server, ensuring a wide range of language coverage and in-depth language support to meet the diverse needs of different user groups.

[0056] Sixth, application scenario support. The call translation and transcription functions of this headset are suitable for face-to-face translation scenarios, simultaneous interpretation scenarios, online conference scenarios, e-sports game scenarios, telecom operator network dial-up call scenario translation, all network signal call scenarios, short video translation scenarios, watching movies and TV with mobile phones / tablets, etc., online video live broadcast translation scenarios for different language audiences, and online audio live broadcast translation scenarios for different language audiences.

[0057] In order to more clearly illustrate the structural features and effects of the present invention, it is described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the translation system principle of the present invention;

[0059] Figure 2 A flowchart of a translation method of a translation system of the present invention;

[0060] Figure 3 Schematic diagram of the translation system workflow of the present invention.

[0061] Description of the accompanying drawings:

[0062] 10. Earphones; 11. Voice pickup module; 12. Bluetooth connection module; 13. Voice mixing module; 14. Voice decoding module; 15. Voice playback module; 16. Voice encoding module; 20. Application APP; 21. Voice decoding and separation module; 22. Network upload module; 23. Voice encoding and transmission module; 24. User interface display module; 30. Cloud server; 31. Translation processing module; 32. Multi-language support module; 33. Database; 34. Machine learning module. DETAILED DESCRIPTION

[0063] The present invention Figures 1 to 3 As shown, a speech translation system, earphone and translation method, the translation system includes earphone 10, application APP 20 and cloud server 30, wherein,

[0064] The headset 10 comprises:

[0065] The Bluetooth connection module 12 is used to establish a Bluetooth connection with a mobile device; in this embodiment, the Bluetooth connection module 12 is a multi-mode Bluetooth with classic Bluetooth and BLE protocols (supported by both iOS and Android phones) and SPP protocol (supported by Android phones); signal transmission is completed between the headset 10 and the mobile device (mobile phone) through BLE protocol, SPP protocol, and HFP protocol communication.

[0066] The voice pickup module 11 is used to pick up the user's voice through a built-in high-sensitivity microphone.

[0067] The voice mixing module 13 is used to mix the voice information received by the microphone and the downlink voice information of the mobile device call module to generate mixed voice information.

[0068] The voice encoding module 16 performs OPUS encoding or MSBC encoding on the mixed voice information and then sends it uplink to the application APP.

[0069] The voice decoding module 14 is used to perform OPUS decoding or MSBC decoding processing on the received downlink voice information that has been OPUS encoded or MSBC encoded.

[0070] The voice playing module 15 is used to play the decoded voice.

[0071] The application APP 20 has:

[0072] The voice decoding and separation module 21 is used to decode the OPUS-encoded or MSBC-encoded mixed voice information sent by the headset 10 and separate it into voice information of independent channels; in this embodiment, it is separated into voice information of two independent channels.

[0073] The network uploading module 22 is used to upload the separated voice information to the cloud server 30 via the mobile data network or Wi-Fi.

[0074] The voice encoding and transmission module 23 is used to encode the voice information translated by the cloud server 30 and send it back to the headset 10 through the mobile device network.

[0075] The user interface display module 24 is used to provide a user operation interface, including language selection, translation result display and system settings.

[0076] The cloud server 30 has:

[0077] The translation processing module 31 is used to perform real-time translation, transcription and text-to-speech processing on the voice information uploaded by the application APP 20; the translation processing module 31 uses TTS technology to generate an audio file in pcm format to convert text into speech.

[0078] The multi-language support module 32 is used to ensure that the system supports multiple different languages.

[0079] The cloud server 30 further includes a database 33 for storing and retrieving translation data and a machine learning module 34 for optimizing the translation algorithm; the database 33 can be used to store multilingual data samples and recorded voice data information.

[0080] The translation system is applicable to Apple iOS system, Android system and Hongmeng system. Moreover, the multilingual support module 32 supports languages ​​of various countries in the world.

[0081] A translation method applied to the speech translation system comprises the following steps:

[0082] S1, establishing a wireless connection with a mobile device through the Bluetooth connection module 12 of the headset 10;

[0083] S2, picking up the user's voice through the voice pickup module 11 of the headset 10;

[0084] S3, the voice mixing module 13 of the earphone 10 mixes the picked-up voice with the downlink voice information of the mobile device; the voice encoding module 16 encodes the mixed voice;

[0085] S4, in the application APP 20, the mixed voice is decoded by the voice decoding and separation module 21 and separated into voice information of independent channels;

[0086] S5, uploading the separated voice information to the cloud server 30 through the network upload module 22 of the application APP 20;

[0087] S6, translating, transcribing and converting text into speech on the uploaded voice information through the translation processing module 31 in the cloud server 30;

[0088] S7, displaying the translated text information on the application APP 20;

[0089] S8, encoding the voice information translated by the cloud server 30 through the voice encoding and transmission module 23 of the application APP 20 and sending it back to the headset 10 through the Bluetooth connection module 12;

[0090] S9, after the voice decoding module 14 of the earphone 10 decodes the voice, the translated mobile device downlink voice is played through the voice playing module 15; at the same time, the translated voice pickup module 11 picks up the voice and encodes it through the voice encoding module 16, and then sends it to the mobile device call module, which transmits it to the remote mobile device.

[0091] A Bluetooth headset 10, comprising

[0092] The Bluetooth connection module 12 is used to establish a Bluetooth connection with a mobile device.

[0093] The voice pickup module 11 is used to pick up the user's voice through a built-in high-sensitivity microphone.

[0094] The voice mixing module 13 is used to mix the voice information received by the microphone with the downlink voice information of the mobile device to generate mixed voice information.

[0095] The voice encoding module 16 performs OPUS encoding or MSBC encoding on the mixed voice information and then sends the mixed voice information to the application APP 20 in an uplink.

[0096] The voice decoding module 14 is used to perform OPUS decoding or MSBC decoding processing on the received downlink voice information that has been OPUS encoded or MSBC encoded.

[0097] The voice playing module 15 is used to play the decoded voice.

[0098] The working principle and process of the translation system are explained below with a specific translation process: The patent of this invention relates to an innovative voice translation system, which realizes efficient communication between earphone 10 and mobile device A mobile phone by combining classic Bluetooth and BLE technology. When mobile phone A is making a call, whether it is making a call to mobile phone B or answering an incoming call, the microphone of earphone 10 can accurately pick up the voice of the user of mobile phone A (named aa). At the same time, the remote voice (named bb) from mobile phone B is transmitted to earphone 10 through the Bluetooth technology of mobile phone A.

[0099] The voice mixing module 13 in the headset 10 mixes the received local voice (aa) and the remote voice (bb) to generate mixed voice information (named cc). This mixed voice information is then processed by OPUS encoding or MSBC encoding through the voice encoding module 16 and sent to the application APP 20 of the A mobile phone through the uplink channel.

[0100] After the application APP 20 receives the encoded mixed voice information, it is first decoded by the voice decoding and separation module 21 to restore it to a mixed voice. Then, the application APP 20 separates the mixed voice into two independent channel voice information, namely the original user voice (aa) and the remote voice (bb). The separated voice information is uploaded to the cloud server 30 through the network for real-time translation and transcription into text, and an audio file in pcm format is generated using TTS technology, generating two different voice information (named aaa and bbb respectively) and corresponding text information (named aaaa and bbbb respectively).

[0101] The text information aaaa and bbbb generated by the cloud server 30 is displayed to the user through the application APP 20, realizing the transcription and display function of the voice information. The application APP 20 encodes the voice information aaa and bbb translated by the cloud server 30, and sends it back to the headset 10 through the mobile phone network. The headset 10 decodes the received encoded voice information, restores it to aaa and bbb voice information, and the voice playback module 15 plays the bbb voice information. At the same time, the aaa voice information is encoded by the voice encoding module 16 and transmitted to the remote mobile device B mobile phone by the mobile device call module to complete the translation function.

[0102] The design of this system enables it to be compatible with various cloud models at home and abroad, providing a wide range of application scenarios. It can support Apple iOS system, Android system and Hongmeng system, ensuring that users of different operating systems can enjoy the services of this system. In addition, the cloud server 30 supports translation and transcription of multiple languages, allowing this system to serve global users and meet the needs of users with different language backgrounds.

[0103] Through this implementation, the speech translation system of the present invention provides users with a real-time, accurate and safe speech translation service, which is particularly suitable for occasions requiring cross-language communication such as international conferences, travel, and business negotiations.

[0104] The design focus of the present invention is to form a translation system by interconnecting the headset, the application APP and the operation server. The translation system has the following advantages:

[0105] First, the real-time translation function of calls. After the headset is connected to the smartphone via Bluetooth, it can realize the real-time translation function of calls during network calls. This function allows users to receive the translation results of the other party's voice in real time during international calls, breaking the language barrier and greatly improving communication efficiency. Whether it is business negotiations, academic seminars or daily communication, users can enjoy seamless language support.

[0106] Second, the simultaneous interpretation function, after pairing the Bluetooth headset with the mobile phone, users can use the simultaneous interpretation function to achieve instant translation of voice information when making calls around the world. This technology can synchronously capture the speaker's voice and quickly translate it into the language set by the user, providing a near real-time translation experience, which is suitable for a variety of scenarios such as multinational meetings, remote work and international travel.

[0107] Third, recording and transcribing functions. In addition to real-time translation and simultaneous interpretation, the headset also has a recording function that can record the content of the call. Combined with the voice information transcribing function, users can convert call recordings into text records for subsequent review and analysis. This function is extremely valuable for meeting records, legal consultations, or any occasions that require recording and review.

[0108] Fourth, the multi-language translation function of the headset covers all countries and regions around the world, and supports multi-language translation during live broadcasts, including voice and text. No matter where the user is, they can get real-time language support during live broadcasts, and realize cross-cultural communication and information sharing. This is particularly important for fields such as international live broadcasts, distance education, and global news reporting.

[0109] Fifth, system compatibility and language support. The call translation and transcription functions of this headset are not only applicable to domestic and international cloud models, but also compatible with Apple iOS, Android and Hongmeng systems. At the same time, it supports all languages ​​supported by the cloud server, ensuring a wide range of language coverage and in-depth language support to meet the diverse needs of different user groups.

[0110] Sixth, application scenario support. The call translation and transcription functions of this headset are suitable for face-to-face translation scenarios, simultaneous interpretation scenarios, online conference scenarios, e-sports game scenarios, telecom operator network dial-up call scenario translation, all network signal call scenarios, short video translation scenarios, watching movies and TV with mobile phones / tablets, etc., online video live broadcast translation scenarios for different language audiences, and online audio live broadcast translation scenarios for different language audiences.

[0111] The above description is only a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any slight modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A speech translation system, characterized in that: include Headphones with: A Bluetooth connection module, used to establish a Bluetooth connection with a mobile device; Voice pickup module, used to pick up user voice through a built-in high-sensitivity microphone; A voice mixing module is used to mix the voice information received by the microphone with the downlink voice information of the mobile device to generate mixed voice information; The voice coding module performs OPUS coding or MSBC coding on the mixed voice information and then sends it uplink to the application APP; The voice decoding module is used to perform OPUS decoding or MSBC decoding processing on the received downlink voice information that has been OPUS encoded or MSBC encoded; Voice playback module, used to play the decoded voice; The application APP has: The voice decoding and separation module is used to decode the mixed voice information of OPUS or MSBC encoding sent by the headset and separate it into voice information of independent channels; A network upload module is used to upload the separated voice information to the cloud server; The voice coding and transmission module is used to encode the voice information translated by the cloud server and send it back to the headset through the mobile device network; User interface display module, used to provide a user operation interface, including language selection, translation result display and system settings; Cloud servers with: The translation processing module is used to perform real-time translation, transcription, and text-to-speech processing on the voice information uploaded by the application APP; The multilingual support module is used to ensure that the system supports multiple different languages.

2. The speech translation system according to claim 1, characterized in that: The cloud server further includes a database for storing and retrieving translation data and a machine learning module for optimizing the translation algorithm.

3. The speech translation system according to claim 1, characterized in that: The network uploading module uploads the separated voice information to the cloud server via the mobile data network or Wi-Fi.

4. The speech translation system according to claim 1, characterized in that: The application APP decodes the coded mixed voice information sent by the headset and separates it into voice information of two independent channels.

5. The speech translation system according to claim 1, characterized in that: The Bluetooth connection module is a multi-mode Bluetooth with classic Bluetooth, BLE protocol and SPP protocol.

6. The speech translation system according to claim 1, characterized in that: The translation processing module uses TTS technology to generate an audio file in pcm format to convert text into speech.

7. The speech translation system according to claim 1, characterized in that: Applicable to Apple iOS, Android and Hongmeng systems.

8. The speech translation system according to claim 1, characterized in that: The multi-language support module supports languages ​​of various countries in the world.

9. A translation method applied to the speech translation system according to any one of claims 1 to 8, characterized in that: The following steps are involved: S1. Establish a wireless connection with the mobile device through the Bluetooth connection module of the headset; S2, picking up the user's voice through the voice pickup module of the earphone; S3, mixing the picked-up voice with the downlink voice information of the mobile device through the voice mixing module of the earphone; encoding the mixed voice through the voice encoding module; S4, in the application APP, the mixed voice is decoded by the voice decoding and separation module and separated into voice information of independent channels; S5. Upload the separated voice information to the cloud server through the network upload module of the application APP; S6. The uploaded voice information is translated, transcribed and converted into text-to-speech by a translation processing module in the cloud server; S7, displaying the translated text information on the application APP; S8, encoding the voice information translated by the cloud server through the voice encoding and transmission module of the application APP and sending it back to the headset through the Bluetooth connection module; S9, after the voice decoding module of the earphone decodes the voice, it plays the translated downlink voice of the mobile device through the voice playing module; at the same time, the voice pickup module picks up the translated voice and encodes it through the voice encoding module, and sends it to the mobile device call module, which transmits it to the remote mobile device.

10. A Bluetooth headset, characterized in that: Included A Bluetooth connection module, used to establish a Bluetooth connection with a mobile device; Voice pickup module, used to pick up user voice through a built-in high-sensitivity microphone; A voice mixing module is used to mix the voice information received by the microphone with the downlink voice information of the mobile device to generate mixed voice information; The voice coding module performs OPUS coding or MSBC coding on the mixed voice information and then sends it uplink to the application APP; The voice decoding module is used to perform OPUS decoding or MSBC decoding processing on the received downlink voice information that has been OPUS encoded or MSBC encoded; The voice playback module is used to play the decoded voice.

Images