Real-time call translation method and device

Abstract

The invention provides a real-time call translation method and device, relates to the technical field of data processing, and is applied to a Bluetooth headset, and the Bluetooth headset establishes communication connection with a mobile terminal through a Bluetooth hands-free protocol and establishes communication connection with a translation application running in the mobile terminal through a Bluetooth low-power-consumption general attribute protocol. The method comprises the following steps: acquiring an original sound to be translated, and performing audio compression coding on the original sound to be translated to obtain a compressed audio; transmitting the compressed audio to the translation application through a Bluetooth low-power-consumption general attribute protocol; receiving a translated audio returned by the translation application through a Bluetooth low-power universal attribute protocol, wherein the translated audio is an audio generated after the translation application translates the compressed audio; and carrying out sound mixing processing on the original sound to be translated and the translated audio to obtain a mixed audio, and outputting the mixed audio.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a real-time call translation method and apparatus. Background Technology

[0002] With the acceleration of globalization, the demand for cross-language communication is growing. In scenarios such as business exchanges, international conferences, and cross-border family communication, users urgently need real-time translation capabilities during calls.

[0003] Therefore, how to effectively implement real-time call translation has become an urgent problem to be solved in the industry. Summary of the Invention

[0004] This invention provides a real-time call translation method and apparatus to solve the problem of how to effectively implement real-time call translation function in the prior art.

[0005] This invention provides a real-time call translation method applied to a Bluetooth headset. The Bluetooth headset establishes a communication connection with a mobile terminal via the Bluetooth hands-free protocol and establishes a communication connection with a translation application running on the mobile terminal via the Bluetooth Low Energy Universal Attribute Protocol. The method includes: Obtain the original audio to be translated, and perform audio compression encoding on the original audio to be translated to obtain compressed audio; The compressed audio is transmitted to the translation application via Bluetooth Low Energy Universal Property Protocol. The translation audio returned by the translation application is received via Bluetooth Low Energy Universal Attribute Protocol (BLP). The translation audio is the audio generated by the translation application after translating the compressed audio. The original audio to be translated is mixed with the translated audio to obtain a mixed audio, which is then output.

[0006] According to a real-time call translation method provided by the present invention, the original audio to be translated includes the user's original voice; the step of obtaining the original audio to be translated includes: acquiring the user's voice through a microphone to obtain the user's original voice; The output of the mixed audio includes: sending the mixed audio to the mobile terminal via the Bluetooth hands-free protocol, so that the mobile terminal can transmit the mixed audio to the other end of the call.

[0007] According to the present invention, a real-time call translation method is provided, wherein the original audio to be translated includes the original audio from the other end; The process of obtaining the original audio to be translated includes: receiving the original audio from the other end forwarded by the mobile terminal via Bluetooth hands-free protocol, wherein the original audio from the other end is the voice sent to the mobile terminal by the other end of the call; The output of the mixed audio includes: playing the mixed audio.

[0008] According to a real-time call translation method provided by the present invention, the Bluetooth headset includes a first headset and a second headset, and the method is performed by the first headset; the method further includes: sending the mixed audio to the second headset, and controlling the second headset to play the mixed audio synchronously with the first headset.

[0009] According to a real-time call translation method provided by the present invention, the step of audio compression encoding of the original audio to be translated includes: compressing and encoding the original audio to be translated using the Opus encoding format.

[0010] According to a real-time call translation method provided by the present invention, the step of mixing the original audio to be translated with the translated audio includes: Obtain volume configuration parameters, and adjust the volume ratio between the original audio to be translated and the translated audio according to the volume configuration parameters; The original audio to be translated, after adjusting the volume ratio, is mixed with the translated audio to obtain the mixed audio.

[0011] The present invention also provides a Bluetooth headset, comprising the following modules: a microphone, a processor, and a memory; The microphone is used to capture the user's original voice; The memory stores a computer program, which, when executed by the processor, implements the aforementioned real-time call translation method.

[0012] According to a real-time call translation method provided by the present invention, the mobile terminal establishes a communication connection with a Bluetooth headset via Bluetooth hands-free protocol, the mobile terminal runs a translation application, and the translation application establishes a communication connection with the Bluetooth headset via Bluetooth Low Energy Universal Attribute Protocol; the method includes: The translation application receives compressed audio transmitted by the Bluetooth headset via Bluetooth Low Energy General Attribute Protocol. The compressed audio is translated using the translation application to obtain the translated audio. The translation application transmits the translated audio to the Bluetooth headset via Bluetooth Low Energy Universal Attribute Protocol (BLP), so that the Bluetooth headset can mix the original audio to be translated with the translated audio before outputting it.

[0013] According to a real-time call translation method provided by the present invention, the compressed audio includes user-compressed audio and / or peer-compressed audio; The user compressed audio is the audio obtained by the Bluetooth headset after compressing and encoding the user's original voice. The compressed audio from the other end is the audio obtained by the Bluetooth headset after compressing and encoding the original audio received from the other end via the Bluetooth hands-free protocol.

[0014] According to a real-time call translation method provided by the present invention, the step of translating the compressed audio to obtain translated audio includes: The compressed audio is decoded, and the decoded audio is subjected to speech recognition to obtain speech text; The speech text is translated from the first language to the second language to obtain the translated text. The translated text is then used for speech synthesis to obtain the translated audio.

[0015] According to a real-time call translation method provided by the present invention, the compressed audio is translated using the translation application to obtain translated audio; including: The translation application decodes the compressed audio, performs speech recognition on the decoded audio, and obtains speech text. The speech text is translated from the first language to the second language to obtain the translated text; the translated text is then synthesized to obtain the translated audio.

[0016] The present invention also provides a mobile terminal, comprising: a processor and a memory; The memory stores a computer program, which, when executed by the processor, implements the aforementioned real-time call translation method.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the real-time call translation method as described above.

[0018] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the real-time call translation method as described above.

[0019] The present invention also provides a real-time call translation system, comprising: A Bluetooth headset, comprising a microphone, an audio codec unit, a mixing unit, and a Bluetooth communication unit; A mobile terminal, wherein a translation application is running on the mobile terminal; The Bluetooth headset establishes a first communication connection with the mobile terminal via the Bluetooth communication unit through the Bluetooth hands-free protocol, and establishes a second communication connection with the translation application via the Bluetooth Low Energy Universal Attribute Protocol; the Bluetooth headset and the mobile terminal are used to execute the real-time call translation method described above.

[0020] The real-time call translation method and apparatus provided by this invention uses a Bluetooth headset as an audio relay node, acquires call audio using the Bluetooth hands-free protocol, and establishes an independent data transmission channel with the translation application through the Bluetooth Low Energy Universal Property Protocol. This overcomes the limitations of mobile terminal systems on call audio acquisition, enabling the translation application to acquire the original audio to be translated and process it. Simultaneously, the Bluetooth headset mixes the original audio and the translated audio before outputting it, allowing the user to hear both the original audio and the corresponding translation, ensuring the continuity and naturalness of the call and achieving real-time translation during the call. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is one of the flowcharts of a real-time call translation method provided by the present invention; Figure 2 This is the second flowchart of a real-time call translation method provided by the present invention; Figure 3 A schematic diagram of the architecture of the real-time call translation system based on Bluetooth headset provided by the present invention; Figure 4 A schematic diagram of a Bluetooth headset provided by the present invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] The real-time call translation method based on Bluetooth headsets provided by this invention can be executed by an electronic device with data processing capabilities, including but not limited to the processor in a Bluetooth headset, the processor in a mobile terminal, an embedded controller, or an application processor.

[0025] In this application, the user's original voice or the other party's voice collected is data obtained in compliance with regulations after obtaining the user's authorization and consent.

[0026] Figure 1 This is one of the flowcharts illustrating a real-time call translation method provided by the present invention, such as... Figure 1 As shown, this is applied to a Bluetooth headset, which establishes a communication connection with a mobile terminal via the Bluetooth hands-free protocol and establishes a communication connection with a translation application running on the mobile terminal via the Bluetooth Low Energy Universal Attribute Protocol. In this application, Bluetooth headsets refer to wireless audio devices with Bluetooth communication capabilities, which may be true wireless stereo (TWS) Bluetooth headsets, over-ear Bluetooth headsets, or other forms of Bluetooth audio devices.

[0027] In this embodiment, the Bluetooth headset establishes a communication connection with the mobile terminal via the Bluetooth Hands-Free Profile (HFP). The Bluetooth Hands-Free Profile is a standard Bluetooth audio transmission protocol used to transmit call audio data between the Bluetooth headset and the mobile terminal.

[0028] Through the communication connection established via the Bluetooth hands-free protocol, Bluetooth headsets can receive call audio forwarded by mobile terminals and can also send processed audio to mobile terminals.

[0029] Meanwhile, the Bluetooth headset also establishes a communication connection with the translation application running on the mobile terminal via the Bluetooth Low Energy Generic Attribute Profile (BLE GATT).

[0030] Bluetooth Low Energy Universal Attribute Protocol (BLE) is a data transmission protocol based on Bluetooth Low Energy technology, characterized by low power consumption and stable transmission.

[0031] Translation apps are applications installed and running on mobile devices to process audio for translation. Through a communication connection established via the Bluetooth Low Energy Universal Attribute Protocol (BLE), Bluetooth headsets can exchange data bidirectionally with the translation app, including sending compressed audio to be translated and receiving the translated audio.

[0032] This invention innovatively employs a dual-protocol parallel communication architecture: Bluetooth hands-free protocol is used for the traditional call audio transmission channel, while Bluetooth Low Energy universal protocol is used for the private data transmission channel with translation applications. This dual-protocol architecture design breaks through the limitations of existing mobile phone systems on call audio acquisition, enabling Bluetooth headsets to act as relay devices to acquire audio data from both parties in a call.

[0033] More specifically, the method includes: Step 110: Obtain the original audio to be translated, and perform audio compression encoding on the original audio to be translated to obtain compressed audio; In this application, the original audio to be translated refers to the raw speech audio that needs to be translated. The original audio to be translated can originate from either party in the call scenario, including the voice of the user on this end or the voice sent by the other end of the call.

[0034] There are several ways to obtain the original audio to be translated: it can be obtained by directly capturing the user's voice through the microphone built into a Bluetooth headset; or it can be obtained by receiving voice sent by the other end of a call from a mobile terminal via Bluetooth hands-free protocol. The specific method for obtaining the original audio depends on the needs of the translation scenario.

[0035] After obtaining the original audio to be translated, the original audio is subjected to audio compression encoding to obtain compressed audio.

[0036] Audio compression coding refers to the use of audio coding technology to compress raw audio data in order to reduce the size of the audio data and facilitate its transmission via Bluetooth Low Energy Universal Attribute Protocol.

[0037] Audio compression coding can be implemented using various encoding formats and algorithms, including but not limited to lossy or lossless compression coding. Through audio compression coding, the amount of data transmitted can be significantly reduced, transmission efficiency improved, and transmission latency lowered while maintaining audio quality.

[0038] Step 120: Transmit the compressed audio to the translation application via Bluetooth Low Energy Universal Attribute Protocol; In this application, after the audio compression encoding is completed, the Bluetooth headset transmits the compressed audio to a translation application running on the mobile terminal through a communication connection established by the Bluetooth Low Energy Universal Property Protocol.

[0039] Bluetooth Low Energy (BLE) Universal Attribute Protocol (UAP) transmission features stable data transmission and low power consumption. During transmission, compressed audio data is encapsulated into data packets conforming to the BLE specification and sent to the mobile terminal via the Bluetooth communication link, where it is received and processed by the translation application.

[0040] Step 130: Receive the translated audio returned by the translation application via Bluetooth Low Energy Universal Attribute Protocol (BLP). The translated audio is the audio generated by the translation application after translating the compressed audio. In this application, translated audio refers to the target language audio generated by a translation application after translating the received compressed audio.

[0041] Translation processing includes processes such as speech recognition, language translation, and speech synthesis on audio. After completing the translation process, the translation application generates translated audio that corresponds to the semantics of the original but is in a different language.

[0042] The Bluetooth headset receives the translated audio returned by the translation application via the Bluetooth Low Energy Universal Attribute Protocol (BLE). The translated audio can be in compressed or uncompressed format, and the Bluetooth headset performs appropriate decoding processing as needed after receiving it.

[0043] Step 140: Mix the original audio to be translated with the translated audio to obtain a mixed audio, and output the mixed audio.

[0044] In this application, audio mixing refers to the process of superimposing and synthesizing two or more audio signals to generate a single-channel mixed audio that contains multiple audio contents. In this embodiment, audio mixing synthesizes the original audio to be translated with the translated audio, so that the output mixed audio simultaneously contains the original speech and the translated speech.

[0045] Through audio mixing, users can hear both the original and translated audio simultaneously, allowing them to experience the original speaker's tone and intonation while also understanding the translated semantic content. This effectively solves the problem of maintaining the rhythm of communication in cross-language interactions.

[0046] In this application, the method of outputting the mixed audio can include various implementations: the mixed audio can be sent to a mobile terminal via Bluetooth hands-free protocol, and then transmitted to the other end of the call; alternatively, the mixed audio can be played directly through the speaker of a Bluetooth headset for the local user to listen to. The choice of output method depends on the source of the original audio to be translated and the actual application scenario.

[0047] In this application, a Bluetooth headset is used as an audio relay node. The Bluetooth hands-free protocol is used to acquire call audio, and an independent data transmission channel is established with the translation application via the Bluetooth Low Energy Universal Property Protocol. This overcomes the limitations of mobile terminal systems on call audio acquisition, enabling the translation application to acquire and process the original audio to be translated. Simultaneously, the Bluetooth headset mixes the original audio and the translated audio before outputting it, allowing the user to hear both the original audio and the corresponding translation, ensuring the continuity and naturalness of the call and achieving real-time translation during the call.

[0048] Optionally, the original audio to be translated includes the user's original voice; obtaining the original audio to be translated includes: capturing the user's voice through a microphone to obtain the user's original voice; The output of the mixed audio includes: sending the mixed audio to the mobile terminal via the Bluetooth hands-free protocol, so that the mobile terminal can transmit the mixed audio to the other end of the call.

[0049] In this application, the original audio to be translated includes the user's original audio. The user's original audio refers to the voice emitted by the wearer of the Bluetooth headset, i.e., the user on this end.

[0050] Accordingly, obtaining the original audio to be translated includes: capturing the user's voice through a microphone to obtain the user's original audio.

[0051] The microphone refers to an audio acquisition device installed on a Bluetooth headset, used to collect the user's voice signal. The microphone converts the sound wave signal emitted by the user into an electrical signal, which is then converted from analog to digital to obtain digital audio data, i.e., the user's original voice.

[0052] In the scenario of processing the user's original voice, the mixed audio is sent to the mobile terminal via the Bluetooth hands-free protocol, so that the mobile terminal can transmit the mixed audio to the other end of the call.

[0053] Specifically, after mixing the user's original voice with the translated audio, the Bluetooth headset sends the mixed audio to the mobile terminal through a communication connection established by the Bluetooth hands-free protocol.

[0054] After receiving the mixed audio, the mobile terminal transmits it to the other end of the call via a call link, such as a GSM network, VoLTE network, or the Internet. This allows the other end to hear both the original voice of the user and the translated audio simultaneously, achieving real-time translation and transmission of the user's speech.

[0055] This application implements a real-time translation function for the user's voice, enabling the other end of the call to understand the translated content while hearing the original audio, effectively solving the language barrier problem for the user to express themselves to the other end in cross-language calls.

[0056] Optionally, the original audio to be translated includes the original audio from the other end; The process of obtaining the original audio to be translated includes: receiving the original audio from the other end forwarded by the mobile terminal via Bluetooth hands-free protocol, wherein the original audio from the other end is the voice sent to the mobile terminal by the other end of the call; The output of the mixed audio includes: playing the mixed audio.

[0057] In this application, the original audio to be translated includes the original audio from the other end. The original audio from the other end refers to the voice sent from the other end of the call to this end, that is, the voice of the remote user.

[0058] Accordingly, obtaining the original audio to be translated may specifically refer to receiving the original audio forwarded by the mobile terminal via Bluetooth hands-free protocol, where the original audio is the voice sent by the other end of the call to the mobile terminal.

[0059] Specifically, during a call, the voice of the other end is sent to the local mobile terminal through the call link.

[0060] After receiving the original audio from the other end, the mobile terminal forwards it to the Bluetooth headset via the Bluetooth hands-free protocol. The Bluetooth headset then uses this original audio as the audio to be translated, performing subsequent compression encoding, transmission, and translation processing.

[0061] In the scenario of processing the original audio from the other end, the output of the mixed audio specifically refers to playing the mixed audio; after the Bluetooth headset mixes the original audio from the other end with the translated audio, it plays the mixed audio through the speaker of the Bluetooth headset, so that the user on the local end can hear both the original audio from the other end and the translated audio at the same time.

[0062] This application implements a real-time translation function for the voice of the other end user, enabling the local user to understand the translated content while hearing the original voice of the other end, effectively solving the problem of comprehension barriers when receiving the other end's voice in cross-language calls.

[0063] Optionally, the Bluetooth headset includes a first headset and a second headset, and the method is performed by the first headset; the method further includes: sending the mixed audio to the second headset, and controlling the second headset to play the mixed audio synchronously with the first headset.

[0064] In this application, the Bluetooth headset includes a first earpiece and a second earpiece, which can respectively correspond to the main earpiece and the secondary earpiece in a TWS Bluetooth headset. The above method is performed by the first earpiece.

[0065] Specifically, after the first earphone completes the mixing process and obtains the mixed audio, it sends the mixed audio to the second earphone through the communication link between the earphones.

[0066] The first and second earphones use a synchronization control mechanism to achieve synchronized playback of mixed audio, ensuring that the audio content heard by the user's two ears is consistent in time.

[0067] The second earphone monitors the audio data of the first earphone to obtain the same mixed audio content; the first and second earphones coordinate the playback timing through a synchronization signal to ensure synchronized playback.

[0068] In this application, by working together with the main and secondary earphones, binaural synchronous playback of mixed audio is achieved, avoiding the problem of asynchronous sound playback between the left and right ears and providing users with a better listening experience.

[0069] Optionally, the audio compression encoding of the original audio to be translated includes: compressing and encoding the original audio to be translated using the Opus encoding format.

[0070] In this application, the Opus encoding format is an open and efficient audio encoding format with low latency, high compression ratio, and excellent sound quality.

[0071] The Opus encoding format is particularly suitable for real-time audio transmission scenarios, as it can maintain high audio quality at low bit rates while having low encoding latency.

[0072] Using the Opus encoding format for compression can significantly reduce the amount of audio data transmitted, reduce the data pressure transmitted via Bluetooth Low Energy Universal Attribute Protocol, while ensuring audio quality and improving the accuracy of subsequent speech recognition.

[0073] In this application, by adopting the Opus encoding format, the amount of data transmitted is reduced while ensuring audio quality, thereby improving transmission efficiency and recognition accuracy, and providing an efficient audio compression solution for real-time call translation.

[0074] Optionally, the mixing process of the original audio to be translated and the translated audio includes: Obtain volume configuration parameters, and adjust the volume ratio between the original audio to be translated and the translated audio according to the volume configuration parameters; The original audio to be translated, after adjusting the volume ratio, is mixed with the translated audio to obtain the mixed audio.

[0075] In this application, the volume configuration parameter is configuration data used to control the volume ratio of the original sound and the translated sound. The volume configuration parameter can be set by the user through the control interface of the translation application or Bluetooth headset, or the system default configuration value can be used.

[0076] Adjust the volume ratio between the original audio and the translated audio according to the volume configuration parameters. The volume ratio can be adjusted by controlling the gain of the audio signal, such as reducing the volume of the original audio, increasing the volume of the translated audio, or keeping both at similar volume levels.

[0077] After adjusting the volume ratio, the adjusted original audio to be translated and the translated audio are mixed to obtain a mixed audio. The mixing process can be achieved through the superposition of audio signals.

[0078] In this application, the volume ratio of the original audio to the translated audio is dynamically adjusted through volume configuration parameters, allowing users to adjust the listening effect according to personal preferences and actual scenario needs, solving the problem of rhythm control in cross-language communication and providing a flexible user experience.

[0079] Figure 2 This is a second flowchart illustrating a real-time call translation method provided by the present invention, as shown below. Figure 2As shown, the mobile terminal establishes a communication connection with the Bluetooth headset via the Bluetooth hands-free protocol, and a translation application runs on the mobile terminal. The translation application establishes a communication connection with the Bluetooth headset via the Bluetooth Low Energy Universal Attribute Protocol. Mobile terminals refer to electronic devices with Bluetooth communication capabilities and application running capabilities, including but not limited to smartphones and tablets.

[0080] The mobile terminal establishes a communication connection with the Bluetooth headset via the Bluetooth hands-free protocol for transmitting call audio.

[0081] The mobile terminal runs a translation application, which is an application with voice translation function that can perform voice recognition, translation, and voice synthesis processing on audio.

[0082] The translation application establishes a communication connection with the Bluetooth headset via Bluetooth Low Energy Universal Attribute Protocol (BLP) to receive compressed audio sent by the Bluetooth headset and send translated audio to the Bluetooth headset.

[0083] The method includes: Step 210: Receive compressed audio transmitted by the Bluetooth headset via the Bluetooth Low Energy Universal Attribute Protocol through the translation application. In this application, the translation application receives compressed audio transmitted by a Bluetooth headset through a communication connection established via Bluetooth Low Energy General Attribute Protocol. The compressed audio is the audio data obtained by the Bluetooth headset after audio compression encoding of the original audio to be translated.

[0084] Step 220: Using the translation application, the compressed audio is translated to obtain the translated audio. The translation application processes received compressed audio, including decoding, speech recognition, language translation, and speech synthesis. The application can utilize AI capabilities to complete the translation, obtaining translated audio in the target language that corresponds to the original meaning.

[0085] Step 230: The translated audio is transmitted to the Bluetooth headset via the Bluetooth Low Energy Universal Attribute Protocol through the translation application, so that the Bluetooth headset can mix the original audio to be translated with the translated audio before outputting it.

[0086] After completing the translation process, the translation application transmits the translated audio to a Bluetooth headset via Bluetooth Low Energy Universal Attribute Protocol (BLE). Upon receiving the translated audio, the Bluetooth headset mixes it with the original audio to be translated and outputs the mixed audio.

[0087] In this application, the processing flow of real-time call translation is described from the mobile terminal side. Together with the processing flow from the Bluetooth headset side, a complete real-time call translation function is realized, breaking through the limitations of existing systems on the acquisition of call audio.

[0088] Optionally, the compressed audio includes user-compressed audio and / or peer-compressed audio; The user compressed audio is the audio obtained by the Bluetooth headset after compressing and encoding the user's original voice. The compressed audio from the other end is the audio obtained by the Bluetooth headset after compressing and encoding the original audio received from the other end via the Bluetooth hands-free protocol.

[0089] In this application, the user-compressed audio is the audio obtained by the Bluetooth headset after compressing and encoding the user's original voice.

[0090] Specifically, the Bluetooth headset uses a microphone to capture the user's voice, compresses and encodes the captured original voice to obtain compressed audio, and then transmits it to a translation application for translation processing.

[0091] The compressed audio from the other end is the audio obtained by the Bluetooth headset after compressing and encoding the original audio received from the other end via the Bluetooth hands-free protocol.

[0092] Specifically, the Bluetooth headset receives the voice from the other end of the call forwarded by the mobile terminal via the Bluetooth hands-free protocol, compresses and encodes the received original audio from the other end to obtain compressed audio, and transmits it to a translation application for translation processing.

[0093] The translation application can process both the user's compressed audio and the other party's compressed audio simultaneously, generating corresponding translated audio for each, thus enabling two-way translation of the voice between the two parties in a call.

[0094] This implementation supports both user-compressed audio and peer-compressed audio, enabling omnidirectional translation of voice between the two parties in a call and providing complete translation support for cross-language calls.

[0095] Optionally, the step of translating the compressed audio to obtain the translated audio includes: The compressed audio is decoded, and the decoded audio is subjected to speech recognition to obtain speech text; The speech text is translated from the first language to the second language to obtain the translated text. The translated text is then used for speech synthesis to obtain the translated audio.

[0096] In this application, decoding refers to the process of decompressing and restoring compressed audio to obtain the original audio data. Decoding is implemented using a decoding algorithm corresponding to the encoding format.

[0097] Speech recognition refers to the process of converting audio signals into text content. Speech recognition can be achieved using Automatic Speech Recognition (ASR) technology, which analyzes audio through acoustic and language models to output corresponding speech text. The speech text is text content in the user's first language.

[0098] The first language refers to the language of the original audio being translated, and the second language refers to the target language. The translation application translates the audio text from the first language to the second language, obtaining the translated text. The translation process can be implemented using machine translation technology, including but not limited to neural network-based translation models.

[0099] Speech synthesis refers to the process of converting text content into spoken audio. Speech synthesis can be achieved using text-to-speech (TTS) technology, which converts translated text into natural and fluent translated audio.

[0100] This implementation achieves the conversion from compressed audio to translated audio through a complete processing flow of decoding, speech recognition, translation, and speech synthesis, providing complete audio processing capabilities for real-time call translation.

[0101] In one alternative embodiment, Figure 3 This is a schematic diagram of the architecture of the real-time call translation system based on Bluetooth headset provided by the present invention, as shown below. Figure 3 As shown, the overall architecture of the system consists of six main components: a main earpiece, a secondary earpiece, a translation app, an AI capability module, a local mobile phone, and a remote mobile phone. These components are interconnected through different communication protocols and data transmission channels to jointly achieve real-time translation for cross-language calls.

[0102] The main earphone integrates a complete audio acquisition and processing chain. Starting from the microphone, the user's original voice captured by the microphone first enters the 16K / 16bit real-time noise reduction and AEC module for audio preprocessing. This module is used to eliminate environmental noise and echo interference and improve the clarity of the acquired audio. The audio processed by the noise reduction algorithm is then sent to the 16K / 16bit voice acquisition module for further audio data processing.

[0103] Meanwhile, the main earphone also has the ability to receive external audio. The original sound received from the other end via Bluetooth hands-free protocol enters the audio processing flow of the main earphone through the speaker channel. A mixing module is set inside the main earphone. This mixing module is responsible for mixing audio from different sources, including mixing and superimposing the original sound to be translated with the translated audio returned from the translation APP to generate a mixed audio containing the original sound and the translated sound.

[0104] The main earpiece and the translation app establish a communication connection via the Bluetooth Low Energy Universal Attribute Protocol interface. This interface is a green communication channel bound to the BLE GATT protocol. The main earpiece sends compressed and encoded audio data to the translation app through this channel and receives the translated audio returned by the translation app through the same channel. In addition, the main earpiece also establishes a communication connection with the local mobile phone based on the Bluetooth hands-free protocol through the BTHeadset module for transmitting regular call audio data.

[0105] In the structure of the secondary earphone, it has a simpler processing chain compared to the primary earphone. The secondary earphone is mainly responsible for receiving the mixed audio data from the primary earphone, processing it locally through its built-in mixing module, and then playing it through the speaker. The secondary earphone also establishes a communication connection with the translation app through the Bluetooth Low Energy Universal Attribute Protocol interface to receive the audio data required for synchronized playback. The primary and secondary earphones achieve data synchronization through the earphone communication link to ensure that the audio content played by both ears is consistent in time and avoid the problem of asynchronous sound playback between the left and right ears.

[0106] In the structure of the translation app, the application runs on the local mobile phone and contains multiple functional modules. The translation app receives compressed audio data from the main and secondary earphones through the Bluetooth Low Energy Universal Attribute Protocol interface. The received compressed audio is first decoded by a single-channel Opus decoding module to restore the original audio data. Then the decoded audio is sent to the speech recognition process.

[0107] The translation app has multiple Opus codec modules to process audio data from different sources and destinations. After decoding the audio, the app sends the audio data to the AI ​​capability module for intelligent processing. The AI ​​capability module includes three core components: a recognition module, a translation module, and a synthesis module. The recognition module uses automatic speech recognition technology to recognize the decoded audio and convert the audio content into speech text in the first language. The translation module receives the speech text output by the recognition module and uses machine translation technology to translate the speech text from the first language to the second language to obtain the translated text.

[0108] The synthesis module receives the translated text output by the translation module and uses speech synthesis technology to convert the translated text into natural and fluent translated audio. The three components in the AI ​​capability module process the data in the order of recognition, translation, and synthesis to form a complete translation processing pipeline. After obtaining the translated audio, the translation app compresses and encodes the translated audio through the Opus encoding module, and then transmits the compressed translated audio back to the main and secondary earphones through the Bluetooth Low Energy Universal Attribute Protocol interface. In addition, the translation app also has the ability to save the audio data during the processing as MP3 or WAV format files, which is convenient for users to play back or archive later.

[0109] In the structure of the local mobile phone, the local mobile phone serves as the operating carrier of the translation APP and the platform for the call function. It establishes a communication connection with the main headset via the Bluetooth hands-free protocol. This connection is used to transmit regular call audio, including receiving mixed audio from the main headset and sending it to the remote end through the call link, as well as receiving the original audio from the remote end and forwarding it to the main headset.

[0110] The phone's calling function supports multiple calling methods, including traditional SIM card calls via GSM network and audio / video calls via the Internet. SIM card calls transmit voice data through the operator's network, while Internet calls use WiFi or mobile data networks to enable audio / video calls with various social apps and conferencing software.

[0111] In the structure of the remote mobile phone, the remote mobile phone represents the terminal device used by the other end of the call. It establishes a call connection with the local mobile phone through the GSM network or the Internet. The remote mobile phone receives the mixed audio sent by the local mobile phone, so that the other end of the call can hear the original voice of the local user and the translated voice at the same time. At the same time, the remote mobile phone sends the voice of the other end user to the local mobile phone, and the translation of the other end's voice is realized through the processing process of the local mobile phone and the main headset.

[0112] From the perspective of the overall data flow, the real-time call translation system of the present invention adopts a dual-channel parallel architecture design. The Bluetooth hands-free protocol channel is used to transmit regular call audio to realize voice communication between the headset and the mobile phone, as well as between the two parties in the call. The Bluetooth Low Energy Universal Property Protocol channel is used to transmit compressed audio and translated audio between the headset and the translation APP to realize bidirectional transmission of translation data.

[0113] This dual-channel design cleverly breaks through the limitations of existing mobile phone systems on call audio acquisition, enabling Bluetooth headsets to act as relay devices to acquire audio data from both parties in a call and send it to a translation app for translation processing. This achieves real-time translation of cross-language calls. The system architecture features low latency and high compatibility, and can simultaneously support SIM card call translation and social app audio and video call translation, providing users with a seamless cross-language communication experience.

[0114] Figure 4 This is a schematic diagram of a Bluetooth headset provided by the present invention, as shown below. Figure 4 As shown, a Bluetooth headset includes a microphone, a processor, and memory.

[0115] The microphone is used to capture the user's original voice. Specifically, the microphone is an audio acquisition device mounted on the Bluetooth headset housing, and can be a microelectromechanical system (MEMS), a microphone, an electret microphone, or other types of sound pickup devices.

[0116] A microphone can convert the sound wave signal emitted by the user into an analog electrical signal. This analog electrical signal is then converted into digital audio data, i.e., the user's original voice, by an analog-to-digital converter.

[0117] In one implementation, the microphone's sampling rate can be set to 16kHz or higher to ensure that the captured user's original voice has sufficient audio quality to meet the accuracy requirements of subsequent speech recognition. The microphone can also be used in conjunction with noise reduction algorithms to suppress environmental noise in the captured audio, improving the clarity of the user's original voice.

[0118] In a call scenario, when a user wears a Bluetooth headset to communicate via voice, the microphone continuously collects the user's voice signal and transmits the collected original voice to the processor for further processing.

[0119] The memory stores a computer program, which, when executed by the processor, implements the real-time call translation method described above for Bluetooth headsets. The memory may include both program memory and data memory.

[0120] The program memory can be Flash memory, read-only memory, or other non-volatile storage media, used to store the firmware and computer programs required for processor execution. The data memory can be random access memory or static random access memory, used to store temporary data during program execution, including the audio buffer of the original audio to be translated, compressed audio data, received translated audio, and intermediate data during the mixing process.

[0121] In one implementation, the computer program stored in the memory includes an audio acquisition module, an audio encoding module, a Bluetooth communication module, an audio decoding module, a mixing module, and an audio playback module. These modules work together to achieve a complete functional process, including acquiring the original audio to be translated, compression encoding, Bluetooth transmission, receiving the translated audio, and mixing and outputting the audio.

[0122] The processor is the core control unit of the Bluetooth headset, used to execute the computer program stored in the memory to implement the real-time call translation method of the present invention.

[0123] The processor can be a microcontroller unit, a digital signal processor, a system-on-a-chip, or a dedicated audio processing chip. In one implementation, the processor integrates Bluetooth communication and audio processing functions, and can simultaneously support communication processing for both Bluetooth hands-free protocol and Bluetooth Low Energy Universal Attribute Protocol.

[0124] The processor's clock speed and computing power need to meet the requirements of real-time audio processing, enabling it to complete audio acquisition, encoding, transmission, decoding, and mixing tasks under low latency. When the processor executes the computer program in memory, it controls the microphone to acquire the user's original voice or receives the other end's original voice via Bluetooth hands-free protocol. It then compresses and encodes the acquired original voice to be translated to generate compressed audio, transmits the compressed audio to the translation application via Bluetooth Low Energy Universal Attribute Protocol, receives the translated audio returned by the translation application, and mixes the original voice to be translated with the translated audio to output a mixed audio.

[0125] In practical applications, Bluetooth headsets also include components such as speakers, Bluetooth antennas, batteries, charging ports, and buttons or touch-sensitive areas.

[0126] The speaker is used to play mixed audio, allowing users to listen to calls containing both the original and translated audio. The Bluetooth antenna is used for Bluetooth signal transmission and reception, ensuring stable communication with the mobile terminal. The battery provides power to the Bluetooth headset, supporting its portable use. A charging port or charging contacts are used for battery charging, and can employ a Type-C interface, magnetic contacts, or wireless charging. Buttons or touch-sensitive areas are used for user interaction control; users can control call answering, volume adjustment, and translation function on / off via button or touch operations.

[0127] When the Bluetooth headset is a TWS Bluetooth headset, it includes a first earpiece and a second earpiece, corresponding to the main earpiece and the secondary earpiece, respectively. Both the first and second earpieces include the aforementioned components such as a microphone, processor, memory, and speaker. The first earpiece, as the main device, is responsible for establishing a communication connection with the mobile terminal using the Bluetooth Hands-free protocol and Bluetooth Low Energy General Attribute Protocol, and performing the main audio processing tasks, including audio acquisition, compression encoding, transmission, receiving and transcribing audio, and mixing.

[0128] The second earpiece acts as a slave device, receiving the mixed audio from the first earpiece and playing it synchronously to ensure that the audio content heard by both ears is consistent in time. Data transmission and synchronization control between the first and second earpieces are achieved through an inter-ear communication link, enabling binaural collaboration.

[0129] The Bluetooth headset provided in this embodiment collects the user's original voice through a microphone, and executes a computer program in the memory through a processor to perform functions such as audio compression encoding, Bluetooth transmission, receiving translated audio, and audio mixing. This breaks through the limitations of existing mobile phone systems on the acquisition of call audio, making the Bluetooth headset a key relay node in a real-time call translation system. It realizes the real-time translation function of cross-language calls and has the technical effects of low latency and high compatibility.

[0130] The present invention also provides a mobile terminal, which is another hardware carrier for implementing the above-mentioned real-time call translation method. The mobile terminal works in conjunction with a Bluetooth headset to form a complete real-time call translation system.

[0131] A mobile terminal consists of a processor and memory. The processor is the core computing unit of the mobile terminal, used to execute computer programs stored in memory, including the operating system, translation applications, and other applications.

[0132] The processor can be an application processor, a central processing unit (CPU), or a SoC (System-on-a-Chip) integrating multiple processing units such as CPU, GPU, and DSP. The processor needs sufficient computing power to support the real-time processing of computationally intensive tasks such as speech recognition, machine translation, and speech synthesis in translation applications. In one implementation, the processor can also work with a neural network accelerator or AI coprocessor to accelerate the inference computation of deep learning-based speech recognition and machine translation models in translation applications, thereby reducing translation processing latency and improving the response speed of real-time call translation.

[0133] The memory stores a computer program, which, when executed by the processor, implements the real-time call translation method described above for mobile terminals. The memory may include internal memory and external memory. Internal memory includes RAM, used to store data and instructions during program execution. External memory includes Flash memory or eMMC storage chips, used to store the operating system, translation application, and related data.

[0134] In one implementation, the translation application stored in the memory includes a Bluetooth communication management module, an audio decoding module, a speech recognition module, a machine translation module, a speech synthesis module, and an audio encoding module. These modules work together to achieve a complete functional process, including receiving compressed audio, decoding, speech recognition, translation, speech synthesis, and returning the translated audio.

[0135] When the processor executes the translation application, the mobile terminal receives compressed audio transmitted from the Bluetooth headset via Bluetooth Low Energy Universal Attribute Protocol (BLE), performs translation processing on the compressed audio to obtain translated audio, and transmits the translated audio to the Bluetooth headset via BLE.

[0136] Translation applications can also communicate with cloud-based AI services via the network, offloading tasks such as speech recognition, machine translation, and speech synthesis to cloud servers for execution, thereby achieving higher processing power and more accurate translation results.

[0137] In this implementation, the translation application acts as a client, establishing a connection with the cloud-based AI service via a mobile network or Wi-Fi network. It sends the decoded audio or speech text to the cloud for processing and receives the translated text or audio from the cloud. The translation application connects to an AI capability module, which provides three core functions: recognition, translation, and synthesis. This module can be deployed locally on the mobile terminal or accessed via a network interface on a cloud server.

[0138] In practical applications, mobile terminals also include components such as Bluetooth modules, communication modules, displays, touchscreens, speakers, microphones, and batteries. The Bluetooth module includes a Bluetooth chip and a Bluetooth antenna, used to establish communication connections with Bluetooth headsets via Bluetooth Hands-Free Protocol and Bluetooth Low Energy General Attribute Protocol.

[0139] The communication module includes a cellular communication module and a Wi-Fi module. The cellular communication module supports GSM, LTE, 5G, and other communication standards, and is used for SIM card voice calls and mobile data transmission. The Wi-Fi module is used for wireless network connectivity. The display screen shows the user interface of the translation application, including translation settings, language selection, and real-time translated text display. Users can configure translation language pairs and adjust parameters such as the volume ratio of the original and translated audio through the user interface. The touchscreen is used for user interaction, allowing users to control various functions of the translation application via touch. The mobile terminal's built-in speaker and microphone are used for audio input and output in non-headphone mode. The battery provides power to the mobile terminal.

[0140] The mobile terminal can be a smartphone, including Android smartphones or iOS smartphones.

[0141] Translation apps can be system apps pre-installed on mobile devices or third-party apps downloaded and installed by users from app stores.

[0142] Translation applications require Bluetooth communication permissions to establish a Bluetooth Low Energy Universal Attribute Protocol (BLE) connection with Bluetooth headsets. The mobile terminal can also be a tablet or other electronic device with Bluetooth communication capabilities and application execution capabilities. When the mobile terminal is a tablet, voice calls can be made via VoIP applications, and the translation application can also provide real-time translation of the call audio.

[0143] The mobile terminal provided in this embodiment, by running a translation application and executing corresponding computer programs, realizes functions such as receiving compressed audio, processing translation, and returning translated audio. Working together with a Bluetooth headset, it constitutes a complete real-time call translation system, which can support SIM card call translation and audio and video call translation of various social apps, and has a highly compatible technical effect.

[0144] In one alternative embodiment, the system includes a Bluetooth headset and a mobile terminal.

[0145] Bluetooth headsets include a microphone, an audio codec unit, a mixing unit, and a Bluetooth communication unit. The microphone is used to capture the user's original voice, converting the user's voice into an electrical signal, which is then converted from analog to digital to obtain digital audio data.

[0146] In one implementation, the Bluetooth headset can be equipped with multiple microphones, using beamforming and noise reduction algorithms to improve the quality of the user's original voice capture and effectively suppress environmental noise interference with voice acquisition. The audio codec unit is used for audio compression encoding and decoding, and can be implemented using either hardware or software codecs.

[0147] During the compression encoding process, the audio codec unit encodes the original audio to be translated to generate compressed audio, thereby reducing the amount of data transmitted through the Bluetooth Low Energy Universal Property Protocol. In one implementation, the audio codec unit adopts the Opus encoding format, which features low latency, high compression ratio, and excellent sound quality.

[0148] During the decoding process, the audio codec unit decodes the received translated audio, restoring it to playable audio data. The mixing unit mixes the original audio and the translated audio, superimposing and mixing them at a certain volume ratio to generate a mixed audio. The mixing unit supports dynamic adjustment of the volume ratio, allowing users to set the volume ratio of the original audio and the translated audio according to their personal preferences. In one implementation, the mixing unit can also perform volume normalization on the mixed audio to prevent clipping distortion after mixing.

[0149] The Bluetooth communication unit is used to communicate with mobile terminals via the Bluetooth protocol. It includes a Bluetooth chip and a Bluetooth antenna, supports two communication modes: Bluetooth hands-free protocol and Bluetooth Low Energy Universal Attribute Protocol, and can maintain communication connections of both protocols simultaneously, enabling parallel processing of call audio transmission and translation data transmission.

[0150] In addition, Bluetooth headsets also include speakers for playing mixed audio, processors for controlling the coordinated work of various components and performing audio processing tasks, memory for storing firmware programs and operating data, and batteries for powering the Bluetooth headset.

[0151] When the Bluetooth earphone is a TWS Bluetooth earphone, the Bluetooth earphone includes a first earphone and a second earphone.

[0152] The first earpiece, acting as the master device, includes a complete microphone, audio codec unit, mixing unit, and Bluetooth communication unit. It is responsible for establishing a communication connection with the mobile terminal and performing the main audio processing tasks. The second earpiece, acting as the slave device, receives the mixed audio data from the first earpiece and plays it synchronously with it, enabling binaural listening and avoiding the problem of asynchronous sound playback between the left and right ears.

[0153] A translation application runs on the mobile device. The mobile device can be a smartphone, tablet, or other electronic device with Bluetooth communication capabilities.

[0154] Translation apps are applications installed on mobile devices that have speech recognition, machine translation, and speech synthesis capabilities.

[0155] The translation application connects to the AI ​​capability module, which provides three core functions: recognition, translation, and synthesis.

[0156] Within the translation application, there is an Opus decoding module for decoding compressed audio, and an interface for interacting with AI capabilities to call speech recognition, translation, and speech synthesis services.

[0157] AI capability modules can be deployed locally on mobile devices to achieve offline translation, or deployed on cloud servers to achieve online translation via network interfaces. Online translation typically achieves higher translation accuracy and richer language support.

[0158] In this embodiment, the Bluetooth headset establishes a dual-channel communication connection with the mobile terminal via a Bluetooth communication unit. The first communication connection is established via the Bluetooth hands-free protocol and is used to transmit call audio, including sending mixed audio to the mobile terminal for call transmission and receiving voice from the other end of the call forwarded by the mobile terminal.

[0159] Bluetooth hands-free calling is a standard Bluetooth calling protocol that is compatible with the phone functions of mobile devices and the audio and video calling functions of various social apps. The local mobile phone connects to the remote mobile phone via a call link, supporting GSM network calls and internet audio and video calls.

[0160] The second communication connection is established via Bluetooth Low Energy (BLE) General Attribute Protocol (GAP) for private data transmission between the Bluetooth headset and the translation application. This includes the Bluetooth headset sending compressed audio to the translation application, and the translation application returning translated audio to the Bluetooth headset. BLE features low power consumption and stable transmission, making it suitable for continuous data transmission scenarios.

[0161] Based on the above system architecture, the Bluetooth headset executes the real-time call translation method applied to the Bluetooth headset, and the mobile terminal executes the real-time call translation method applied to the mobile terminal. Taking a SIM card call translation scenario as an example, the complete workflow of the system is illustrated.

[0162] In the local audio processing flow, the local user wears a Bluetooth headset to make a call. When the user speaks, the microphone of the Bluetooth headset captures the user's original voice. The audio codec unit compresses and encodes the user's original voice to generate local opus data. The Bluetooth communication unit transmits the local opus data to the translation application through a second communication connection.

[0163] The translation application decodes the local Opus data to restore the original audio, uses the AI ​​capability module's recognition function to perform speech recognition to obtain speech text, uses the translation function to translate the speech text from the first language to the second language to obtain translated text, and uses the synthesis function to synthesize the translated text into local translated audio. The translation application returns the local translated audio to the Bluetooth headset via a second communication connection. The mixing unit mixes the user's original voice with the local translated audio to generate mixed audio, and the Bluetooth communication unit sends the mixed audio to the mobile terminal via a first communication connection.

[0164] The mobile terminal sends the mixed audio to the other end of the call via the call link, and then to the remote mobile phone via the GSM network or the Internet. The other end of the call hears the original voice of the local user and the translated voice, and can simultaneously feel the tone and intonation of the local user and understand the translated semantic content.

[0165] In the remote audio processing flow, the voice from the other end of the call is sent to the local mobile terminal via the call link. The mobile terminal then forwards the original audio from the other end to the Bluetooth headset via the first communication connection. The audio codec unit compresses and encodes the original audio from the other end to generate remote OPUS data, and the Bluetooth communication unit transmits the remote OPUS data to the translation application via the second communication connection.

[0166] The translation application performs the same decoding, speech recognition, translation, and speech synthesis processes on the remote Opus data as on the local audio processing end, generating remote translated audio. The translation application returns the remote translated audio to the Bluetooth headset via a second communication connection. The mixing unit mixes the original audio from the remote end with the remote translated audio to generate a mixed audio. The first headset plays the mixed audio and simultaneously sends it to the second headset. The first and second headsets play the mixed audio synchronously, allowing the local user to hear both the original audio from the remote end and the translated audio simultaneously in both ears.

[0167] It also supports translation of audio and video calls from social apps, including WeChat voice calls, video calls, Tencent Meeting, etc.

[0168] In these scenarios, the workflow is the same as that of SIM card call translation. Audio and video call audio from the social app are transmitted between the Bluetooth headset and the mobile terminal via Bluetooth hands-free protocol. The Bluetooth headset acquires the audio from both parties and transmits it to the translation application for translation processing via Bluetooth Low Energy Universal Attribute Protocol (BLE), achieving real-time cross-language translation. Since Bluetooth hands-free protocol is a standard Bluetooth calling protocol, it is compatible with the audio interfaces of various social apps. Therefore, the real-time call translation system provided by this invention has high compatibility and can implement call translation functionality without modifying the social app.

[0169] The real-time call translation system based on Bluetooth headset provided in this embodiment innovatively uses a Bluetooth headset as a relay device and acquires audio from both parties in a call through a dual-protocol parallel communication architecture. This breaks through the limitations of existing mobile phone systems on acquiring call audio and solves the problem that non-system applications cannot acquire call audio.

[0170] Employing audio compression encoding and dual-channel parallel transmission technology, the system reduces audio transmission and processing latency, enabling real-time translation. It supports SIM card call translation and audio / video call translation for various social media apps, offering broad application scenarios and high compatibility.

[0171] The system employs a highly efficient audio compression encoding format, reducing the amount of data transmitted while maintaining audio quality and improving speech recognition accuracy. Through the coordinated operation of primary and secondary earphones, the system achieves synchronized binaural playback of mixed audio, providing a superior user experience. The system supports dynamic adjustment of the original and translated audio volume ratio to meet individual user needs, and the mixing of original and translated audio solves the problem of rhythm control in cross-language communication.

[0172] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a real-time call translation method, which includes: acquiring the original audio to be translated, performing audio compression encoding on the original audio to be translated, and obtaining compressed audio; The compressed audio is transmitted to the translation application via Bluetooth Low Energy Universal Property Protocol. The translation audio returned by the translation application is received via Bluetooth Low Energy Universal Attribute Protocol (BLP). The translation audio is the audio generated by the translation application after translating the compressed audio. The original audio to be translated is mixed with the translated audio to obtain a mixed audio, which is then output.

[0173] Alternatively, the translation application may receive compressed audio transmitted by the Bluetooth headset via Bluetooth Low Energy General Attribute Protocol. The compressed audio is translated using the translation application to obtain the translated audio. The translation application transmits the translated audio to the Bluetooth headset via Bluetooth Low Energy Universal Attribute Protocol (BLP), so that the Bluetooth headset can mix the original audio to be translated with the translated audio before outputting it.

[0174] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute the real-time call translation method provided by the above methods, the method including: acquiring the original audio to be translated, performing audio compression encoding on the original audio to be translated, and obtaining compressed audio; The compressed audio is transmitted to the translation application via Bluetooth Low Energy Universal Property Protocol. The translation audio returned by the translation application is received via Bluetooth Low Energy Universal Attribute Protocol (BLP). The translation audio is the audio generated by the translation application after translating the compressed audio. The original audio to be translated is mixed with the translated audio to obtain a mixed audio, which is then output.

[0176] Alternatively, the translation application may receive compressed audio transmitted by the Bluetooth headset via Bluetooth Low Energy General Attribute Protocol. The compressed audio is translated using the translation application to obtain the translated audio. The translation application transmits the translated audio to the Bluetooth headset via Bluetooth Low Energy Universal Attribute Protocol (BLP), so that the Bluetooth headset can mix the original audio to be translated with the translated audio before outputting it.

[0177] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the real-time call translation method provided by the above methods, the method comprising: acquiring the original audio to be translated, performing audio compression encoding on the original audio to be translated, and obtaining compressed audio; The compressed audio is transmitted to the translation application via Bluetooth Low Energy Universal Property Protocol. The translation audio returned by the translation application is received via Bluetooth Low Energy Universal Attribute Protocol (BLP). The translation audio is the audio generated by the translation application after translating the compressed audio. The original audio to be translated is mixed with the translated audio to obtain a mixed audio, which is then output.

[0178] Alternatively, the translation application may receive compressed audio transmitted by the Bluetooth headset via Bluetooth Low Energy General Attribute Protocol. The compressed audio is translated using the translation application to obtain the translated audio. The translation application transmits the translated audio to the Bluetooth headset via Bluetooth Low Energy Universal Attribute Protocol (BLP), so that the Bluetooth headset can mix the original audio to be translated with the translated audio before outputting it.

[0179] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0180] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time call translation method, characterized in that, The method is applied to Bluetooth headsets, wherein the Bluetooth headset establishes a communication connection with a mobile terminal via the Bluetooth hands-free protocol and establishes a communication connection with a translation application running on the mobile terminal via the Bluetooth Low Energy Universal Attribute Protocol; the method includes: Obtain the original audio to be translated, and perform audio compression encoding on the original audio to be translated to obtain compressed audio; The compressed audio is transmitted to the translation application via Bluetooth Low Energy Universal Property Protocol. The translation audio returned by the translation application is received via Bluetooth Low Energy Universal Attribute Protocol (BLP). The translation audio is the audio generated by the translation application after translating the compressed audio. The original audio to be translated is mixed with the translated audio to obtain a mixed audio, which is then output.

2. The real-time call translation method according to claim 1, characterized in that, The original audio to be translated includes the user's original audio; The process of obtaining the original audio to be translated includes: capturing the user's voice through a microphone to obtain the user's original audio. The output of the mixed audio includes: sending the mixed audio to the mobile terminal via the Bluetooth hands-free protocol, so that the mobile terminal can transmit the mixed audio to the other end of the call.

3. The real-time call translation method according to claim 1, characterized in that, The original audio to be translated includes the original audio from the other end; the acquisition of the original audio to be translated includes: receiving the original audio from the other end forwarded by the mobile terminal via Bluetooth hands-free protocol, wherein the original audio from the other end is the voice sent to the mobile terminal by the other end of the call; The output of the mixed audio includes: playing the mixed audio.

4. The real-time call translation method according to claim 3, characterized in that, The Bluetooth headset includes a first headset and a second headset, and the method is performed by the first headset; the method further includes: sending the mixed audio to the second headset, and controlling the second headset to play the mixed audio synchronously with the first headset.

5. The real-time call translation method according to claim 1, characterized in that, The audio compression encoding of the original audio to be translated includes: compressing and encoding the original audio to be translated using the Opus encoding format.

6. The real-time call translation method according to claim 1, characterized in that, The process of mixing the original audio to be translated with the translated audio includes: Obtain volume configuration parameters, and adjust the volume ratio between the original audio to be translated and the translated audio according to the volume configuration parameters; The original audio to be translated, after adjusting the volume ratio, is mixed with the translated audio to obtain the mixed audio.

7. A real-time call translation method, characterized in that, The method is applied to a mobile terminal, wherein the mobile terminal establishes a communication connection with a Bluetooth headset via a Bluetooth hands-free protocol, and a translation application runs on the mobile terminal, the translation application establishing a communication connection with the Bluetooth headset via a Bluetooth Low Energy Universal Attribute Protocol; the method includes: The translation application receives compressed audio transmitted by the Bluetooth headset via Bluetooth Low Energy General Attribute Protocol. The compressed audio is translated using the translation application to obtain the translated audio. The translation application transmits the translated audio to the Bluetooth headset via Bluetooth Low Energy Universal Attribute Protocol (BLP), so that the Bluetooth headset can mix the original audio to be translated with the translated audio before outputting it.

8. The real-time call translation method according to claim 7, characterized in that, The compressed audio includes user-compressed audio and / or peer-compressed audio; The user compressed audio is the audio obtained by the Bluetooth headset after compressing and encoding the user's original voice. The compressed audio from the other end is the audio obtained by the Bluetooth headset after compressing and encoding the original audio received from the other end via the Bluetooth hands-free protocol.

9. The real-time call translation method according to claim 7, characterized in that, The compressed audio is translated using the translation application to obtain translated audio, including: The translation application decodes the compressed audio, performs speech recognition on the decoded audio, and obtains speech text. The speech text is translated from the first language to the second language to obtain the translated text; the translated text is then synthesized to obtain the translated audio.

10. A Bluetooth headset, characterized in that, include: Microphone, processor, and memory; The microphone is used to capture the user's original voice; The memory stores a computer program, which, when executed by the processor, implements the real-time call translation method according to any one of claims 1 to 6.

11. A mobile terminal, characterized in that, Includes: processor, memory; The memory stores a computer program, which, when executed by the processor, implements the real-time call translation method according to any one of claims 7 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the real-time call translation method as described in any one of claims 1 to 9.

13. A real-time call translation system, characterized in that, include: A Bluetooth headset, comprising a microphone, an audio codec unit, a mixing unit, and a Bluetooth communication unit; A mobile terminal, wherein a translation application is running on the mobile terminal; The Bluetooth headset establishes a first communication connection with the mobile terminal via the Bluetooth communication unit through the Bluetooth hands-free protocol, and establishes a second communication connection with the translation application via the Bluetooth Low Energy Universal Attribute Protocol. The Bluetooth headset is used to perform the real-time call translation method according to any one of claims 1 to 6; the mobile terminal is used to perform the real-time call translation method according to any one of claims 7 to 9.

Images