A multilingual, multi-terminal simultaneous interpretation system based on TWS earphones and its control method

Abstract

This invention provides a multilingual, multi-terminal simultaneous interpretation system and its control method based on TWS earphones. The system includes: TWS earphones for acquiring audio to be translated and performing voice encoding to obtain encoded audio, which is then sent to a corresponding central control device; receiving decoded translation audio sent by the central control device; the central control device for receiving the encoded audio sent by the TWS earphones, performing audio digitization processing and encapsulation to obtain encapsulated audio, which is then sent to a translation terminal; receiving translation audio sent by the translation terminal, independently splitting and decoding the translation audio to obtain decoded translation audio for each language, and sending it to the corresponding language TWS earphones; and the translation terminal for receiving the encapsulated audio sent by the central control device, inputting the encapsulated audio into a translation model, outputting the translation audio, and sending the translation audio to the central control device. This invention enables simultaneous interpretation across multiple languages ​​and terminals and is capable of use in a private local area network environment.

Description

Technical Field

[0001] This invention relates to the field of voice translation control technology, and in particular to a multilingual, multi-terminal simultaneous interpretation system based on TWS earphones and its control method. Background Technology

[0002] Currently, domestic technical solutions for simultaneous interpretation access and control can block the original audio and only hear the target language translation, improving the user experience. They can also achieve linkage control between wired headphones and voice systems, providing headphone translation capabilities and solving the problems of low accuracy and poor privacy in simultaneous interpretation with open-back headphones in noisy environments.

[0003] These methods primarily utilize software algorithms or a combination of hardware and software to achieve simultaneous interpretation functionality on a single Bluetooth headset device. However, none of them address the issue of multiple Bluetooth headset devices simultaneously accessing the translation system for multiple languages, failing to meet the needs of multilingual teaching and training, multilingual international conferences, and other similar scenarios. Furthermore, existing methods all require internet access to function, making simultaneous interpretation impossible on privately deployed translation systems.

[0004] Existing simultaneous interpretation systems can only connect to one headset device, which cannot meet the translation needs of multiple people interacting simultaneously, such as multilingual teaching and training, and international conferences; existing simultaneous interpretation systems all require access to the Internet to be used, and do not have the ability to be used in a private local area network environment, which cannot meet the needs of use with confidentiality requirements; existing simultaneous interpretation systems have high requirements for the headset devices connected, making them relatively inconvenient to use. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multilingual, multi-terminal simultaneous interpretation system and its control method based on TWS earphones, which can realize simultaneous interpretation of multiple languages ​​and multiple terminals and has the ability to be used in a local area network private environment.

[0006] To achieve the above objectives, the present invention is implemented using the following technical solution: On one hand, the present invention provides a multilingual, multi-terminal simultaneous interpretation system based on TWS earphones, comprising: TWS earphones are used to collect audio to be translated, perform voice encoding on the audio to be translated to obtain encoded audio, and send the encoded audio to the corresponding central control device; and receive decoded translated audio sent by the central control device; The central control device is used to receive encoded audio sent by TWS earbuds, perform audio digitization processing on the encoded audio to obtain pre-processed audio, encapsulate the pre-processed audio to obtain encapsulated audio, and send the encapsulated audio to the translation end; it also receives translated audio sent by the translation end, performs independent streaming of the translated audio according to the unique language tag bound to the TWS earbuds to obtain translated audio in each language, decodes the translated audio in each language to obtain decoded translated audio in each language, and sends the decoded translated audio in each language to the TWS earbuds of the corresponding language; The translation end is used to receive the encapsulated audio sent by the central control device, input the encapsulated audio into the translation model, output the translated audio, and send the translated audio to the central control device.

[0007] When applied, this invention enables simultaneous translation across multiple languages ​​and multiple earphone terminals via a central control device, meeting the needs of scenarios such as cross-border conferences and foreign language teaching and training. It significantly improves translation efficiency, solves the problem of simultaneous multilingual translation, and ensures that all TWS earphones connected to this device are in the same translation model session. This allows the translation model to perform better semantic analysis, thereby improving translation accuracy. It allows ordinary TWS earphones to access translation and meets the translation needs of private deployments with confidentiality requirements or in professional fields, reducing the cost of access devices while expanding the scope of application.

[0008] Optionally, the TWS earphones are further configured to extract the user's voice data and mask the voice data of non-users to obtain noise-reduced audio for translation. This invention utilizes AI noise reduction to achieve noise masking and, based on the extracted language information, automatically binds language tags, improving convenience.

[0009] Optionally, the central control device includes a Bluetooth radio frequency unit, an audio codec encoding / decoding chip, a main control SoC processor, and a network communication module; The Bluetooth radio frequency unit is used to receive encoded audio sent by the TWS earphones and send the encoded audio to the audio codec chip; receive decoded and translated audio in various languages ​​sent by the audio codec chip and send the decoded and translated audio in various languages ​​to the TWS earphones of the corresponding languages; The audio codec chip is used to receive encoded audio sent by the Bluetooth radio frequency unit, perform audio digitization processing on the encoded audio to obtain pre-processed audio, and send the pre-processed audio to the main control SoC processor; receive translated audio in various languages ​​sent by the main control SoC processor, decode the translated audio in various languages ​​to obtain decoded translated audio in various languages, and send the decoded translated audio in various languages ​​to the Bluetooth radio frequency unit. The main control SoC processor is used to receive pre-processed audio sent by the audio codec encoding / decoding chip, perform parallel processing and encapsulation on the pre-processed audio to obtain encapsulated audio, and send the encapsulated audio to the network communication module; receive translated audio sent by the network communication module, and independently split the translated audio according to the unique language tag bound to the TWS earphone to obtain translated audio in each language, and send the translated audio in each language to the audio codec encoding / decoding chip; The network communication module is used to receive encapsulated audio sent by the main control SoC processor and send the encapsulated audio to the translation end; and to receive translated audio sent by the translation end and send the translated audio to the main control SoC processor.

[0010] Optionally, the Bluetooth radio frequency unit includes an antenna array, a BLE-Mesh control module, and a multi-channel Bluetooth audio module; The antenna array is used to receive encoded audio sent by TWS earphones and send the encoded audio to the BLE-Mesh control module; receive decoded and translated audio in various languages ​​sent by the BLE-Mesh control module and send the decoded and translated audio in various languages ​​to the corresponding TWS earphones; The BLE-Mesh control module is used to receive coded audio transmitted by the antenna array, allocate time slots and number channels sequentially according to the unique language tag to obtain coded audio for each channel, and send the coded audio for each channel to the multi-channel Bluetooth audio module; it also receives decoded and translated audio for each language transmitted by the multi-channel Bluetooth audio module, and sends the decoded and translated audio for each language to the antenna array according to the unique language tag. The multi-channel Bluetooth audio module is used to receive encoded audio from each channel sent by the BLE-Mesh control module, and send the encoded audio from each channel to the audio codec chip; it also receives decoded audio from the audio codec chip and sends the decoded audio to the BLE-Mesh control module.

[0011] Optionally, the BLE-Mesh control module is also used to bind a unique language tag based on the unique ID of the TWS earphone, or to bind a unique language tag based on the language information of the audio to be translated collected by the TWS earphone. The BLE-Mesh control module is responsible for device network access, pairing, allocation of timing slots, channel numbering, online device detection, disconnection and reconnection, permission management, prevention of Bluetooth channel conflicts, and ensuring no crosstalk or interference when multiple devices are connected concurrently.

[0012] Optionally, the network communication module includes a wired connection module and a wireless WIFI module; The wired connection module is used for wired data transmission with the translation model deployed offline or in the cloud at the translation terminal; The wireless WIFI module is used for wireless data transmission with the translation model deployed offline or in the cloud at the translation terminal.

[0013] Optionally, network interconnection devices are also included to establish data transmission between the central control device and the translation terminal.

[0014] Optionally, the translation model can be an internet translation model or a local area network translation model. The translation model is designed to allow access to both public and private translation models, thus expanding its applicability.

[0015] On the other hand, the present invention provides a control method for a multilingual, multi-terminal simultaneous interpretation system based on TWS earphones, applied to the system described in the first aspect, the method comprising: TWS earphones collect the audio to be translated, perform voice encoding on the audio to be translated, obtain encoded audio, and send the encoded audio to the corresponding central control device; The central control device receives encoded audio sent by TWS earphones, performs audio digitization processing on the encoded audio to obtain pre-processed audio, encapsulates the pre-processed audio to obtain encapsulated audio, and sends the encapsulated audio to the translation end; The translation terminal receives the encapsulated audio sent by the central control device, inputs the encapsulated audio into the translation model, outputs the translated audio, and sends the translated audio back to the central control device. The central control device receives the translated audio sent by the translation terminal, and independently splits the translated audio according to the unique language tag bound to the TWS earphone to obtain translated audio in each language. The translated audio in each language is then decoded to obtain decoded translated audio in each language, and the decoded translated audio in each language is sent to the TWS earphone in the corresponding language. TWS earbuds receive decoded and translated audio sent by a central control device.

[0016] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: This invention enables simultaneous multilingual and multi-terminal translation via a central control device. Audio in the language of this headset can be translated into corresponding audio in other headset languages ​​and played on those headsets, meeting the needs of cross-border conferences and foreign language teaching and training scenarios. It significantly improves translation efficiency, solves the problem of simultaneous multilingual translation, and ensures that all TWS headsets connected to this device are in the same translation model session, allowing the translation model to perform better semantic analysis and thus improve translation accuracy. It allows ordinary TWS headsets to access translation and meets the translation needs of private deployments with confidentiality requirements or in professional fields, reducing access device costs while expanding applicability. It lowers the cost of accessing translation devices and increases access flexibility and applicability. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of a multilingual, multi-terminal simultaneous interpretation system based on TWS earphones provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the control method for a multilingual, multi-terminal simultaneous interpretation system based on TWS earphones provided in an embodiment of the present invention. In the picture: 1. TWS earphones; 2. Central control device; 3. Network interconnection device; 4. Translation terminal. Detailed Implementation

[0018] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0019] The term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0020] Example 1

[0021] like Figure 1 As shown, this embodiment introduces a multilingual, multi-terminal simultaneous interpretation system based on TWS earphones, including: The True Wireless Stereo (TWS) earphone 1 is used to capture the audio to be translated, encode the audio to be translated into speech, obtain the encoded audio, and send the encoded audio to the corresponding central control device 2; and receive the decoded translated audio sent by the central control device 2. The central control device 2 is used to receive the encoded audio sent by the TWS earphone 1, perform audio digitization processing on the encoded audio to obtain pre-processed audio, encapsulate the pre-processed audio to obtain encapsulated audio, and send the encapsulated audio to the translation terminal 4; it receives the translated audio sent by the translation terminal 4, performs independent streaming of the translated audio according to the unique language tag bound to the TWS earphone 1 to obtain translated audio in each language, decodes the translated audio in each language to obtain decoded translated audio in each language, and sends the decoded translated audio in each language to the TWS earphone 1 of the corresponding language; Translation terminal 4 is used to receive the encapsulated audio sent by central control device 2, input the encapsulated audio into the translation model, output the translated audio, and send the translated audio back to central control device 2.

[0022] This embodiment enables simultaneous translation across multiple languages ​​and multiple earphone terminals via a central control device. Audio in the language of one earphone can be translated into the corresponding audio in other earphone languages ​​and played on those earphones. This meets the needs of scenarios such as cross-border conferences and foreign language teaching and training, significantly improving translation efficiency and solving the problem of simultaneous multilingual translation. It ensures that all TWS earphones connected to this device are in the same translation model session, allowing the translation model to perform better semantic analysis and thus improve translation accuracy. It allows ordinary TWS earphones to access translation and meets the translation needs of private deployments with confidentiality requirements or in professional fields, reducing access device costs while expanding applicability. It lowers the cost of accessing translation devices and increases access flexibility and applicability.

[0023] Example 2

[0024] Based on Example 1, this example introduces a multilingual, multi-terminal simultaneous interpretation system based on TWS earphones, such as... Figure 1 As shown, it specifically includes TWS earphones 1, central control device 2, network interconnection device 3, and translation terminal 4, wherein: TWS earphone 1 is used to collect the audio to be translated, perform voice encoding on the audio to be translated to obtain encoded audio, and send the encoded audio to the corresponding central control device 2; and receive the decoded translated audio sent by the central control device 2.

[0025] In this embodiment, a central control device can connect to multiple TWS earphones, such as three TWS earphones, including earphone 1, earphone 2, and earphone 3, which correspond to language 1, language 2, and language 3 respectively. There can be multiple central control devices, and each central control device can connect to multiple TWS earphones.

[0026] The TWS earphone 1 is also used to extract the user's voice data and mask the voice data of non-users to obtain noise-reduced audio for translation. Subsequent translations will use the noise-reduced audio.

[0027] To reduce environmental noise interference, artificial intelligence (AI) noise reduction is employed. Before starting the translation dialogue, the user uses TWS earphones to conduct a few seconds of sound testing, extracting the user's voice data and masking the voice data of non-users, thereby achieving the AI ​​noise reduction function.

[0028] The central control device 2 includes a Bluetooth radio frequency unit, an audio codec (coder-decoder) chip, a system-on-chip (SoC) processor, and a network communication module. The central control device 2 enables the TWS earphones 1 connected to it to be in the same translation model session, allowing the translation model to perform better semantic analysis and thus improve translation accuracy.

[0029] The Bluetooth radio frequency unit includes an antenna array, a Bluetooth Low Energy Mesh (BLE-Mesh) control module, and a multi-channel Bluetooth audio module.

[0030] The Bluetooth radio frequency unit is used to receive encoded audio sent by TWS earbuds 1 and send the encoded audio to the audio codec chip; it also receives decoded and translated audio in various languages ​​sent by the audio codec chip and sends the decoded and translated audio in each language to the corresponding TWS earbuds 1.

[0031] Specifically: The antenna array is used to receive encoded audio sent by TWS earbuds 1 and transmit the encoded audio to the BLE-Mesh control module; it also receives decoded audio in various languages ​​sent by the BLE-Mesh control module and transmits the decoded audio in each language to the corresponding TWS earbuds 1. This antenna array is a 2.4G diversity antenna array.

[0032] The BLE-Mesh control module receives encoded audio transmitted by the antenna array, allocates time slots and assigns channel numbers to the encoded audio according to the unique language tag, obtains the encoded audio for each channel, and sends the encoded audio for each channel to the multi-channel Bluetooth audio module; it also receives decoded and translated audio for each language transmitted by the multi-channel Bluetooth audio module, and sends the decoded and translated audio for each language to the antenna array according to the unique language tag.

[0033] The BLE-Mesh control module is also used to bind a unique language tag (i.e., the language of the audio emitted by the TWS earphone 1) based on the unique ID of the TWS earphone 1, or to bind a unique language tag based on the language information of the audio to be translated collected by the TWS earphone 1. The audio to be translated is noise-reduced audio. Based on the language information obtained when extracting the timbre during AI noise reduction, the module automatically binds the corresponding language tag to achieve the translation of the target language and the transmission of the translated language audio data, thereby realizing the simultaneous multilingual translation function.

[0034] The BLE-Mesh control module is responsible for device network access, pairing, allocating timing slots and channel numbers, online device detection, disconnection and reconnection, access control, preventing Bluetooth channel conflicts, and ensuring that multiple devices can operate concurrently without crosstalk or interference.

[0035] The multi-channel Bluetooth audio module is used to receive encoded audio from each channel sent by the BLE-Mesh control module and send the encoded audio from each channel to the audio codec chip; it also receives decoded audio from the audio codec chip and sends the decoded audio to the BLE-Mesh control module.

[0036] Then: The audio codec chip is used to receive encoded audio sent by the multi-channel Bluetooth audio module in the Bluetooth RF unit, perform audio digitization processing on the encoded audio to obtain pre-processed audio, and send the pre-processed audio to the main control SoC processor; it also receives translated audio in various languages ​​sent by the main control SoC processor, decodes the translated audio in various languages ​​to obtain decoded translated audio in various languages, and sends the decoded translated audio in various languages ​​to the multi-channel Bluetooth audio module in the Bluetooth RF unit.

[0037] The audio codec chip is responsible for the digital preprocessing of all uplink audio from the headphones, which is then handed over to the central control SoC processor. The downlink translated audio is decoded and amplified before being sent back to the Bluetooth radio frequency for distribution to the corresponding headphones.

[0038] The main control SoC processor receives pre-processed audio sent by the audio codec chip, performs parallel processing and encapsulation on the pre-processed audio to obtain encapsulated audio, and sends the encapsulated audio to the network communication module; it also receives translated audio sent by the network communication module, and independently splits the translated audio according to the unique language tag bound to the TWS earphone to obtain translated audio in each language, and sends the translated audio in each language to the audio codec chip.

[0039] The central control SoC processor is responsible for parallel processing of multiple audio tasks, protocol encapsulation, and translation model docking. Each channel is configured with an independent buffer queue to completely eliminate crosstalk, audio dropouts, and latency jitter.

[0040] The network communication module is used to receive the encapsulated audio sent by the main control SoC processor and send the encapsulated audio to the translation terminal 4; and to receive the translated audio sent by the translation terminal 4 and send the translated audio to the main control SoC processor.

[0041] The network communication module includes a wired connection module and a Wireless Fidelity (WIFI) module. It is a universal network module and supports high-speed cache storage.

[0042] The wired connection module is used for wired data transmission with the translation model deployed offline or in the cloud in translation terminal 4.

[0043] The wireless WIFI module is used for wireless data transmission with the translation model deployed offline or in the cloud in translation terminal 4.

[0044] Network interconnection device 3 is used to establish data transmission between central control device 2 and translation terminal 4. Network interconnection device 3 can be a router or a switch.

[0045] Both the wired connection module and the wireless WIFI module transmit data to the translation model in the translation terminal 4 through the network interconnection device 3.

[0046] The network communication module can connect to routers or switches via wired and wireless means, enabling wired / wireless access to translation models deployed offline (i.e., cloud-based). It can satisfy the need to access translation models on the external network, as well as the need to access self-deployed translation models for confidentiality purposes.

[0047] Translation terminal 4 is used to receive encapsulated audio sent by the network communication module in the central control device 2 through the network interconnection device 3, input the encapsulated audio into the translation model, output translated audio, and send the translated audio to the network communication module in the central control device 2 through the network interconnection device 3.

[0048] The encapsulated audio is input into the translation model, and the output translated audio includes: First, the encapsulated audio is processed by the Automatic Speech Recognition (ASR) module to obtain the speech recognition result.

[0049] Then, the speech recognition results are translated by AI through the Machine Translation (MT) module to obtain the speech translation result.

[0050] Finally, the text-to-speech (TTS) module is used to synthesize the speech translation results to obtain the translated audio, which is then output.

[0051] The translation model in translation terminal 4 is an Internet translation model or a local area network translation model. Translation terminal 4 can be in the cloud or on a server to support cloud or private translation models.

[0052] This embodiment integrates a Bluetooth module into a central control device, enabling a highly efficient translation solution for simultaneous multilingual and multi-terminal translation. This improves translation efficiency, solves the problem of simultaneous multi-terminal and multilingual translation, and significantly enhances translation quality and efficiency. It can access both public and private translation models, expanding its applicability, reducing the cost of connecting translation devices, and increasing access flexibility and applicability. Utilizing AI noise reduction, it automatically extracts and binds language tags to timbre, achieving noise shielding. Furthermore, based on the extracted language information, it can autonomously bind language tags, improving convenience.

[0053] Example 3

[0054] Based on Embodiment 2, this embodiment introduces a control method for a multilingual, multi-terminal simultaneous interpretation system based on TWS earphones, including the following steps: Step 1: TWS earphone 1 collects the audio to be translated, performs voice encoding on the audio to be translated, obtains the encoded audio, and sends the encoded audio to the corresponding central control device 2.

[0055] Step 2: The central control device 2 receives the encoded audio sent by the TWS earphone 1, performs audio digitization processing on the encoded audio to obtain pre-processed audio, encapsulates the pre-processed audio to obtain encapsulated audio, and sends the encapsulated audio to the translation terminal 4.

[0056] Step 3: The translation terminal 4 receives the encapsulated audio sent by the central control device 2, inputs the encapsulated audio into the translation model, outputs the translated audio, and sends the translated audio back to the central control device 2.

[0057] Step 4: The central control device 2 receives the translated audio sent by the translation terminal 4, and independently splits the translated audio according to the unique language tag bound to the TWS earphone 1 to obtain the translated audio in each language. The translated audio in each language is then decoded to obtain the decoded translated audio in each language, and the decoded translated audio in each language is sent to the TWS earphone 1 of the corresponding language.

[0058] Step 5: TWS earphone 1 receives the decoded and translated audio sent by the central control device 2.

[0059] In one specific embodiment, such as Figure 2 As shown, the functional link is divided into uplink and downlink.

[0060] Uplink includes: Each TWS earphone 1 uses its microphone to collect audio to be translated, performs voice encoding on the audio to be translated, obtains encoded audio, and sends the encoded audio to the corresponding central control device 2.

[0061] The central control device 2 includes a Bluetooth radio frequency unit, an audio codec chip, a main control SoC processor, and a network communication module. The Bluetooth radio frequency unit includes an antenna array, a BLE-Mesh control module, and a multi-channel Bluetooth audio module. The antenna array is a 2.4G diversity antenna array.

[0062] The 2.4G diversity antenna array receives the encoded audio sent by the corresponding TWS earphone 1 and sends the encoded audio to the BLE-Mesh control module.

[0063] The BLE-Mesh control module receives the encoded audio transmitted by the 2.4G diversity antenna array. Based on the unique language tag, it sequentially allocates time slots and assigns channel numbers to the encoded audio to obtain the encoded audio for each channel. The encoded audio for each channel is then sent to the multi-channel Bluetooth audio module.

[0064] The multi-channel Bluetooth audio module receives the encoded audio from each channel sent by the BLE-Mesh control module and sends the encoded audio from each channel to the audio codec chip.

[0065] The audio codec chip receives encoded audio sent by the multi-channel Bluetooth audio module, performs audio digitization processing on the encoded audio to obtain pre-processed audio, and sends the pre-processed audio to the main control SoC processor.

[0066] The main control SoC processor receives pre-processed audio sent by the audio codec chip, performs parallel processing and encapsulation on the pre-processed audio to obtain encapsulated audio, and sends the encapsulated audio to the network communication module.

[0067] The network communication module receives the packaged audio sent by the main control SoC processor and sends the packaged audio to the translation terminal 4 through the network interconnection device 3.

[0068] Translation terminal 4 receives encapsulated audio sent by network communication module through network interconnection device 3, inputs the encapsulated audio into translation model, and outputs translated audio, i.e.: First, the encapsulated audio is processed by the ASR module to obtain the speech recognition result.

[0069] Then, the speech recognition results are translated by AI through the MT module to obtain the speech translation results.

[0070] Finally, the TTS module is used to synthesize the speech translation results to obtain the translated audio, which is then output.

[0071] Downlink includes: The translation terminal 4 sends the translated audio to the network communication module via the network interconnection device 3.

[0072] The network communication module receives the translated audio sent by the translation terminal 4 and sends the translated audio to the main control SoC processor.

[0073] The main control SoC processor receives the translated audio sent by the network communication module, and performs independent streaming of the translated audio according to the unique language tag bound to the TWS earphones to obtain the translated audio in each language. The translated audio in each language is then sent to the audio codec encoding and decoding chip.

[0074] The audio codec chip receives translated audio in various languages ​​sent by the main control SoC processor, decodes the translated audio in each language to obtain decoded translated audio in each language, and sends the decoded translated audio in each language to the multi-channel Bluetooth audio module.

[0075] The multi-channel Bluetooth audio module receives decoded and translated audio from the audio codec chip and sends the decoded and translated audio to the BLE-Mesh control module.

[0076] The BLE-Mesh control module receives decoded and translated audio in various languages ​​from the multi-channel Bluetooth audio module, and sends the decoded and translated audio in each language to the 2.4G diversity antenna array based on the unique language tag.

[0077] The 2.4G diversity antenna array receives decoded and translated audio in various languages ​​sent by the BLE-Mesh control module, and sends the decoded and translated audio in various languages ​​to the corresponding TWS earphone 1.

[0078] TWS earbuds 1 receive decoded and translated audio sent by central control device 2.

[0079] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A multilingual, multi-terminal simultaneous interpretation system based on TWS earphones, characterized in that, include: TWS earphones are used to collect audio to be translated, perform voice encoding on the audio to be translated, obtain encoded audio, and send the encoded audio to the corresponding central control device; Receive decoded and translated audio sent by the central control device; The central control device is used to receive encoded audio sent by TWS earphones, perform audio digitization processing on the encoded audio to obtain pre-processed audio, encapsulate the pre-processed audio to obtain encapsulated audio, and send the encapsulated audio to the translation end; The system receives translated audio sent by the translation terminal, and independently splits the translated audio according to the unique language tag bound to the TWS earphone to obtain translated audio in each language. The system then decodes the translated audio in each language to obtain decoded translated audio in each language, and sends the decoded translated audio in each language to the TWS earphone in the corresponding language. The translation end is used to receive the encapsulated audio sent by the central control device, input the encapsulated audio into the translation model, output the translated audio, and send the translated audio to the central control device.

2. The multilingual multi-terminal simultaneous interpretation system based on TWS earphones according to claim 1, characterized in that, The TWS earphones are also used to extract the user's voice data and mask the voice data of non-users to obtain noise-reduced audio to be translated.

3. The multilingual, multi-terminal simultaneous interpretation system based on TWS earphones according to claim 1, characterized in that, The central control device includes a Bluetooth radio frequency unit, an audio codec encoding / decoding chip, a main control SoC processor, and a network communication module; The Bluetooth radio frequency unit is used to receive encoded audio sent by the TWS earphones and send the encoded audio to the audio codec chip; receive decoded and translated audio in various languages ​​sent by the audio codec chip and send the decoded and translated audio in various languages ​​to the TWS earphones of the corresponding languages; The audio codec chip is used to receive encoded audio sent by the Bluetooth radio frequency unit, perform audio digitization processing on the encoded audio to obtain pre-processed audio, and send the pre-processed audio to the main control SoC processor. The system receives translated audio in various languages ​​sent by the main control SoC processor, decodes the translated audio in various languages ​​to obtain decoded translated audio in various languages, and sends the decoded translated audio in various languages ​​to the Bluetooth radio frequency unit. The main control SoC processor is used to receive pre-processed audio sent by the audio codec encoding / decoding chip, perform parallel processing and encapsulation on the pre-processed audio to obtain encapsulated audio, and send the encapsulated audio to the network communication module. The system receives translated audio sent by the network communication module, and independently splits the translated audio according to the unique language tag bound to the TWS earphone to obtain translated audio in each language. The translated audio in each language is then sent to the audio codec encoding / decoding chip. The network communication module is used to receive encapsulated audio sent by the main control SoC processor and send the encapsulated audio to the translation end; The system receives the translated audio sent by the translation terminal and sends the translated audio to the main control SoC processor.

4. The multilingual multi-terminal simultaneous interpretation system based on TWS earphones according to claim 3, characterized in that, The Bluetooth radio frequency unit includes an antenna array, a BLE-Mesh control module, and a multi-channel Bluetooth audio module; The antenna array is used to receive encoded audio sent by TWS earphones and send the encoded audio to the BLE-Mesh control module; receive decoded and translated audio in various languages ​​sent by the BLE-Mesh control module and send the decoded and translated audio in various languages ​​to the corresponding TWS earphones; The BLE-Mesh control module is used to receive coded audio transmitted by the antenna array, allocate time slots and number channels sequentially according to the unique language tag to obtain coded audio for each channel, and send the coded audio for each channel to the multi-channel Bluetooth audio module; it also receives decoded and translated audio for each language transmitted by the multi-channel Bluetooth audio module, and sends the decoded and translated audio for each language to the antenna array according to the unique language tag. The multi-channel Bluetooth audio module is used to receive encoded audio from each channel sent by the BLE-Mesh control module, and send the encoded audio from each channel to the audio codec chip; it also receives decoded audio from the audio codec chip and sends the decoded audio to the BLE-Mesh control module.

5. The multilingual multi-terminal simultaneous interpretation system based on TWS earphones according to claim 4, characterized in that, The BLE-Mesh control module is also used to bind a unique language tag based on the unique ID of the TWS earphone, or to bind a unique language tag based on the language information of the audio to be translated collected by the TWS earphone.

6. The multilingual multi-terminal simultaneous interpretation system based on TWS earphones according to claim 3, characterized in that, The network communication module includes a wired connection module and a wireless WIFI module; The wired connection module is used for wired data transmission with the translation model deployed offline or in the cloud at the translation terminal; The wireless WIFI module is used for wireless data transmission with the translation model deployed offline or in the cloud at the translation terminal.

7. The multilingual, multi-terminal simultaneous interpretation system based on TWS earphones according to claim 1, characterized in that, It also includes network interconnection devices for establishing data transmission between the central control device and the translation terminal.

8. The multilingual multi-terminal simultaneous interpretation system based on TWS earphones according to claim 1, characterized in that, The translation model is either an internet translation model or a local area network translation model.

9. A control method for a multilingual, multi-terminal simultaneous interpretation system based on TWS earphones, characterized in that, Applied to the system according to any one of claims 1 to 8, the method comprises: TWS earphones collect the audio to be translated, perform voice encoding on the audio to be translated, obtain encoded audio, and send the encoded audio to the corresponding central control device; The central control device receives encoded audio sent by TWS earphones, performs audio digitization processing on the encoded audio to obtain pre-processed audio, encapsulates the pre-processed audio to obtain encapsulated audio, and sends the encapsulated audio to the translation end; The translation terminal receives the encapsulated audio sent by the central control device, inputs the encapsulated audio into the translation model, outputs the translated audio, and sends the translated audio back to the central control device. The central control device receives the translated audio sent by the translation terminal, and independently splits the translated audio according to the unique language tag bound to the TWS earphone to obtain translated audio in each language. The translated audio in each language is then decoded to obtain decoded translated audio in each language, and the decoded translated audio in each language is sent to the TWS earphone in the corresponding language. TWS earbuds receive decoded and translated audio sent by a central control device.

Images