Translation terminal, translation method and translation system

Abstract

The invention relates to the technical field of speech translation, and provides a translation terminal, a translation method and a translation system.The translation terminal comprises a first wireless module used for establishing a first wireless communication link with peripheral audio equipment; the second wireless module is used for establishing a second wireless communication link with the mobile terminal; the main control module is used for receiving the first audio stream from the mobile terminal through the second wireless module, generating a first translated audio stream based on the first audio stream, and sending the first translated audio stream to the peripheral audio equipment through the first wireless module; and the main control module is also used for receiving the second audio stream, generating a second translated audio stream based on the second audio stream, and sending the second translated audio stream to the mobile terminal through the second wireless module. The two independent wireless modules are adopted to construct a double-link architecture, parallel real-time transmission of bidirectional audio streams is achieved, translation delay can be reduced, and it is guaranteed that equipment is portable and convenient to use.

Description

Technical Field

[0001] This invention relates to the field of speech translation technology, and in particular to a translation terminal, translation method, and translation system. Background Technology

[0002] With the increasing demand for cross-language communication, portable real-time translation devices have emerged. Currently, translation devices typically connect to smartphones via wireless technologies such as Bluetooth to transmit and process audio signals.

[0003] However, existing translation devices have significant shortcomings in achieving two-way real-time translation. On the one hand, using a single Bluetooth module for a one-to-one connection with a mobile phone presents compatibility issues. For example, some mobile phones do not support Bluetooth master-slave mode switching and default to the phone as the master device. In this case, it is impossible to simultaneously process real-time translation and playback of two-way audio streams, thus failing to achieve functions such as real-time call translation, real-time voice or video translation, resulting in fragmented translation and discontinuous communication. On the other hand, even when Bluetooth master-slave mode switching is supported, the single Bluetooth link becomes a bottleneck for data transmission, making it difficult to efficiently process two-way audio streams concurrently, resulting in high translation latency and noticeable stuttering during conversations, severely impacting the fluency of communication and user experience. On the other hand, the hybrid wired and wireless connection solution adopted to address this bottleneck—connecting one end wirelessly (e.g., Bluetooth) and the other end wired (e.g., Type-C)—sacrifices device portability. Summary of the Invention

[0004] This invention provides a translation terminal, translation method, and translation system to address the shortcomings of existing translation devices that cannot portablely handle real-time translation and broadcasting of two-way audio streams simultaneously.

[0005] This invention provides a translation terminal, comprising: The first wireless module is used to establish a first wireless communication link with the peripheral audio device; The second wireless module is used to establish a second wireless communication link with the mobile terminal; The main control module is used to receive a first audio stream from the mobile terminal via the second wireless module, generate a first translated audio stream based on the first audio stream, and control the playback of the first translated audio stream, or to send the first translated audio stream to the peripheral audio device via the first wireless module; The main control module is further configured to receive a second audio stream, generate a second translated audio stream based on the second audio stream, and send the second translated audio stream to the mobile terminal via the second wireless module.

[0006] According to a translation terminal provided by the present invention, the first wireless module is further configured to: The system receives hardware coordination signals sent by the second wireless module and performs communication arbitration on the channel resource occupancy between the first wireless module and the second wireless module based on the hardware coordination signals.

[0007] According to a translation terminal provided by the present invention, the hardware coordination signal includes a priority signal and a request signal, and the first wireless module is specifically used for: Receive the priority signal and the request signal sent by the second wireless module; In response to the request signal, based on the transmission priority of the priority signal and its own internal signals, it is determined whether to generate an authorization signal; When the authorization signal is generated, the authorization signal is sent to the second wireless module to authorize the second wireless module to occupy channel resources.

[0008] According to a translation terminal provided by the present invention, the main control module is further configured to: Receive the arbitration request sent by the first wireless module; In response to the arbitration request, the priority of the channel resources occupied by the first wireless module and the second wireless module is adjusted based on at least one of the buffer status of the bidirectional audio stream, the working status of the translation engine, and historical communication statistics.

[0009] According to a translation terminal provided by the present invention, the main control module is further configured to: The software coexistence interface is invoked to coordinate the communication timing between the first wireless module and the second wireless module.

[0010] According to a translation terminal provided by the present invention, the main control module includes a processing unit, a translation engine, and a hardware abstraction layer connecting the processing unit and the translation engine; The processing unit is configured to perform audio preprocessing on the first audio stream received from the second wireless module, copy the preprocessed first audio stream, and transmit the copied first audio stream to the translation engine through the hardware abstraction layer. The translation engine is used to translate the first audio stream, generate the first translated audio stream, and send the first translated audio stream to the processing unit through the hardware abstraction layer; The processing unit is further configured to receive the first translated audio stream and transmit the first translated audio stream to the first wireless module.

[0011] According to a translation terminal provided by the present invention, the processing unit is further configured to: The first audio stream and the first translated audio stream are mixed to generate a mixed audio stream; Control the playback of the mixed audio stream, or send the mixed audio stream to the peripheral audio device via the first wireless module.

[0012] A translation terminal according to the present invention further includes an audio input module, and the processing unit is further configured to: Receive the second audio stream from the audio input module and perform audio preprocessing on the second audio stream; The preprocessed second audio stream is transmitted to the translation engine through the hardware abstraction layer, so that the translation engine generates the second translated audio stream.

[0013] According to a translation terminal provided by the present invention, the audio input module includes a microphone array; Alternatively, the audio input module may be the first wireless module, and the second audio stream may be received from the peripheral audio device through the first wireless module.

[0014] According to a translation terminal provided by the present invention, the audio preprocessing includes at least one of noise reduction processing, echo cancellation processing, and adaptive processing.

[0015] The present invention also provides a translation method, the method being applied to a translation terminal, the translation terminal including a first wireless module and a second wireless module, the method comprising: The first audio stream is received from the mobile terminal via the second wireless module. Based on the first audio stream, generate a first translated audio stream and play the first translated audio stream, or send the first translated audio stream to a peripheral audio device connected to the first wireless module via the first wireless module; Receive the second audio stream and generate a second translated audio stream based on the second audio stream; The second translated audio stream is sent to the mobile terminal via the second wireless module.

[0016] The present invention also provides a translation system, including a peripheral audio device, a mobile terminal, and a translation terminal as described above.

[0017] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the translation method as described above.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the translation method as described above.

[0019] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the translation method as described above.

[0020] The translation terminal, translation method, and translation system provided by this invention establish dedicated wireless communication links with peripheral audio devices and with mobile terminals by setting up independent first and second wireless modules. This dual-link hardware architecture completely separates the uplink and downlink audio streams at the physical level, allowing them to be transmitted in parallel and without interference on their respective communication links in real time. This design not only solves the data congestion and high latency bottlenecks caused by single-link transmission in traditional translation devices, achieving a smooth and natural two-way real-time dialogue experience; at the same time, this all-wireless solution also ensures the portability and compatibility of the device, and the operation mode of the two independent links enhances the system's communication stability and anti-interference capability in complex wireless environments. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is one of the structural schematic diagrams of the translation terminal provided by the present invention; Figure 2 This is one of the application scenario diagrams of the translation terminal provided by the present invention; Figure 3 This is the second schematic diagram of the translation terminal provided by the present invention; Figure 4 This is a schematic diagram of the main control module of the translation terminal provided by the present invention; Figure 5 This is the second schematic diagram of the application scenario of the translation terminal provided by the present invention; Figure 6 This is a diagram illustrating the overall architecture and workflow of the translation terminal provided by this invention. Figure 7 This is a flowchart illustrating the translation method provided by the present invention; Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] With the growing demand for cross-language communication, various forms of translation devices have emerged to eliminate language barriers. Among them, real-time translation terminals that work with smartphones have received widespread attention due to their convenience.

[0025] Currently, translation devices generally employ wireless communication technologies, especially Bluetooth, to connect with devices such as mobile phones and headphones. However, existing translation devices have certain shortcomings when performing two-way real-time translation.

[0026] First, a single Bluetooth link cannot meet the demands of concurrent processing of bidirectional real-time audio streams. Traditional translation devices typically use a single Bluetooth module for a one-to-one connection with a smartphone. This architecture, when handling bidirectional dialogue translation, must process two concurrent business flows time-sharing on the same link: a downlink audio stream of "listening to the other party's speech → translating → playing back to the user," and an uplink audio stream of "listening to the user's speech → translating → sending back to the other party." The single Bluetooth link becomes a bottleneck for data transmission, making it difficult to efficiently process these two tasks concurrently. This results in high translation latency, causing noticeable stuttering during the dialogue and severely impacting the fluency of communication and user experience.

[0027] Secondly, to address the aforementioned issues, some solutions employ a compromise: a hybrid approach combining wireless (e.g., Bluetooth connection to headphones) and wired (e.g., Type-C connection to a phone) methods to achieve simultaneous processing of bidirectional audio streams. However, this semi-wireless, semi-wired solution adds an extra physical connection, sacrificing device portability and flexibility, making it extremely inconvenient, especially in scenarios requiring mobility.

[0028] Furthermore, regardless of whether a single Bluetooth module is used, or a WCN (Wireless Connectivity Network) chip or module integrating WiFi and Bluetooth functions is employed, both solutions face severe interference problems in complex wireless environments. Bluetooth technology primarily operates in the 2.4GHz public frequency band, which is also used by WiFi and numerous other wireless devices, resulting in channel congestion. Although the AFH (Adaptive Frequency Hopping) mechanism in the Bluetooth protocol can mitigate interference from fixed-frequency signals such as WiFi to some extent, this method cannot effectively avoid co-channel interference caused when two or more Bluetooth modules operate simultaneously. Especially in complex network environments with numerous wireless devices, such as conferences and exhibitions, co-channel interference can lead to packet loss and connection instability, thereby affecting the real-time performance and accuracy of translation.

[0029] In response, this invention provides a translation terminal that can portablely achieve low-latency, high-efficiency, two-way real-time translation, while also possessing strong interference resistance and good compatibility, thereby overcoming the aforementioned shortcomings.

[0030] Figure 1 This is one of the structural schematic diagrams of the translation terminal provided by the present invention, such as... Figure 1 As shown, the translation terminal 100 includes: The first wireless module 110 is used to establish a first wireless communication link with the peripheral audio device 200; The second wireless module 120 is used to establish a second wireless communication link with the mobile terminal 300; The main control module 130 is used to receive a first audio stream from the mobile terminal 300 through the second wireless module 120, generate a first translated audio stream based on the first audio stream, and control the playback of the first translated audio stream, or send the first translated audio stream to the peripheral audio device 200 via the first wireless module 110. The main control module 130 is also used to receive a second audio stream, generate a second translated audio stream based on the second audio stream, and send the second translated audio stream to the mobile terminal 300 via the second wireless module 120.

[0031] Specifically, the first wireless module 110 and the second wireless module 120 are the hardware foundation for achieving efficient concurrent bidirectional data transmission of the translation terminal 100. In one implementation, these two wireless modules can be dual Bluetooth modules, i.e., two independently operating Bluetooth chips or two independent Bluetooth cores integrated on the same chip. Of course, in other possible implementations, other short-range wireless communication technology modules can also be used, such as a Bluetooth module and a satellite flash module, etc., and this embodiment of the invention does not specifically limit this. It should be understood that the design of using dual Bluetooth modules is to physically isolate the communication links connecting the translation terminal 100 to different devices, thereby realizing parallel processing of bidirectional audio streams.

[0032] Specifically, the first wireless module 110 is used to establish a first wireless communication link with the peripheral audio device 200. Here, the peripheral audio device 200 refers to a user-side device used for listening to and / or picking up audio. For example, the peripheral audio device 200 can be a Bluetooth headset, which can both play sound (listen) and typically integrates a microphone (pick up sound). The establishment of the first wireless communication link allows the translation terminal 100 to act as a master device, sending audio data to the peripheral audio device 200, which acts as a slave device.

[0033] The second wireless module 120 is used to establish a second wireless communication link with the mobile terminal 300. Here, the mobile terminal 300 refers to the source device for the call or communication, such as a smartphone, personal computer (PC), or tablet computer. The mobile terminal 300 can run various operating systems, such as iOS, Android, and HarmonyOS, and can run calling applications (such as cellular calling), social software (such as WeChat), and conferencing systems. In the second wireless communication link, the translation terminal 100 typically acts as a slave device, while the mobile terminal 300 acts as the master device. All system audio or application audio on the mobile terminal 300 can be transmitted to the translation terminal 100 through this link.

[0034] The main control module 130 is the control and processing core of the entire translation terminal 100, equivalent to the brain of the device. It can be a system-on-chip (SoC) that integrates a microcontroller unit (MCU), a digital signal processor (DSP), and other components. Specifically, the main control module 130 may further include key components such as a digital signal processing unit and a translation engine, responsible for the reception, processing, translation, and routing of all audio streams.

[0035] The working process of the main control module 130 can be divided into two parallel paths. Figure 2 This is one of the application scenario diagrams of the translation terminal provided by the present invention, such as... Figure 2 As shown in (a), the first path processes the downlink audio stream, i.e., the audio stream from the other party to the local user. The main control module 130 receives the first audio stream from the mobile terminal 300 (such as the local user's mobile phone) through the second wireless module 120. This first audio stream can be understood as the downlink audio data that needs to be translated for the local user. For example, when making a WeChat voice call with a foreign friend, the foreign language speech spoken by the other party (i.e., the other party's original voice) will be transmitted from the other party's mobile phone to the local user's mobile phone, and then transmitted from the local user's mobile phone (i.e., the mobile terminal 300) to the main control module 130 of the translation terminal 100 via Bluetooth. This part of the data stream is the first audio stream.

[0036] Upon receiving the first audio stream, the main control module 130 generates a first translated audio stream based on it. Specifically, the translation engine inside the main control module 130 translates the received first audio stream in real time, for example, translating the English audio stream into a Chinese audio stream, and the generated content is the first translated audio stream.

[0037] Subsequently, the main control module 130 can send the first translated audio stream to the peripheral audio device 200 via the first wireless module 110. In this way, the user wearing a Bluetooth headset (i.e., the peripheral audio device 200) can directly hear the Chinese translation of the other party's speech, achieving the effect of simultaneous interpretation. It should be understood that the translation terminal 100 can transmit not only the first translated audio stream to the user's Bluetooth headset, but also the first audio stream (i.e., the other party's original voice) to the user's Bluetooth headset, so that the user can simultaneously hear the original voice and the corresponding translated audio. Furthermore, the translation terminal 100 can also be equipped with a playback module (such as a speaker). In scenarios where the user is not connected to the peripheral audio device 200 via the first wireless module 110, after the first translated audio stream is generated, the main control module 130 can directly control the playback module to play the first translated audio stream, so that the first translated audio stream is directly broadcast or displayed on the translation terminal 100 for the user's convenience.

[0038] like Figure 2As shown in (b), the second path processes the uplink audio stream, i.e., the audio stream from "local user to other party". The main control module 130 is also used to receive the second audio stream, which can be understood as the uplink audio data spoken by the local user that needs to be translated to the other party, i.e., the original audio stream of the local user. The source of this second audio stream can be in various ways. For example, it can be directly picked up by the user's voice through the microphone (or microphone array) integrated into the translation terminal 100; or, for example, when the user is wearing a peripheral audio device 200 with a microphone function (such as a Bluetooth headset), the user's voice can be received from the peripheral audio device 200 through the first wireless module 110. The main control module 130 can automatically or manually select the sound source according to the device connection status.

[0039] Upon receiving the second audio stream, the main control module 130 also generates a second translated audio stream based on the second audio stream. For example, the user's spoken Chinese speech (second audio stream) is translated into an English audio stream (second translated audio stream) in real time by the translation engine within the main control module 130.

[0040] Finally, the main control module 130 sends the second translated audio stream to the mobile terminal 300 (such as the user's mobile phone) via the second wireless module 120. After receiving the translated audio stream, the mobile terminal 300 sends it to the other party in the call through its own communication application (such as WeChat). In this way, the other party can hear the English translation of the user's speech.

[0041] It is understandable that the first audio stream can be an audio stream received by mobile terminal 300 from another mobile terminal (such as in a two-way call translation scenario), or it can be an audio stream played by mobile terminal 300 itself, such as foreign language news, video conferencing, online courses, podcasts, audiobooks, music lyrics, game voice, etc. For example... Figure 2As shown in (c), in this real-time translation scenario, the mobile terminal 300 can transmit the first audio stream (or original audio stream) played from itself to the main control module 130 of the translation terminal 100 via the second wireless module 120. After receiving the audio stream, the main control module 130 translates it and transmits the resulting synthesized audio to the user's Bluetooth headset via the first wireless module 110 so that the user can listen to the translated content. In addition, when the user does not connect the Bluetooth headset via the first wireless module 110, the main control module 130 can also directly control the playback module on the translation terminal 100 to play the translated synthesized audio. The translation terminal provided in this embodiment of the invention establishes dedicated wireless communication links with peripheral audio devices and with the mobile terminal by setting up independent first and second wireless modules. This dual-link hardware architecture completely separates the uplink and downlink audio streams at the physical level, allowing them to be transmitted in parallel and without interference in real time on their respective communication links. This design not only solves the data congestion and high latency bottlenecks caused by single-link transmission in traditional translation devices, achieving smooth, natural, and millisecond-level two-way real-time translation; at the same time, this all-wireless solution also ensures the portability and compatibility of the device, and the two independent link operation modes also enhance the system's communication stability and anti-interference capabilities in complex wireless environments.

[0042] Based on the above embodiments, the first wireless module 110 is further configured to: The system receives a hardware coordination signal sent by the second wireless module 120 and performs communication arbitration on the channel resource occupancy between the first wireless module 110 and the second wireless module 120 based on the hardware coordination signal.

[0043] It should be noted that in the dual wireless module architecture, since the first wireless module 110 and the second wireless module 120 may operate simultaneously in the same frequency band (e.g., the 2.4GHz band commonly used in Bluetooth technology), co-channel interference may occur when the two modules transmit and receive data simultaneously, affecting the transmission quality and stability of the audio stream, and consequently causing translation delays or stuttering. To address this potential problem, this embodiment of the invention proposes a hardware-level cooperative anti-interference mechanism.

[0044] Specifically, in this embodiment of the invention, the first wireless module 110 is assigned the role of arbitrator, and is also used to receive hardware coordination signals sent by the second wireless module 120, and to conduct communication arbitration on the channel resource occupation between the first wireless module 110 and the second wireless module 120 based on the hardware coordination signals.

[0045] Here, hardware coordination signals refer to level signals transmitted through a direct physical connection (such as GPIO ports) between two wireless modules. Compared to transmitting messages through a software protocol stack, this direct hardware connection offers nanosecond-level transmission speeds and extremely high reliability, enabling near real-time state synchronization and collaboration between the two modules, thus laying the foundation for efficient arbitration.

[0046] Understandably, since a wireless channel can only accommodate one data packet at a time on the same frequency, when two modules want to use the same wireless channel, an arbitrator is needed to decide who gets to use the channel resource first. In this embodiment of the invention, the first wireless module 110 assumes this role. Based on the coordination signal obtained from the second wireless module 120 and its own communication needs, it coordinates and allocates the right to use the channel.

[0047] For example, when the second wireless module 120 needs to send the second translated audio stream to the mobile terminal 300, it first notifies the first wireless module 110 via a hardware coordination signal. Upon receiving the signal, the first wireless module 110 determines whether it is currently performing critical data transmission (such as sending the first translated audio stream to a Bluetooth headset). If the channel is idle or its transmission task has a low priority, it can authorize the second wireless module 120 to use the channel; otherwise, it will ask the second wireless module 120 to wait.

[0048] In this embodiment of the invention, a communication arbitration mechanism based on hardware-coordinated signals is introduced to establish an ultra-low latency and continuously connected communication link between two wireless modules, enabling refined and real-time management of shared channel resources. This hardware-level arbitration mechanism can quickly and effectively avoid co-frequency conflicts between the two modules, ensuring the stability and high quality of bidirectional audio stream transmission, thereby significantly improving the anti-interference capability and robustness of the translation terminal in complex electromagnetic environments.

[0049] Based on any of the above embodiments, the hardware coordination signal includes a priority signal and a request signal, and the first wireless module 110 is specifically used for: Receive the priority signal and the request signal sent by the second wireless module 120; In response to the request signal, based on the transmission priority of the priority signal and its own internal signals, it is determined whether to generate an authorization signal; When the authorization signal is generated, the authorization signal is sent to the second wireless module 120 to authorize the second wireless module 120 to occupy channel resources.

[0050] Specifically, Figure 3 This is the second structural schematic diagram of the translation terminal provided by the present invention, as shown below. Figure 3As shown, the hardware coordination signals may specifically include a priority signal and a request signal. Correspondingly, the first wireless module 110 generates a grant signal. These three signals can be implemented through three GPIO pins connected between the first wireless module 110 and the second wireless module 120.

[0051] The first wireless module 110 continuously monitors the corresponding pins connected to the second wireless module 120 to receive priority signals and request signals sent by the second wireless module 120. When the second wireless module 120 has data to send (e.g., translated user voice to be transmitted to a mobile phone), it actively pulls the request signal high (or low, depending on the circuit design) to indicate a request to use a channel (e.g., a channel in the 2.4GHz band). Simultaneously, it sets the priority signal level according to the importance of the data to be transmitted; for example, it sets high priority for real-time audio data packets and low priority for non-critical control signaling.

[0052] Upon detecting a valid request signal, the first wireless module 110 will respond immediately. It will read the level of the priority signal to understand the urgency of the request from the second wireless module 120. Then, it will combine this information with its own internal signal transmission priority to make a judgment. For example, if the first wireless module 110 is also preparing to send a high-priority audio data packet to the Bluetooth headset, it may determine that its own priority is higher and thus temporarily refrain from generating an authorization signal. Conversely, if it is in an idle state, or if the priority of its internal pending task is lower than the received external priority signal, it will determine that authorization is possible.

[0053] If authorization is deemed valid, the first wireless module 110 generates an authorization signal, for example, by pulling up (or pulling down) the level of the pin corresponding to the authorization signal (GRANT), and sends this signal to the second wireless module 120. Upon receiving a valid authorization signal, the second wireless module 120 gains access to the channel and can immediately begin its data transmission. After transmission is complete, it withdraws its request signal, and the first wireless module 110 also withdraws its authorization signal, completing one arbitration cycle.

[0054] In this embodiment of the invention, a clear, complete, and efficient closed-loop arbitration protocol is constructed by refining the hardware coordination signals into three specific signals: request, priority, and authorization. This protocol not only solves the problem of co-channel interference but also addresses the issue of who should occupy channel resources first by introducing a priority mechanism. This enables dynamic and intelligent allocation of channel resources, ensuring the priority transmission of critical audio data and thus guaranteeing low latency and smoothness in bidirectional real-time translation.

[0055] Based on any of the above embodiments, this embodiment of the invention provides a more intelligent and robust conflict resolution strategy on the basis of communication arbitration performed by the first wireless module 110, forming a two-level arbitration architecture to cope with more complex communication conflict scenarios.

[0056] In practical applications, especially in environments with extremely congested wireless signals, the first wireless module 110, acting as a hardware arbitrator, may encounter conflicting situations that it cannot resolve independently. For example, the first wireless module 110 and the second wireless module 120 may both issue the highest-priority channel usage request at the same time. In this situation, simple hardware priority logic may not be sufficient to make the optimal judgment, because from the perspective of each module, its own task is crucial. Improper handling may lead to an arbitration deadlock or a suboptimal choice, thereby affecting the real-time performance of at least one side's audio stream.

[0057] To address this issue, this embodiment of the invention introduces a two-level arbitration mechanism assisted by the main control module 130. Under this mechanism, the first wireless module 110 is configured to: when encountering a preset conflict condition that it cannot resolve on its own (e.g., receiving a continuous request from the second wireless module 120 with the same task priority as its own within a set time threshold), stop autonomous arbitration and instead send an arbitration request to the main control module 130.

[0058] The arbitration request here can be a specific signal or message. To ensure the highest response speed, preferably, the request can be triggered via a dedicated hardware interrupt line connecting the first wireless module 110 and the main control module 130. When the first wireless module 110 triggers the interrupt, the main control module 130 will immediately suspend its current secondary task and enter the interrupt service routine to handle the arbitration request, thereby ensuring timely processing.

[0059] Upon receiving an arbitration request from the first wireless module 110, the main control module 130 immediately intervenes in the arbitration process. Unlike the first wireless module 110, which can only make judgments based on instantaneous priority signals, the main control module 130 has a global system perspective and can make a final decision based on richer, higher-level application information. For example, the main control module 130 will comprehensively evaluate at least one of the following: the buffer status of the bidirectional audio stream, the working status of the translation engine, historical communication statistics, etc.

[0060] Specifically, regarding the buffer status of the bidirectional audio streams, the main control module 130 checks the buffer queue lengths of the first audio stream (downlink) and the second audio stream (uplink) within the main control module 130. If it finds that the downlink audio stream buffer is about to run out (which may cause stuttering in the translated audio heard by the user), it will prioritize the transmission of the first wireless module 110. Conversely, if the uplink audio stream buffer is about to overflow (which may cause partial loss of the user's voice), it will prioritize the transmission of the second wireless module 120.

[0061] Furthermore, by obtaining the translation engine's operational status, we can understand whether it is currently processing downlink or uplink audio, and the processing progress. Analyzing historical communication statistics allows us to determine the recent channel occupancy times of the two wireless modules, ensuring long-term fairness and preventing any link from being continuously suppressed.

[0062] Through a comprehensive evaluation of the aforementioned global information, the main control module 130 dynamically adjusts the allocation weights (priorities) and makes a final decision. For example, if it determines that ensuring the user's listening experience is more critical at present, it will send an instruction to the first wireless module 110 via the control bus, forcing it to obtain the right to use the channel for this time, and instructing the second wireless module 120 to wait. This dynamic weight adjustment based on the application layer context ensures that at any given time, the data stream that has the greatest impact on the overall user experience receives the highest priority processing.

[0063] The method provided in this invention combines the low latency and high efficiency advantages of hardware arbitration with the high flexibility of software control in the main control module by constructing a two-level collaborative architecture of hardware-based rapid arbitration and main control intelligent adjudication. This mechanism not only effectively resolves arbitration deadlocks that are difficult to handle at the hardware level, but more importantly, it can make optimal resource allocation decisions based on the real-time operating status of the entire translation application, thereby maximizing the timeliness and reliability of critical audio signal processing even under various complex communication conditions.

[0064] Based on any of the above embodiments, the main control module 130 is further configured to: The software coexistence interface is invoked to coordinate the communication timing between the first wireless module 110 and the second wireless module 120.

[0065] Specifically, to address the co-channel interference problem associated with dual wireless modules (such as dual Bluetooth modules), this invention provides another optional or supplementary anti-interference solution, which coordinates at the software level. This solution can be used independently of the hardware arbitration solution in the above embodiments, or it can be combined with it to form a three-dimensional anti-interference system with hardware and software collaboration. Furthermore, the dual wireless modules can also be designed as Bluetooth and a star-flash module to utilize the stronger anti-interference characteristics of star-flash technology, further enhancing the system's anti-interference capability.

[0066] In this embodiment of the invention, the main control module 130 is used to call a software coexistence interface to coordinate the communication timing between the first wireless module 110 and the second wireless module 120. Here, the software coexistence interface typically refers to a series of functions or application programming interfaces pre-built into the software development kit (SDK) provided by the wireless communication chip manufacturer, such as the Coexistence (coex) interface commonly found in the Bluetooth SDK. The firmware or operating system on the main control module 130 can pass coordination information to the underlying protocol stack of the two wireless modules by calling these interfaces.

[0067] Coordinated communication timing refers to actively staggering the time windows for data transmission and reception between two wireless modules through software scheduling. For example, the main control module 130 can use the COEX interface to divide a complete communication cycle into multiple time slices and allocate different time slices to the first wireless module 110 and the second wireless module 120. During the time slice allocated to the first wireless module 110, the second wireless module 120 remains silent or only receives data; and vice versa. This achieves the effect of time division multiplexing on a macroscopic level, ensuring that although the two modules share the same frequency band, they avoid time conflicts.

[0068] Specifically, the main control module 130 can dynamically adjust the time slice length and scheduling strategy allocated to each module based on the data volume of the two audio streams and real-time requirements, in order to optimize the overall communication efficiency and latency performance.

[0069] This invention provides a software-based collaborative coexistence scheme. This scheme requires no additional hardware connection; by calling a standard software interface, the communication timing of the two wireless modules can be coordinated, effectively avoiding co-channel interference. This approach is flexible, low-cost, and can complement hardware arbitration mechanisms to jointly enhance the communication reliability of the translation terminal in various complex environments, ensuring a stable and smooth two-way translation experience.

[0070] Based on any of the above embodiments Figure 4 This is a schematic diagram of the main control module of the translation terminal provided by the present invention, as shown below. Figure 4As shown, the main control module 130 includes a processing unit 131, a translation engine 132, and a hardware abstraction layer 133 connecting the processing unit 131 and the translation engine 132; The processing unit 131 is used to perform audio preprocessing on the first audio stream received from the second wireless module 120, copy the preprocessed first audio stream, and transmit the copied first audio stream to the translation engine 132 through the hardware abstraction layer 133. The translation engine 132 is used to translate the first audio stream, generate the first translated audio stream, and send the first translated audio stream to the processing unit 131 through the hardware abstraction layer 133; The processing unit 131 is further configured to receive the first translated audio stream and transmit the first translated audio stream to the first wireless module 110.

[0071] Specifically, the main control module 130 is further divided into three core parts: the processing unit 131, the translation engine 132, and the hardware abstraction layer (HAL) 133 that connects the two.

[0072] The processing unit 131 can be one or more digital signal processor (DSP) cores, such as an audio-specific DSP (ADSP). It is responsible for the audio data input / output (I / O), routing, and general audio signal processing tasks of the entire main control module.

[0073] Translation engine 132 is the module that performs the core translation tasks. It can be a dedicated hardware accelerator, such as a neural network processing unit, specifically designed for efficiently running artificial intelligence models such as Automatic Speech Recognition (ASR), machine translation, and Text-to-Speech (TTS). In other embodiments, it can also be a purely software algorithm module running on a high-performance processor. This embodiment of the invention does not specifically limit it in this way.

[0074] The hardware abstraction layer 133 is a software interface or middleware layer between the processing unit 131 and the translation engine 132. Its function is to decouple the upper-layer audio processing logic (executed by the processing unit 131) from the lower-layer specific translation implementation (executed by the translation engine 132). The processing unit 131 only needs to call the standardized API provided by HAL to send the audio to be translated and receive the translation results, without needing to care about the specific hardware model, driver, or internal working mechanism of the translation engine 132.

[0075] The following section will describe in detail the working principle of each functional module within the main control module 130 regarding the processing of the downlink audio stream. Specifically, the processing unit 131 receives the first audio stream (i.e., the original speech of the other party) from the mobile terminal 300 through the second wireless module 120. This raw audio stream may contain echoes, environmental noise, etc., from the call link, and its audio format (such as SBC or AAC encoding format) and sampling rate may not meet the requirements of the translation engine. Therefore, the processing unit 131 first performs audio preprocessing on the first audio stream, which may include echo cancellation, noise reduction, adaptive adjustment, etc.

[0076] After preprocessing, processing unit 131 copies the clean first audio stream, generating two identical audio data streams. One of them (the copied first audio stream) is transmitted to translation engine 132 for translation via the interface of hardware abstraction layer 133. The other original audio stream is temporarily retained by processing unit 131.

[0077] After receiving the audio stream from the hardware abstraction layer 133, the translation engine 132 executes the complete translation process. First, it converts the audio into text through speech recognition, then it translates the text into the target language text through machine translation, and finally it converts the target language text into speech through speech synthesis, thereby generating the first translated audio stream (i.e., the translated speech).

[0078] After the translation engine 132 completes the translation, it sends the generated first translated audio stream back to the processing unit 131 through the hardware abstraction layer 133. After receiving the translated first audio stream, the processing unit 131 transmits it to the first wireless module 110, and it is finally played by the user's peripheral audio device 200 (such as Bluetooth headphones).

[0079] In this embodiment of the invention, a modular architecture consisting of a processing unit, a translation engine, and a hardware abstraction layer is introduced, achieving a high degree of decoupling and flexibility of system functions. The existence of the hardware abstraction layer allows the translation engine to be replaced or upgraded as an independent module, without requiring extensive modifications to the processing unit software code. This design significantly reduces system development and maintenance costs and enhances product scalability.

[0080] Based on any of the above embodiments, the processing unit 131 is further configured to: The first audio stream and the first translated audio stream are mixed to generate a mixed audio stream; Control the playback of the mixed audio stream, or send the mixed audio stream to the peripheral audio device 200 via the first wireless module 110.

[0081] It should be noted that in the application scenarios of the above embodiments, the user usually hears the pure translated speech. However, in some scenarios, the user may want to hear both the original voice (to perceive its tone and emotion) and the translated speech simultaneously. To meet this need, the embodiments of the present invention have extended the functionality of the processing unit 131.

[0082] Specifically, in the above embodiment, it was mentioned that while sending one audio stream for translation, the processing unit 131 retained another pre-processed first audio stream (i.e., the other party's original audio). In this embodiment, this retained audio stream will be used for subsequent processing.

[0083] Specifically, after receiving the first translated audio stream (translation) returned from the hardware abstraction layer 133, the processing unit 131 will call its internal mixing algorithm to mix its own first audio stream (original text) with the first translated audio stream (translation) just received to generate a mixed audio stream.

[0084] Here, audio mixing can be understood as superimposing two audio signals according to a certain weighting ratio. For example, the volume of the first audio stream (original text) can be attenuated to a low level (e.g., 20%-40% of the original volume) to serve as background noise; while the first translated audio stream (translated text) is kept at a normal or high volume level (e.g., 80%-100%) to serve as the foreground main sound. In this way, in the final mixed audio stream, the user can clearly hear the translated text while also vaguely perceiving the intonation and rhythm of the original text. In some preferred embodiments, this mixing ratio can also be dynamically adjustable, allowing users to personalize it through a companion mobile app or buttons on the translation terminal.

[0085] After generating the mixed audio stream, the processing unit 131 no longer simply sends the first translated audio stream, but instead sends the newly generated mixed audio stream to the peripheral audio device 200 for playback via the first wireless module 110. Alternatively, the first translated audio stream can be played directly through the playback module on the translation terminal.

[0086] This invention provides users with clearer and richer auditory feedback by adding a mixing step between the original and translated text. Users obtain accurate translations without completely losing the speaker's emotional nuances and linguistic context, greatly enhancing their communication experience and immersion.

[0087] Based on any of the above embodiments, the translation terminal 100 further includes an audio input module, and the processing unit 131 is further configured to: Receive the second audio stream from the audio input module and perform audio preprocessing on the second audio stream; The preprocessed second audio stream is transmitted to the translation engine 132 through the hardware abstraction layer 133, so that the translation engine 132 generates the second translated audio stream.

[0088] It should be noted that the above embodiments describe the processing path inside the main control module 130 for the processing of the downlink audio stream. The embodiments of the present invention will describe the processing of the uplink audio stream in parallel with it, thereby fully demonstrating the bidirectional data flow process under the modular architecture.

[0089] Specifically, the translation terminal 100 also includes an audio input module, which is the physical entry point for capturing the local user's voice, i.e., the second audio stream.

[0090] When the user speaks, the upstream audio stream is processed as follows: First, the processing unit 131 receives the raw second audio stream (i.e., the user's speech) from the audio input module. Similar to the downstream audio stream, this raw audio stream may be mixed with ambient noise. Therefore, the processing unit 131 also performs audio preprocessing on the second audio stream, such as noise reduction and echo cancellation, to obtain a clean, clear, and suitable voice data stream for machine translation.

[0091] After preprocessing, processing unit 131 transmits the clean second audio stream to translation engine 132 by calling the standardized interface provided by hardware abstraction layer 133. It should be understood that the uplink data stream here shares the same hardware abstraction layer interface as the downlink data stream described in the above embodiments, reflecting the uniformity and consistency of the architecture. Processing unit 131 only needs to call the interface provided by hardware abstraction layer 133 and pass the uplink audio stream as a parameter to complete data delivery, without needing to concern itself with the specific implementation of the translation engine.

[0092] After receiving the user's voice stream, the translation engine 132 performs a translation task and generates a second translated audio stream (e.g., translating Chinese speech into English speech). After the translation is completed, the result (the second translated audio stream) is also returned to the processing unit 131 through the interface of the hardware abstraction layer 133.

[0093] After receiving the translated second audio stream, the processing unit 131 routes and transmits it to the second wireless module 120, and finally sends it to the mobile terminal 300 through the second wireless communication link, thereby transmitting it to the other party in the call.

[0094] Based on any of the above embodiments, the audio input module includes a microphone array; Alternatively, the audio input module may be the first wireless module 110, and the second audio stream may be received from the peripheral audio device 200 through the first wireless module 110.

[0095] Specifically, the embodiments of the present invention describe in detail two implementation methods of the audio input module, which provide the translation terminal with flexible sound pickup options to adapt to different user scenarios.

[0096] In one implementation, the audio input module includes a microphone array. In this configuration, the microphone array is directly integrated into the housing of the translation terminal 100. Here, a microphone array refers to a pickup system consisting of two or more microphones arranged in a specific geometric layout. When the user places the translation terminal on a table or wears it around their neck, the processing unit 131 can use the multiple audio signals acquired by the microphone array to synthesize a high-quality, high signal-to-noise ratio second audio stream through an algorithm.

[0097] In another implementation, the audio input module can be a first wireless module 110. In this configuration, the translation terminal 100 itself may not pick up sound, but instead delegates the task to a user-worn peripheral audio device 200 (e.g., a Bluetooth headset with a microphone). The user speaks into the microphone of the Bluetooth headset, and the voice signal is captured by the Bluetooth headset and transmitted to the translation terminal 100 via a first wireless communication link (e.g., Bluetooth HFP protocol). At this time, the first wireless module 110 of the translation terminal acts as the audio input module, responsible for receiving the user's voice data from the Bluetooth headset, which is the second audio stream. Subsequently, the first wireless module 110 delivers the received audio stream to the processing unit 131 for further processing.

[0098] In practical applications, the system can make intelligent selections. For example, the system can use the built-in microphone array by default; when it detects that a peripheral audio device 200 with microphone functionality has successfully connected through the first wireless module 110, the system can automatically switch the audio input source to that external device, providing users with a better call experience.

[0099] This invention provides two sound pickup methods: a built-in microphone array and an external microphone, which enhances the flexibility and adaptability of the translation terminal, meets diverse needs under different environments and habits, and improves the product's practicality and user experience.

[0100] Based on any of the above embodiments, the audio preprocessing includes at least one of noise reduction processing, echo cancellation processing, and adaptive processing.

[0101] Specifically, noise reduction processing aims to separate and remove background noise from the original audio stream. For example, when a user uses a translation terminal 100 in a noisy environment, their speech will be mixed with a large amount of environmental noise. Noise reduction algorithms (such as traditional spectral subtraction or advanced deep learning-based noise reduction models) can identify and suppress these steady-state or non-steady-state noise components, extracting a cleaner human voice signal, thereby helping to improve the recognition accuracy of translation.

[0102] Echo cancellation is primarily used to solve the echo problem in full-duplex communication. In the application scenario of this invention, when a peripheral audio device 200 (such as a Bluetooth headset) plays the other party's translation (first translated audio stream), the sound it plays may be picked up again by the microphone of the same device and mixed with the user's own speech (second audio stream), forming an echo. The echo cancellation algorithm eliminates the echo by using the playing audio as a reference signal and subtracting the delayed and distorted version of the reference signal from the mixed signal picked up by the microphone, thus ensuring that the other party will not hear the echo of their own speech and guaranteeing a smooth call experience.

[0103] Adaptive processing typically refers to a series of algorithms that dynamically adjust parameters based on signal characteristics, the most typical of which is Automatic Gain Control (AGC). AGC automatically adjusts the amplitude (volume) of the audio signal. When the user's voice is too soft, AGC amplifies the signal appropriately to prevent the speech from being drowned out; when the user's voice is too loud or too close to the microphone, AGC attenuates the signal appropriately to prevent clipping distortion. Through AGC processing, the audio volume input to the translation engine can be ensured to be moderate and stable, further improving the system's robustness.

[0104] In a specific implementation, the processing unit 131 can selectively enable or combine these processing algorithms according to the current operating state and environment to achieve the best audio purification effect.

[0105] In this embodiment of the invention, by integrating one or more advanced audio signal processing algorithms such as noise reduction, echo cancellation, and adaptive processing, the original audio can be deeply purified and optimized before translation. This not only improves the quality of the audio signal input to the translation engine and enhances the accuracy of speech recognition and translation, but also solves the echo problem in full-duplex calls, ultimately providing users with clear, reliable, and smooth cross-language communication in various complex acoustic environments.

[0106] Based on any of the above embodiments, this invention provides a low-interference real-time translation terminal and its operating method based on a dual Bluetooth architecture. The translation terminal includes dual Bluetooth modules, namely a first Bluetooth module and a second Bluetooth module. The second Bluetooth module is paired and connected to a mobile phone, taking over all audio input and output of the phone; the first Bluetooth module is connected to a Bluetooth headset or other peripherals and outputs the translation results.

[0107] In addition, the translation terminal may also include a display module, an audio processing module, and a control module. The display module is used to display the original text and translation of the user and the other party in real time. The audio processing module supports the pickup and playback of Bluetooth headsets, wired headsets, and built-in microphones, integrates text-to-speech synthesis, and supports seamless switching of multi-channel audio links. The control module has independent on / off control of the original / translated text playback and translation functions.

[0108] The mobile device is equipped with a Bluetooth communication module and a data forwarding module. After the Bluetooth communication module is paired with the translation terminal, it takes over all audio input and output, including audio from calls, WeChat, conferences, etc. The data forwarding module is used to send the received original audio to the translation terminal and send the translated audio back to the other party in the call.

[0109] Figure 5 This is a schematic diagram illustrating an application scenario of the translation terminal provided by the present invention, such as... Figure 5 As shown, the following describes the workflow of the translation terminal provided in this embodiment of the invention, taking the longest information link—mobile terminal (such as PC, smartphone, etc.), translation terminal, and Bluetooth device (such as Bluetooth headset)—as an example.

[0110] Figure 6 This is a diagram illustrating the overall architecture and workflow of the translation terminal provided by this invention, as follows: Figure 6 As shown, it displays two audio streams processed in parallel: one is the other party's speech (downlink audio stream), and the other is the user's own speech (uplink audio stream).

[0111] The downlink data link illustrates how the user hears the other party's speech during translation. Specifically, the other party's voice enters the terminal device via a phone or app like WeChat, serving as the downlink data stream interface to the translation device. Upon entering the digital signal processing unit (DSP), this raw downlink audio stream is split into two. One path passes through the audio algorithm processing module, then through the Hardware Abstraction Layer (HAL), and is sent to the translation engine for processing. The other path (not shown in detail in the diagram, but logically possible) is sent to the MIX (mixer) module within the DSP (Digital Signal Processing Unit). Figure 6 The data selection unit shown (waiting to be mixed with the translated audio) is used to enable simultaneous playback of the original and translated texts.

[0112] The translation engine translates the received audio (such as English) into the target language (such as Chinese), and then sends the translated Chinese audio back to the playback module of the digital signal processing unit via HAL. The Chinese translation from the playback module can be mixed with the original English audio. The mixed audio stream is then routed to the designated output channel via PCM (Pulse Code Modulation, a digital audio format) and MUX (Multiple-Selector).

[0113] Audio data is sent to the first Bluetooth module via IIS (a serial audio bus interface). The first Bluetooth module then wirelessly transmits the final audio (plain audio or mixed audio) to the Bluetooth headset worn by the user via Bluetooth.

[0114] The uplink data link demonstrates how a user's spoken words are translated and transmitted to the other party. Specifically, the user's voice is captured by a microphone array (i.e., a sound pickup module). The microphone signal enters the digital signal processing unit via PDM (Pulse Density Modulation, Digital Microphone Interface). First, it undergoes noise reduction and echo cancellation processing by the audio algorithm processing module to remove sound transmitted from headphones or speakers, preventing the other party from hearing their own echo.

[0115] The pre-processed clean vocals (upstream audio stream) are also split in two. One path passes through the recording module, then through HAL, and is sent to the translation engine for processing. The other path goes directly to the mixing module (i.e., the data selection unit).

[0116] The translation engine translates the received Chinese into English, and then sends the translated English audio back to the data selection unit of the digital signal processing unit via HAL. After selection, the translated English uplink audio stream (original text only, translation only, or original text and translation) is sent to the terminal via the data interface and finally transmitted to the other party in the call.

[0117] In this embodiment of the invention, the digital signal processing unit (DSP) simultaneously routes the audio at high speed and copies the audio data to the translation engine for processing. This is a bypass monitoring + processing mode that does not affect the real-time performance of the call while completing the translation task. This call translation architecture, through the collaborative working mode of the translation engine and the DSP, delegates the highly real-time data routing to the processing unit and the computationally intensive translation task to the translation engine. Furthermore, through data copying and audio mixing mechanisms, it seamlessly embeds bidirectional real-time translation functionality without affecting the basic call quality.

[0118] The translation method provided by this invention is described below. The translation method described below can be referred to in correspondence with the translation terminal described above.

[0119] Based on any of the above embodiments Figure 7This is a flowchart illustrating the translation method provided by the present invention, as shown below. Figure 7 As shown, the method is applied to a translation terminal, which includes a first wireless module and a second wireless module. The method includes: Step 710: Receive the first audio stream from the mobile terminal via the second wireless module; Step 720: Based on the first audio stream, generate a first translated audio stream and play the first translated audio stream, or send the first translated audio stream to the peripheral audio device connected to the first wireless module via the first wireless module; Step 730: Receive the second audio stream and generate a second translated audio stream based on the second audio stream; Step 740: The second translated audio stream is sent to the mobile terminal via the second wireless module.

[0120] Specifically, before performing step 710, the translation terminal 100 has typically established a first wireless communication link with the user's peripheral audio device 200 (such as a Bluetooth headset) through its first wireless module 110, and simultaneously established a second wireless communication link with the user's mobile terminal 300 (such as a smartphone) through its second wireless module 120, thereby constructing a dual-path parallel communication foundation.

[0121] When a user engages in cross-language communication via mobile terminal 300 (e.g., making a voice call with a foreign friend using WeChat), the other party's voice is transmitted as system audio by mobile terminal 300 through the second wireless communication link. The audio data received by translation terminal 100 through its second wireless module 120 constitutes the first audio stream. This audio stream is the downlink voice that needs to be translated for the local user.

[0122] Upon receiving the first audio stream, the main control module 130 inside the translation terminal 100 immediately processes it. This processing involves translating the first audio stream to generate the corresponding translated speech, i.e., the first translated audio stream. For example, if the first audio stream is English speech, the main control module will translate it into Chinese speech.

[0123] After generating the first translated audio stream, it can be played directly through the playback module inside the translation terminal 100. Alternatively, it can be transmitted in real time to the user's external audio device 200 via the first wireless module 110 along the first wireless communication link. In this way, the user can directly hear the translated text of the other party's speech through headphones.

[0124] When a user wants to reply, their speech needs to be captured and translated. Steps 730 and 740 describe this uplink processing. It should be noted that there is no time restriction between steps 730 and 740 and steps 710 and 720.

[0125] This method first requires receiving a second audio stream from the translation terminal or an established external audio device, i.e., the original speech of the user on the local end. Upon receiving the second audio stream, the main control module 130 also generates a second translated audio stream based on it. For example, if the user speaks in Chinese, the main control module will translate it into English in real time, generating the second translated audio stream.

[0126] After generating the second translated audio stream, the method sends the translated audio stream back to the mobile terminal 300 via the second wireless module 120 along the second wireless communication link. Upon receiving this audio stream, the mobile terminal 300 uses it as microphone input and sends it to the other party in the call through a running communication application (such as WeChat). In this way, the other party can hear the translated version of the user's speech.

[0127] The translation method provided in this invention utilizes the dual wireless module hardware architecture of the translation terminal, allocating the processing paths of the downlink audio stream and the uplink audio stream on two independent wireless links. This dual-path concurrent processing mechanism fundamentally solves the latency and conflict problems caused by bidirectional audio transmission under a single link, achieving true full-duplex, low-latency, bidirectional real-time translation, and providing users with a smooth, natural, and uninterrupted cross-language communication experience.

[0128] It should be noted that other embodiments of the translation method provided by the present invention can refer to the various embodiments of the translation terminal described above, and will not be repeated here.

[0129] Based on any of the above embodiments, this invention provides a translation system, which includes a peripheral audio device, a mobile terminal, and a translation terminal as described in any of the above embodiments.

[0130] It should be noted that the specific workflow of the translation system provided in this embodiment of the invention can be referred to the above embodiments of the translation terminal and translation amplification, and will not be repeated here.

[0131] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8As shown, the electronic device may include a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a translation method. This method is applied to a translation terminal, which includes a first wireless module and a second wireless module. The method includes: receiving a first audio stream from a mobile terminal via the second wireless module; generating a first translated audio stream based on the first audio stream and playing the first translated audio stream, or sending the first translated audio stream to a peripheral audio device connected to the first wireless module via the first wireless module; receiving a second audio stream and generating a second translated audio stream based on the second audio stream; and sending the second translated audio stream to the mobile terminal via the second wireless module.

[0132] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the translation methods provided by the above methods. The method is applied to a translation terminal, which includes a first wireless module and a second wireless module. The method includes: receiving a first audio stream from a mobile terminal via the second wireless module; generating a first translated audio stream based on the first audio stream and playing the first translated audio stream, or sending the first translated audio stream to a peripheral audio device connected to the first wireless module via the first wireless module; receiving a second audio stream and generating a second translated audio stream based on the second audio stream; and sending the second translated audio stream to the mobile terminal via the second wireless module.

[0134] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the translation methods provided by the above methods. The method is applied to a translation terminal, the translation terminal including a first wireless module and a second wireless module. The method includes: receiving a first audio stream from a mobile terminal via the second wireless module; generating a first translated audio stream based on the first audio stream and playing the first translated audio stream, or sending the first translated audio stream to a peripheral audio device connected to the first wireless module via the first wireless module; receiving a second audio stream and generating a second translated audio stream based on the second audio stream; and sending the second translated audio stream to the mobile terminal via the second wireless module.

[0135] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A translation terminal, characterized by, The application comprises: a first wireless module, configured to establish a first wireless communication link with an external audio device; a second wireless module, configured to establish a second wireless communication link with a mobile terminal; a master module, configured to receive a first audio stream from the mobile terminal through the second wireless module, generate a first translated audio stream based on the first audio stream, and control the first translated audio stream to be played or send the first translated audio stream to the external audio device via the first wireless module; the master module is further configured to receive a second audio stream, generate a second translated audio stream based on the second audio stream, and send the second translated audio stream to the mobile terminal via the second wireless module.

2. The translation terminal according to claim 1, characterized in that, The first wireless module is further configured to: receive a hardware coordination signal sent by the second wireless module, and perform communication arbitration on the occupation of channel resources between the first wireless module and the second wireless module based on the hardware coordination signal.

3. The translation terminal according to claim 2, characterized in that, The hardware coordination signal comprises a priority signal and a request signal, and the first wireless module is specifically configured to: receive the priority signal and the request signal sent by the second wireless module; in response to the request signal, determine whether to generate an authorization signal based on the transmission priority of the priority signal and an internal signal; in the case of generating the authorization signal, send the authorization signal to the second wireless module to authorize the second wireless module to occupy the channel resources.

4. The translation terminal of claim 2, wherein, The master module is further configured to: receive an arbitration request sent by the first wireless module; in response to the arbitration request, adjust the priority of the first wireless module and the second wireless module in occupying the channel resources based on at least one of the buffer status of the bidirectional audio stream, the working status of the translation engine, and historical communication statistical data.

5. The translation terminal of claim 1, wherein, The master module is further configured to: invoke a software coexistence interface to coordinate the communication timing between the first wireless module and the second wireless module.

6. The translation terminal of claim 1, wherein, The master module comprises a processing unit, a translation engine, and a hardware abstraction layer connecting the processing unit and the translation engine; the processing unit is configured to perform audio preprocessing on the first audio stream received from the second wireless module, copy the preprocessed first audio stream, and transmit the copied first audio stream to the translation engine through the hardware abstraction layer; the translation engine is configured to translate the first audio stream, generate the first translated audio stream, and send the first translated audio stream to the processing unit through the hardware abstraction layer; the processing unit is further configured to receive the first translated audio stream and transmit the first translated audio stream to the first wireless module.

7. The translation terminal of claim 6, wherein, The processing unit is further configured to: perform audio mixing processing on the first audio stream and the first translated audio stream to generate a mixed audio stream; control the mixed audio stream to be played or send the mixed audio stream to the external audio device via the first wireless module.

8. The translation terminal of claim 6, wherein, The application further comprises an audio input module, and the processing unit is further configured to: receive the second audio stream from the audio input module and perform audio preprocessing on the second audio stream; transmitting the preprocessed second audio stream to the translation engine through the hardware abstraction layer, so that the translation engine generates the second translated audio stream.

9. The translation terminal of claim 8, wherein, The audio input module comprises a microphone array. Alternatively, the audio input module is the first wireless module, and the second audio stream is received from the external audio device through the first wireless module.

10. The translation terminal according to any one of claims 6 to 9, characterized in that, The audio preprocessing comprises at least one of noise reduction processing, echo cancellation processing, and adaptive processing.

11. A method of translation, characterized by, The method is applied to a translation terminal, which comprises a first wireless module and a second wireless module, and the method comprises: receiving a first audio stream from a mobile terminal through the second wireless module; generating a first translated audio stream based on the first audio stream, and playing the first translated audio stream, or alternatively, sending the first translated audio stream to an external audio device connected to the first wireless module via the first wireless module; receiving a second audio stream, and generating a second translated audio stream based on the second audio stream; sending the second translated audio stream to the mobile terminal via the second wireless module.

12. A translation system, characterized by The system comprises an external audio device, a mobile terminal, and a translation terminal as claimed in any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the translation method as claimed in claim 11.

14. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the translation method as claimed in claim 11.

Images