Conversation simultaneous transmission method, earphone and readable storage medium

Abstract

The invention provides a conversation simultaneous transmission method, an earphone and a readable storage medium, and belongs to the technical field of data processing, and the method comprises the steps: building and maintaining a Bluetooth connection with a mobile terminal, and operating a Bluetooth audio protocol on the Bluetooth connection; in response to a simultaneous transmission mode starting instruction, suspending the Bluetooth audio protocol and activating a general attribute configuration file GATT data protocol in a state of keeping the Bluetooth connection uninterrupted; and the GATT data channel and the mobile terminal are utilized to carry out data interaction of the simultaneous transmission service. According to the method, the audio protocol and the GATT data protocol are dynamically switched on the single Bluetooth physical connection, so that seamless switching between a common mode and a simultaneous transmission mode can be realized without disconnecting and reconnecting; meanwhile, data are transmitted through a GATT channel, the limitation that a traditional Bluetooth audio protocol cannot separate double-person sound sources is broken through, and low-delay and high-precision independent simultaneous transmission under the double-person dialogue scene is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a dialogue simultaneous interpretation method, an earphone and a readable storage medium. BACKGROUND

[0002] With the development of cross-language communication needs, face-to-face simultaneous interpretation applications based on Bluetooth earphones, especially True Wireless Stereo (TWS) Bluetooth earphones, are increasingly popular. In such applications, users often want to conveniently switch between daily music listening, calling functions and translation functions, and realize real-time translation of two-person dialogue.

[0003] Bluetooth earphones usually use standard audio protocols, such as Advanced Audio Distribution Profile (A2DP) or Hands Free Profile (HFP), to support audio playback and voice collection. When data services such as translation are needed, the existing solution is usually to directly multiplex the above audio protocols for voice transmission, or the user needs to manually disconnect the current Bluetooth connection, re-pair or switch to other connection modes.

[0004] However, the above existing methods have deficiencies in actual use. First, during the switching between ordinary audio mode and translation mode, there is often a connection interruption or a long waiting time, which cannot realize smooth transition. Second, based on the transmission mode of the existing audio protocol, it is difficult to meet the real-time requirements of independent interaction of two-way voice data when processing two-person dialogue scenarios, which affects the fluency of simultaneous interpretation experience. SUMMARY

[0005] The present application provides a dialogue simultaneous interpretation method, an earphone and a readable storage medium, to solve the problems of connection interruption or delay when switching between ordinary audio services and simultaneous interpretation services of Bluetooth earphones in the prior art, and the difficulty of traditional transmission methods to meet the real-time interaction requirements of two-person dialogue, realizing seamless switching between ordinary mode and simultaneous interpretation mode and efficient and independent dialogue data interaction.

[0006] The present application provides a dialogue simultaneous interpretation method applied to a master earphone, which is in communication connection with a slave earphone, mainly comprising: establishing and maintaining a Bluetooth connection with a mobile terminal, and running a Bluetooth audio protocol on the Bluetooth connection; in response to a simultaneous interpretation mode opening instruction from the mobile terminal, suspending the Bluetooth audio protocol while keeping the Bluetooth connection unbroken, and activating a Generic Attribute Profile (GATT) data protocol based on the Bluetooth connection; Interact with the mobile terminal through the GATT data channel for data interaction of the simultaneous interpretation service.

[0007] According to the application, a dialogue simultaneous interpretation method is provided, which interacts with the mobile terminal through the GATT data channel for data interaction of the simultaneous interpretation service, and comprises the following steps. The first audio data to be translated collected by the main earphone is sent to the mobile terminal through the GATT data protocol. The first translated audio data returned by the mobile terminal is received through the GATT data protocol, which is obtained by translating the first audio data to be translated. The first translated audio data is forwarded to the secondary earphone through a private link between the main earphone and the secondary earphone.

[0008] According to the application, a dialogue simultaneous interpretation method is provided, which interacts with the mobile terminal through the GATT data protocol for data interaction of the simultaneous interpretation service, and further comprises the following steps. The second audio data to be translated collected by the secondary earphone is received through a private link between the main earphone and the secondary earphone. The second audio data to be translated is sent to the mobile terminal through the GATT data protocol. The second translated audio data returned by the mobile terminal is received through the GATT data protocol, which is obtained by translating the second audio data to be translated. The second translated audio data is decoded and played.

[0009] According to the application, a dialogue simultaneous interpretation method is provided, which interacts with the mobile terminal through the GATT data protocol for data interaction of the simultaneous interpretation service, and further comprises the following steps. The routing identifier of the received translated audio data is parsed. If it is determined that the routing identifier indicates that the target is the secondary earphone, it is determined that the translated audio data is the first translated audio data. If it is determined that the routing identifier indicates that the target is the main earphone, it is determined that the translated audio data is the second translated audio data.

[0010] According to the application, a dialogue simultaneous interpretation method is provided, which interacts with the mobile terminal through the GATT data protocol for data interaction of the simultaneous interpretation service, and further comprises the following steps. When detecting that any earphone establishes the Bluetooth connection with the mobile terminal, the any earphone is taken as the main earphone. The role configuration instruction is sent to the other earphone through the communication link between the main earphone and the other earphone, so as to configure the other earphone as the secondary earphone. Establish a private link with the secondary earphone.

[0011] According to a simultaneous interpretation method provided by the present invention, the step of sending the first audio data to be translated, collected by the main earpiece, to the mobile terminal via the GATT data protocol includes: The first audio data to be translated is compressed and encoded using the Opus encoder to generate a first audio data packet, and the first audio data packet is sent to the mobile terminal through the GATT data protocol. Accordingly, after receiving the first translated audio data returned by the mobile terminal via the GATT data protocol, the method further includes: using the Opus decoder to decode the first translated audio data.

[0012] According to a simultaneous interpretation method provided by the present invention, the step of sending the second audio data to be translated to the mobile terminal via the GATT data protocol includes: The second audio data to be translated is compressed and encoded using the Opus encoder to generate a second audio data packet, and the second audio data packet is sent to the mobile terminal through the GATT data protocol. Accordingly, after receiving the second translated audio data returned by the mobile terminal via the GATT data protocol, the method further includes: using the Opus decoder to decode the second translated audio data.

[0013] According to a simultaneous interpretation method provided by the present invention, the first audio data to be translated is obtained based on the following steps: The first vibration signal is acquired using the bone conduction microphone of the main earphone; The first sound wave signal is acquired using the air conduction microphone array of the main earphone; The first vibration signal and the first sound wave signal are weighted, fused, and denoised to generate the first audio data to be translated. Accordingly, the second audio data to be translated is obtained based on the following steps: The second vibration signal is acquired using the bone conduction microphone of the secondary earphone; The second sound wave signal is acquired using the air conduction microphone array of the secondary earphone; The second vibration signal and the second sound wave signal are weighted, fused, and denoised to generate the second audio data to be translated.

[0014] The simultaneous interpretation method provided by the present invention further includes: In response to a simultaneous transmission mode shutdown command from the mobile terminal, a protocol recovery request is sent to the mobile terminal; the protocol recovery request is used to request the reactivation of the Bluetooth audio protocol and to stop data interaction based on the GATT data channel.

[0015] According to a simultaneous interpretation method provided by the present invention, the step of establishing and maintaining a Bluetooth connection with a mobile terminal and running the Bluetooth audio protocol on the Bluetooth connection includes: Establish and maintain a Bluetooth connection with the mobile terminal based on Basic Rate or Enhanced Data Rate (BR / EDR) technology; The Advanced Audio Distribution Protocol (A2DP) and the Hands-Free Protocol (HFP) run on the Bluetooth connection.

[0016] This invention also provides another simultaneous interpretation method for dialogue, applied to mobile terminals, mainly including: Establish and maintain a Bluetooth connection with the main earphone, and run the Bluetooth audio protocol on the Bluetooth connection; In response to the simultaneous interpretation mode activation operation, a simultaneous interpretation mode activation command is sent to the main earphone, and the Bluetooth audio protocol is suspended in consultation with the main earphone, and the GATT data protocol based on the Bluetooth connection is activated. The GATT data protocol is used to perform simultaneous interpretation data interaction with the main headset.

[0017] The present invention also provides a Bluetooth headset, comprising: a first earpiece and a second earpiece; The first and second earpieces are configured such that either earpiece that establishes a connection with the mobile terminal is designated as the primary earpiece, and the other earpiece is designated as the secondary earpiece. The main earpiece includes a memory and a processor, which executes a computer program to implement the simultaneous interpretation method as described above.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the simultaneous interpretation method as described above.

[0019] The simultaneous interpretation method, headset, and readable storage medium provided by this invention can achieve seamless switching between normal mode and simultaneous interpretation mode by dynamically switching between audio protocol and GATT data protocol on a single Bluetooth physical connection without disconnection and reconnection; at the same time, by using GATT channel to transmit data, it breaks through the limitation of traditional Bluetooth audio protocol that cannot separate two voice sources, and realizes low latency and high precision independent simultaneous interpretation in two-person dialogue scenarios. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is one of the flowcharts of the simultaneous interpretation method for dialogue provided by the present invention.

[0022] Figure 2 This is a schematic diagram illustrating the data interaction process between the secondary earphone and the mobile terminal provided by the present invention.

[0023] Figure 3 This is a schematic diagram of the data interaction process between the main earphone and the mobile terminal provided by the present invention.

[0024] Figure 4 This is the second flowchart of the simultaneous interpretation method for dialogue provided by the present invention.

[0025] Figure 5 This is a schematic diagram of the hardware architecture and audio signal processing logic of the headphone side provided by the present invention.

[0026] Figure 6 This is a schematic diagram of the data processing logic on the mobile terminal side provided by the present invention.

[0027] Figure 7 This is a schematic diagram of the connection status and protocol operation in the normal TWS mode provided by the present invention.

[0028] Figure 8 This is a schematic diagram of the connection status and data interaction in the simultaneous interpretation mode provided by the present invention.

[0029] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] It should be noted that in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0032] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more.

[0033] With the increasing frequency of cross-language communication scenarios, face-to-face simultaneous interpretation systems based on mobile terminals and wireless earphones (especially TWS true wireless Bluetooth earphones) have been widely used in business negotiations, travel, and daily communication. In these application scenarios, users expect to be able to flexibly switch between normal usage modes such as listening to music and making calls and face-to-face translation modes using their worn Bluetooth earphones, and obtain a real-time and smooth conversation translation experience.

[0034] Existing TWS Bluetooth earbuds primarily establish a physical connection with mobile terminals via classic Bluetooth technology and rely on predefined audio profiles in the Bluetooth standard to transmit data. For example, when a user listens to music, the system runs the Advanced Audio Distribution Protocol (A2DP); when a user makes a voice call or uses a traditional voice assistant, the system runs the Hands-free Protocol (HFP). When performing translation tasks, existing solutions typically reuse the aforementioned audio protocols used for calls to collect and transmit voice signals, or require the user to disconnect the current classic Bluetooth connection and re-establish a separate Bluetooth Low Energy (BLE) connection to transmit data.

[0035] However, in practical applications, it has been found that this technical solution based on traditional connection architecture has obvious limitations.

[0036] On the one hand, standard Bluetooth audio protocols (such as A2DP and HFP) and data transmission protocols are usually mutually exclusive or managed independently. When users need to switch between ordinary audio services and dedicated translation services, it often involves the disconnection and reconnection of the underlying connection link or a complex mode handshake process, resulting in obvious delays or even connection interruptions when switching services, which disrupts the continuity of user experience.

[0037] On the other hand, traditional call protocols are usually designed for mono audio stream transmission, and the signals collected by the microphone are often mixed at the system level. This transmission mechanism is difficult to meet the high real-time requirements of independent and concurrent transmission of multiple audio data in two-person dialogue scenarios, which limits the interactive efficiency of simultaneous interpretation services.

[0038] In summary, existing solutions cannot seamlessly switch between ordinary TWS Bluetooth earbuds and dedicated translation earbuds, limiting their widespread adoption and flexibility, and impacting user experience. Furthermore, the high real-time requirements of two-person dialogue scenarios mean that the processing delays in traditional solutions disrupt the dialogue rhythm, affecting the naturalness and fluency of communication. Users often expect to hear the translation result immediately after the other person speaks. Therefore, this invention provides a new simultaneous interpretation method, earbuds, and a readable storage medium that effectively solves the above-mentioned shortcomings. The following describes the invention in conjunction with... Figures 1-9 A detailed explanation will be provided.

[0039] This invention provides a simultaneous interpretation method for dialogue, applicable to wireless audio systems supporting two-person simultaneous interpretation. The wireless audio system typically includes a mobile terminal and a TWS Bluetooth headset connected to it. The TWS Bluetooth headset includes two independent earpieces; logically, one earpiece is configured as the primary earpiece, and the other as the secondary earpiece. The primary earpiece, as the core node for communication with the mobile terminal, is responsible for establishing a Bluetooth connection and data aggregation and forwarding; the secondary earpiece communicates with the primary earpiece via a private link to achieve audio data synchronization and transmission. The mobile terminal can be a smartphone, tablet, smartwatch, or other electronic device with Bluetooth communication and data processing capabilities.

[0040] Figure 1 This is one of the flowcharts illustrating the simultaneous interpretation method provided by the present invention. In this embodiment, the main earpiece is the subject executing the method. Figure 1 As shown, the method includes the following steps: Step 11: Establish and maintain a Bluetooth connection with the mobile terminal, and run the Bluetooth audio protocol on the Bluetooth connection.

[0041] In this embodiment, with both the mobile terminal and the headset powered on and the user having enabled automatic Bluetooth connection, the main headset will typically automatically pair, handshake, and establish a link with the mobile terminal via its built-in Bluetooth communication module, according to Bluetooth technology standards. Specifically, this Bluetooth connection usually refers to a traditional Bluetooth physical connection (ACL Link) based on Basic Rate (BR) or Enhanced Data Rate (EDR) technology. This physical connection method offers high bandwidth stability and compatibility, and can support various upper-layer application protocols.

[0042] After a Bluetooth connection is established, the primary headset, acting as an audio playback device, will by default run the Bluetooth audio protocol on that connection to support the user's daily audio needs. The Bluetooth audio protocol refers to a standardized Bluetooth profile used for transmitting high-quality audio streams or voice call data. For example, when a user plays music on a mobile device, the Advanced Audio Distribution Profile (A2DP) is used, while when the user makes a phone call, the Hands-Free Profile (HFP) is used. In these cases, the Bluetooth connection's bandwidth resources are primarily occupied by the aforementioned Bluetooth audio protocol for transmitting audio streams in formats such as SBC and AAC.

[0043] Step 12: In response to the simultaneous transmission mode activation command from the mobile terminal, while keeping the Bluetooth connection open, suspend the Bluetooth audio protocol and activate the GATT data protocol based on the Bluetooth connection.

[0044] The simultaneous interpretation mode activation command refers to the specific control signal sent by the mobile terminal to the main headset after detecting that the user has started the simultaneous interpretation service, such as when the user clicks the "Start Simultaneous Interpretation" button in the translation app.

[0045] After receiving the instruction to enable simultaneous interpretation mode, the main earpiece dynamically adjusts the current connection status through a specific protocol negotiation mechanism.

[0046] Specifically, suspending the Bluetooth audio protocol can include: the main headset sending a protocol pause request to the mobile terminal, or responding to a pause request initiated by the mobile terminal, causing the current A2DP or HFP protocol to enter an inactive state and stopping the transmission of audio streams via the Bluetooth audio protocol, thereby freeing up bandwidth resources of the physical connection. It is important to note that during this process, the underlying Bluetooth connection (i.e., the physical link) remains connected without disconnection or reconnection, thus avoiding the link rebuilding time required for switching to BLE mode in traditional solutions.

[0047] Simultaneously or subsequently, the main headset activates the Generic Attribute Profile (GATT) data protocol based on the same Bluetooth connection. While the GATT data protocol is typically used for low-power data transmission, in this embodiment, it is configured to run over the current Bluetooth connection (BR / EDR), i.e., GATTover BR / EDR, to carry subsequent high-real-time simultaneous transmission service data.

[0048] Activating the GATT data protocol can be achieved by establishing a GATT connection, negotiating the maximum transmission unit (MTU) size, and configuring parameters such as the connection interval, in order to build a high-speed channel for data pass-through.

[0049] Step 13: Use the GATT data channel to perform simultaneous transmission service data interaction with the mobile terminal.

[0050] Simultaneous interpretation services refer to the business process of real-time acquisition, transmission, translation, and playback of voice in a two-person dialogue scenario. Using the GATT data protocol for data interaction means that the main headset uses this protocol as a data carrier, constructing a specific data packet format at the application layer for bidirectional communication.

[0051] Data interaction generally includes two directions: uplink and downlink. In the uplink direction, the main earpiece sends the acquired audio data to be translated to the mobile terminal. This audio data can be collected by the main earpiece itself or from other devices, such as the secondary earpiece; this step does not limit this. After receiving the audio data, the mobile terminal uses a local or cloud-based translation engine for speech recognition and machine translation.

[0052] In the downlink direction, the main earpiece receives translated audio data returned by the mobile terminal. This translated audio data is a data stream obtained by translating and synthesizing speech from the aforementioned audio data to be translated. After receiving the audio data to be translated, the main earpiece can perform local playback processing or send it to other devices for playback according to business logic.

[0053] The technical solutions of the above embodiments effectively solve the problems of cumbersome device function switching and insufficient real-time transmission in the prior art.

[0054] First, addressing the issue of device function switching, this embodiment innovatively employs a dynamic protocol management mechanism on the same physical link. Existing solutions typically require users to manually disconnect the classic Bluetooth connection and then re-search and pair to establish a low-power Bluetooth connection for simultaneous interpretation. This process is not only cumbersome but also involves a disconnection period of several seconds. However, the simultaneous interpretation method provided in this embodiment, upon receiving the simultaneous interpretation mode activation command, does not sever the underlying classic Bluetooth physical connection. Instead, it merely suspends the currently running A2DP or HFP audio protocol at the protocol stack level and quickly activates the GATT data protocol. This protocol-level soft switching is instantaneous and imperceptible to the user, achieving a seamless experience from ordinary music / call mode to professional simultaneous interpretation mode, greatly improving the convenience and user stickiness of the solution.

[0055] Secondly, regarding real-time requirements, this embodiment suspends the high-bandwidth Bluetooth audio protocol, freeing up ample channel resources for the GATT data protocol. Compared to traditional solutions that attempt to cram control commands into congested audio protocol channels or rely on bandwidth-limited standard BLE connections, this embodiment utilizes the higher-bandwidth Classic Bluetooth Connection (BR / EDR) physical layer to carry the GATT protocol, constructing a dedicated high-speed data channel. This eliminates the need for complex handshake verification, resulting in a more streamlined and efficient data interaction process. It significantly reduces the overall latency of audio data from acquisition to transmission, thus meeting the millisecond-level real-time simultaneous interpretation requirements in two-person dialogue scenarios and ensuring a natural and smooth dialogue rhythm.

[0056] Based on the above embodiments, as an optional embodiment, this invention further describes how the simultaneous interpretation method utilizes the GATT data channel to perform data interaction for simultaneous interpretation services with the mobile terminal. It should be understood that in actual implementations, the data interaction for simultaneous interpretation services is bidirectional, including both uploading the audio data to be translated collected by the headset to the mobile terminal and receiving the translated audio data sent by the mobile terminal. The following embodiment will specifically describe one direction, namely, the process of the main headset collecting the first audio data to be translated, and the secondary headset ultimately presenting the first translated audio data after one-way translation.

[0057] Figure 2 This is a schematic diagram illustrating the data interaction process between the secondary earphone and the mobile terminal provided by the present invention, as shown below. Figure 2 As shown, the specific steps include, but are not limited to, the following: Step 131: The first audio data to be translated, collected by the main earphone, is sent to the mobile terminal via the GATT data protocol.

[0058] Upon receiving the simultaneous interpretation mode activation command from the mobile terminal and confirming that the simultaneous interpretation mode is activated, the main earpiece will initiate the audio acquisition function. The first audio data to be translated refers to the speech signal emitted by the user wearing the main earpiece, which is captured by one or more microphones on the main earpiece and converted into a digital audio data stream. The microphones of the main earpiece convert the captured digital audio data stream into analog electrical signals, which are then sampled and quantized by the analog-to-digital converter (ADC) in the audio codec according to a preset sampling rate (e.g., 16kHz) and bit depth (e.g., 16bit) to generate the original digital audio data, which constitutes the first audio data to be translated.

[0059] Subsequently, the main earphone does not use Bluetooth audio protocols (such as A2DP or HFP) to transmit the first audio data to be translated. Instead, based on the established Bluetooth connection, it utilizes the activated GATT data protocol. Specifically, the main earphone packages the acquired first audio data to be translated into data packets (Packet Data Units, PDUs) conforming to the GATT data protocol specifications. The transmission process involves the main earphone's Bluetooth radio frequency unit sending these data packets to the paired mobile terminal via the GATT data channel in a data pass-through manner. This method cleverly bypasses the fixed processing mode of traditional audio protocols for audio streams, transmitting audio as pure data, laying the foundation for realizing dual-channel independent audio interaction.

[0060] Step 132: Receive the first translated audio data returned by the mobile terminal through the GATT data protocol. The first translated audio data is obtained by translating the first audio data to be translated.

[0061] Step 133: Forward the first translated audio data to the secondary earphone via a private link between the secondary earphone and the secondary earphone.

[0062] The private link is a dedicated communication channel pre-established between the main and secondary earpieces for efficient collaboration. It is independent of the public Bluetooth communication between the main earpiece and the mobile terminal. The forwarding is executed by the main earpiece. After receiving the first translated audio data, it does not process it for playback locally but immediately transmits it wirelessly via the private link to the secondary earpiece worn by user B. The secondary earpiece, upon receiving the first translated audio data, then decodes and plays it, allowing user B to clearly hear the corresponding translation of user A's words.

[0063] Furthermore, within the simultaneous interpretation framework, this invention clearly defines a unidirectional audio translation flow path: the first audio data to be translated is collected by the main earpiece, uploaded to the mobile terminal for translation via the GATT data channel, and the translated audio data is then returned to the main earpiece via the GATT data channel. Finally, it is cleverly routed to the secondary earpiece for playback via a private link between the main and secondary earpieces. By transmitting and forwarding the first translated audio data as a GATT data packet, not only is data interaction for simultaneous interpretation services achieved under a single Bluetooth connection, but more importantly, it constructs a directional information flow where one earpiece collects the first audio data to be translated, and the other earpiece receives the first translated audio data. This lays a solid technical foundation for achieving a natural, interference-free, and immersive two-person real-time dialogue translation experience, greatly enhancing the product's practicality and user experience.

[0064] The previous embodiment described a one-way communication process where user A speaks while user B listens to the translation through a secondary earpiece. The following embodiment will further describe the reverse audio processing flow, showing how, after user B responds, the secondary earpiece collects the second audio data to be translated, and the corresponding translation result (second translated audio data) is decoded and played back on user A's primary earpiece.

[0065] Figure 3 This is a schematic diagram illustrating the data interaction process between the main earphone and the mobile terminal provided by the present invention, as shown below. Figure 3 As shown, this embodiment will continue to be described based on the scenario in the previous embodiment.

[0066] When user B (assuming their language is Chinese) hears the first translated audio data corresponding to user A's first audio data to be translated, they begin to respond in Chinese, for example, saying, "I'm fine, thank you." At this point, the main earpiece will interact with the mobile terminal to execute a reverse data interaction process, which may include the following steps: Step 134: Receive the second audio data to be translated collected by the secondary earphone via a private link with the secondary earphone.

[0067] When user B speaks, the secondary earphone he wears will capture his voice through its own microphone and other audio acquisition components, and obtain second audio data to be translated, which includes the Chinese voice "I am fine, thank you".

[0068] It's important to note that the secondary earpiece does not attempt to establish direct communication with the mobile terminal. Instead, it sends the received second audio data to be translated to the primary earpiece via a private link already established between the primary and secondary earpieces. Therefore, the primary earpiece receives the second audio data to be translated from the secondary earpiece through this private link.

[0069] Step 135: The second audio data to be translated is sent to the mobile terminal via the GATT data protocol.

[0070] After receiving the second audio data to be translated, the main earphone sends the second audio data to the mobile terminal via the GATT data protocol.

[0071] In this step, the main earpiece acts as a data aggregator and forwarder. It encapsulates the second audio data to be translated received from the secondary earpiece according to the GATT data protocol specifications, and then sends it to the mobile terminal via the activated GATT data channel between the main earpiece and the mobile terminal. Thus, the first and second audio data to be translated for two different users, A and B, are both uploaded by the main earpiece, sharing the same Bluetooth connection and GATT data channel with the mobile terminal.

[0072] Step 136: Receive the second translated audio data returned by the mobile terminal via the GATT data protocol.

[0073] After the mobile terminal receives the second audio data to be translated, the pre-installed translation application (APP) recognizes that it is a Chinese speech and calls the translation engine to perform "Chinese to English" processing, generating the corresponding English speech "I'm fine, thank you". This string of English speech data constitutes the second translated audio data, which is obtained by translating the second audio data to be translated.

[0074] Step 137: Decode and play the second translated audio data.

[0075] After receiving and parsing the second translated audio data, namely the English voice "I'm fine, thank you," the main earphone performs a decoding and playback operation. For example, it restores the compressed and encoded second translated audio data into playable raw pulse code modulation (PCM) data. The playback method can be to drive the main earphone's built-in speaker to convert the decoded audio data into sound waves, so that user A wearing the main earphone can clearly hear the English translation of user B's words.

[0076] The dialogue simultaneous interpretation method provided by the present invention differentiates between the first audio data to be translated and the second audio data to be translated and their corresponding translated audio data, and uses the main earphone as a data interaction hub, combined with the private link between the main and auxiliary earphones, to implement the logic of two-person face-to-face dialogue translation and playback. This complete two-way interaction process is completed under a single Bluetooth connection, without frequent disconnection or protocol switching, thus achieving low-latency, high-efficiency, and smooth experience face-to-face dialogue simultaneous interpretation on ordinary TWS Bluetooth earphones.

[0077] In order to further clarify how the mobile terminal in the present invention accurately distributes the translation result to the corresponding earphone, a specific embodiment of the present invention details the structure of the translated audio data and the processing logic of the main earphone.

[0078] Based on the above two-way dialogue scenario, this embodiment introduces an audio stream distribution mechanism based on a routing identifier. The first translated audio data or the second translated audio data contains a routing identifier. The dialogue simultaneous interpretation method further includes: Analyzing the routing identifier of the received translated audio data; If it is determined that the routing identifier indicates the target as the auxiliary earphone, it is determined that the translated audio data is the first translated audio data; [[ID=!2]]If it is determined that the routing identifier indicates the target as the main earphone, it is determined that the translated audio data is the second translated audio data.

[0079] Continuing with the example of the conversation between user A (English speaker, wearing the main earphone) and user B (Chinese speaker, wearing the auxiliary earphone).

[0080] In the interaction process of the foregoing embodiment, whether it is the first audio data to be translated uploaded by the main earphone or the second audio data to be translated uploaded by the auxiliary earphone via the main earphone, when the translation application APP on the mobile terminal processes these data, it not only performs the translation operation, but also adds a specific identification information to the generated translated audio data. This identification information is called a routing identifier.

[0081] Specifically, when the mobile terminal receives the first audio data to be translated from user A (such as the English "Hello, how are you?") and translates it into the first translated audio data (Chinese "你好,你怎么样?"), since the mobile terminal knows that the first translated audio data is obtained by translating the first audio data to be translated of user A, it must be for user B wearing the auxiliary earphone to listen to. Therefore, a routing identifier clearly pointing to the auxiliary earphone is embedded in the GATT data packet encapsulating the first translated audio data. For example, this routing identifier can be a specific binary value (such as 0x01) or a fragment of a device address.

[0082] Correspondingly, when the mobile terminal receives the second audio data to be translated (Chinese "I'm fine, thank you") from user B and translates it into the second translated audio data (English "I'm fine, thank you"), since the mobile terminal knows that the second translated audio data must be for user A wearing the main headset to listen to, it will embed a routing identifier pointing to the main headset (such as 0x00) in the GATT data packet encapsulating the second translated audio data.

[0083] Thus, each of the first translated audio data and the second translated audio data contains a routing identifier for indicating its playback destination.

[0084] When the main headset receives any translated audio data from the mobile terminal through the GATT data channel, it can directly execute the following steps: First, the main headset needs to parse the routing identifier of the received translated audio data, including checking the header or specific field of each received GATT data packet and extracting the routing identifier information contained therein.

[0085] Then, it will make a judgment and decision based on the parsed routing identifier: If it is determined that the routing identifier indicates the target as the secondary headset, for example, the parsed value is 0x01, then it is determined that the translated audio data is the first translated audio data. In this case, the main headset knows that this data is for the secondary headset to play, so it will perform a forwarding operation, that is, send this GATT data packet to the secondary headset through the private link, and the main headset itself will not play.

[0086] If it is determined that the routing identifier indicates the target as the main headset, for example, the parsed value is 0x00, then it is determined that the translated audio data is the second translated audio data. In this case, the main headset knows that this data is for itself to play, so it will perform a local decoding and playback operation.

[0087] The dialogue simultaneous translation method provided by the present invention provides a simple and efficient decision-making basis for the main headset by introducing a routing identifier mechanism, enabling it to accurately distinguish and process two different-direction audio streams in the upstream and downstream directions, realizing the transformation of the complex audio stream distribution logic from passive guessing at the headset end to precise guidance actively controlled by the mobile terminal, thereby ensuring that the translated audio can be accurately delivered to the correct listener's ear, completely avoiding the possibility of sound confusion between the left and right ears, and making the logic of the entire two-person dialogue simultaneous translation system clearer, more reliable and stable in operation.

[0088] To clarify how the roles of the primary and secondary earbuds are determined in this invention, a specific embodiment of the invention provides a detailed description of the initial connection and role assignment process for TWS Bluetooth earbuds. This process typically occurs during the normal usage phase when the user begins using the earbuds but has not yet activated the simultaneous transmission mode.

[0089] Assume a user has a pair of TWS Bluetooth earbuds (referred to as the left and right earbuds for simplicity) and a mobile device. When the user removes either earbud from the charging case, for example, the right earbud first, and intends to connect to the mobile device, the system will perform the following steps to determine the primary and secondary earbuds: When a Bluetooth connection is established between any one of the earphones and the mobile terminal, that earphone is designated as the master earphone. A role configuration command is sent to the other earpiece via a communication link to configure the other earpiece as the secondary earpiece; Establish a private link with the secondary earphone.

[0090] In the above scenario, because the right earphone is removed first, it enters a discoverable and connectable state. The user's mobile terminal will then search for the right earphone and complete the Bluetooth pairing and connection process. Once this Bluetooth connection is successfully established, the system of this invention dynamically configures the right earphone, which was the first to successfully connect to the mobile terminal, as the primary earphone. This means that the determination of the primary earphone is not fixed at the factory, but is flexibly determined based on the user's actual connection with the mobile terminal during use.

[0091] After the right earpiece is designated as the primary earpiece, the user may then remove the left earpiece. At this point, the primary earpiece (right earpiece) needs to configure the left earpiece as the secondary earpiece and establish communication between them.

[0092] Specifically, the primary earpiece (right earpiece) actively searches for and discovers the unpaired left earpiece, and sends a role configuration command to the left earpiece via their internal communication protocol. This role configuration command explicitly informs the left earpiece that its role is that of a secondary earpiece. Upon receiving this role configuration command, the left earpiece confirms its subordinate status and will subsequently communicate primarily with the primary earpiece, rather than engaging in complex interactions directly with the mobile terminal.

[0093] After the main and secondary earpieces are assigned, in order to achieve subsequent data synchronization and command forwarding, such as audio data flow in the same transmission mode, this embodiment will establish a stable and efficient internal channel between the main and secondary earpieces, for example, by establishing a private link between the main and secondary earpieces.

[0094] The establishment of a private link refers to the process where the main earpiece (right earpiece) and the newly configured secondary earpiece (left earpiece) negotiate and activate a dedicated, low-latency data transmission channel using their communication link. This private link is independent of the public Bluetooth connection between the main earpiece and the mobile terminal; it is dedicated to internal collaboration between the two earpieces, ensuring the implementation of key functions such as synchronized audio playback, sensor data exchange, and efficient forwarding of translated audio in simultaneous interpretation mode.

[0095] The simultaneous interpretation method provided by this invention employs a dynamic and flexible TWS earphone master / slave role allocation mechanism, allowing either earphone to act as the master earphone, greatly improving the user's convenience for single-ear use and simplifying the complexity of the initial connection. Simultaneously, by establishing a private link between the master and slave earphones, the necessary internal communication foundation is built for the subsequent implementation of complex binaural collaboration functions.

[0096] The embodiments of the present invention will further describe in detail how the main earphone efficiently sends the first audio data to be translated to the mobile terminal, so as to clearly explain that the present invention can effectively solve the bandwidth bottleneck and high latency problems that may be encountered in the prior art when transmitting real-time audio data on Bluetooth data channels, especially on the relatively bandwidth-limited GATT channels.

[0097] When user A (wearing the main earpiece) says "Hello, how are you?" in English, the main earpiece picks up the first audio data to be translated (e.g., a 16kHz sampling rate, 16-bit depth PCM format data stream). If this unprocessed raw audio data is transmitted directly, the data volume is enormous, specifically 6000 * 16 / 8 = 32000 bytes / second, which would quickly fill the available bandwidth of the GATT data channel, leading to data congestion, packet loss, and unacceptable transmission delays, failing to meet the real-time requirements of simultaneous interpretation services. To address this technical deficiency, this embodiment employs an audio compression encoding technique at the earpiece end, specifically including: The first audio data to be translated is compressed and encoded using the Opus encoder to generate a first audio data packet, which is then sent to the mobile terminal via the GATT data protocol.

[0098] In this embodiment, the main earphone does not immediately encapsulate and send the first acquired audio data to be translated. Instead, it first calls its built-in Opus encoder. The Opus encoder is an open-source and advanced audio codec designed for real-time audio communication over networks. Its core advantage lies in maintaining high voice quality at extremely low bit rates and having very low algorithm latency.

[0099] By inputting the first audio data to be translated in PCM format into the Opus encoder for compression encoding, the lengthy audio data is compressed into a series of compact data units containing the same speech information, which constitute the first audio data packet. Through compression encoding, the size of the audio data is significantly reduced, for example, it can be compressed to one-tenth or even less of the original size, while the key information of the human voice is preserved.

[0100] Because the first audio data packet size becomes very small after Opus encoding, the main headset can easily send these small packets through the GATT data channel without causing channel congestion. This makes the audio data transmission process fast and smooth, fundamentally reducing the latency from audio acquisition to the data leaving the sending end.

[0101] Accordingly, after the mobile terminal returns the translated audio data to the main earpiece, the main earpiece also needs to be able to restore the compressed data to sound. Therefore, this embodiment also includes a corresponding decoding step: After receiving the first translated audio data returned by the mobile terminal via the GATT data protocol, the method further includes: using the Opus decoder to decode the first translated audio data.

[0102] In this step, the first translated audio data received by the primary earpiece from the GATT data channel is the Chinese translation sent to the secondary earpiece, which is also a data packet encoded and compressed using Opus. Before forwarding these data packets to the secondary earpiece via the private link, or after the secondary earpiece receives them, a decoding operation needs to be performed. The secondary earpiece's processor calls its built-in Opus decoder to decode the received data packets, accurately restoring them into a playable PCM audio stream.

[0103] This invention innovatively introduces the Opus encoding and decoding process into the end-side processing of TWS earphones, performing efficient compression on the audio data before it enters the Bluetooth transmission link, greatly reducing the amount of data to be transmitted, thus cleverly bypassing the inherent bandwidth limitation of the GATT channel.

[0104] First, it solves the technical challenge of transmitting real-time, high-quality voice over the GATT channel, making simultaneous interpretation services using GATT possible—a significant breakthrough for traditional Bluetooth protocol applications. Second, due to the substantial reduction in transmitted data and the smooth transmission channel, the entire uplink latency from voice acquisition to data transmission to the mobile terminal is reduced to milliseconds, providing crucial technical support for achieving a seamless, instant translation experience for users. Finally, the high-quality audio characteristics of Opus encoding ensure that even at low bitrates, the accuracy of speech recognition is not significantly affected, guaranteeing the final effectiveness of the entire translation system.

[0105] Based on the above embodiments, the present invention further optimizes the audio data uplink transmission process involving the secondary earphone side in the same way, which can ensure that when the user wearing the secondary earphone speaks, his voice data can also be transmitted to the mobile terminal in a low-latency and high-efficiency manner, thereby forming a complete and symmetrical two-way simultaneous interpretation system.

[0106] Continuing with the scenario described in the previous embodiment, when user B (wearing the secondary earpiece) responds with the Chinese phrase "I'm fine, thank you," the secondary earpiece collects the second audio data to be translated. If the secondary earpiece directly sends this raw PCM audio data to the primary earpiece via a private link, it will also put enormous bandwidth pressure on this internal communication link, potentially leading to internal transmission delays and data loss, thereby affecting the overall system response speed.

[0107] To address this issue and maintain consistency with the uplink processing flow of the main earphone, this embodiment also employs audio compression coding technology for the data processing of the secondary earphone and the forwarding and subsequent processing of the second audio data to be translated by the main earphone. Specifically, this includes: The second audio data to be translated is compressed and encoded using the Opus encoder to generate a second audio data packet, which is then sent to the mobile terminal via the GATT data protocol.

[0108] First, at the secondary earpiece, after acquiring the second audio data to be translated, the secondary earpiece uses the Opus encoder to compress and encode the second audio data, generating a second audio data packet. This process is exactly the same as how the primary earpiece processes the acquired audio data, compressing the original PCM audio stream into a smaller second audio data packet using the built-in Opus encoder.

[0109] After encoding, the secondary earpiece sends this lightweight second audio data packet to the primary earpiece via a private link. Upon receiving the packet, the primary earpiece, since the data is already in compressed format, requires no further processing and can directly send it to the mobile terminal via the GATT data channel.

[0110] Accordingly, when the mobile terminal returns the translated second audio data, such as the English phrase "I'm fine, thank you," to the main earphone, the main earphone needs to decode this data, which is also encoded using Opus, before it can be played. Therefore, this embodiment also includes a corresponding decoding step: after receiving the second translated audio data returned by the mobile terminal via the GATT data protocol, it further includes: using the Opus decoder to decode the second translated audio data.

[0111] In this step, after the main earphone receives the second translated audio data directed to itself from the GATT data channel, it calls its built-in Opus decoder to decode these compressed data packets in real time, restore them to a high-quality PCM audio stream, and then drive the speaker to play it out.

[0112] This invention extends and applies end-to-end audio compression to the uplink data stream of the secondary earpiece. For example, by efficiently compressing the second audio data to be translated as it leaves the secondary earpiece, the data load on all subsequent transmission links, including private links and the GATT channel, is minimized. First, it ensures complete symmetry in technical implementation and performance between the two uplink audio links originating from the primary and secondary earpieces, guaranteeing consistency and fairness in the two-way dialogue experience. Second, by compressing the internal transmission data from the secondary earpiece to the primary earpiece, the bandwidth pressure on the private link is greatly reduced, ensuring stable and low-latency internal communication, which is crucial for the primary earpiece to timely aggregate and forward data. Finally, together with the solution in the previous embodiment, it constitutes a complete end-to-end two-way low-latency audio transmission solution, ensuring that regardless of which user speaks, their voice can be processed by the system with the lowest latency and highest efficiency, laying a solid foundation for truly fluent and natural two-person simultaneous interpretation.

[0113] This embodiment further explains in more detail the generation methods of the first and second audio data to be translated, namely the audio acquisition and initial processing stages.

[0114] In real-world conversation scenarios, the microphones of TWS Bluetooth earbuds not only pick up the wearer's voice but also inevitably capture various ambient noises, and even echoes from the other party's translated speech played through the earbud speakers. If these interference signals mix with the wearer's voice, they will severely impact the performance of the subsequent speech recognition engine. To address this challenge, this embodiment employs a dual-microphone fusion pickup scheme combining bone conduction and air conduction technologies. This effectively solves the technical challenge of accurately picking up and separating the wearer's own voice in complex acoustic environments, especially when there is background noise, interference from others, or echoes. This is a crucial prerequisite for improving speech recognition accuracy and ensuring the final translation quality.

[0115] Specifically, the acquisition of the first audio data to be translated by the main earphone mentioned in the above embodiments is based on the following steps: The first vibration signal is acquired using the bone conduction microphone of the main earphone; The first sound wave signal is acquired using the air conduction microphone array of the main earphone; The first vibration signal and the first sound wave signal are weighted, fused, and denoised to generate the first audio data to be translated.

[0116] A bone conduction microphone is a special type of sensor that does not pick up sound through the air. Instead, it directly senses and collects the bone vibration signals caused by the vibration of the vocal cords when speaking, by being placed close to the user's skull (such as the skull around the auricle). A significant characteristic of this primary vibration signal is that it contains almost only the wearer's own voice information and is insensitive to ambient noise from the outside air and the voices of others, thus possessing an extremely high signal-to-noise ratio and natural sound source separation characteristics.

[0117] An air conduction microphone array refers to multiple traditional microphones on the main earphone that face the external environment. It collects a mixed sound wave signal, i.e., the first sound wave signal, which is transmitted through the air and contains the wearer's voice as well as all background environmental sounds. Although the first sound wave signal contains rich environmental information, it also preserves the most authentic and natural timbre of the wearer's voice.

[0118] The main earphone's processor or dedicated audio processing unit (Digital Signal Processor, DSP) performs a complex fusion process on the two different signal sources: using the first vibration signal as a clean anchor point for the wearer's speech, it can accurately determine which sounds are emitted by the wearer; then, combining the rich speech details and timbre information in the first sound wave signal, it performs weighted fusion using specific algorithms, such as adaptive filters or deep learning-based speech separation models. During the fusion process, based on the first vibration signal, the parts of the first sound wave signal that match the wearer's speech are enhanced, while background noise, wind noise, and echoes are suppressed. This process, which combines weighted fusion and noise reduction, ultimately generates high-quality first audio data for translation that is both pure and retains its natural timbre.

[0119] Accordingly, in order to ensure the consistency of the two-way call experience, the acquisition process of the second audio data to be translated collected by the secondary earpiece also adopts the same technical solution: that is, the bone conduction microphone of the secondary earpiece is used to collect the second vibration signal; the air conduction microphone array of the secondary earpiece is used to collect the second sound wave signal; and the second vibration signal and the second sound wave signal are weighted, fused and noise-reduced to generate the second audio data to be translated.

[0120] The simultaneous interpretation method provided by this invention introduces advanced dual-channel sensor fusion technology at the front end of audio acquisition. By combining the high directivity of bone conduction signals and the high fidelity of air conduction signals, it fundamentally solves the problem of distinguishing the wearer's voice from background noise in noisy environments, greatly improving the audio signal-to-noise ratio input to the speech recognition engine, thereby directly improving the accuracy of speech recognition and the precision of translation. It is particularly effective for scenarios where two people speak simultaneously or take turns speaking rapidly, clearly separating their respective speech streams and providing high-quality audio input for achieving truly high-precision independent simultaneous interpretation.

[0121] To address the challenge of seamlessly and quickly switching the headset from dedicated simultaneous interpretation data mode back to general audio mode, such as listening to music or making phone calls, after a user completes a translation conversation, this invention further provides an implementation method for exiting simultaneous interpretation mode and restoring normal Bluetooth audio functionality, mainly including: In response to a simultaneous transmission mode shutdown command from the mobile terminal, a protocol recovery request is sent to the mobile terminal; the protocol recovery request is used to request the reactivation of the Bluetooth audio protocol and to stop data interaction based on the GATT data channel.

[0122] Continuing with the scenario described in the previous embodiment, when user A and user B have finished their conversation and need to end the simultaneous interpretation function, user A can perform a shutdown operation on their mobile terminal's translation app, such as clicking the "End Simultaneous Interpretation" virtual button. This operation will trigger the mobile terminal to generate and send a simultaneous interpretation mode shutdown command.

[0123] When the main earpiece responds to a simultaneous interpretation mode disabling command from the mobile terminal, it initiates a negotiation process to exit simultaneous interpretation mode. This response process is real-time; the main earpiece's Bluetooth module continuously listens for GATT commands from the mobile terminal, and once it recognizes this specific simultaneous interpretation mode disabling command, it triggers subsequent actions.

[0124] The master headset sends a protocol recovery request to the mobile terminal. This protocol recovery request is a signaling action initiated by the master headset. Its principle is to explicitly inform the mobile terminal through a predefined protocol interaction: "Request to stop the current GATT data transmission service and prepare to reactivate the previously suspended Bluetooth audio protocol." This protocol recovery request can also be sent through the GATT channel, and is the reverse operation of the command to enable simultaneous transmission mode.

[0125] After receiving this protocol recovery request, the mobile terminal will conduct a quick protocol handshake with the main headset, and the two parties will reach an agreement to prepare for the switch of the protocol stack.

[0126] Specifically, reactivating the Bluetooth audio protocol means that previously suspended Bluetooth audio protocols such as A2DP for music playback or HFP for calls will be restored to an active state. The Bluetooth protocol stack will reallocate link resources, allowing audio streams to be transmitted again through these standard protocols. Simultaneously, stopping data interaction on the GATT data channel means that both the headset and the mobile phone will disable the dedicated GATT service used for simultaneous interpretation, ceasing the transmission and reception of audio data to be translated.

[0127] The process of switching from GATT data mode to standard audio protocol mode is just as crucial as entering simultaneous interpretation mode. It is essential that the underlying Bluetooth physical connection (ACL Link) remains uninterrupted at all times.

[0128] This invention proposes a method to guide the Bluetooth protocol stack to perform a soft switch of the upper-layer protocol without interrupting the physical connection by clearly defining the simultaneous interpretation mode shutdown command protocol recovery request at the application layer. This achieves a seamless transition from professional translation mode to daily use mode, and the user does not experience any delay in connection interruption or reconnection. It feels like an instant switch of application functions, which greatly improves the smoothness of the user experience and the ease of use of the product. Moreover, since it avoids the time-consuming and power-consuming operation of disconnection and reconnection, this solution is not only fast but also more energy-efficient.

[0129] As an optional embodiment, when a user uses the TWS Bluetooth earphones described in this invention in daily life, such as listening to music or making voice calls, the connection status between the earphones and the mobile terminal is as follows: Establish and maintain a Bluetooth connection with the mobile terminal based on Basic Rate or Enhanced Data Rate (BR / EDR) technology; The Advanced Audio Distribution Protocol (A2DP) and the Hands-Free Protocol (HFP) run on the Bluetooth connection.

[0130] BR / EDR technology is a core component of the Bluetooth standard, designed to provide relatively high transmission bandwidth and stable point-to-point or point-to-multipoint connections. Unlike Bluetooth Low Energy (BLE), which is primarily used for transmitting small amounts of data, BR / EDR technology is well-suited for transmitting continuous audio streams with large data volumes. This invention establishes and maintains a BR / EDR Bluetooth connection, forming a stable and reliable asynchronous connectionless link (ACLLink) between the main headset and the mobile terminal. This is the physical foundation for the operation of all subsequent upper-layer protocols.

[0131] One or more standard Bluetooth audio protocols will run on this Bluetooth connection based on BR / EDR technology.

[0132] The A2DP protocol is specifically designed for unidirectional transmission of high-quality stereo audio streams, forming the foundation for users to enjoy high-quality music using Bluetooth headphones. When a user plays music on a mobile device, the audio data flows from the mobile device to the primary earphone via the A2DP protocol, and then the primary earphone synchronously transmits the data to the secondary earphone, achieving stereo playback for both ears.

[0133] The HFP protocol is designed to enable Bluetooth hands-free calling. It supports two-way, real-time mono voice transmission and includes a set of commands for controlling calls (such as answering, hanging up, and rejecting calls). When a user receives an incoming call or makes a call, the system automatically or manually switches to the HFP protocol. At this time, the microphone of the main headset picks up the user's voice and sends it to the mobile terminal. Simultaneously, the mobile terminal also sends the other party's voice to the main headset for playback via the HFP protocol.

[0134] This invention does not reinvent a separate connection technology. Instead, it makes full use of the high-bandwidth physical link provided by BR / EDR technology, as well as A2DP and HFP, two standardized audio protocols that are widely supported in all smartphones and TWS earphones. This allows the earphones to behave and perform exactly the same as any high-end TWS Bluetooth earphones on the market when they are not performing simultaneous transmission functions, and can be seamlessly integrated into the user's existing usage habits.

[0135] This invention also provides another simultaneous interpretation method, which, together with the aforementioned simultaneous interpretation method executed on the main earpiece side, constitutes a complete and collaborative system. In this embodiment, the subject executing the simultaneous interpretation method is a mobile terminal, such as a smartphone or tablet computer. Figure 4 This is the second flowchart of the simultaneous interpretation method provided by the present invention, as shown below. Figure 4 As shown, the method mainly includes the following steps: Step 21: Establish and maintain a Bluetooth connection with the main earphone, and run the Bluetooth audio protocol on the Bluetooth connection.

[0136] Step 22: In response to the simultaneous interpretation mode activation operation, send a simultaneous interpretation mode activation command to the main earphone, and negotiate with the main earphone to suspend the Bluetooth audio protocol and activate the GATT data protocol based on the Bluetooth connection.

[0137] Step 23: Use the GATT data protocol to perform simultaneous interpretation data interaction with the main headset.

[0138] In the initial state, the mobile terminal establishes and maintains a Bluetooth connection with the main headset, and runs the Bluetooth audio protocol on the Bluetooth connection. The mobile terminal, as the central device, establishes an asynchronous connectionless link with the main headset, which is the peripheral device, through the classic Bluetooth technology BR / EDR, and runs standard Bluetooth audio protocols such as A2DP or HFP on this asynchronous connectionless link according to user operations (such as playing music or making a phone call).

[0139] When a user needs to activate the simultaneous interpretation function on a mobile device, for example, when a user clicks the "Start Simultaneous Interpretation" virtual button in a translation app.

[0140] After detecting the user's explicit intent, the mobile terminal proactively sends a specific formatted simultaneous interpretation mode activation command to the main earpiece. This command is transmitted to the main earpiece via the established Bluetooth link. Once the main earpiece confirms the activation command, the mobile terminal further negotiates with the main earpiece to suspend the currently running A2DP or HFP protocol, thereby freeing up bandwidth resources on the physical link. Following this, both parties collaboratively activate the GATT data protocol over the same Bluetooth physical connection.

[0141] After the GATT data protocol is successfully activated, it enters the simultaneous interpretation working mode and uses the GATT data protocol to conduct data interaction with the main headset for simultaneous interpretation services.

[0142] During this phase, the mobile terminal begins bidirectional data interaction with the main headset through the newly activated GATT data channel. This includes receiving audio data packets to be translated from the two users uploaded by the main headset; and sending the translated audio data packets generated locally to the main headset through this channel, whereby the main headset is responsible for routing, distribution, and playback.

[0143] The simultaneous interpretation method provided by this invention achieves seamless switching between normal mode and simultaneous interpretation mode by dynamically switching between audio protocol and GATT data protocol on a single Bluetooth physical connection without disconnection and reconnection; at the same time, it uses GATT channel to transmit data, breaking through the limitation of traditional Bluetooth audio protocol that cannot separate two voice sources, and realizes low latency and high precision independent simultaneous interpretation in two-person dialogue scenarios.

[0144] To comprehensively and clearly demonstrate the technical solution of this invention, a complete embodiment will be used below in conjunction with... Figures 5 to 8 As shown, the present invention provides a detailed description of the switching from ordinary TWS Bluetooth mode to simultaneous interpretation mode in the simultaneous interpretation system, and how a complete two-way dialogue audio stream is acquired, transmitted, processed and played in simultaneous interpretation mode.

[0145] User A, who speaks English, and User B, who speaks Chinese, are conducting business negotiations using a TWS Bluetooth headset as described in this invention. User A wears the right earpiece, and User B wears the left earpiece; assuming the right earpiece is the primary earpiece and the left earpiece is the secondary earpiece. User A's smartphone serves as the mobile terminal, and the accompanying translation app of this invention is installed on it.

[0146] Figure 5 This is a schematic diagram of the hardware architecture and audio signal processing logic on the headphone side provided by the present invention, as shown below. Figure 5 As shown, each earphone of the present invention (whether it is the main earphone or the secondary earphone) has advanced end-side audio processing capabilities.

[0147] When a user wearing the headphones speaks, their voice is captured, processed, and transmitted upstream. The headphones acquire the skull vibration signal when the user speaks via their bone conduction microphone, and simultaneously acquire a mixed sound wave signal containing the user's voice and ambient noise via an air conduction microphone array. These two signals are then sent to an audio processing unit that integrates AEC+ENC noise reduction algorithms.

[0148] The audio processing unit uses bone conduction signals as a reference and adaptively filters air conduction signals to accurately extract the user's pure voice and generate a high-quality pulse code modulation (PCM) audio stream. To accommodate subsequent Bluetooth transmission, this PCM audio stream is not used directly but is instead compressed in real-time by a high-efficiency Opus encoding module, forming a small Opus audio data packet suitable for network transmission.

[0149] When the headphones need to play the translation results from the mobile terminal, the downlink process is executed: the headphones receive the Opus-encoded audio data packets, which are first restored to PCM audio streams by the Opus decoding module. Then, the audio streams are processed by an ANC noise reduction algorithm to eliminate low-frequency environmental noise in the user's ear canal. Finally, the audio streams are played out through the speakers to provide the user with an immersive and clear listening experience.

[0150] In addition, the uplink Opus data packets processed by the secondary earpiece are forwarded to the primary earpiece forwarding module, while the downlink data received by the primary earpiece can also be distributed to the secondary earpiece as needed. This forms the basis for the collaborative work between the two earpieces.

[0151] Figure 6 This is a schematic diagram of the data processing logic on the mobile terminal side provided by the present invention, such as... Figure 6 As shown, when the translation app on the mobile terminal receives any uplink Opus data packet from the main earphone via Bluetooth, each data packet is first decoded by Opus to restore the original PCM audio. Subsequently, this PCM audio is sent to the AI ​​capability module, which may be deployed in the cloud. To communicate efficiently with the cloud, PCM data may first undergo Speex encoding. In the cloud, the data stream sequentially goes through three core steps: transcription (ASR), translation (NMT), and text-to-speech (TTS).

[0152] The translated and synthesized audio is transmitted back to the mobile terminal. After receiving the audio, the mobile terminal will call the Opus encoding module again to compress the PCM audio into Opus data packets so that it can be efficiently transmitted back to the headphones via Bluetooth.

[0153] During Opus encoding, a crucial step is that the mobile terminal embeds a clear routing identifier in the data packet based on the target audience (user A or user B) of the translated speech.

[0154] Optionally, Figure 6 It also demonstrated a parallel local recording function, where the mobile terminal can mix the decoded original conversations of both parties to obtain Mix audio, compress it into an MP3 file, and save it as meeting minutes.

[0155] Figure 7 This is a schematic diagram of the connection status and protocol operation in normal TWS mode provided by the present invention. Figure 8 This is a schematic diagram of the connection status and data interaction in the simultaneous interpretation mode provided by the present invention, combined with... Figure 7 and Figure 8 As shown, this invention demonstrates a core improvement: the ability to seamlessly switch between two modes to adapt to different user scenarios.

[0156] Reference Figure 7 As shown, when simultaneous audio transmission is not enabled, the earbuds are in normal TWS mode. In this mode, the mobile device and the main earbud connect via standard classic Bluetooth, running either the A2DP or HFP protocol depending on whether the user is listening to music or making a call. The secondary earbud synchronizes audio by monitoring the main earbud. At this time, the user experience is completely consistent with any mainstream TWS earbuds on the market, ensuring its versatility and basic performance as an everyday consumer electronics product.

[0157] refer to Figure 8As shown, when a user activates the simultaneous interpretation function in the translation app, the system seamlessly switches to conversational simultaneous interpretation mode. In this mode, the Bluetooth protocol between the mobile terminal and the main earpiece switches from A2DP / HFP to GATT over BR / EDR. This means that the original high-speed classic Bluetooth physical connection (BR / EDR) between the main earpiece and the mobile terminal is not disconnected; instead, the standard Bluetooth protocol is suspended on this physical link, activating the GATT protocol as a flexible data channel. This technical solution creatively utilizes the high-bandwidth physical layer of classic Bluetooth to carry GATT data transmission, thereby constructing a communication link that is both high-speed and capable of bidirectional, free transmission of customized data packets, perfectly solving the bandwidth and latency bottlenecks of real-time audio streaming transmission.

[0158] Under the GATT over BR / EDR architecture, a complete bidirectional translation data stream is achieved: (1) such as Figure 5 and Figure 8 As shown, the main and secondary earpieces each collect and encode the user's voice. The Opus data packets from the secondary earpiece are sent to the main earpiece for aggregation via an internal private link. The main earpiece then sends these two independent Opus data packets to transcription path 1 and transcription path 2 on the mobile terminal via the GATT over BR / EDR channel.

[0159] (2) such as Figure 6 As shown, after the mobile terminal completes the translation process, it sends two OPUS translation data packets with routing identifiers to the main headset through the same GATT over BR / EDR channel.

[0160] (3) The main earphone, acting as the system's intelligent gateway, parses the routing identifier after receiving a downlink data packet. If the identifier points to the secondary earphone, the main earphone forwards the OPUS translated data packet to the secondary earphone for playback via a private link and discards the local copy to avoid crosstalk. If the routing identifier points to itself, it decodes and plays the data directly locally.

[0161] As an optional embodiment, the present invention also provides a Bluetooth headset, mainly comprising a first earpiece and a second earpiece; The first and second earpieces are configured such that either earpiece that establishes a connection with the mobile terminal is designated as the primary earpiece, and the other earpiece is designated as the secondary earpiece. The main earpiece includes a memory and a processor. When the processor executes a computer program, it implements the simultaneous interpretation method provided in any of the above embodiments, which will not be described in detail here.

[0162] The earphones provided by this invention can achieve seamless switching between normal mode and simultaneous interpretation mode by dynamically switching between audio protocol and GATT data protocol on a single Bluetooth physical connection without disconnecting and reconnecting; at the same time, by using GATT channel to transmit data, it breaks through the limitation of traditional Bluetooth audio protocol that cannot separate two voice sources, and realizes low latency and high precision independent simultaneous interpretation in two-person dialogue scenarios.

[0163] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 9 As shown, the electronic device may include a processor 910, a communications interface 920, a memory 930, and a communication bus 940, wherein the processor 910, communications interface 920, and memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute a simultaneous interpretation method, which includes: establishing and maintaining a Bluetooth connection with a mobile terminal, running a Bluetooth audio protocol on the Bluetooth connection; responding to a simultaneous interpretation mode activation command from the mobile terminal, suspending the Bluetooth audio protocol while maintaining the Bluetooth connection, and activating the GATT data protocol based on the Bluetooth connection; and using the GATT data channel to perform simultaneous interpretation service data interaction with the mobile terminal.

[0164] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the simultaneous interpretation method provided in the above embodiments, the method comprising: establishing and maintaining a Bluetooth connection with a mobile terminal, running a Bluetooth audio protocol on the Bluetooth connection; responding to a simultaneous interpretation mode activation instruction from the mobile terminal, suspending the Bluetooth audio protocol while keeping the Bluetooth connection open, and activating the GATT data protocol based on the Bluetooth connection; and performing simultaneous interpretation service data interaction with the mobile terminal using the GATT data channel.

[0166] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the simultaneous interpretation method provided in the above embodiments. The method includes: establishing and maintaining a Bluetooth connection with a mobile terminal, running a Bluetooth audio protocol on the Bluetooth connection; responding to a simultaneous interpretation mode activation command from the mobile terminal, suspending the Bluetooth audio protocol while keeping the Bluetooth connection open, and activating the GATT data protocol based on the Bluetooth connection; and using the GATT data channel to perform simultaneous interpretation service data interaction with the mobile terminal.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A simultaneous interpretation method for dialogue, applied to a main earpiece, wherein the main earpiece and a secondary earpiece are communicatively connected, characterized in that, include: Establish and maintain a Bluetooth connection with the mobile terminal, and run the Bluetooth audio protocol on the Bluetooth connection; In response to the simultaneous transmission mode activation command from the mobile terminal, while keeping the Bluetooth connection open, the Bluetooth audio protocol is suspended and the GATT data protocol based on the Bluetooth connection is activated. The GATT data channel is used to perform simultaneous transmission data interaction with the mobile terminal.

2. The simultaneous interpretation method according to claim 1, characterized in that, The data interaction for simultaneous interpretation services between the mobile terminal and the GATT data channel includes: The first audio data to be translated, collected by the main earphone, is sent to the mobile terminal via the GATT data protocol. The first translated audio data is received from the mobile terminal via the GATT data protocol. The first translated audio data is obtained by translating the first audio data to be translated. The first translated audio data is forwarded to the secondary earpiece via a private link between the secondary earpiece and the secondary earpiece.

3. The simultaneous interpretation method according to claim 2, characterized in that, The data interaction for simultaneous interpretation services with the mobile terminal using the GATT data protocol further includes: The second audio data to be translated is received by the secondary earpiece via a private link with the secondary earpiece. The second audio data to be translated is sent to the mobile terminal via the GATT data protocol; The second translated audio data is received from the mobile terminal via the GATT data protocol. The second translated audio data is obtained by translating the second audio data to be translated. The second translated audio data is decoded and played.

4. The simultaneous interpretation method according to claim 3, characterized in that, The first or second translated audio data contains a routing identifier, and the simultaneous interpretation method further includes: Parse the routing identifier of the received translated audio data; If it is determined that the routing identifier indicates the target as the secondary earphone, then the translated audio data is determined to be the first translated audio data; If the routing identifier indicates that the target is the main earphone, then the translated audio data is determined to be the second translated audio data.

5. The simultaneous interpretation method according to claim 1, characterized in that, Prior to responding to the simultaneous transmission mode activation command from the mobile terminal, the method further includes: When a Bluetooth connection is established between any one of the earphones and the mobile terminal, that earphone is designated as the master earphone. A role configuration command is sent to the other earpiece via a communication link to configure the other earpiece as the secondary earpiece; Establish a private link with the secondary earphone.

6. The simultaneous interpretation method according to claim 2, characterized in that, The step of sending the first audio data to be translated, collected by the main earphone, to the mobile terminal via the GATT data protocol includes: The first audio data to be translated is compressed and encoded using the Opus encoder to generate a first audio data packet, and the first audio data packet is sent to the mobile terminal through the GATT data protocol. Accordingly, after receiving the first translated audio data returned by the mobile terminal via the GATT data protocol, the method further includes: using the Opus decoder to decode the first translated audio data.

7. The simultaneous interpretation method according to claim 3, characterized in that, The step of sending the second audio data to be translated to the mobile terminal via the GATT data protocol includes: The second audio data to be translated is compressed and encoded using the Opus encoder to generate a second audio data packet, and the second audio data packet is sent to the mobile terminal through the GATT data protocol. Accordingly, after receiving the second translated audio data returned by the mobile terminal via the GATT data protocol, the method further includes: using the Opus decoder to decode the second translated audio data.

8. The simultaneous interpretation method according to claim 3, characterized in that, The first audio data to be translated was obtained based on the following steps: The first vibration signal is acquired using the bone conduction microphone of the main earphone; The first sound wave signal is acquired using the air conduction microphone array of the main earphone; The first vibration signal and the first sound wave signal are weighted, fused, and denoised to generate the first audio data to be translated. Accordingly, the second audio data to be translated is obtained based on the following steps: The second vibration signal is acquired using the bone conduction microphone of the secondary earphone; The second sound wave signal is acquired using the air conduction microphone array of the secondary earphone; The second vibration signal and the second sound wave signal are weighted, fused, and denoised to generate the second audio data to be translated.

9. The simultaneous interpretation method according to claim 1, characterized in that, Also includes: In response to the simultaneous transmission mode shutdown command from the mobile terminal, a protocol recovery request is sent to the mobile terminal; The protocol recovery request is used to request the reactivation of the Bluetooth audio protocol and to stop data interaction based on the GATT data channel.

10. The simultaneous interpretation method according to claim 1, characterized in that, The establishment and maintenance of a Bluetooth connection with the mobile terminal, and the operation of the Bluetooth audio protocol on the Bluetooth connection, include: Establish and maintain a Bluetooth connection with the mobile terminal based on Basic Rate or Enhanced Data Rate (BR / EDR) technology; The Advanced Audio Distribution Protocol (A2DP) and the Hands-Free Protocol (HFP) run on the Bluetooth connection.

11. A simultaneous interpretation method for dialogue, applied to a mobile terminal, characterized in that, include: Establish and maintain a Bluetooth connection with the main earphone, and run the Bluetooth audio protocol on the Bluetooth connection; In response to the simultaneous interpretation mode activation operation, a simultaneous interpretation mode activation command is sent to the main earphone, and the Bluetooth audio protocol is suspended in consultation with the main earphone, and the GATT data protocol based on the Bluetooth connection is activated. The GATT data protocol is used to perform simultaneous interpretation data interaction with the main headset.

12. A Bluetooth headset, characterized in that, include: Including the first and second earphones; The first and second earpieces are configured such that either earpiece that establishes a connection with the mobile terminal is designated as the primary earpiece, and the other earpiece is designated as the secondary earpiece. The main earpiece includes a memory and a processor, which executes a computer program to implement the simultaneous interpretation method as described in any one of claims 1-10.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the simultaneous interpretation method as described in any one of claims 1 to 10 or the simultaneous interpretation method as described in claim 11.

Images