Method and system for face-to-face simultaneous interpretation and Bluetooth earphone

Abstract

The invention provides a face-to-face simultaneous interpretation method, a face-to-face simultaneous interpretation system and a Bluetooth earphone, the face-to-face simultaneous interpretation system comprises a mobile terminal and a Bluetooth earphone pair, and the Bluetooth earphone pair comprises a first Bluetooth earphone and a second Bluetooth earphone. In the translation mode, the mobile terminal only maintains the existing Bluetooth data connection with the first earphone, and the second earphone does not directly interact with the mobile terminal, but sends the uplink voice to the first earphone through inter-ear communication. The first earphone converges and encapsulates the two paths of voice to form an uplink composite frame and sends the uplink composite frame to the mobile terminal; the composite frame comprises a session ID, a source identifier, a serial number, a timestamp and an acoustic state mark, and the terminal splits the voice stream according to the composite frame and executes recognition, error correction and translation to generate two translations. And the terminal establishes a routing binding table and issues a downlink data packet containing two translations to the first earphone, the first earphone filters and plays the translations according to the binding relation and forwards the translations belonging to the second earphone to the second earphone through the inter-ear link, and bidirectional directional playing and self-listening filtering are realized.

Description

Technical Field

[0001] This invention relates to the fields of speech translation and Bluetooth audio communication technology, specifically to a method, system, and Bluetooth headset for face-to-face simultaneous interpretation. Background Technology

[0002] Face-to-face simultaneous interpretation is a fusion application of voice interaction and machine translation. It typically involves microphones capturing speech, speech recognition converting speech to text, machine translation generating target language text, and then outputting it to the listener via speech synthesis or text playback. With the improvement of mobile terminal computing power, enhanced edge-cloud collaboration capabilities, and iterations in Bluetooth audio technology, the product form has gradually evolved from early standalone translation devices to a collaborative model of "phone / tablet + headphones," and is further developing towards lightweight, low-latency, and private playback in true wireless Bluetooth headphones (TWS). The demand for such products continues to grow in scenarios such as cross-border communication, business meetings, and travel inquiries. However, face-to-face real-time dialogue is characterized by two-way alternation, close distance, strong interference, and latency sensitivity, placing higher demands on link organization, audio routing, echo control, and session management.

[0003] In existing technologies, the use of headphones in conjunction with electronic devices for translation is quite common. For example, the solution shown in patent document CN110602675A is based on establishing a Bluetooth link between a first headphone and an electronic device. The first headphone collects the first user's voice and uploads it to the electronic device to complete the translation. The headphone then receives and plays the translated signal, while a second headphone is allowed to participate in the translation process by listening or uploading. This type of solution can reduce the frequency of handheld operation by the user. However, in face-to-face simultaneous interpretation scenarios, the two users often need to hear each other's translation separately. Without a binding mechanism between the uplink voice source and the downlink translation, it is easy for the two headphones to receive the same translation, or for the user to hear their own translation, leading to comprehension interference. At the same time, when voice, music, and prompts coexist, the system needs to switch between different bearers such as SCO / ACL / BLE. Link reconstruction and resource contention can cause latency jitter or even short-term interruptions, affecting the continuous dialogue experience.

[0004] To achieve two-way simultaneous interpretation, some solutions employ a structure where the terminal establishes separate voice links with the main and secondary earpieces. For example, publication CN110381485B proposes establishing a first SCO link between the terminal and the main earpiece, and a second SCO link with the secondary earpiece in a simultaneous interpretation scenario. Both SCO links support bidirectional voice data transmission. The terminal translates the voice from each link and transmits it back to the corresponding earpiece for playback, logically forming two independent translation paths. This structure can clearly distinguish between the two voice sources. However, in the actual terminal ecosystem, differences in the number of concurrent SCOs, HFP resource usage, and system Bluetooth stack strategies may lead to failure in establishing concurrent voice links or preemption by the system during scene switching. Furthermore, SCOs have limited bandwidth and encoding formats, and parallel multi-link operation further compresses the effective load and anti-interference margin. Coupled with multiple reconnections and clock drift, this can easily lead to problems such as increased translation latency, lost voice segments, or mismatched playback.

[0005] Other solutions expand application scenarios from the perspective of "separate connection" between the left and right earphones. For example, CN113411712A proposes establishing separate communication channels between the left and right earphones and the terminal based on the usage status of the left and right earphones and the terminal device information. It also proposes using a backend server to virtualize the left and right earphones as different accounts, allowing different audio to be sent to different account earphones, thus achieving dual-ear collaboration. This approach helps achieve differentiated audio distribution, but its implementation often relies on the terminal having dual Bluetooth modules or requires modifications to the backend account system, resulting in high adaptation boundaries and deployment costs. Furthermore, this type of solution focuses on "distribution capabilities" and lacks sufficient coverage of closed-loop mechanisms necessary for face-to-face simultaneous interpretation, such as bidirectional parallel voice acquisition, uplink timing alignment, targeted playback of translated text based on the recipient's identity, and echo loop management. This makes it difficult to guarantee end-to-end real-time performance and consistency in complex on-site environments.

[0006] Some technologies also extend translation capabilities to the headset side. For example, CN111696552B discloses an implementation method where the headset acquires source speech and translates it into target speech, allowing users to complete translations without relying on external devices under certain conditions. While end-device integration helps reduce dependence on networks and external computing power, face-to-face simultaneous interpretation typically requires parallel input and translation output from two wearers, and needs to handle issues such as double-talking, accent differences, and near-field noise. If each headset independently performs recognition and translation, it can easily lead to computational and power consumption pressures, difficulty in ensuring model consistency, and risks such as missing session synchronization and routing strategies. Furthermore, the lack of session-level state transitions and responsibility handover can amplify the probability of frame drops, out-of-order playback, and repeated broadcasts in the event of master-slave switching, removal of the headset, or obstruction causing link rerouting.

[0007] Regarding recognition accuracy, existing technologies have also proposed speech recognition error correction mechanisms to improve translation quality. For example, CN113779972B obtains the pronunciation sequence to be detected, constructs the FST to be detected, and combines the keyword FST with the Chinese character confusion set to correct the recognized text. It uses logical score comparison to determine the replacement, thereby improving the error correction effect of the recognized text. This type of method can alleviate some recognition errors caused by similar pronunciations, but it mainly works at the text level after recognition. Face-to-face simultaneous interpretation is also affected by the combined effects of link jitter, double-person superimposed speech, and playback echo: on the one hand, the translation played by the headphones may be picked up again by the microphone of the local machine or the other party, forming echo and crosstalk, reducing the signal-to-noise ratio of the recognition front end, so that the error cannot be corrected by simply "replacing the confusion set"; on the other hand, if the timestamp alignment, packet reordering, and packet loss compensation of the dual-path uplink speech are not handled properly, the input sentences in the error correction and translation stages will be truncated or spliced ​​across sentences, thereby causing semantic deviation of the translation.

[0008] In summary, existing technologies typically address the aforementioned issues through several approaches: First, employing a collaborative translation process between headphones and electronic devices, utilizing Bluetooth links to upload voice and transmit translated signals back, and providing options such as monitoring, uploading, or multi-bearer switching to improve usability; Second, maintaining independent voice links with both the main and secondary headphones simultaneously on the terminal, enabling differentiation of the source of bidirectional voice and separate transmission of translated text; Third, using separate connections for the left and right headphones and backend account mapping to allow different audio to be distributed to different headphones; Fourth, improving the user experience from the perspectives of device form factor and text quality through enhanced on-device translation or error correction algorithms. While these approaches improve usability to some extent, there is still room for improvement in areas such as "maximizing the reuse of existing link structures to achieve dual uplink convergence, establishing stable routing binding on the downlink side of the translated text, maintaining session continuity during master-slave / link changes, and suppressing near-field echo and dual-talk interference." Summary of the Invention

[0009] To address the aforementioned technical issues, this invention provides a method, system, and Bluetooth headset for face-to-face simultaneous interpretation. The aim is to achieve parallel acquisition and aggregation of dual-channel audio without significantly altering the existing Bluetooth connection configuration, and to deliver the translated text to the corresponding headset after establishing a stable binding relationship between the translated text and the audio source. This ensures continuity and consistency in interpretation even during scene switching, link jitter, and changes in master-slave roles.

[0010] A face-to-face simultaneous interpretation method, applied to a terminal and a pair of wireless earpieces, the wireless earpieces including a first earpiece and a second earpiece, comprising:

[0011] In translation mode, the existing Bluetooth data connection between the first earphone and the terminal remains active, and a physical link between the first earphone and the second earphone is established.

[0012] An application layer session protocol runs on the physical link between the ears. The protocol frame of the session protocol includes a frame header, payload, and checksum. The frame header includes at least a type field, a source identifier, a sequence number, and a timestamp. The type field is used to distinguish between translated control signaling and translated audio data.

[0013] The first earphone and the second earphone respectively collect the voice signals of their respective wearers and perform front-end voice processing to obtain the processed voice signals;

[0014] The first and second earpieces respectively perform local audio encoding on the processed voice signal frame by frame with a fixed frame length to obtain uplink audio data frames. The first earpiece does not decode or re-encode the uplink audio data frames received from the second earpiece, but only performs aggregation encapsulation and relay forwarding, and encapsulates the source identifier of the corresponding earpiece in the uplink audio data frames. The second earpiece sends the uplink audio data frames carrying the source identifier of the second earpiece to the first earpiece through the session protocol. The first earpiece aggregates its own uplink audio data frames and the uplink audio data frames received from the second earpiece according to a predefined multiplexing encapsulation format to form a composite data packet. The composite data packet includes at least the source identifier, sequence number, timestamp, and length fields of each of the two audio sub-packets, and uploads it to the terminal through the general attribute protocol data channel of the Bluetooth data connection.

[0015] The terminal sends the composite data packet to the translation processing module, performs speech recognition and translation on the two uplink audio streams to generate two downlink translated audio data frames, and carries a routing identifier bound to the uplink source identifier in the downlink translated audio data frames; the terminal sends a downlink data packet containing the two downlink translated audio data frames to the first earpiece; after the first earpiece decodes the downlink data packet to obtain the two downlink translated audio data frames, it filters according to its own logical role, outputs and plays the downlink translated audio data frame bound to the second earpiece source identifier, and forwards the downlink translated audio data frame bound to the first earpiece source identifier to the second earpiece on the ear-to-ear physical link; the second earpiece decodes the received downlink translated audio data frame and plays it.

[0016] Furthermore, the type field of the session protocol is used to distinguish different types of signaling and data, including at least:

[0017] Session start and end signaling: Used to identify the start and end of a session, ensuring the start and stop of the translation process;

[0018] Session synchronization signaling: used to synchronize data or status between two headsets to ensure timing consistency during translation;

[0019] Role Switching Request Signaling: Used to switch the master / slave role between two headsets, ensuring that the system can flexibly adjust the device role according to needs;

[0020] Synchronization anchor signaling: used to synchronize the position of the audio stream during translation, ensuring that there are no desynchronization issues when playing audio through dual headphones;

[0021] Uplink audio data: Audio data from the user, uploaded to the system for translation via the first earpiece;

[0022] Downlink translated audio data: The translated audio data is transmitted to the headphones via a session protocol for the user to play.

[0023] Furthermore, the translation control signaling has a higher transmission priority than the translation audio data in the session protocol, and adopts a preemptive transmission mechanism to ensure that the preemption delay satisfies ∈ [1, 20] milliseconds. The preemptive transmission mechanism uses a priority queue or signal interruption mechanism. During data transmission, when the control signaling arrives, the system will interrupt the transmission of the current audio data and send the control signaling first. After the signaling is sent, the audio data transmission will continue.

[0024] Furthermore, at the start of the translation session, the master earpiece synchronizes the recording start point identifier, encoding parameter set, and timing reference identifier to the slave earpiece via session synchronization signaling; wherein, the encoding parameter set includes at least the sampling rate, encoding format, and bit rate; the master earpiece and slave earpiece attach a unified timestamp to the audio data during audio acquisition according to the session synchronization signaling; in the downlink phase, playback alignment is performed based on the sequence number and timestamp of the translated audio data frame to ensure that the binaural translation playback synchronization error is no greater than the error threshold, where ∈ [5,50] milliseconds;

[0025] In this system, both the master and slave earphones use fixed-length audio frames as the smallest processing unit. Each audio frame carries a frame sequence number and a sampling timestamp or logical clock count value. The master earphone synchronizes the audio frames from both earphones based on the frame sequence number, and when a frame sequence deviation is detected between the two audio frames, it performs synchronization adjustment through buffer alignment and / or frame-level delay compensation. The master earphone periodically sends synchronization control frames to the slave earphone, which carry a reference frame sequence number and a reference count value. The slave earphone performs audio sampling count compensation and / or frame generation beat adjustment based on the deviation between its local count value and the reference count value to achieve progressive alignment of the logical clocks of the master and slave earphones.

[0026] Furthermore, the audio data frames of the session protocol carry forward error correction redundancy information and checksums; the receiving end performs frame reordering based on the sequence number and timestamp of the audio data frames. The frame reordering includes: establishing a reordering window buffer for out-of-order detection and buffer alignment, deduplicating and outputting audio data frames that fall into the reordering window buffer in order, and discarding audio data frames that exceed the reordering window buffer and / or setting missing tags for missing audio data frames; the first earphone performs buffering and reordering on the uplink audio data frames from the second earphone before forwarding them to the terminal, so as to restore the out-of-order transmission process from the second earphone to the first earphone and from the first earphone to the terminal. The end-to-end target delay corresponding to the buffer satisfies ∈ [80, 250] milliseconds; when a master-slave role switch occurs, the new master earphone inherits the frame order state, synchronization information and reordering window buffer context of the original master earphone, and continues to perform the buffering and reordering after inheritance.

[0027] Furthermore, the multiplexing encapsulation format of the composite data packet includes a composite packet header and at least two sub-packet segments. The composite packet header includes at least a composite packet sequence number and a composite packet timestamp. Each sub-packet segment includes at least a source identifier, a sub-packet sequence number, a sub-packet timestamp, an encoding format identifier, and a length field.

[0028] Furthermore, the downlink translated audio data frame carries its bound uplink source identifier; the first earpiece only outputs and plays the translated audio data frame whose source identifier points to the second earpiece, and the second earpiece only outputs and plays the translated audio data frame whose source identifier points to the first earpiece; unmatched translated audio data frames are discarded.

[0029] Furthermore, the master-slave role switching process is executed when any of the following trigger conditions are met: the remaining battery power of the master headset is lower than the threshold, where ∈ [0.10, 0.30]; or the difference between the master and slave battery power is greater than the threshold, where ∈ [0.03, 0.08], and the translation application is in the foreground active state; or the received signal strength of the master headset and the terminal is lower than the threshold for a duration of [2, 20] seconds and ∈ [-85, -60] dB / mW; or the silence duration of one party is detected to exceed the threshold, where ∈ [3, 30] seconds.

[0030] Furthermore, when a master-slave role switch is triggered, the current master earphone sends a role switch request signaling carrying a synchronization anchor point through the session protocol. The synchronization anchor point includes at least the sequence number anchor point of the current uplink audio data frame and the current downlink translation playback progress anchor point. The role switch request signaling is configured as a control signaling higher than the audio data signaling and supports preemptive transmission. Before sending the role switch request signaling, the sending end selects to reduce the modulation and coding order or adopt a robust transmission method based on link quality parameters. After sending the role switch request signaling, the sending end opens an acknowledgment listening window. If no acknowledgment information is received from the receiving end within a preset time slot, non-backoff fast retransmission is performed. Additionally, the sending end avoids frequency hopping channels with high bit error rates based on the channel quality mapping table, scheduling the role switch request signaling to a channel that meets the quality threshold for transmission.

[0031] Furthermore, the master-slave role switching adopts a non-disconnect migration process, including a parallel connection overlap period. During the parallel connection overlap period, the original master earphone maintains a first general attribute protocol connection with the terminal, while the new master earphone establishes a second general attribute protocol connection with the terminal. During the overlap period, the terminal switches its uplink and downlink transmission channels to the second general attribute protocol connection and then releases the first general attribute protocol connection. Before or during the master-slave role switching, the communication context information of the original master earphone is continuously synchronized from the earphone backend. The communication context information includes at least the audio stream sequence number, link encryption offset, Bluetooth clock phase, and decoding status. During the parallel connection overlap period, the new master earphone simultaneously receives downlink data packets from the terminal and residual downlink data packets forwarded by the original master earphone. The data packets from both sources are matched and deduplicated based on timestamps to achieve continuous takeover of downlink data.

[0032] Furthermore, during the parallel connection overlap period, the terminal simultaneously sends the same downlink data packets to both the original master headset and the new master headset; the original master headset and the new master headset align the downlink translation playback progress according to the synchronization anchor point and complete the playback responsibility switch; wherein, the receiving end maintains a jitter buffer of preset depth, and during the link switching transition period, it prioritizes outputting the cached pre-read data from the jitter buffer for playback; and, when jitter is detected in the audio samples or timestamps before and after the switch, fade-in and fade-out processing is performed on the audio samples before and after the switch to reduce transient noise caused by sample discontinuity.

[0033] Furthermore, the front-end speech processing includes short-range directional noise reduction processing, which includes: performing beamforming based on dual-microphone signals to form a main lobe and a null in the interference direction; and applying gain suppression or null suppression to the sound source in the interference direction based on directional gating and distance gating, wherein the directional gating corresponds to the azimuth angle interval. ∈[0°,30°], Distance gate threshold satisfy ∈[0.20,0.40] meters.

[0034] Furthermore, the short-range directional noise reduction processing is performed according to frequency band differences, with the upper frequency limit for the low-frequency band. satisfy ∈[300,700] Hz, conservative gain suppression and superimposed spectral shaping are performed in the low-frequency band; the lower limit frequency in the mid-to-high frequency band... With upper limit frequency satisfy ∈[700,1300] Hertz and ∈[3000,5000] Hz, null enhancement and Wiener-type suppression are performed in the mid-to-high frequency band; and the duration is triggered when a sudden far-field high-energy noise event is detected. A short-time deep suppression mode ∈ [50,500] milliseconds, in which instantaneous gain reduction and limiting are performed.

[0035] Furthermore, the front-end speech processing also includes mode switching and state management. When switching between directional suppression mode and normal noise reduction mode, the mode switching performs fade-in and fade-out operations on the beam pointing, null depth, and adaptive update step size of frequency domain adaptive noise reduction and echo cancellation. It also saves and maintains two sets of filter parameters and target curves corresponding to directional suppression mode and normal noise reduction mode, and reuses noise statistics and secondary path estimation results during switching to shorten the convergence time. During translation playback, echo cancellation and playback leakage estimation driven by playback reference signal are enabled, frequency domain nulls and gain compression are applied to the self-return of the playback segment, and state interlock control is performed when a state of strong directional suppression and playback coexistence is detected, including increasing the echo cancellation reference weight, reducing the far-end return gain, and enabling a strong dual-talk detection strategy. When a user is detected speaking, the playback volume is automatically reduced or a short-term attenuation is performed to ensure uplink speech priority and suppress echo residue.

[0036] Furthermore, the translation processing module performs error correction processing after obtaining the recognized text from speech recognition and before generating the translation. This error correction processing includes rule-based error correction and neural network error correction. The rule-based error correction includes: segmenting the recognized text to obtain a word sequence; converting the word sequence into a pinyin sequence aligned with the word sequence based on a dictionary, while retaining non-Chinese characters as anchor points; calling a pre-compiled finite-state converter on the pinyin sequence to perform pinyin-Chinese rule matching and replacement, and applying the longest match priority for replacement rules with inclusion relationships; and performing dynamic rule post-processing after the finite-state converter output, which performs longest string priority matching on the pinyin sequence and maps the pinyin offset back to the word sequence index to complete forced replacement. The neural network error correction includes: obtaining a query vector from the corresponding audio segment or its acoustic feature input parameter-shared acoustic dual-tower model, and retrieving a candidate entity set from a pre-built entity acoustic vector index library, with a candidate number... satisfy [3,10]; The candidate entity and the recognized text are concatenated to construct a semantic model input sequence for replacement localization and semantic discrimination. The semantic model outputs the start and end positions of the segment to be replaced and the semantic confidence score; and a weighted fusion score is obtained based on the acoustic similarity score and the semantic confidence score. When the fusion score exceeds a threshold... The corresponding segments of the identified text are replaced with candidate entities at the specified start and end positions, and the threshold value is... ∈[0.65,0.85].

[0037] Furthermore, a face-to-face simultaneous interpretation system includes a terminal, an earpiece pair, and a translation processing module. The earpiece pair includes a first earpiece and a second earpiece, with a physical link established between them. The earpiece pair runs an application-layer session protocol on the physical link, and the frame header of the session protocol includes at least a type field, a source identifier, a sequence number, and a timestamp. The first earpiece includes a connection maintenance module, an uplink aggregation relay module, and a multiplexing encapsulation module, used to maintain the existing Bluetooth data connection with the terminal, receive uplink audio data frames from the second earpiece, and aggregate them with the uplink audio data frames from the first earpiece itself to form a composite data packet. The data is uploaded via a general attribute protocol data channel; the second earpiece includes a voice acquisition and ear-to-ear transmission module, used to generate uplink audio data frames carrying the source identifier of the second earpiece and send them to the first earpiece via the ear-to-ear physical link; the translation processing module is used to perform speech recognition and translation on the two uplink audio streams and generate two downlink translated audio data frames, and the downlink translated audio data frames carry routing identifiers bound to the uplink source identifiers; the first earpiece and the second earpiece each include a downlink decoding and filtering playback module, used to decode the downlink data packets and filter them according to logical roles and only play downlink translated audio data frames bound to the other party's source identifier.

[0038] Furthermore, the session protocol configuration control signaling has a higher priority than audio data and supports preemptive transmission with preemption delay. The system includes a session synchronization mechanism for synchronizing the recording start point identifier, encoding parameter set, and timing reference identifier. The system also includes a role management module and a status synchronization module. The role management module initiates role switching when at least one of the following conditions is met: power threshold, power difference threshold, received signal strength duration threshold, or silent window threshold. The status synchronization module synchronizes the uplink sequence number anchor point and the downlink playback progress anchor point. The terminal includes a routing table update module for completing the uplink and downlink channel switching and releasing the old connection during the parallel connection overlap period.

[0039] Furthermore, the translation processing module includes an error correction submodule, which comprises a rule-based error correction unit and a neural network error correction unit. The rule-based error correction unit includes an offline model generation component and an online inference component. The offline model generation component constructs a finite-state converter from the set of pinyin-Chinese character replacement rules and performs deterministic, minimization, and context rewriting compilation to generate a binary error correction model file. The online inference component loads the binary error correction model file and constructs a pinyin sequence aligned with the word sequence obtained from word segmentation. It then performs streaming matching and replacement on the pinyin sequence and supports dynamic longest-matching rules and mapping replacement from pinyin offset to word index. The neural network error correction... The unit includes an offline database construction module and an online cascaded model module. The offline database construction module generates standard speech from the entity vocabulary and inputs it into a key-value tower network to obtain entity acoustic vectors. It then stores the entity acoustic vectors in a vector index library and constructs a nearest neighbor retrieval index. The online cascaded model module includes a parameter-sharing acoustic dual-tower recall model and a semantic localization and discrimination model. The acoustic dual-tower recall model generates query vectors from the input audio segment and retrieves a set of candidate entities from the vector index library. The semantic localization and discrimination model outputs the replacement start position, replacement end position, and semantic confidence score. Based on the weighted fusion result of the acoustic similarity score and the semantic confidence score, and a threshold condition, it controls whether to perform the replacement.

[0040] Furthermore, the device includes a first earpiece and a second earpiece. The first earpiece includes a Bluetooth communication module, a first microphone array, a speaker, a processor, and a memory. The second earpiece includes an inter-ear communication module, a second microphone array, a speaker, a processor, and a memory. The first and second earpieces run an application layer session protocol on the inter-ear physical link. The frame header of the session protocol includes at least a type field, a source identifier, a sequence number, and a timestamp. The processor of the first earpiece is used to multiplex and encapsulate two uplink audio data frames into a composite data packet and upload it to the terminal through a general attribute protocol data channel. The processors of the first and second earpieces are respectively used to decode downlink data packets and filter them according to logical roles, outputting only the downlink translated audio data frames bound to the source identifier of the other party and playing them. The first and second earpieces each include a front-end voice processing module, which includes at least short-range directional noise reduction, mode switching and state management, and echo correction for translated playback.

[0041] Furthermore, a computer-readable storage medium stores a computer program thereon, which, when executed by a processor, is used to implement the steps of the face-to-face simultaneous interpretation method.

[0042] The face-to-face simultaneous interpretation method, system, and Bluetooth headset provided by this invention have the following advantages: In translation mode, the terminal maintains the existing Bluetooth data connection only with the first headset. The second headset does not directly establish a translation data channel with the terminal. Instead, it sends its uplink voice to the first headset via a physical link between the ears and based on an application layer session protocol. The first headset then aggregates and encapsulates the dual-path uplink voice to form a composite data packet, which is then uploaded to the terminal. This avoids the Bluetooth stack differences, concurrency limitations, and bandwidth contention issues caused by the terminal concurrently maintaining two voice links, improving cross-device compatibility and connection stability. It is particularly suitable for face-to-face two-way simultaneous interpretation environments where "both parties take turns speaking, the distance is close, and the conversation is fast-paced."

[0043] This invention introduces fields such as session identifier, source identifier, sequence number, and timestamp into composite data packets / protocol frames, enabling the terminal to reliably split two voice streams and perform out-of-order reordering, synchronization alignment, and translation processing. The terminal establishes a routing binding table and carries the routing identifier corresponding to the source identifier in the downlink translation frame. The two earpieces perform self-listening filtering based on logical roles, only playing the translation corresponding to the "other party's source," thus achieving "whoever speaks, the translation is directed to the corresponding earpiece." This significantly reduces the probability of translation crosstalk, mis-sending, hearing one's own translation, and mismatched playback in face-to-face communication, improving the naturalness and comprehensibility of the dialogue. Simultaneously, the receiver's frame reordering and reordering window buffer, based on the sequence number and timestamp, achieves out-of-order detection, deduplication, and in-order output, and discards out-of-window frames and / or sets missing markers for missing frames, ensuring continuous input and stable recognition of uplink voice even when short sentences frequently switch or the speech rate varies significantly.

[0044] To ensure uninterrupted connection even during momentary link fading caused by head rotation, gesture obstruction, or human coupling in face-to-face scenarios, this invention sends a role switch request signaling carrying synchronization anchors when a master-slave role switch is triggered. These synchronization anchors include at least the sequence number anchor of the uplink audio data frame and the downlink translation playback progress anchor, enabling the terminal and the new master earpiece to quickly align based on these anchors. Furthermore, a parallel connection overlap period mechanism without link migration is employed to achieve smooth transfer of relay responsibilities between the first and second earpieces. Even under triggering conditions such as battery level, signal strength, wearing / removing, or reconnection, the session identifier, uplink sequence, and downlink playback progress remain continuous, reducing the risk of sentence breaks, repeated playback, and translation interruptions caused by the switch. During the switchover transition period, context synchronization, dual-channel reception, and timestamp matching deduplication ensure that translation playback is "non-overlapping and non-gap" at the handover moment. The receiving end maintains a jitter buffer of preset depth to mask the handshake and switching delays with buffer delays, and performs fade-in and fade-out smoothing when jitter of samples or timestamps before and after switching is detected, thereby reducing transient noise and achieving a smooth transition in sound, meeting the requirements of face-to-face communication for "natural sound, no interruption, and no popping".

[0045] Furthermore, echo suppression / gating, dual-talk detection, and playback attenuation strategies on the earphone side can reduce the pollution of uplink acquisition caused by translated text playback. Combined with optional forward error correction and cache reordering mechanisms, these enhance resistance to packet loss and out-of-order delivery, improving the quality of recognition and translation input from the source and stabilizing end-to-end latency. On the terminal side, after recognition and before translation, a collaborative process of rule-based error correction and neural network error correction is introduced to constrain and correct errors caused by proper nouns, homophone confusion, and accent differences, making the accuracy and consistency of the translation more stable. In summary, this invention, with "the first earphone interacting with the terminal, and the second earphone relaying via an inter-ear link (or taking over the interaction with the terminal after switching)" as its core, combines features such as session protocols, composite frame encapsulation, route binding and filtering, seamless switching, and acoustic management to achieve low latency, uninterrupted operation, and high intelligibility in face-to-face two-way simultaneous interpretation without increasing the user's operational burden. Attached Figure Description

[0046] Appendix Figure 1 A schematic diagram illustrating a face-to-face simultaneous interpretation application scenario;

[0047] Appendix Figure 2 This is a block diagram of the overall system structure.

[0048] Appendix Figure 3 This is a structural block diagram of a Bluetooth headset.

[0049] Appendix Figure 4 A flowchart illustrating the methods of face-to-face simultaneous interpretation;

[0050] Appendix Figure 5 A schematic diagram of the data format encapsulated for uplink voice aggregation;

[0051] Appendix Figure 6 This is a diagram illustrating translation routing binding and targeted delivery.

[0052] Appendix Figure 7 This is a schematic diagram illustrating the state of link / role switching;

[0053] Appendix Figure 8 This is a schematic diagram of near-field echo and dual-talk suppression;

[0054] Appendix Figure 9 This is a schematic diagram of the collaborative processing flow for speech recognition error correction.

[0055] Figure Labels: Simultaneous Interpretation System - 100 Mobile Terminal - 200 Bluetooth Headset Pair - 300 First Bluetooth Headset - 310 Second Bluetooth Headset - 320 Inter-ear Communication Link - 330 First User - 400 Second User - 500 Voice Acquisition / Reception Module - 210 Uplink Aggregation Encapsulation Module - 220 Voice Recognition Module - 230 Translation Module - 240 Error Correction Module - 250 Downlink Routing Binding Module - 260 Playback Control Module - 270 Session Management Module - 280 Link Management Module - 290 First Headset Microphone - 311 First earphone speaker - 312 First earphone Bluetooth transceiver module - 313 First earphone audio codec unit - 314 First earphone echo suppression / gating unit - 315 First earphone buffer / sequence control unit - 316 First earphone control unit - 317 Second earphone microphone - 321 Second earphone speaker - 322 Second earphone Bluetooth transceiver module - 323 Second earphone audio codec unit - 324 Second earphone echo suppression / gating unit - 325 Second earphone buffer / sequence control unit - 326 Second earphone control unit - 327 Detailed Implementation

[0056] The technical solution of this embodiment will now be clearly and completely described with reference to the accompanying drawings. In the description of this embodiment, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings and are only for the convenience of describing this embodiment and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this embodiment. The terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying their relative importance.

[0057] In the following embodiments, the face-to-face simultaneous interpretation method, system, and Bluetooth headset proposed by the present invention will be described in detail with reference to the accompanying drawings, enabling those skilled in the art to implement them without inventive effort. It should be understood that the following description is merely an exemplary embodiment and does not constitute a limitation on the scope of protection of the present invention.

[0058] This embodiment focuses on the needs of two-way simultaneous interpretation under face-to-face communication conditions. For example... Figure 1As shown, the simultaneous interpretation system 100 is composed of a mobile terminal 200 and a Bluetooth headset pair 300. The Bluetooth headset pair 300 includes a first Bluetooth headset 310 and a second Bluetooth headset 320. In the default "relay uplink / directed downlink" working mode, the mobile terminal 200 only establishes and maintains an existing Bluetooth data connection (e.g., BLEGATT or ACL data channel) with the first Bluetooth headset 310. The second Bluetooth headset 320 does not directly establish a translation data channel with the mobile terminal 200, but exchanges session control and audio data with the first Bluetooth headset 310 through an inter-ear communication link 330. The first Bluetooth headset 310, acting as a relay aggregation node, receives the uplink voice from both users and uploads it to the terminal. After the terminal 200 completes recognition and translation, it establishes a routing binding according to the voice source and directs the translation downlink, so that the first user 400 mainly hears the translation from the second user 500, and the second user 500 mainly hears the translation from the first user 400. When a master-slave switch or link switch trigger condition occurs (such as changes in battery level, signal strength, or wearing status), the second Bluetooth headset 320 can be optionally switched to the new first headset, which establishes a new existing Bluetooth data connection with the mobile terminal 200 and takes over the relay function, so as to realize the interaction between the second headset and the terminal after the link switch.

[0059] like Figure 2 As shown, the mobile terminal 200 includes a voice acquisition and receiving module 210, an uplink composite frame parsing / convergence management module 220, a voice recognition module 230, a translation module 240, an error correction module 250, a downlink routing binding module 260, a playback control module 270, a session management module 280, and a link management module 290 on the system side. Specifically, the voice acquisition and receiving module 210 receives composite uplink data packets through a single uplink entry point to the first Bluetooth headset 310 and splits the composite data packets into two logical voice streams based on the source identifier; the downlink routing binding module 260 maintains a routing binding table based on the source identifier and the headset's logical role, determining the target headset for translation delivery and the filtering strategy; the link management module 290 only needs to maintain one existing Bluetooth data connection between "terminal ↔ first headset" and works with the session management module 280 to complete connection migration and session state migration during master-slave switching.

[0060] like Figure 3As shown, on the earpiece side, the first Bluetooth earpiece 310 assumes the primary connection responsibility for interacting with the mobile terminal 200 and acts as a relay node to aggregate uplink voice data from the second Bluetooth earpiece 320. In default mode, the second Bluetooth earpiece 320 does not directly interact with the mobile terminal 200, but instead runs an application-layer session protocol with the first Bluetooth earpiece 310 on the inter-ear communication link 330 to transmit control signaling, uplink audio data frames, and necessary synchronization information. To accommodate link switching scenarios, the second Bluetooth earpiece 320 also has the ability to establish an existing Bluetooth data connection with the mobile terminal 200, but this connection is only enabled when it becomes the new first earpiece after a link switch. The first Bluetooth headset 310 internally includes a first headset microphone 311, a first headset speaker 312, a first headset Bluetooth transceiver module 313, a first headset audio codec unit 314, a first headset echo suppression / gating unit 315, a first headset buffer / sequence control unit 316, and a first headset control unit 317; the second Bluetooth headset 320 internally includes a second headset microphone 321, a second headset speaker 322, a second headset Bluetooth transceiver module 323, a second headset audio codec unit 324, a second headset echo suppression / gating unit 325, a second headset buffer / sequence control unit 326, and a second headset control unit 327.

[0061] In the specific session execution process, the method flow starts with the session establishment step S101: the mobile terminal 200 sends a control signal to the first Bluetooth headset 310 to enter the translation mode through the session management module 280. The first Bluetooth headset 310 then synchronizes the same session entry information to the second Bluetooth headset 320 through the inter-ear communication link 330. The synchronization includes at least a recording start point identifier, an encoding parameter set, and a timing reference identifier, and instructs the second Bluetooth headset 320 to start acquisition and encoding from the frame boundary corresponding to the recording start point identifier. After the synchronization is completed, the first Bluetooth headset 310 sends a session start instruction to the second Bluetooth headset 320, so that the two headsets enter the acquisition and encoding state at the same frame boundary. Subsequently, in step S102, the first Bluetooth headset 310 and the second Bluetooth headset 320 establish an application layer session protocol on the inter-ear link. The protocol frame header of the session protocol includes at least a type field, a source identifier, a sequence number, and a timestamp, and may include a logical clock count value field. The sequence number is a monotonically increasing frame ID generated by a local logical counter to avoid uncertainties caused by clock drift. The type field is also used to distinguish between session synchronization control frames (SYNC) and role switching request frames (ROLE_SWITCH_REQ). The source identifier is used to distinguish between "voice from the first headset" and "voice from the second headset". In step S103, the two headsets collect the voice of their respective wearers in parallel and perform front-end voice processing. The processed voice signal is then framed with a preset fixed frame length, which can be aligned with the Bluetooth transmission cycle. In step S104, the second Bluetooth headset 320 sends an uplink audio data frame carrying its source identifier to the first Bluetooth headset 310. The first Bluetooth headset 310 and the second Bluetooth headset 320 respectively perform local audio encoding on their respective framed voices to obtain the uplink audio data frame. The first Bluetooth headset 310 does not decode or re-encode the uplink audio data frame received from the second Bluetooth headset 320. Instead, it performs buffering and rearrangement based on the sequence number and timestamp before aggregation, encapsulation, and relay forwarding. The first Bluetooth headset 310 multiplexes the two uplink audio data frames into a composite data packet and uploads it through the existing Bluetooth data connection between it and the terminal 200. In steps S105-S107, terminal 200 completes identification, optional error correction, and translation to generate two translations. In step S108, terminal 200 delivers the translation from source A to the other party's headset for playback according to the routing binding table, and avoids hearing its own translation on the headset side through self-listening filtering rules. Finally, in steps S109-S110, the session state is maintained and resources are released upon exit.

[0062] like Figure 5As shown, to support source differentiation and terminal-side parsing of uplink voice, the uplink aggregation encapsulation adopts a composite frame or data packet structure. The uplink composite frame data packet includes at least a frame header and a payload field, and may include a check field for consistency verification, where the check field may include a CRC checksum. Furthermore, the uplink audio data frame may optionally carry forward error correction (FEC) redundancy information to improve recovery capabilities in packet loss scenarios. The frame header includes a session ID field to identify the current interpretation session, a source identifier field to indicate that the voice data belongs to the first user 400 or the second user 500, a sequence number field to provide frame order information, a timestamp field to provide timing reference, and an acoustic status flag field to represent acoustic status information such as simultaneous acquisition / playback, dual-talk detection, and noise level, allowing the terminal or headset to select processing strategies as needed. The payload field includes at least payload A audio segment and payload B audio segment, used to carry two uplink voice segments respectively or a single voice segment when one side is silent. After receiving the uplink composite frame, the mobile terminal reads the aforementioned fields to complete session attribution, source segmentation, out-of-order recovery, and buffer reordering. Buffer reordering includes: establishing a reordering window buffer for out-of-order detection and buffer alignment; deduplicating and outputting audio frames falling within the reordering window in order; discarding audio frames exceeding the reordering window and / or setting missing frames with missing tags; and outputting a source-distinguished speech stream to the speech recognition module. The end-to-end target latency corresponding to the reordering window buffer can be configured from 80 to 250 milliseconds to strike a balance between low latency and robustness.

[0063] like Figure 6 As shown, during the downlink phase, to ensure stable binding between the translation and the source and reduce crosstalk risk, the mobile terminal 200 maintains a routing binding table. The routing binding table includes at least rules for delivering from source A to the second headset and from source B to the first headset, and may include self-listening filtering rules to restrict the translation from being delivered or played to the same side that generated the translation. The downlink routing binding module 260 selects a first translation delivery channel and a second translation delivery channel when generating downlink data packets based on the routing binding table: the first translation delivery channel delivers the translation corresponding to the first user's voice source to the second Bluetooth headset, and the second translation channel delivers the translation corresponding to the second user's voice source to the first Bluetooth headset 310. After receiving downlink data, the first Bluetooth headset 310 and the second Bluetooth headset 320 respectively perform filtering according to their local logical roles, sending only the translated audio corresponding to the "other party source identifier" for decoding and outputting to the speaker; unmatched translated audio data frames are discarded.

[0064] like Figure 7As shown, to address potential changes in business modes and links during face-to-face conversations, the simultaneous interpretation system 100 employs a state machine to describe the switching relationships of the system's working states during conversation maintenance. The state machine includes at least music mode, call mode, and simultaneous interpretation mode states. The conversation management module, in conjunction with the link management module, switches to simultaneous interpretation mode upon entering a translation conversation and maintains key states such as the conversation ID, source identifier, and routing binding table. When a master-slave switch, removal / reconnection event, or reconnection event occurs, the system enters a conversation state synchronization process and completes the transfer of uplink aggregation and downlink playback responsibilities during the handover process. Subsequently, through an exception recovery process, the system returns to simultaneous interpretation mode to continue interpretation.

[0065] like Figure 8 As shown, to suppress the interference of translated text playback on voice acquisition in near-field conditions, the first earphone echo suppression / gating unit 315 and the second earphone echo suppression / gating unit 325 suppress echo components on the earphone side and cooperate with gating control. The translated text is output from the speaker through the playback path and may be picked up by the microphone through the echo acquisition path; the user's speech enters the microphone through the near-field speech path to form valid voice input. The echo cancellation processing, combined with the playback reference signal, cancels the acquired components, and the gating control signal reduces the playback volume or performs a short-term attenuation when it detects that the user is speaking, thereby making the uplink output more suitable for entering the encoding and uplink links and improving the signal-to-noise ratio of the speech recognition input.

[0066] like Figure 9 As shown, when the quality of the recognized text needs improvement, the error correction module 250 of the terminal 200 executes an error correction coordination process. This process takes the recognized output text as input and determines whether to enter the error correction branch based on an error correction trigger. When the error correction conditions are met, error correction candidates are generated, and a score comparison and replacement selection are performed on these candidates. The corrected text is then output as the translation input. When the error correction conditions are not met, the recognized output text is directly used as the translation input. The translated output is then written into the routing binding table by the downlink routing binding module 260 based on the source identifier and directed to the recipient's headset for playback.

[0067] Through the above structure and process, the mobile terminal 200, while maintaining only the existing single Bluetooth data connection of "terminal ↔ first earphone", can still achieve parallel acquisition, aggregation, uploading, and source-distinguished processing of dual-path uplink voice through the inter-ear link 330. In the downlink phase, it can achieve targeted delivery of translations and self-listening filtering according to the routing binding table. When the conditions for switching links are met, the system can smoothly switch the second earphone to the new first earphone and interact with the terminal 200, thereby maintaining session continuity in the event of link changes.

[0068] To further illustrate the working details of this embodiment under the core structure of "the first earphone interacts with the terminal, and the second earphone does not directly interact with the terminal", several more vivid embodiments and data interaction examples are given below in conjunction with specific application scenarios.

[0069] In this embodiment, the first user 400 (e.g., Chinese businessperson A) wears the first Bluetooth headset 310, and the second user 500 (e.g., American businessperson B) wears the second Bluetooth headset 320. The mobile terminal 200 (e.g., a mobile phone) is carried by A and runs the simultaneous interpretation application. The two users sit on opposite sides of a conference table, approximately 0.6 meters apart, with the sounds of keyboards and air conditioning present. After the system enters simultaneous interpretation mode, the mobile phone only establishes a BLEGATT connection with the first Bluetooth headset 310; the second Bluetooth headset 320 does not establish a translation data channel with the mobile phone.

[0070] 1) Session Establishment and Synchronization: User A clicks "Start Face-to-Face Simultaneous Interpretation" on their mobile phone. Terminal 200 sends a session start / end signaling (Type=START) to the first earpiece 310, which carries session ID=0xA13F, language pair=Chinese↔English, sampling rate=16kHz, encoding format identifier=LC3 (or other low-latency encoding), target bit rate=24kbps, frame length=10ms, and initial timestamp T0. The first earpiece 310 forwards the session start / end signaling to the second earpiece 320 on the inter-ear link 330, along with a timing reference identifier (e.g., using the local clock of the first earpiece as the master clock), so that both earpieces start acquisition and encoding near T0.

[0071] 2) Uplink Relay Aggregation: When Person B says "WirbraucheneineLieferzeitvonzweiWochen.", the second earpiece 320 generates an uplink audio data frame (Type=UL_AUDIO, Src=0x02, Seq=12001, TS=T0+120010ms) every 10ms and sends it to the first earpiece 310 via the inter-ear conversation protocol. The first earpiece 310 simultaneously collects Person A's voice, for example, when Person A says "Okay, but quality must be guaranteed.", generating a frame (Type=UL_AUDIO, Src=0x01, Seq=3051, TS=T0+120020ms). The first earpiece 310 aggregates the two data frames into a composite data packet using a predefined multiplexed encapsulation format: the composite packet header contains CompSeq=5002 and CompTS=T0+120020ms; sub-packet A carries Src=0x01 and its Seq / TS / Len; sub-packet B carries Src=0x02 and its Seq / TS / Len. The first earpiece 310 then uploads this composite data packet through its GATT data channel with the terminal 200. Therefore, from the terminal's perspective, the uplink entry always originates from the first earpiece 310, without relying on the terminal to concurrently connect to the second earpiece.

[0072] 3) Terminal splitting, recognition, translation and feedback: After receiving the composite data packet, the uplink aggregation encapsulation module 220 in the terminal 200 splits the two voice streams according to the Src field and sends them to the voice recognition module 230 respectively. Recognition results are as follows: for Src = 0x02, the English text "Wir brauchen eine Lieferzeit von zwei Wochen." is output; for Src = 0x01, the Chinese text "可以,但要保证质量" is output. The translation module 240 generates translations respectively: translates Src = 0x02 into Chinese "我们需要两周的交付周期", and translates Src = 0x01 into English "Das ist möglich, aber die Qualität muss gewährleistet sein.".

[0073] 4) Routing binding and directional playback: The terminal 200 carries the routing identifier Route = Bind(Src) in the downlink translated audio data frame and performs delivery according to the routing binding table: when Src = 0x02 (voice of B), the target earphone is the first earphone 310; when Src = 0x01 (voice of A), the target earphone is the second earphone 320. To reduce the multicast complexity of the terminal, the terminal can also send the same "downlink combined packet" containing the two translations to the first earphone 310, and the first earphone 310 then forwards the translated frame belonging to the second earphone via the inter-aural link; at this time, the second earphone still does not need to interact directly with the terminal. The first earphone 310 and the second earphone 320 play only the translations of each other according to the self-hearing filtering rules respectively. For example, B only hears the English translation and does not hear the playback of his own English, and A only hears the Chinese translation and does not hear the playback of his own Chinese, thus avoiding understanding interference.

[0074] 5) Example of the playback rhythm of the translation: When the conversation alternates frequently, the playback control module 270 can gate the playback according to the "close talk detection" result of the acoustic state marker field: if it is detected that the wearer is speaking, the playback of the local translation is attenuated for a short time (e.g., -12dB, lasting 120ms) or paused and queued, and continues to play after the close talk ends; if it is detected that both parties are speaking simultaneously (double talk), the translation is segmented and a prompt tone or a short pause is inserted between the two segments to prevent the translations from overlapping and making it inaudible.

[0075] During a long session, the first earpiece 310 may experience battery depletion or signal obstruction. For example, if the battery of the first earpiece 310 drops to 15% and the RSSI remains below -80dBm for 8 consecutive seconds, the master-slave switchover trigger condition is met. At this point, the system initiates a non-disconnect migration process: the first earpiece 310 sends a role switch request signaling (Type=ROLE_SWITCH_REQ) to the second earpiece 320 via the inter-ear link 330, carrying a synchronization anchor. The ROLE_SWITCH_REQ is configured as a control signaling higher than the audio data and supports preemptive transmission. Before transmission, a robust transmission mode is selected based on link quality parameters. After transmission, an acknowledgment listening window is opened; if no acknowledgment is received within a preset time slot, a non-backoff fast retransmission is performed. Scheduling is then performed to avoid frequency-hopping channels with high bit error rates based on the channel quality mapping table. Anchor_UL={Src=0x02,Seq=12600,TS=T0+126000ms} With Anchor_DL={PlayPos=3.24s,DLSeq=8801}.

[0076] 1) Parallel Connection Overlap Period: The second earpiece 320 begins establishing a new existing Bluetooth data connection (second GATT connection) with the terminal 200. Before the connection is established, the first earpiece 310 continues to act as a relay to upload uplink composite data packets to ensure uninterrupted communication. After detecting that the second connection is ready, the terminal 200 enters the overlap period: on the one hand, it continues to receive the upload from the first earpiece; on the other hand, it simultaneously sends the same downlink composite packets to both the first and second earpieces. During the overlap period, it configures the scheduling priority of control signaling and uplink data to be higher than that of downlink translation data, and automatically allocates or limits the transmission bandwidth of the two connections to avoid frame loss and interference caused by insufficient bandwidth when the two connections coexist.

[0077] 2) Routing table and session state migration: In the session management module 280, the terminal 200 switches the role marker of "first earphone = connection holder" to the second earphone 320 and updates the delivery target in the routing binding table to the new role; at the same time, the second earphone 320 aligns the downlink playback progress to the PlayPos=3.24s position according to the synchronization anchor point to avoid repeated broadcasts or skipping, and continuously mirrors and synchronizes the session context information of the original first earphone 310 in the background (including at least the uplink sequence state, encryption offset and downlink playback progress) so as to close and align the protocol stack state at the moment of takeover.

[0078] 3) Responsibility Handover and Old Connection Release: After the terminal confirms that the second earpiece 320 has stably received the uplink composite data packet and completed the downlink playback alignment, the terminal switches the uplink and downlink channels to the second GATT connection and releases the first GATT connection. Afterward, the second earpiece 320 becomes the new first earpiece (relay aggregation node), and the original first earpiece 310 degenerates into the new second earpiece, interacting only with the second earpiece 320 via an inter-ear link. This ensures that the entire switching process appears as a continuous translation without noticeable breaks in the audio for the user. The receiving end maintains a preset depth jitter buffer and prioritizes outputting cached pre-read data during the link switching transition period. When jitter is detected in the samples or timestamps before and after the switch, fade-in and fade-out smoothing is performed on the audio samples before and after the switch to reduce transient noise.

[0079] In practical use, users may briefly remove the earphones, open the charging case, or experience a short-term disconnection. This embodiment provides an anomaly recovery strategy: when any earphone detects a removal / wearing event (e.g., the in-ear detection changes from IN to OUT), the device immediately sets Wear=OUT in the acoustic status flag field and sends a status reporting signal (Type=STATE_REPORT) to the relay earphone via the session protocol; the relay earphone then reports to terminal 200. Terminal 200 accordingly suspends or reduces the delivery frequency of translations to that earphone and caches the translations in the playback queue, resuming playback according to the latest timestamp after detecting re-wearing (Wear=IN).

[0080] If a reconnection event occurs, such as an inter-ear link interruption lasting more than 300ms, the repeater headset will enter an abnormal recovery process: First, it will continue uploading local voice using a conservative buffering strategy, and set the length field of the other sub-packet in the uplink composite packet to zero to indicate its absence; simultaneously, it will periodically send session synchronization signaling (Type=SYNC) to probe for recovery at the other end. When the other end recovers, the repeater headset will request the other end to retransmit several frames (e.g., the most recent 20 frames) from the specified Seq according to the sequence number anchor point, and perform frame rearrangement and deduplication locally to ensure that the recovered voice stream does not splice across sentences.

[0081] Both the inter-ear link and the terminal link may experience packet loss and out-of-order delivery. In this embodiment, the second earphone 320 inserts a lightweight FEC redundancy information frame every N frames (e.g., N=5). The redundancy information can be a simple XOR check or RS encoded segment, and a CRC checksum is carried at the end of each frame. After receiving the data, the first earphone 310 buffers it according to the sequence number and completes the rearrangement in a jitter buffer with an end-to-end target delay of 120ms: when a Seq is detected but the FEC is recoverable, it is recovered first and then submitted for uplink composite encapsulation; when the missing segment is unrecoverable and exceeds the buffer window, the segment is marked as lost and Loss=1 is set in the acoustic status flag field to prompt the terminal-side identification module to perform sentence segmentation protection (e.g., to avoid forcibly splicing the text before and after the missing position into the same sentence).

[0082] On the terminal side, the uplink convergence encapsulation module 220 can also perform secondary correction based on the CompSeq / CompTS of the composite packet: if the timestamp deviation between the two sub-packets is found to exceed the threshold (e.g., >40ms), then a silence pad is inserted for the earlier packet or a small time stretch is performed for the later packet to reduce the voice alignment error between the two packets, thereby making the correspondence of "A speaks a sentence - B listens to the translation a sentence" more stable.

[0083] When the face-to-face distance is close, the translation played by earphone B may be picked up by earphone A's microphone and re-uploaded, forming an "echo re-translation". In this embodiment, while playing the translation, the earphone side simultaneously inputs the playback reference signal into the echo canceller (AEC) and estimates the leakage path; when a large playback energy is detected and the proximity detection is true, a strong dual-talk detection strategy is activated: increasing the AEC reference weight, reducing the far-end return gain, and performing gated compression on the speech before uplink encoding (e.g., attenuating non-proximity components by -15dB within 50ms).

[0084] For example: While A is speaking, B's earpiece plays the Chinese translation "We need two weeks to deliver." Earpiece B first suppresses backsound sampling locally. Then, when earpiece 310 receives the uplink frame from earpiece B, it reads the Play=1 and DT=1 (dual-talk) flags in the acoustic status flag field and selects a more conservative uploading strategy for that frame. For example, it reduces the coding gain and prompts the terminal-side recognition module to use a stronger echo robust model, ultimately reducing the probability of the echo being misidentified as A's speech.

[0085] In cross-border business scenarios, proper nouns such as company names, place names, and model numbers frequently appear. Taking "Haden301" as an example, the pronunciation might be "Hay-denthree-oh-one," easily leading to variations like "Haden301" or "Hadden301." The error correction module 250 can simultaneously enable rule-based and neural error correction: the rule-based side maps potentially confusing segments to a candidate set based on pinyin / phoneme rules; the neural side retrieves the candidate entity 'Haden301' from the entity acoustic vector index library through acoustic dual-tower retrieval and uses a semantic localization discrimination model to determine the start and end positions to be replaced. When the fusion score exceeds a threshold (e.g., 0.78), the erroneous segments in the recognized text are replaced with standard entities, and then sent to the translation module 240 to ensure consistency of the company name in the translation.

[0086] Meanwhile, to avoid incorrect replacements introduced by error correction, the terminal can maintain an independent user vocabulary and confidence threshold for each source in the routing binding table: for example, the threshold for enabling English accent adaptation for source B (Src=0x02) is 0.72, and the threshold for enabling Chinese accent adaptation for source A (Src=0x01) is 0.80, thereby optimizing the error correction stability of both sides in bidirectional simultaneous interpretation.

[0087] The above embodiments illustrate that this embodiment uses a "single terminal link + ear-to-ear conversation protocol" structure to organize both the uplink and downlink of the second earphone within the relay framework of the first earphone; the second earphone only takes over and interacts with the terminal when switching links, thus maintaining high stability and low latency despite differences in Bluetooth stacks across multiple models and systems.

[0088] To facilitate the explanation of the determinism of downlink targeted delivery, this embodiment provides a specific organization method for the routing binding table. The routing binding table uses (Session ID, Source Identifier Src) as the key and (Target Headset Logical Role TargetRole, Delivery Channel Channel, FilterPolicy FilterPolicy, ValidFrom) as the value. TargetRole, for example, takes the value {ROLE_FIRST, ROLE_SECOND}, indicating that delivery should be made to the device currently playing the role of the first or second headset; Channel can be mapped to a GATT feature handle or an ACL data channel identifier; FilterPolicy includes at least three types of rules: "self-listening filtering," "silent filtering," and "wearing / removing suppression." Each time the terminal 200 parses an uplink composite data packet, it reads the Session ID field and the Source Identifier field. If the key does not exist in the routing binding table, it writes the following according to the default rules: Src=0x01→TargetRole=ROLE_SECOND, Src=0x02→TargetRole=ROLE_FIRST, and records ValidFrom=current CompTS. When a master-slave switch occurs, the terminal does not immediately delete the old record during the overlap period. Instead, it interprets TargetRole as a "logical role" rather than a "physical device" and synchronously updates the Channel to point to the new connection. In this way, even if two connections exist simultaneously during the switch, the routing binding table can still maintain semantic stability, avoiding the situation where "table entries still point to the old physical headset," which would lead to misdelivery of the translation.

[0089] Furthermore, to prevent mismatched playback in extreme cases, the terminal can simultaneously carry two fields, Src and Dst, in the downlink translated audio data frame: Src represents the uplink source corresponding to the translation, and Dst represents the target logical role determined by the terminal. After receiving the downlink frame, the earphone first checks whether Dst matches the current logical role of the device, and then checks whether Src is the "other party's source." Only after this double matching does it proceed to decoding and playback. For example, during the overlap period of a switching link, both the original first earphone and the new first earphone may receive the same downlink combined packet. In this case, both will make a consistency judgment based on Dst / the device's role, and only the side responsible for playback will actually output, while the other side will discard it or only use it for buffer alignment, thereby achieving a seamless handover in terms of listening experience.

[0090] In actual products, headphones often simultaneously handle music playback and call functions. This embodiment prioritizes reusing existing data connections and avoids additional SCO channel usage in simultaneous interpretation mode: when the system is in music mode and the user initiates simultaneous interpretation, the session management module 280 prioritizes the BLE / ACL data channel to carry uplink composite data packets and downlink translations, avoiding contention with the music A2DP for the same real-time voice transmission. When an incoming call is received or the system enters call mode, it can adopt two strategies: first, directly terminate the simultaneous interpretation session and release the routing binding table; second, put the simultaneous interpretation session into a paused state (Type=PAUSE), maintaining the session ID but stopping uplink acquisition and downlink broadcasting, resuming via Type=RESUME after the call ends. During the pause / resume process, control signaling has higher priority within the session protocol and can be preemptively sent, keeping the control delay for state switching at the millisecond level, avoiding the need for the user to re-pair or re-enter the session after the call ends.

[0091] In terms of priority scheduling, the first headset 310, acting as a relay node, can maintain three types of queues: a control signaling queue, an uplink audio queue, and a downlink translation queue. The control signaling queue has the highest priority and supports preemption; the uplink audio queue and the downlink translation queue can be scheduled by polling based on timestamps or by a "proximity priority" strategy. For example, when the wearer is detected speaking, the scheduling weight of the uplink audio queue is increased, and the continuous transmission length of the downlink translation queue is reduced (e.g., a maximum of 2 frames of translation are sent each time before forcibly relinquishing the transmission opportunity), in order to reduce echo re-collection and end-to-end latency jitter. The above scheduling strategy, combined with the aforementioned acoustic gating, can maintain a stable simultaneous interpretation experience even in noisy environments.

[0092] To further clarify the implementation method of "the second earpiece not directly interacting with the terminal," this embodiment provides an example of a frame format for the inter-ear session protocol: the frame header includes Version (1 byte), Type (1 byte), Src (1 byte), Flags (1 byte), Seq (2 bytes), TS (4 bytes), and Len (2 bytes), the payload is audio data or control parameters, and the tail is CRC (2 bytes). The Flags can indicate whether it is a retransmission frame, whether it contains FEC, or whether it is in a two-way talk state. Regarding reliable transmission, the inter-ear link can adopt a "lightweight acknowledgment + window retransmission" mechanism: the second earpiece 320 continuously sends 8 uplink audio frames in a window W=8, and the first earpiece 310 returns a cumulative acknowledgment ACK equal to the maximum continuous received Seq according to Seq; when the second earpiece 320 does not receive an ACK within a timeout Δ=30ms, it only retransmits the missing frame instead of retransmitting the entire window, thereby controlling latency while ensuring reliability. For control signaling (such as ROLE_SWITCH_REQ and SYNC), a double-transmission + acknowledgment mechanism can be adopted, that is, the same signaling is sent twice within a 5ms interval and an explicit ACK is required, which further reduces the probability of loss of critical control signaling in interference environments.

[0093] Taking tourist inquiries as an example: the first user, 400, is a Chinese-speaking tourist, wearing a first earpiece 310 and holding a terminal 200; the second user, 500, is a local staff member, wearing a second earpiece 320. When a tourist asks, "How do I get to the Brandenburg Gate?" the second earpiece 320 does not need to be paired or connected to the tourist's mobile phone, but can still upload the staff member's English answer via an inter-ear link and obtain a Chinese translation for playback. In continuous dialogue, the system can recognize the "question-answer" structure and segment the playback: when a question ends with a filler word or a pause exceeds 300ms, a translation synthesis is triggered; when the answer is long, it is divided into multiple segments according to a comma / parallel structure and played sequentially, with very short silences (e.g., 40ms) inserted between segments to improve comprehensibility. Throughout the process, the terminal only needs to maintain its connection with the first earpiece; the staff member can complete two-way communication without any mobile phone operation, demonstrating the ease of use and stability of this embodiment in face-to-face scenarios.

[0094] To meet the real-time requirements of simultaneous interpretation, this embodiment can perform end-to-end latency estimation for acquisition, encoding, relay, transmission, recognition translation, and synthesis playback. Taking a frame length of 10ms as an example: acquisition and front-end processing at the earphone side takes approximately 10-20ms, encoding and encapsulation take approximately 5-15ms; the ear-to-ear transmission from the second earphone to the first earphone can be controlled within 5-20ms under good channel conditions; the upload transmission from the first earphone to the terminal takes approximately 10-30ms; terminal-side ASR and translation can use streaming inference, with the first packet latency of approximately 80-200ms, and subsequent translations output incrementally every 40-120ms; translation TTS synthesis or translation audio splicing takes approximately 20-80ms, and downlink delivery and earphone decoding playback take approximately 10-40ms. Under the above configuration, the typical end-to-end total latency can fall within the range of 200-450ms, and since the terminal only maintains one data connection, the jitter caused by link reconstruction and resource contention is significantly reduced. This embodiment also allows for selection between "lower latency" and "higher robustness" depending on the scenario: for example, in a quiet indoor environment, the jitter buffer can be reduced from 120ms to 80ms for faster feedback; in a strong interference environment such as a subway, the buffer can be increased to 200ms and a stronger FEC can be enabled to improve continuity.

[0095] To ensure that the second earpiece 320 does not directly interact with the terminal 200 in the default mode, this embodiment can set constraints at the device and protocol layers. Firstly, in simultaneous interpretation mode state 930, the second earpiece 320 disables or does not broadcast GATT services for data services, retaining only the advertising or synchronization channels required for inter-ear communication. Secondly, during the simultaneous interpretation session, the terminal 200 only accepts application data feature writes / notifications from the current first earpiece and returns rejection or delayed processing for connection requests for services with the same name from other devices. Thirdly, the first earpiece 310 can send a "direct connection prohibition flag" (LinkLock=1) to the second earpiece 320, and the second earpiece is only allowed to enable direct connection capability after receiving ROLE_SWITCH_REQ and completing synchronization anchor point verification. Through these measures, even if the user accidentally touches the connection or the system automatically reconnects, routing chaos caused by the second earpiece suddenly establishing a concurrent connection with the terminal midway through the session can be avoided. Only when the link switching process is explicitly triggered and the session state transition is completed will the second earpiece become the new first earpiece and interact with the terminal, thus balancing stability and recoverability.

[0096] Face-to-face simultaneous interpretation often involves business and personal privacy content. This embodiment, with its structure of "the second earpiece not directly connected to the terminal, but only relayed through the first earpiece," further reduces the session exposure surface: by default, only one terminal data connection exists. The terminal can perform session-level encryption and integrity verification on the session ID, source identifier, and translated data, and reuse the same session key or derived key in the earpiece session protocol to protect data transmission from the second earpiece to the first earpiece. For example, at the start of the session, the terminal sends a random number (Nonce) and a session key (KeyS) to the first earpiece. The first earpiece then synchronizes the derived parameters to the second earpiece. Both earpieces perform symmetric encryption on the payload field of the uplink audio data frame and carry a message authentication code at the end of the frame. Simultaneously, the terminal can choose to complete speech recognition and translation locally (offline model or edge-side inference), or use edge-cloud collaboration but only upload the encrypted feature representation, thereby balancing privacy and computing power across different product forms. When switching links, the second earpiece must complete key migration and session consistency verification before taking over and interacting with the terminal to ensure that the switch does not cause session key mismatch or the appearance of data packets that can be replayed by a third party. The above optional implementations do not change the basic link organization and routing binding logic of this embodiment, but can provide stronger protection capabilities in scenarios with higher privacy requirements, such as meetings and medical consultations.

[0097] Furthermore, in scenarios involving rapid multilingual switching (such as continuous switching between Chinese, English, and English), the session management module 280 can maintain a list of language pairs under the same session ID and carry the current language pair index in the control signaling, enabling the terminal and the two headsets to complete language switching without exiting the session, further improving the convenience of actual use.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for face-to-face simultaneous interpretation, applied to a terminal and a pair of wireless earpieces, said wireless earpieces comprising a first earpiece and a second earpiece, characterized in that, include: In translation mode, the existing Bluetooth data connection between the first earphone and the terminal remains active, and a physical link between the first earphone and the second earphone is established. An application layer session protocol runs on the physical link between the ears. The protocol frame of the session protocol includes a frame header, payload, and checksum. The frame header includes at least a type field, a source identifier, a sequence number, and a timestamp. The type field is used to distinguish between translated control signaling and translated audio data. The first earphone and the second earphone respectively collect the voice signals of their respective wearers and perform front-end voice processing to obtain the processed voice signals; The first and second earpieces respectively perform local audio encoding on the processed voice signal frame by frame with a fixed frame length to obtain uplink audio data frames. The first earpiece does not decode or re-encode the uplink audio data frames received from the second earpiece, but only performs aggregation encapsulation and relay forwarding, and encapsulates the source identifier of the corresponding earpiece in the uplink audio data frames. The second earpiece sends the uplink audio data frames carrying the source identifier of the second earpiece to the first earpiece through the session protocol. The first earpiece aggregates its own uplink audio data frames and the uplink audio data frames received from the second earpiece according to a predefined multiplexing encapsulation format to form a composite data packet. The composite data packet includes at least the source identifier, sequence number, timestamp, and length fields of each of the two audio sub-packets, and uploads it to the terminal through the general attribute protocol data channel of the Bluetooth data connection. The terminal sends the composite data packet to the translation processing module, performs speech recognition and translation on the two uplink audio streams to generate two downlink translated audio data frames, and carries a routing identifier bound to the uplink source identifier in the downlink translated audio data frames; the terminal sends a downlink data packet containing the two downlink translated audio data frames to the first earpiece; after the first earpiece decodes the downlink data packet to obtain the two downlink translated audio data frames, it filters according to its own logical role, outputs and plays the downlink translated audio data frame bound to the second earpiece source identifier, and forwards the downlink translated audio data frame bound to the first earpiece source identifier to the second earpiece on the ear-to-ear physical link; the second earpiece decodes the received downlink translated audio data frame and plays it.

2. The method according to claim 1, characterized in that: The type field of the session protocol is used to distinguish different types of signaling and data, and includes at least: Session start and end signaling: Used to identify the start and end of a session, ensuring the start and stop of the translation process; Session synchronization signaling: used to synchronize data or status between two headsets to ensure timing consistency during translation; Role Switching Request Signaling: Used to switch the master / slave role between two headsets, ensuring that the system can flexibly adjust the device role according to needs; Synchronization anchor signaling: used to synchronize the position of the audio stream during translation, ensuring that there are no desynchronization issues when playing audio through dual headphones; Uplink audio data: Audio data from the user, uploaded to the system for translation via the first earpiece; Downlink translated audio data: The translated audio data is transmitted to the headphones via a session protocol for the user to play.

3. The method according to claim 2, characterized in that: The translation control signaling has a higher transmission priority than the translated audio data in the session protocol, and employs a preemptive transmission mechanism to reduce preemption delay. satisfy For milliseconds ∈ [1,20], the preemptive transmission mechanism uses a priority queue or signal interruption mechanism. During data transmission, when a control signal arrives, the system will interrupt the transmission of the current audio data, prioritize sending the control signal, and continue transmitting the audio data after the signal is sent.

4. The method according to claim 2, characterized in that: At the start of the translation session, the master earpiece synchronizes the recording start point identifier, encoding parameter set, and timing reference identifier to the slave earpiece via session synchronization signaling. The encoding parameter set includes at least the sampling rate, encoding format, and bit rate. Based on the session synchronization signaling, the master and slave earpieces append a unified timestamp to the audio data during audio acquisition. During the downlink phase, playback alignment is performed based on the sequence number and timestamp of the translated audio data frame to ensure that the binaural translation playback synchronization error does not exceed an error threshold. ,in ∈[5,50] milliseconds; In this system, both the master and slave earphones use fixed-length audio frames as the smallest processing unit. Each audio frame carries a frame sequence number and a sampling timestamp or logical clock count value. The master earphone synchronizes the audio frames from both earphones based on the frame sequence number, and when a frame sequence deviation is detected between the two audio frames, it performs synchronization adjustment through buffer alignment and / or frame-level delay compensation. The master earphone periodically sends synchronization control frames to the slave earphone, which carry a reference frame sequence number and a reference count value. The slave earphone performs audio sampling count compensation and / or frame generation beat adjustment based on the deviation between its local count value and the reference count value to achieve progressive alignment of the logical clocks of the master and slave earphones.

5. The method according to claim 2, characterized in that: The audio data frames of the session protocol carry forward error correction redundancy information and checksums. The receiving end performs frame reordering based on the sequence number and timestamp of the audio data frames. The frame reordering includes: establishing a reordering window buffer for out-of-order detection and buffer alignment, deduplicating and outputting audio data frames falling into the reordering window buffer in order, and discarding audio data frames exceeding the reordering window buffer and / or setting missing tags for missing audio data frames. The first earphone performs buffering and reordering on the uplink audio data frames from the second earphone before forwarding them to the terminal to restore out-of-order transmission during the transmission process from the second earphone to the first earphone and from the first earphone to the terminal. The end-to-end target delay corresponding to the buffer satisfies the following conditions. ∈[80,250] milliseconds; when a master-slave role switch occurs, the new master earphone inherits the frame order state, synchronization information and rearranged window buffer context of the original master earphone, and continues to execute the buffering and rearrangement after inheritance.

6. The method according to claim 1, characterized in that: The multiplexing and encapsulation format of the composite data packet includes a composite packet header and at least two sub-packet segments. The composite packet header includes at least a composite packet sequence number and a composite packet timestamp. Each sub-packet segment includes at least a source identifier, a sub-packet sequence number, a sub-packet timestamp, an encoding format identifier, and a length field.

7. The method according to claim 1, characterized in that: The downlink translated audio data frame carries its bound uplink source identifier; the first earpiece only outputs and plays the translated audio data frame whose source identifier points to the second earpiece, and the second earpiece only outputs and plays the translated audio data frame whose source identifier points to the first earpiece; unmatched translated audio data frames are discarded.

8. The method according to claim 1, characterized in that: The master-slave role switching process will be executed when any of the following trigger conditions are met: the master earphone has remaining battery power. Below the threshold ,in ∈[0.10,0.30]; or the difference in master-slave charge levels. Greater than the threshold ,in ∈[0.03,0.08], and the translation application is in the foreground active state; or the received signal strength of the main headset and the terminal is within the duration Below the threshold in [2,20] seconds and ∈[-85,-60] dB / mW; or the duration of silence detected by one party. Exceeding the threshold ,in ∈[3,30] seconds.

9. The method according to claim 8, characterized in that: When a master-slave role switch is triggered, the current master earphone sends a role switch request signaling carrying a synchronization anchor point through the session protocol. The synchronization anchor point includes at least the sequence number anchor point of the current uplink audio data frame and the current downlink translation playback progress anchor point. The role switch request signaling is configured as a control signaling higher than the audio data signaling and supports preemptive transmission. Before sending the role switch request signaling, the sending end selects to reduce the modulation and coding order or adopt a robust transmission method based on link quality parameters. After sending the role switch request signaling, the sending end opens an acknowledgment listening window. If no acknowledgment information is received from the receiving end within a preset time slot, non-backoff fast retransmission is performed. Furthermore, the sending end avoids frequency hopping channels with high bit error rates based on the channel quality mapping table and schedules the role switch request signaling to a channel that meets the quality threshold for transmission.

10. The method according to claim 9, characterized in that: The master-slave role switching adopts a non-disconnect migration process, including a parallel connection overlap period; during the parallel connection overlap period, the original master headset maintains a first general attribute protocol connection with the terminal, and the new master headset establishes a second general attribute protocol connection with the terminal. During the overlap period, the terminal switches the uplink and downlink transmission channels to the second general attribute protocol connection and then releases the first general attribute protocol connection; wherein, before or during the master-slave role switch, the communication context information of the original master earphone is continuously synchronized from the earphone background, and the communication context information includes at least the audio stream sequence number, link encryption offset, Bluetooth clock phase and decoding status; Furthermore, during the parallel connection overlap period, the new master headset simultaneously receives downlink data packets from the terminal and residual downlink data packets forwarded by the original master headset. The data packets from both sources are matched and deduplicated based on timestamps to achieve continuous takeover of downlink data.

11. The method according to claim 10, characterized in that: During the parallel connection overlap period, the terminal simultaneously sends the same downlink data packets to both the original master headset and the new master headset. The original master headset and the new master headset align the downlink translation playback progress according to the synchronization anchor point and complete the playback responsibility switch. The receiving end maintains a jitter buffer of a preset depth and prioritizes outputting the cached pre-read data from the jitter buffer for playback during the link switching transition period. Furthermore, when jitter is detected in the audio samples or timestamps before and after the switch, fade-in and fade-out processing is performed on the audio samples before and after the switch to reduce transient noise caused by sample discontinuity.

12. The method according to claim 1, characterized in that: The front-end speech processing includes short-range directional noise reduction processing, which includes: performing beamforming based on dual-microphone signals to form a main lobe and a null in the direction of interference; and applying gain suppression or null suppression to the sound source in the direction of interference based on directional gating and distance gating, wherein the azimuth angle interval corresponding to directional gating is... ∈[0°,30°], Distance gate threshold satisfy ∈[0.20,0.40] meters.

13. The method according to claim 12, characterized in that: The short-range directional noise reduction process is performed according to frequency band differences, with the upper frequency limit in the low-frequency band. satisfy ∈[300,700] Hz, conservative gain suppression and superimposed spectral shaping are performed in the low-frequency band; the lower limit frequency in the mid-to-high frequency band... With upper limit frequency satisfy ∈[700,1300] Hertz and ∈[3000,5000] Hz, null enhancement and Wiener-type suppression are performed in the mid-to-high frequency band; and the duration is triggered when a sudden far-field high-energy noise event is detected. A short-time deep suppression mode ∈ [50,500] milliseconds, in which instantaneous gain reduction and limiting are performed.

14. The method according to claim 13, characterized in that: The front-end speech processing also includes mode switching and state management. When switching between directional suppression mode and normal noise reduction mode, the mode switching performs fade-in and fade-out on the beam pointing, null depth and the adaptive update step size of frequency domain adaptive noise reduction and echo cancellation. It also saves and maintains two sets of filter parameters and target curves corresponding to directional suppression mode and normal noise reduction mode, and reuses noise statistics and secondary path estimation results during switching to shorten the convergence time. During translation playback, echo cancellation and playback leakage estimation driven by the playback reference signal are enabled. Frequency domain nulls and gain compression are applied to the self-return of the playback segment. When a state of strong directional suppression and playback coexisting is detected, state interlock control is performed, including increasing the echo cancellation reference weight, reducing the far-end return gain, and enabling a strong dual-talk detection strategy. When a user is detected speaking, the playback volume is automatically reduced or a short-term attenuation is performed to ensure uplink speech priority and suppress echo remnants.

15. The method according to claim 1, characterized in that: The translation processing module performs error correction processing after obtaining the recognized text from speech recognition and before generating the translation. The error correction processing includes rule-based error correction and neural network error correction. The rule-based error correction includes: segmenting the identified text to obtain a word sequence; converting the word sequence into a pinyin sequence aligned with the word sequence based on a dictionary and retaining non-Chinese characters as anchors; calling a pre-compiled finite-state converter on the pinyin sequence to perform pinyin-Chinese character rule matching and replacement, and performing longest-match priority for replacement rules with inclusion relationships; and performing dynamic rule post-processing after the output of the finite-state converter, which performs longest-string priority matching on the pinyin sequence and maps the pinyin offset back to the word sequence index to complete the forced replacement. The neural network error correction includes: obtaining a query vector from the acoustic twin-tower model that shares the acoustic feature input parameters of the corresponding audio segment or its acoustic feature; and retrieving a set of candidate entities from a pre-built entity acoustic vector index library, with a candidate number... satisfy [3,10]; The candidate entity and the recognized text are concatenated to construct a semantic model input sequence for replacement localization and semantic discrimination. The semantic model outputs the start and end positions of the segment to be replaced and the semantic confidence score; and a weighted fusion score is obtained based on the acoustic similarity score and the semantic confidence score. When the fusion score exceeds a threshold... The corresponding segments of the identified text are replaced with candidate entities at the specified start and end positions, and the threshold value is... ∈[0.65,0.85].

16. A face-to-face simultaneous interpretation system, employing the method of claim 1, characterized in that: A physical link is established between the first and second earpieces. The earpieces run an application layer session protocol on the physical link. The frame header of the session protocol includes at least a type field, a source identifier, a sequence number, and a timestamp. The first earpiece includes a connection maintenance module, an uplink aggregation relay module, and a multiplexing encapsulation module, used to maintain the existing Bluetooth data connection with the terminal, receive uplink audio data frames from the second earpiece, aggregate them with its own uplink audio data frames to form a composite data packet, and then upload it through a general attribute protocol data channel. The second earpiece includes a voice acquisition and inter-ear transmission module, used to generate uplink audio data frames carrying the source identifier of the second earpiece and send them to the first earpiece via the physical link. The translation processing module is used to perform speech recognition and translation on the two uplink audio streams and generate two downlink translated audio data frames, and the downlink translated audio data frames carry routing identifiers bound to the uplink source identifiers; the first earphone and the second earphone respectively include downlink decoding and filtering playback modules, which are used to decode downlink data packets and filter them according to logical roles and only play downlink translated audio data frames bound to the other party's source identifier.

17. The system according to claim 16, characterized in that: The session protocol configuration prioritizes control signaling over audio data and supports preemptive transmission with preemption delay. The system includes a session synchronization mechanism for synchronizing the recording start point identifier, encoding parameter set, and timing reference identifier. The system also includes a role management module and a status synchronization module. The role management module initiates role switching when at least one of the following conditions is met: power threshold, power difference threshold, received signal strength duration threshold, or silent window threshold. The status synchronization module synchronizes the uplink sequence number anchor point and the downlink playback progress anchor point. The terminal includes a routing table update module for completing the uplink and downlink channel switching and releasing the old connection during the parallel connection overlap period.

18. The system according to claim 16, characterized in that: The translation processing module includes an error correction submodule, which comprises a rule-based error correction unit and a neural network error correction unit. The rule-based error correction unit includes an offline model generation component and an online inference component. The offline model generation component constructs a finite-state converter from the set of pinyin-Chinese character replacement rules and performs deterministic, minimization, and context rewriting compilation to generate a binary error correction model file. The online inference component loads the binary error correction model file and constructs a pinyin sequence aligned with the word sequence obtained from word segmentation. It performs streaming matching and replacement on the pinyin sequence and supports dynamic longest rule matching and mapping replacement from pinyin offset to word index. The neural network error correction unit includes... The system includes an offline database construction module and an online cascaded model module. The offline database construction module generates standard speech from the entity vocabulary and inputs it into a key-value pyramid network to obtain entity acoustic vectors. It then stores the entity acoustic vectors in a vector index library and constructs a nearest neighbor retrieval index. The online cascaded model module includes a parameter-sharing acoustic dual-tower recall model and a semantic localization and discrimination model. The acoustic dual-tower recall model generates query vectors from the input audio segments and retrieves a set of candidate entities from the vector index library. The semantic localization and discrimination model outputs the replacement start position, replacement end position, and semantic confidence score. Based on the weighted fusion result of the acoustic similarity score and the semantic confidence score, and a threshold condition, it controls whether to perform the replacement.

19. A Bluetooth headset, using the method of claim 1, characterized in that: The first earphone includes a Bluetooth communication module, a first microphone array, a speaker, a processor, and a memory; the second earphone includes an inter-ear communication module, a second microphone array, a speaker, a processor, and a memory. The first and second earphones run an application layer session protocol on the physical link between their ears. The frame header of the session protocol includes at least a type field, a source identifier, a sequence number, and a timestamp. The processor of the first earpiece is used to multiplex and encapsulate the two uplink audio data frames into a composite data packet and upload it to the terminal through the general attribute protocol data channel; the processors of the first earpiece and the second earpiece are respectively used to decode the downlink data packet and filter it according to the logical role, and only output and play the downlink translated audio data frame bound to the other party's source identifier. The first and second earphones each include a front-end voice processing module, which includes at least short-range directional noise reduction, mode switching and status management, and echo correction for translated text playback.

20. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method steps of any one of claims 1-15.

Images