An audio processing method and related device

Abstract

This application provides an audio processing method and related device applied to a first earphone case for housing a first earphone. The method includes: establishing a communication connection with a second earphone case in response to a trigger operation; obtaining a first language based on first original speech collected by the first earphone; receiving a second audio signal sent by the second earphone case, the second audio signal including the second original speech collected by the second earphone and used to indicate the second language corresponding to the second original speech; translating the second original speech into a second translated speech in the first language using a local translation file corresponding to the first and second languages; and sending the second translated speech to the first earphone for playback. The translation of the second original speech by the first earphone case is based on the first language corresponding to the first original speech and the second language corresponding to the second original speech, independent of the user's setting of the language required for translation, thus reducing the complexity of user operation.

Description

Technical Field

[0001] This application relates to the field of audio processing, and more particularly to an audio processing method and related equipment. Background Technology

[0002] With increasingly frequent global exchanges, users of different languages ​​need to communicate in real time in daily life and work. To overcome language barriers and ensure smooth communication, various translation tools have emerged. When using electronic devices for translation, users can set the desired translation language to ensure that the translated speech meets their needs.

[0003] When using the earphone component as a translation tool, users need to set the translation language each time they use it. This makes the user operation rather complex in cross-language communication scenarios. Summary of the Invention

[0004] This application provides an audio processing method and related equipment that can reduce the complexity of user operation.

[0005] In a first aspect, an audio processing method is provided, applied to a first earphone case, the first earphone case being used to store a first earphone. The method includes: in response to a trigger operation, establishing a communication connection with a second earphone case, the second earphone case being used to store a second earphone; acquiring a first original speech collected by the first earphone, and obtaining a first language corresponding to the first original speech based on the first original speech; receiving a second audio signal sent by the second earphone case, the second audio signal including a second original speech collected by the second earphone and second language type information, the second language type information being used to indicate the second language corresponding to the second original speech; calling a corresponding local translation file based on the first language and the second language; translating the second original speech into the first language through the local translation file to obtain a second translated speech; and sending the second translated speech to the first earphone for playback.

[0006] In some possible implementations, obtaining the first language corresponding to the first original speech based on the first original speech includes: identifying the language type used in the first original speech to obtain a language identification result; when the language identification result indicates that there are multiple candidate languages, and the difference in the proportion of the speech frames corresponding to the multiple candidate languages ​​in the first original speech is less than or equal to a preset difference threshold, prompting the user to specify a language; and using the language specified by the user as the first language.

[0007] In some possible implementations, obtaining the first language corresponding to the first original speech based on the first original speech includes: identifying the language type used in the first original speech to obtain a language identification result; when the language identification result indicates that there are multiple candidate languages, determining a target language from the multiple candidate languages, wherein the speech frame corresponding to the target language has the largest proportion in the first original speech; and using the target language as the first language.

[0008] In some possible implementations, establishing a communication connection with the second earphone box in response to a trigger operation includes: switching the first earphone box to a translation pairing mode and establishing a communication connection with the second earphone box in response to a trigger operation; acquiring the first original voice collected by the first earphone box is: acquiring the first original voice collected by the first earphone box in the translation pairing mode.

[0009] In some possible implementations, the first and second earphone cases are configured with near-field communication (NFC) functionality; the establishment of a communication connection with the second earphone case in response to a trigger operation includes: establishing a communication connection with the second earphone case based on the NFC functionality in response to the distance between the first and second earphone cases being less than or equal to the NFC sensing distance.

[0010] In some possible implementations, the first and second earphone cases are also equipped with Bluetooth communication functionality; the step of establishing a communication connection with the second earphone case in response to a trigger operation includes: in response to the distance between the first and second earphone cases being less than or equal to the sensing distance of near-field communication, establishing a communication connection with the second earphone case based on the Bluetooth communication function if a communication connection cannot be established based on the near-field communication function.

[0011] In some possible implementations, the step of acquiring the first original speech collected by the first earphone and obtaining the first language type information based on the first original speech further includes: sending a first audio signal to the second earphone box, the first audio signal including the first original speech and the first language type information.

[0012] In a second aspect, an earphone case is provided, including a housing having a chamber for storing earphones; and further including a memory and a processor disposed within the housing; the processor being configured to retrieve and run a computer program from the memory, such that the earphone case, as a first earphone case, performs the method described in any one of the first aspects.

[0013] In some possible implementations, the earphone case further includes a near-field communication module, and the processor is further configured to establish a communication connection with other earphone cases sensed by the near-field communication module.

[0014] The earphone case may also include a Bluetooth communication module, and the processor is further configured to establish a communication connection between the communication module and the earphone case based on the trigger operation indication, in the event that the near-field communication module does not respond to the trigger operation.

[0015] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed, causes the method described in any of the first aspects to be performed.

[0016] Fourthly, embodiments of this application provide a computer program product, the computer program product including computer program instructions, which, when executed, cause the method of any one of the first aspects to be performed.

[0017] Fifthly, embodiments of this application provide a chip including a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to implement the method of any of the first aspects described above.

[0018] In a sixth aspect, embodiments of this application provide a communication system, including a first earphone box, a first earphone, a second earphone box, and a first earphone as described in any of the methods of the first aspect above.

[0019] The beneficial effects of this application embodiment compared to the prior art are as follows: In response to a trigger operation, the first earphone box establishes a communication connection with the second earphone component. Based on the first language corresponding to the first original speech collected by the first earphone and the second language corresponding to the second original speech represented by the received second language type information, it calls the local translation files corresponding to the first and second languages. The second original speech is translated into a second translated speech in the first language using the local translation files, and then sent to the first earphone. Therefore, the translation of the second original speech by the first earphone box is based on the first language corresponding to the first original speech and the second language corresponding to the second original speech, without relying on the user's setting of the language required for translation, reducing user operation complexity and improving convenience. Attached Figure Description

[0020] Figure 1 This is a schematic structural diagram of an earphone component; Figure 2 This is a schematic flowchart of a communication method; Figure 3 This is a flowchart illustrating a communication method provided in an embodiment of this application; Figure 4 This is a schematic structural diagram of a communication system applicable to embodiments of this application; Figure 5 This is a schematic flowchart illustrating another communication method provided in an embodiment of this application; Figure 6 This is a schematic flowchart illustrating another communication method provided in the embodiments of this application; Figure 7 This is a schematic structural diagram of a communication device provided in an embodiment of this application; Figure 8 This is a schematic structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0022] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0023] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0024] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0027] Wireless headphones offer great convenience as an audio playback tool.

[0028] Figure 1 This is a schematic structural diagram of an earphone component.

[0029] The headphone assembly 100 may include one or a pair of earphones and an earphone case 110. For example, the headphone assembly 100 may include a third earphone 121 and a fourth earphone 122. The earphone case 110 is used to store the earphones in the headphone assembly 100. The earphones in the headphone assembly 100 may be wireless earphones in the form of earbuds, in-ear headphones, over-ear headphones, or ear-hook headphones. When not in use, the earphones in the headphone assembly 100 can be stored in the earphone case 110. The earphone case 110 can charge the earphones in the headphone assembly 100.

[0030] When a user uses the third earphone 121 and the fourth earphone 122 in the headphone assembly 100, the third earphone 121 and the fourth earphone 122 can be worn on the user's left and right ears respectively. The third earphone 121 and the fourth earphone 122 can be connected by a connector, for example, the third earphone 121 and the fourth earphone 122 can be neckband wireless headphones. Alternatively, the third earphone 121 and the fourth earphone 122 can also be two independent parts.

[0031] When the headphone assembly 100 includes a single earpiece, that earpiece can be universal. That is, when using the headphone, the user can wear the earpiece in either ear. Alternatively, the earpiece can be unilateral, such as ear-only or ear-only.

[0032] Each earpiece in the headphone assembly 100 may have a microphone and a speaker. The microphone is used to capture audio, and the speaker is used to play audio.

[0033] In some embodiments, the earphone case 110 can communicate with other electronic devices or servers via a communication network. The communication network can be a wired network or a wireless network. For example, the communication network can be a local area network (LAN) or a wide area network (WAN). When the communication network is a LAN, such as a wireless local area network (WLAN), the communication network can be a wireless fidelity (WiFi) hotspot network, a Bluetooth network, a Zigbee network, etc., or a short-range point-to-point communication technology, such as near field communication (NFC). When the communication network is a wide area network (WAN), the communication system used can be, for example: Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Long Term Evolution (LTE), LTE Frequency Division Duplex (FDD), LTE Time Division Duplex (TDD), Universal Mobile Telecommunication System (UMTS), Worldwide Interoperability for Microwave Access (WiMAX), non-terrestrial network (NTN), 5th generation (5G) mobile communication system, or new radio access technology (NR). Among these, 5G mobile communication systems can include non-standalone (NSA) and / or standalone (SA) networks.

[0034] Upon detecting that the earphones have been removed from the earphone case 110, the earphone case 110 and the earphones can establish a communication connection to control the earphones' functions, such as adjusting the playback volume or content. The communication connection between the earphone case 110 and the earphones can be a Bluetooth connection, such as a Bluetooth Low Energy (BLE) connection.

[0035] In the earphone assembly 100, the third earphone 121 and the earphone case 110, and the fourth earphone 122 and the earphone case 110, can be paired before leaving the factory. Therefore, when the user first removes the third earphone 121 and the fourth earphone 122 from the earphone assembly 100 and wears them, both the third earphone 121 and the fourth earphone 122 can establish a wireless connection with the earphone case 110, allowing the user to control the third earphone 121 and the fourth earphone 122 through the earphone case 110.

[0036] The earphones in the earphone assembly 100 can also connect to other electronic devices besides the earphone case 110 to play audio from the connected electronic device, or to transmit the collected audio to the connected electronic device.

[0037] With increasingly frequent global exchanges, users of different languages ​​need to communicate in real time in daily life and work. In order to overcome language barriers and ensure smooth communication, various translation tools have emerged.

[0038] When using electronic devices for translation, users can pre-set the desired translation language to ensure that the translated speech uses the language that meets their needs.

[0039] Figure 2 This is a schematic flowchart illustrating a communication method. User A and User B use different languages. For example, User A's native language is Chinese, and User B's native language is English. User A and User B are conversing, and the headset component can be used as a translation tool.

[0040] Step S201: The earphone box 110 obtains the first language and the second language set by the user.

[0041] Users who can set the first and second languages ​​can be User A and / or User B. For example, User A can set the first language to be used by User A, and User B can set the second language to be used by User B.

[0042] The second language can be the language used by user A, such as Chinese. The second language can be the language used by user B, such as English.

[0043] For example, the headphone case 110 may include a display screen. User A can set the first language to Chinese and the second language to English in the translation settings interface displayed on the headphone case 110. Alternatively, the headphone case 110 can establish a communication connection with other electronic devices. The user can set the first and second languages ​​in the translation settings interface displayed on the other electronic device. The other electronic device can send language information to the headphone case 110. The language information can indicate the first and second languages.

[0044] User A can hand the third earphone 121 to user B. After that, steps S202 to S208, as well as steps S209 and S215, can be performed.

[0045] In step S202, the third earphone 121 acquires the second language speech of user B, obtaining the first acquired audio. The speech content of user B is expressed in the second language, namely English.

[0046] In step S203, the third earphone 121 sends the first audio recording to the earphone box 110.

[0047] In step S204, the headphone box 110 translates the first acquired audio, converting it from the second language to the first language to obtain the first translated audio.

[0048] The content of the first translated audio is expressed in the first language, namely Chinese. The semantics expressed by the first collected audio and the first translated audio are consistent.

[0049] In other words, the earphone box 110 translates the first acquired audio, so that the content in the first acquired audio expressed in the second language is translated into the first translated audio expressed in the first language. The semantics expressed in the second language in the first acquired audio and the semantics expressed in the first language in the first translated audio can be the same.

[0050] In step S205, the earphone box 110 sends the first translated audio to the fourth earphone 122.

[0051] In step S206, the fourth earphone 122 plays the first translated audio.

[0052] In step S207, the fourth earpiece 122 collects the user A's speech in the first language to obtain the second audio recording. The content of the second audio recording is expressed in the first language, namely Chinese.

[0053] In step S208, the fourth earphone 122 sends the second audio recording to the earphone box 110.

[0054] In step S209, the headphone box 110 translates the second acquired audio, converting it from the first language to the second language to obtain the second translated audio.

[0055] The content of the second translated audio is expressed in a second language, namely English. The semantics expressed by the second acquired audio and the second translated audio are consistent.

[0056] In step S210, the earphone box 110 sends the first translated audio to the third earphone 121.

[0057] In step S211, the third earphone 121 plays the first translated audio.

[0058] Through steps S202 to S206, the earphone component can capture user B's voice and play the translation of user B's voice back to user A. Through steps S207 to S211, the earphone component captures user A's voice and plays the translation of user A's voice back to user B. Thus, user A and user B can achieve real-time cross-language communication.

[0059] However, the language used by both parties in the translation process needs to be set by the user, which is quite complicated for the user.

[0060] In view of this, embodiments of this application provide an audio processing method that reduces the complexity of user operations and improves user experience.

[0061] The communication method provided in this application will be explained in detail below.

[0062] Figure 3 This is a schematic flowchart illustrating an audio processing method provided in an embodiment of this application. The audio processing method provided in this embodiment can be applied to a first earphone box in a communication system. Figure 4 As shown, the communication system includes a first earphone assembly 410 and a second earphone assembly 420. The first earphone assembly 410 may include a first earphone 411 and a first earphone case 412. The first earphone case 412 is used to store the first earphone 411. The second earphone assembly may include a second earphone 421 and a second earphone case 422. The second earphone case 422 is used to store the second earphone 421.

[0063] like Figure 3 As shown, the audio processing method includes steps S310 to S360, and each step is explained in detail below.

[0064] In step S310, in response to the trigger operation, a communication connection is established with the second headphone box.

[0065] The communication connection between the first and second earphone cases can also be called a communication connection, which can be an NFC connection or a Bluetooth connection.

[0066] Both the first and second earphone cases can contain Bluetooth and NFC modules. The NFC and Bluetooth modules can be used for short-range communication.

[0067] A triggering operation can be an action that brings the first and second earphone cases close together. For example, a triggering operation can make the distance between the first and second earphone cases less than or equal to the NFC sensing distance. In other words, in response to the distance between the first and second earphone cases being less than or equal to the NFC sensing distance, the first and second electronic devices can establish an NFC connection based on NFC functionality. The NFC sensing distance can be understood as the NFC communication distance threshold.

[0068] The position of the earphone case is controlled by the user. Both the user of the first earphone unit and the user of the second earphone unit can independently move their respective earphone cases. NFC's sensing distance is extremely short, typically less than or equal to 4 centimeters (cm). Therefore, if the distance between the first and second earphone cases is less than the NFC sensing distance, the users of both earphone units must actively bring their cases closer together; this action is an explicit and conscious operation. An NFC connection is only triggered when the user intentionally touches the earphone cases. The establishment of an NFC connection can be understood as a connection based on the user's triggering action. The user of the first earphone case and / or the user of the second earphone case can adjust the relative positions of the first and second earphone cases, bringing them closer together.

[0069] Both the first and second earphone cases can transmit Bluetooth signals periodically or non-periodically. When one earphone case detects that the signal strength of the first Bluetooth signal is greater than or equal to a signal strength threshold, it can establish a Bluetooth connection with the other earphone case that transmitted the first Bluetooth signal. The Bluetooth connection established between the first and second earphone cases can be understood as a connection between the first and second earphone cases.

[0070] The earphone case transmits a Bluetooth signal; the closer the earphone is to the earphone case, the stronger the signal. The signal strength threshold can be the highest signal strength when the distance between the earphone cases is less than or equal to a preset distance threshold. The preset distance threshold could be, for example, 10cm. If the first earphone case detects a Bluetooth signal with a signal strength greater than or equal to the signal strength threshold, then the distance between the first earphone case and the earphone case transmitting the Bluetooth signal is less than or equal to the preset distance threshold. The signal transmission distance that makes the Bluetooth signal strength less than or equal to the preset distance threshold can be understood as the preset distance threshold corresponding to that signal strength threshold.

[0071] When the distance between the first and second earphone cases is small enough that the signal strength of the first Bluetooth signal received by the first earphone case is greater than or equal to a signal strength threshold, the second earphone case, which sent the first Bluetooth signal, establishes a Bluetooth connection, serving as a communication link between the first and second earphone cases. Therefore, the triggering operation where the distance between the first and second earphone cases is less than or equal to a preset distance threshold can be understood as a conscious user action. Thus, the establishment of a Bluetooth connection between the first and second earphone cases can be understood as a response of the first and second earphone cases to the user's action, meeting the user's needs.

[0072] For example, if an earphone case detects that the signal strength of a first Bluetooth signal is greater than or equal to a signal strength threshold, and no NFC connection has been established with the earphone case, it can establish a Bluetooth connection with the earphone case that sent the first Bluetooth signal. The preset distance threshold corresponding to the signal strength threshold can be less than or equal to the NFC communication distance threshold.

[0073] For example, the earphone case can periodically send Bluetooth signals. If an earphone case detects that the signal strength of the first Bluetooth signal is greater than or equal to a signal strength threshold, and has not established an NFC connection with other earphone cases, it can establish a Bluetooth connection with the earphone case that sent the first Bluetooth signal.

[0074] Compared to NFC, Bluetooth communication consumes more power. Two headphone cases can prioritize establishing an NFC connection.

[0075] If the preset distance threshold corresponding to the signal strength threshold is less than or equal to the communication distance threshold of near-field communication (NFC), and the distance between the two headphone cases is less than or equal to the NFC sensing distance, but the two headphone cases do not establish an NFC connection, it may be due to an abnormality in the establishment of the NFC connection, for example, the NFC module in one of the headphone cases may be damaged. To respond to user triggers that bring the two headphone cases close together, the two headphone cases can establish a Bluetooth connection, thereby responding to user actions and performing subsequent processing, avoiding impact on subsequent communication processes.

[0076] For example, if an earphone box detects that the signal strength of the first Bluetooth signal is greater than or equal to the signal strength threshold, and no NFC connection has been established with other earphone boxes, the earphone can also output a prompt message.

[0077] After establishing an NFC connection, the first and second earphone cases can determine a pairing identifier through the NFC connection.

[0078] In some embodiments, the second earphone case may send a second device identifier of the second earphone case to the first earphone case. The first earphone case may send a first device identifier of the first earphone case to the second earphone case. The pairing identifier may include a first device identifier box and a second device identifier.

[0079] The device identifier of the headphone case can be the headphone case's media access control (MAC) address, Bluetooth address, universally unique identifier (UUID), etc.

[0080] By utilizing the existing NFC connection, the first and second earphone components can quickly exchange key parameters required for Bluetooth connection, avoiding the time-consuming Bluetooth broadcast scanning or the user operation required for pairing code input, thus improving the efficiency of establishing a Bluetooth connection.

[0081] In other embodiments, the first and second earphone cases can negotiate via NFC connection to determine a pairing identifier.

[0082] Based on the pairing identifier, the first and second earphone cases can establish a Bluetooth connection. For example, one of the earphone cases can send a Bluetooth connection request. The Bluetooth connection request may include the pairing identifier. Upon receiving a Bluetooth connection request, the earphone case receiving the request can establish a Bluetooth connection with the earphone case that sent the request.

[0083] Compared to the sensing distance of NFC, Bluetooth communication has a larger communication distance threshold. By using Bluetooth for data transmission during subsequent translation processes, users do not need to maintain a distance between the first and second earphone cases that is consistently less than or equal to the NFC sensing distance. This reduces the demands on user operation, lowers operational complexity, and improves the user experience.

[0084] By establishing an NFC connection between the first and second earphone cases, a pairing identifier is determined. During the Bluetooth connection establishment process, the pairing target is identified based on the pairing identifier, thus avoiding interference from other devices.

[0085] Step S320: Obtain the first original speech collected by the first earphone, and obtain the first language corresponding to the first original speech based on the first original speech.

[0086] For example, in response to a trigger operation, the first earphone box can switch to a translation pairing mode. In translation pairing mode, the first earphone box can establish a communication connection with the second earphone box. In translation pairing mode, the first earphone box can process the first original speech to obtain the first language corresponding to the first original speech.

[0087] The first earphone box can perform language identification (LID) on the first original speech, and obtain the language identification result of the first original speech. Based on the language identification result of the first original speech, the first earphone box can determine the first language corresponding to the first original speech.

[0088] In the process of language identification, the original audio is processed through windowing, fast Fourier transform (FFT), and Mel filter to extract acoustic features that reflect the essence of sound, such as Mel-frequency cepstral coefficients (MFCCs). By processing these acoustic features using a language identification model, the language used in the original audio can be determined.

[0089] Language identification models can include an extraction model based on a Gaussian mixture model (GMM)-universal background model (UBM), a global variability space, and a classifier. In the extraction model, the GMM-UBM is trained using massive amounts of speech data from multiple speakers and languages, with the number of internal Gaussian components typically between 256 and 2048. The UBM is not specific to any particular speaker or language; it is used only to describe the acoustic feature distribution of general speech. For a speech segment, its acoustic features are calculated relative to each Gaussian component of the UBM, yielding zero-order and first-order statistics (posterior probability summation and weighted feature sum). These statistics are then mapped to a low-dimensional vector, i.e., an i-vector, through a pre-trained global variability space (also known as a T matrix). Thus, a speech segment is transformed into a fixed-length, compact feature vector, i-vector. Classifying the i-vector using a classifier determines the language of the speech segment.

[0090] Language recognition models can also be deep neural networks, such as convolutional neural network (CNN) models, time delay neural network (TDNN) models, or Transformer models.

[0091] Language identification models distinguish languages ​​by learning the subtle differences in pronunciation habits, phoneme distribution, and prosodic features among different languages.

[0092] The first original speech can use one or more languages. That is, the language identification result of the first original speech can represent one or more candidate languages.

[0093] When the language recognition result of the first original speech indicates multiple candidate language types, the first earphone box can determine the first language from among the multiple candidate language types. The speech frame corresponding to the first language accounts for the largest proportion in the first original speech.

[0094] A speech frame corresponding to a candidate language can be understood as a speech frame within an audio segment of that candidate language that has been identified. An audio segment may include one or more speech frames. The proportion of a speech frame corresponding to a candidate language in the first original speech can be understood as the proportion of the duration of the speech frame corresponding to that candidate language to the total duration of the first original speech.

[0095] Actual speech may contain environmental noise, brief slips of the tongue, or a small number of loanwords (such as a mix of Chinese and English). Choosing the language with the highest percentage can avoid misjudging the primary language due to these brief segments and reduce mistranslation. In continuous dialogue, the language may change, but usually one language is dominant. Based on the criterion of the highest percentage, the translation language can remain stable, avoiding frequent switching of the target language due to recognition fluctuations in individual frames.

[0096] If the language recognition result indicates that there are multiple candidate language types, and the difference in the proportion of the speech frames corresponding to these multiple candidate language types in the first original speech is less than or equal to a preset difference threshold, the first earphone box can prompt the user to specify the language type. The first earphone box can then use the language specified by the user as the first language.

[0097] If the difference in the proportion of speech frames corresponding to multiple candidate languages ​​in the first original speech is less than or equal to a preset difference threshold, meaning the frequency difference between the various candidate languages ​​used in the first original speech is small, the first earpiece component can output a prompt message. The prompt message can indicate the multiple candidate languages. The user can select from the multiple candidate languages ​​or set other languages ​​besides the multiple candidate languages.

[0098] Addressing balanced mixed-language scenarios: In some speech, multiple languages ​​are mixed, with no clearly defined dominant language. In such cases, if the system automatically selects the language with the slightly higher proportion, it may still make an incorrect selection due to statistical fluctuations or noise. Once an incorrect selection is made, the subsequent translation results will be completely wrong, requiring the user to start over. User-specified translations can accurately reflect the user's translation intent.

[0099] Step S330: Receive a second audio signal sent by the second earphone box. The second audio signal includes the second original speech collected by the second earphone and the second language type information. The second language type information is used to indicate the second language corresponding to the second original speech.

[0100] After receiving the second original voice message from the second earphone, the second earphone box can perform language recognition on the second original voice message to obtain the language recognition result. Based on the language recognition result of the second original voice message, the first earphone box can determine the second language corresponding to the second original voice message.

[0101] Step S340: Call the corresponding local translation file based on the first language and the second language.

[0102] The headphone case can contain multiple translation files, each corresponding to a different language combination. Based on the correspondence between these combinations and the corresponding translation files, the translation file for the first and second language combinations can be determined.

[0103] Step S350: Translate the second original speech into the first language using a local translation file to obtain the second translated speech.

[0104] Step S360: Send the second translated voice message to the first earphone for playback.

[0105] The first earphone box can send a second translated voice message to the first earphone, so that the first earphone plays the second translated voice message.

[0106] The first earphone box can also send a first audio signal to the second earphone box. The first audio signal includes a first original speech and first language type information. The first language type information indicates the first language corresponding to the first original speech.

[0107] Therefore, the second earphone box can contain multiple translation files. The second earphone box can call upon the translation file corresponding to the combination of the first and second languages ​​to translate the original first speech, obtaining the first translated speech. The second earphone box can then send the first translated speech to the second earphone. The second earphone box can then play the first translated speech.

[0108] It should be understood that the communication system can continuously collect, transmit, and translate raw speech. In some embodiments, for each instance of raw speech collected by the first earphone, the first earphone box can perform language identification to determine a first language. In other embodiments, after detecting a trigger operation, the first earphone box determines the first language based on the first raw speech transmitted by the first earphone for the first time. For subsequent transmissions of raw speech by the first earphone, language identification may not be performed. In still other embodiments, the first earphone box can perform language identification on the raw speech collected by the first earphone periodically or non-periodically.

[0109] The first earphone box can send first speech type information to the second earphone box each time a first language is determined. The first speech type information indicates the first language determined based on the new first original speech.

[0110] Alternatively, the first earphone box may transmit the new first original speech if the first language determined based on the new first original speech is the same as the previously determined first language, without transmitting the first speech category information. If the first language determined based on the new first original speech is different from the previously determined first language, the first earphone box may transmit both the new first original speech and the first speech category information. The first speech category information indicates the first language determined based on the new first original speech.

[0111] Therefore, if the first and second languages ​​remain unchanged, the second earphone box continues to translate the first original speech using the previously determined translation file. If the first and / or second languages ​​change, the second earphone box determines a new translation file based on the new combination of the first and second languages ​​and uses the newly determined translation file to translate the first original speech. The first language not changing could mean that no new first language type information has been received, or that the language indicated by the newly received first language type information is the same as the language indicated by the previously received first language type information.

[0112] Similar to the processing of the first earphone box, in some embodiments, the second earphone box can perform language recognition to determine the second language for each second original voice recording captured by the second earphone. In other embodiments, after detecting a trigger operation, the second earphone box determines the second language based on the first second original voice recording sent by the second earphone. Language recognition is not required for subsequent second original voice recordings sent by the second earphone. In still other embodiments, the second earphone box can perform language recognition on the second original voice recording captured by the second earphone periodically or non-periodically.

[0113] If the second language determined based on the new second original speech is the same as the previously determined second language, the second earphone box can send the new second original speech but not the second speech type information. If the second language determined based on the new second original speech is different from the previously determined second language, the second earphone box can send the new second original speech and the second speech type information, whereby the second speech type information indicates the second language determined based on the new second original speech.

[0114] Therefore, if the first earphone box does not change the first or second language, it continues to translate the second original speech using the previously determined translation file. If the first and / or second language changes, the first earphone box determines a new translation file based on the new combination of the first and second languages ​​and uses the newly determined translation file to translate the second original speech. The second language not changing could mean that no new second language information has been received, or that the language indicated by the newly received second language information is the same as the language indicated by the previously received second language information.

[0115] For the first earphone case, if the first earphone is detected being housed in the first earphone case, the translation pairing mode can be exited, and the communication connection with the second earphone case can be disconnected. Similarly, for the second earphone case, if the second earphone is detected being housed in the second earphone case, the translation pairing mode can be exited, and the communication connection can be disconnected.

[0116] The earbuds are contained within the charging case, meaning they are not removed by the user. When the earbuds are inside the charging case, exiting the translation pairing mode and disconnecting from other earbud components via Bluetooth reduces power consumption.

[0117] The headphone component can also update translation files.

[0118] The translation file, also known as a translation model, translation agent, or translation service module, has a built-in independent translation algorithm model, language library, and speech recognition and synthesis unit. The translation file can be deployed locally on the headphone box, completing speech translation without cloud intervention.

[0119] In some embodiments, the communication system may further include a cloud device 430. A cloud device can be understood as a cloud server.

[0120] The first headset component can send a first version number of a first translation file in the first headset component to the server. Upon receiving the first version number, the server can determine whether the version represented by the first version number is the latest version. For example, the server can compare the first version number with a recorded second version number. If the first version number is less than the second version number, the server can determine that the version represented by the first version number is not the earliest version. If the first version number is not the latest version number, the server can send a second translation file to the first headset component. The second version number of the second translation file represents the latest version. That is, the second translation file is the latest version file.

[0121] Upon receiving the second translation file, the first earpiece assembly can install the second translation file and, after installation, translate the speech sent by other earpiece assemblies using the second translation file.

[0122] When in translation pairing mode, the earphone case can avoid updating translation files, thus preventing interference with ongoing voice translation. In translation pairing mode, the earphone case can also disconnect from the server, thereby reducing power consumption.

[0123] In other embodiments, the first earphone component can send a first version number of a first translation file from the first earphone component to the second earphone component. The second earphone component can also send a third version number of a third translation file from the second earphone component to the first earphone component. If the first and third version numbers are the same, the first earphone component can use the first translation file to translate the second original speech sent by the second earphone component. If an earphone component receives a version number lower than the version number of its translation file, that earphone component can also send its translation file to another earphone component. This other earphone component can receive and install the translation file with the larger version number. Thus, both earphone components can use the newer version of the translation file from the first and third translation files for translation.

[0124] It should be understood that the translation files corresponding to different language combinations in the headphone component can have different version numbers. The headphone component can update each translation file individually. Alternatively, the collection of translation files can have a single version number. The translation files include multiple translation files corresponding to multiple language combinations. The headphone component can update the collection of translation files.

[0125] The communication method provided in this application embodiment allows the first earphone component to determine a first language based on the collected first raw speech, call the translation file corresponding to the first language and the second language sent by the second earphone box to translate the second raw speech sent by the second earphone box into the first language, and send the translation result to the first earphone for playback. Therefore, the user does not need to set the language used for translation, reducing the complexity of user operation and improving the user experience.

[0126] The following is combined with Figure 5 The audio processing method provided in the embodiments of this application will be described in detail.

[0127] Figure 5 This is a schematic flowchart of an audio processing method provided in this application.

[0128] Upon detecting a trigger operation, the first earphone case in the first earphone assembly can perform step S501 with the second earphone case in the second earphone assembly. The trigger operation can be an operation that causes the distance between the first earphone case and the second earphone case to be less than or equal to a distance threshold. The distance threshold can be, for example, the sensing distance of NFC.

[0129] Step S501: The first earphone case and the second earphone case establish an NFC connection.

[0130] In step S502, the first and second earphone cases are connected via NFC to determine a pairing identifier.

[0131] In step S503, the first and second earphone boxes establish a Bluetooth connection based on a pairing identifier.

[0132] One of the first and second earphone components can establish a unique point-to-point binding relationship based on a pairing identifier, locking the pairing object and avoiding interference from other devices during the Bluetooth connection establishment process.

[0133] Step S504: The first earphone collects the first raw voice.

[0134] Step S505: The first earphone box receives the first original voice message sent by the first earphone box.

[0135] Step S506: The second earphone collects the second original voice.

[0136] In step S507, the second earphone box receives the second original voice message sent by the second earphone box.

[0137] If a trigger operation is detected, the first earphone box can also proceed to steps S508 to S509.

[0138] The first original speech can be the speech collected by the first earphone.

[0139] Step S508: The first earphone box determines the first language based on the first original voice.

[0140] In step S509, the first earphone box sends a first audio signal to the second earphone box.

[0141] The first audio signal includes information about the first language type and the first original speech. The information about the first language type indicates the first language.

[0142] If a trigger operation is detected, the second earphone box can also proceed to steps S507 to S509.

[0143] Step S510: The second earphone box determines the second language based on the second original speech.

[0144] In step S511, the second earphone box sends a second audio signal to the first earphone box.

[0145] The second audio signal includes second language information and the second original speech. The second language information indicates the second language.

[0146] Upon receiving the second audio signal sent by the second earphone box, the first earphone box can proceed to steps S512 to S514.

[0147] In step S512, the first earphone box calls the local translation files corresponding to the first and second languages.

[0148] In step S513, the first earphone box translates the second original speech using a local translation file to obtain the second translated speech in the first language.

[0149] In step S514, the first earphone box sends the second translated voice to the first earphone.

[0150] Step S515: The first earphone plays the second translated audio.

[0151] Upon receiving the first audio signal sent by the first headphone box, the second headphone box can proceed to step S510.

[0152] In step S516, the second earphone box calls the local translation files corresponding to the first and second languages.

[0153] In step S517, the second earphone box translates the first original speech using a local translation file to obtain the first translated speech in the second language.

[0154] In step S518, the second earphone box sends the first translated voice to the first earphone.

[0155] Step S519: The second earphone plays the first translated audio.

[0156] The audio processing method provided in this application can reduce the complexity of operation for users during voice translation using a headphone component.

[0157] Language recognition requires processing speech over a relatively long period. Therefore, translation based on the language determined through language recognition suffers from poor timeliness. To improve translation timeliness, after establishing a communication connection between the first and second earphone components, the first earphone component can send first language information to the second earphone component. This first language information indicates the first language of the first earphone component.

[0158] The language of the headphone component can be the system language of the first headphone component or other preset languages. For example, the user of the headphone component can set the language of the headphone component before establishing a communication connection with other headphone components. The user of the headphone component can set the language of the headphone component through buttons or a touch screen on the headphone component's charging case. The user of the headphone component can also set the language of the first headphone component through an electronic device that has a communication connection with the headphone component.

[0159] The system language of the headphone assembly can be the language used for voice prompts within the headphones. Voice prompts might include phrases like "Connected" or "Low battery." If the headphone case includes a display, the system language of the headphone assembly can also be the language of the interface displayed on the screen. The system language of the headphone assembly is usually uniform; that is, the voice prompts and screen display can use the same language.

[0160] The first earpiece assembly can perform language detection on the second original speech transmitted by the second earpiece to determine the language used in the second original speech. Alternatively, before translating the second original speech, the first earpiece assembly can also receive second language type information transmitted by the second earpiece assembly. The second language type information indicates the second language of the second earpiece assembly.

[0161] The second original speech sent by the second earphone component is translated based on the second language type information indicated by the second language type information sent by the second earphone component, thereby reducing the processing complexity of the first earphone component.

[0162] When the second earpiece transmits the second original voice message, the connection between the first and second earpiece components may be the same or different from when the second earpiece transmits the second language information. For example, the connection between the first and second earpiece components can be Bluetooth when both the second earpiece transmits the second original voice message and when it transmits the second language information. Alternatively, the connection between the first and second earpiece components can be NFC when the second earpiece transmits the second language information, while the connection can be Bluetooth when the second earpiece transmits the second original voice message.

[0163] When a user brings the first earphone assembly and the second earphone assembly close together, such that the distance between the first earphone assembly and the second earphone assembly is less than or equal to the NFC communication distance threshold, the first earphone assembly and the second earphone assembly can establish an NFC connection.

[0164] The second earpiece can send second language information to the first earpiece via NFC connection. If the second language indicated by the second language information differs from the first language of the first earpiece, the first and second earpieces establish a Bluetooth connection. The second earpiece can then send a second original voice message to the first earpiece via Bluetooth.

[0165] Compared to Bluetooth communication, NFC has a smaller communication distance threshold. During conversations between the user of the first earphone assembly and the user of the second earphone assembly, maintaining the NFC connection between them requires a relatively small distance between the two earphone assemblies. This places higher demands on the user's control over the relative positions of the first and / or second earphone assemblies, making user operation more complex.

[0166] If the second language information received indicates that the second language of the second earphone component differs from the first language of the first earphone component, the first earphone component can determine that a translation requirement exists and thus establish a Bluetooth connection with the second earphone component. Maintaining this Bluetooth connection between the first and second earphone components requires less distance between them, thereby reducing the complexity of user operation.

[0167] For example, the second earphone component can also send its second device identifier to the first earphone component via NFC connection. Based on the second device identifier, the first earphone component can establish a Bluetooth connection with the second earphone component. Alternatively, the first earphone component can also send its first device identifier to the second earphone component via NFC connection to establish a Bluetooth connection with the second earphone component.

[0168] The following is combined with Figure 6 The audio processing method for the first and second earphone components when they interact with language information is described in detail.

[0169] Figure 6 This is a schematic flowchart of an audio processing method provided in an embodiment of this application.

[0170] This method can be applied to a communication system. The communication system includes a first earphone assembly and a second earphone assembly. The user of the first earphone assembly and the user of the second earphone assembly can be two parties engaged in a conversation. When the earphones in each earphone assembly are housed in an earphone case within that assembly, they can pair with the earphone case via Bluetooth. Therefore, after the earphones are removed from the earphone case, they can establish a Bluetooth connection with the earphone case.

[0171] The earphones in the headphone assembly have functions such as voice capture, voice playback, and Bluetooth communication.

[0172] The earphone case has a built-in high-performance main control chip, a translation document intelligent agent / translation service module that supports multiple language combinations, an NFC communication module, a Bluetooth communication module, a wireless networking module, a storage module, and a power management module. It has independent translation calculation, device pairing, data forwarding, and model update capabilities.

[0173] Upon detecting a trigger operation, the first earphone assembly and the second earphone assembly can proceed to step S601. The trigger operation can be to make the distance between the first earphone case and the second earphone case less than or equal to the NFC sensing distance.

[0174] Step S601: The first earphone assembly establishes an NFC connection with the second earphone assembly.

[0175] The distance between the first and second earphone components is less than or equal to the NFC communication distance threshold. Specifically, the distance between the first earphone case in the first earphone component and the second earphone case in the second earphone component is less than or equal to the NFC sensing distance. In other words, a user can establish an NFC connection by gently touching the first and second earphone components together.

[0176] In step S602, the first earphone component sends first language type information to the second earphone component via NFC connection. The first language type information indicates the first language as the system language type of the first earphone component.

[0177] In step S603, the second earphone component sends second language type information to the first earphone component via NFC connection. The second language type information indicates the system language type of the second earphone component.

[0178] In step S604, the first and second earphone cases are connected via NFC to determine a pairing identifier.

[0179] In step S605, the first and second earphone boxes establish a Bluetooth connection based on a pairing identifier.

[0180] If the distance between the first earphone assembly and the second earphone assembly is greater than a preset distance, the NFC connection between the first earphone assembly and the second earphone assembly can be disconnected.

[0181] In some cases, both the first and second earphone components can connect to the server. After a Bluetooth connection is established between the first and second earphone components, audio data transmission between them can proceed without going through the server.

[0182] After step S602, the second earphone assembly can proceed to step S606.

[0183] In step S606, the second earphone component determines the corresponding translation file based on the first language and the second language.

[0184] After step S603, the first earphone assembly can proceed to step S607.

[0185] In step S607, the first earphone component determines the corresponding translation file based on the first language and the second language.

[0186] Step S608: The first earphone component acquires the first raw voice.

[0187] The first earphone assembly can capture first raw speech through the first earphone in the first earphone assembly. The first earphone is worn by the user of the first earphone assembly.

[0188] In step S609, the first earphone assembly sends the first original voice to the second earphone assembly via Bluetooth connection.

[0189] The first earphone in the first earphone assembly can send the first original voice to the first earphone box in the first earphone assembly. The first earphone box can send the first original voice to the second earphone assembly via Bluetooth connection.

[0190] In step S610, the second earphone component translates the first original speech using the determined translation file to obtain the first translated speech in the second language.

[0191] The second earphone box in the second earphone assembly can translate the first original speech.

[0192] In step S611, the second earphone component plays the first translated voice.

[0193] The second earphone box in the second earphone assembly can send the first translated voice message to the second earphone in the second earphone assembly. The second earphone can then play the first translated voice message.

[0194] Step S612: The second earphone component acquires the second original voice.

[0195] The second earphone assembly can capture a second original voice through the second earphone.

[0196] In step S613, the second earphone assembly sends the second original voice to the first earphone assembly via Bluetooth connection.

[0197] The second earphone can send a second original voice message to the second earphone box. The second earphone box can send the second original voice message to the first earphone assembly.

[0198] In step S614, the first earphone component translates the second original speech using the determined translation file to obtain the second translated speech.

[0199] The first earphone box in the first earphone assembly can translate the second original speech.

[0200] In step S615, the first earphone component plays the second translated voice.

[0201] The first earphone box can send a second translated voice message to the first earphone. The first earphone can then play the second translated voice message.

[0202] For the headphone unit, upon detecting that the headphones have been placed in the charging case, the Bluetooth connection with other headphone units can be disconnected, the translation model can be put into sleep mode, and power consumption can be reduced. If the user of the headphone unit needs to communicate across languages ​​with other people again, the headphone unit can be touched to another person's headphone unit to enable the two headphone units to connect and perform translation processing.

[0203] With the headphone component connected to the server, the server can update the translation model in the headphone component.

[0204] The server is responsible for storing the latest translation models, algorithm patches, and language packages, remotely pushing updates to the smart earphone box, and can also store device identifiers to assist in device authorization and management. The server does not participate in real-time translation calculations.

[0205] The server can detect whether the translation module version set in the headset component is the latest version and send an update package to the headset component. The update package may include one or more of the following: the latest translation model, a new language package, and algorithm optimization patches. After receiving the update package, the headset component completes the local installation and upgrade. After the update is complete, the cloud connection is disconnected, but subsequent translation continues to run locally without affecting communication between users. By updating the translation model, improvements can be made in translation accuracy, support for new minority languages, dialect recognition, and speech synthesis tone, enhancing the translation experience without replacing hardware.

[0206] The audio processing method provided in this application embodiment enables real-time translation of the speech between the user of the first earphone component and the user of the second earphone component, with latency down to the millisecond level. The fluency of the communication is close to that of native language conversation, thus improving the user experience.

[0207] To improve the accuracy of the selected translation files and the overall accuracy of the translation results, the headset component can authenticate the speaker in the acquired audio.

[0208] After acquiring the second original speech, the second earpiece component can extract features from the second original speech to obtain second speech features. The second earpiece component can compare these second speech features with the second user's speech features. The second user can be a user who has set the language of the second earpiece component to their second language. If the similarity between the second speech features and the second user's speech features is greater than or equal to a preset similarity threshold, i.e., if the second speech features match the second user's speech features, the speaker in the audio acquired by the second earpiece component is considered to be the second user, and the second earpiece component can send second verification information to the first earpiece component.

[0209] The second earpiece component performs feature extraction on the second original speech, which can be done on all or part of the speech segments. For example, the second earpiece component can continuously collect speech. Each time the second original speech is transmitted by the second earpiece component, it can be understood as a speech segment in the speech collected by the second earpiece component. The second earpiece component can perform feature extraction on speech segments of a preset duration in each collected second original speech. The second earpiece component can also perform feature extraction on the collected speech periodically or non-periodically. For example, the second earpiece component can perform feature extraction on the most recently collected second original speech after a preset interval.

[0210] Speech features obtained through feature extraction can include voiceprint information and / or vocal behavior features. Voiceprint information can be understood as a set of feature parameters extracted from speech signals through algorithms that can characterize the speaker's unique identity. Voiceprint information can include one or more of the following: timbre, fundamental frequency profile, speech rate, phoneme duration, and prosody. Vocal behavior features, also known as articulation behavior features or articulation habit features, refer to personalized features related to the movement patterns, coordination methods, and dynamic timing of the speaker's articulation organs (such as the tongue, lips, palate, and glottis).

[0211] When the second user sets the language of the second earphone component to the second language, the second earphone component can collect the second user's voice and extract features from the second user's voice to obtain the second user's voice features.

[0212] For example, when the second user sets the language of the second earphone component to the second language, the second earphone component can output a prompt message to remind the second user to speak, so that the second earphone component can capture the second user's voice.

[0213] For example, when the second user sets the language of the second earphone assembly to a second language through the second earphone box of the second earphone assembly or an electronic device communicatively connected to the second earphone box, the second earphone assembly can collect the second user's voice through the second earphone in the second earphone assembly.

[0214] The first earpiece component can also extract features from the first original speech to obtain first speech features. If the similarity between the first speech features and the first user's first user speech features is greater than or equal to a preset similarity threshold, the first earpiece component can send first verification information and the first original speech to the second earpiece component. If the similarity between the first speech features and the first user's first user speech features is less than the preset similarity threshold, the first earpiece component can send the first original speech to the second earpiece component, but will no longer send the first verification information.

[0215] When the similarity between the first voice feature and the first user's voice feature is greater than or equal to a preset similarity threshold, and the second verification information is received, the first earphone component can call the translation file corresponding to the combination of the second language and the first language to translate the second original voice and obtain the second translated voice.

[0216] The second earpiece component compares the second speech features of the second original speech with the second user's speech features of the second user whose language is set to the second language. If the comparison result indicates a high similarity, meaning that the second original speech is the second user's speech, the second verification information is sent to the first earpiece component to instruct the first earpiece component to translate the second original speech based on the second language, thereby making the translation of the first earpiece component more accurate.

[0217] If the first earphone component does not receive the second verification information, it can perform language recognition on the second original speech to determine the language used by the second original speech, and then translate the second original speech to obtain the second translated speech based on the recognized language used by the second original speech.

[0218] If the first similarity between the first speech feature and the first user's first user speech feature is less than a preset similarity threshold, the first earphone component may also output a prompt message to remind the user of the first earphone component to set the first translation language. The operation of setting the first translation language can be understood as setting the first language of the first earphone component. If the first language of the first earphone component is the system language of the first earphone component, the operation of setting the first translation language can also be understood as setting the language required for the current translation, rather than setting the first language of the first earphone component.

[0219] Upon detecting a setting operation for a first translation language, the first earpiece component can send third language information to the second earpiece component. The third language information indicates the first translation language specified in the setting operation.

[0220] Upon receiving third language information and the first original speech, the second earphone component can, based on the first translation language indicated by the third language information and the identity verification result of the second original speech, invoke a translation file to translate the first original speech, obtaining a first translated speech. The second earphone component can then play the first translated speech. Therefore, the translation result from the second earphone component is more accurate.

[0221] In response to the trigger operation, the second earphone assembly can also authenticate the user of the second earphone assembly.

[0222] If the second similarity between the second voice feature and the second user's second voice feature is less than a preset similarity threshold, the second earpiece component can also output a prompt message to remind the user of the second earpiece component to set a second translation language. In response to the second setting operation for the second translation language, the first earpiece component can send fourth language type information to the second earpiece component. The fourth language type information indicates the second translation language set by the second setting operation.

[0223] Upon receiving the fourth language type information and the second original speech sent by the second earphone component, the first earphone component can, based on the second translation language represented by the fourth language type information and the first language or the first translation language, call the corresponding translation file to translate the second original speech and obtain the second translated speech.

[0224] The above text combined Figures 1 to 6 The audio processing method of the embodiments of this application is described in detail below. Figure 7 and Figure 8 This document describes in detail the device embodiments of this application. It should be understood that the audio processing device in the embodiments of this application can execute the various methods described in the foregoing embodiments of this application. That is, the specific working processes of the various products described below can be referred to the corresponding processes in the foregoing method embodiments.

[0225] Figure 7 This is a schematic structural diagram of an audio processing device provided in an embodiment of this application. The audio processing device 1000 may include a communication unit 1010 and a processing unit 1020.

[0226] The communication unit 1010 is used to establish a communication connection with the second earphone box in response to a trigger operation, the second earphone box being used to store the second earphone.

[0227] The processing unit 1020 is used to acquire the first original speech collected by the first earphone, and to obtain the first language corresponding to the first original speech based on the first original speech.

[0228] The communication unit 1010 is further configured to receive a second audio signal sent by the second earphone box, the second audio signal including a second original voice collected by the second earphone and second language type information, the second language type information being used to indicate the second language corresponding to the second original voice.

[0229] The processing unit 1020 is also configured to call the corresponding local translation file based on the first language and the second language.

[0230] The processing unit 1020 is further configured to translate the second original speech into the first language using the local translation file to obtain the second translated speech.

[0231] The communication unit 1010 is also used to send the second translated voice to the first earphone for playback.

[0232] It should be noted that the aforementioned audio processing device 1000 is embodied in the form of a functional unit. The term "unit" here can be implemented in software and / or hardware, without specific limitations.

[0233] For example, a “unit” can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor) and memory for executing one or more software or firmware programs, combined logic circuitry, and / or other suitable components that support the described functions.

[0234] Therefore, the units of the various examples described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0235] Figure 8 This is a schematic diagram of the structure of an electronic device 1100 provided in an embodiment of this application. The electronic device 1100 can be an earphone assembly or an earphone case within an earphone assembly. Figure 8 As shown, the electronic device 1100 of this embodiment includes: at least one processor ( Figure 8 The document shows only one processor, namely a microcontroller unit (MCU) 1110, a battery manager 1120, an earphone charging module 1130, an interface module 1140, a status display module 1150, a first wireless communication module 1160, a second wireless communication module 1170, a memory 1180, a battery 1190, etc.

[0236] The memory 1180 is used to store a computer program. The computer program can run on the MCU 1110. When the MCU 1110 executes the computer program, it can implement the steps of any of the above method embodiments.

[0237] Those skilled in the art will understand that Figure 8This is merely an example of electronic device 1100 and does not constitute a limitation on electronic device 1100. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0238] In electronic device 1100, the processor can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0239] In some embodiments, memory 1180 may be an internal storage unit of electronic device 1100, such as a hard disk or memory of electronic device 1100. In other embodiments, memory 1180 may be an external storage device of electronic device 1100, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on electronic device 1100. Furthermore, memory 1180 may include both internal and external storage units of electronic device 1100. Memory 1180 is used to store operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory 1180 may also be used to temporarily store data that has been output or will be output.

[0240] The memory 1180 can have built-in flash memory for storing device identifiers, translation files, temporary voice cache, and update package data, ensuring fast reading of translation data.

[0241] The first wireless communication module 1160 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on electronic devices 1100. The first wireless communication module 1160, also known as a wireless networking module, is used to connect to a cloud management server to download the latest translation models, algorithm update packages, and language expansion packages. It can be turned off during normal real-time communication to reduce power consumption.

[0242] The second wireless communication module 1170 can provide wireless communication solutions for use on the electronic device 1100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The second wireless communication module 1170 can also be called a short-range point-to-point communication module. For example, the second wireless communication module 1170 can integrate an NFC communication unit and a BLE Bluetooth communication unit. The NFC communication unit is used to quickly exchange device IDs and language information with other headphone boxes when triggered by a user's touch operation. The BLE communication unit is used for real-time point-to-point direct transmission of voice data, with high transmission rates and low latency, replacing traditional cloud relays.

[0243] The battery manager 1120 manages the built-in battery 1190, providing voltage-divided power to each module, optimizing low power consumption, and ensuring battery life for touch pairing, local translation, and data transmission.

[0244] The status display module 1150 can be a light emitting diode (LED) indicator light used to display power status, network connection status, touch pairing status, translation module running status, and update progress.

[0245] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0246] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0247] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement any of the above-described method embodiments.

[0248] This application provides a computer program product that, when running, can implement any of the above-described method embodiments.

[0249] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some possible implementations, the computer-readable medium cannot be an electrical carrier signal or a telecommunication signal.

[0250] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0251] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0252] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0253] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0254] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An audio processing method, characterized in that, Applied to a first earphone case, the first earphone case being used to store a first earphone, the method includes: In response to a trigger operation, a communication connection is established with a second earphone case, which is used to store the second earphone; Acquire the first original speech collected by the first earphone, and obtain the first language corresponding to the first original speech based on the first original speech; The system receives a second audio signal sent by the second earphone box. The second audio signal includes a second original voice collected by the second earphone and second language type information. The second language type information is used to indicate the second language corresponding to the second original voice. Based on the first language and the second language, call the corresponding local translation file; The second original speech is translated into the first language using the local translation file to obtain the second translated speech; The second translated audio is sent to the first earphone for playback.

2. The method according to claim 1, characterized in that, The step of obtaining the first language corresponding to the first original speech based on the first original speech includes: The language used in the first original speech is identified to obtain the language identification result; If the language identification result indicates that there are multiple candidate languages, and the difference in the proportion of the speech frames corresponding to the multiple candidate languages ​​in the first original speech is less than or equal to a preset difference threshold, the user is prompted to specify a language. Use the language specified by the user as the first language.

3. The method according to claim 1, characterized in that, The step of obtaining the first language corresponding to the first original speech based on the first original speech includes: The language used in the first original speech is identified to obtain the language identification result; When the language identification result indicates that there are multiple candidate languages, the target language is determined from the multiple candidate languages, and the speech frame corresponding to the target language has the largest proportion in the first original speech. The target language is designated as the first language.

4. The method according to claim 1, characterized in that, The step of establishing a communication connection with the second earphone box in response to the trigger operation includes: In response to the trigger operation, switch to translation pairing mode and establish a communication connection with the second earphone box; The step of obtaining the first original voice collected by the first earphone is: in the translation pairing mode, obtaining the first original voice collected by the first earphone.

5. The method according to any one of claims 1 to 4, characterized in that, Both the first and second earphone cases are equipped with near-field communication functionality; The step of establishing a communication connection with the second earphone box in response to the trigger operation includes: In response to the distance between the first and second earphone boxes being less than or equal to the sensing distance of near-field communication, a near-field communication connection is established with the second earphone box based on the near-field communication function; The pairing identifier is determined through the near-field communication connection; Based on the pairing identifier, a Bluetooth connection is established with the second earphone case; Receiving the second audio signal sent by the second earphone box includes: receiving the second audio signal via the Bluetooth connection.

6. The method according to claim 5, characterized in that, The first and second earphone cases are also equipped with Bluetooth communication functionality; The step of establishing a communication connection with the second earphone box in response to the trigger operation includes: In response to the distance between the first and second earphone boxes being less than or equal to the sensing distance of near-field communication, if a communication connection cannot be established with the second earphone box based on the near-field communication function, a communication connection is established with the second earphone box based on the Bluetooth communication function.

7. The method according to any one of claims 1 to 4, characterized in that, After acquiring the first raw speech collected by the first earphone and obtaining the first language corresponding to the first raw speech based on the first raw speech, the method further includes: A first audio signal is sent to the second earphone box. The first audio signal includes the first original voice and first language type information, wherein the first language type information indicates the first language.

8. An earphone case, characterized in that, include: The housing has a cavity for storing the earphones; The memory and processor are located within the housing; The processor is configured to retrieve and run a computer program from the memory, causing the headphone case to perform the audio processing method of any one of claims 1 to 7 in the form of a first headphone case.

9. The earphone case according to claim 8, characterized in that, The earphone case also includes a near-field communication module. The processor is also configured to; Based on the near-field communication module, a communication connection is established with other headphone boxes sensed by the near-field communication module.

10. A communication system, characterized in that, It includes a first earphone case, a first earphone, a second earphone case, and a second earphone, wherein the first earphone case is used to perform the method of any one of claims 1-7.

Images