Audio processing method, earphone and terminal equipment

Abstract

The invention provides an audio processing method, an earphone and a terminal device, and relates to the technical field of audio, the earphone comprises a plurality of microphones, in the scheme, after the earphone establishes a wireless communication connection with the terminal device, multipath audio signals collected by the plurality of microphones are transmitted to the terminal device through the wireless communication connection; and after the terminal equipment receives the multiple paths of audio signals transmitted by the earphone, noise reduction processing is carried out on the multiple paths of audio signals to obtain a target audio signal. According to the scheme, noise reduction and other processing can be carried out on the audio signal collected by the earphone by utilizing the large computing power of the terminal equipment, so that the noise reduction effect can be effectively improved, and the pickup effect is further improved.

Description

Technical Field

[0001] The present application relates to the field of audio technology, and in particular to an audio processing method, earphones, and terminal equipment. Background Art

[0002] Headphones are commonly used accessories for mobile phones, computers and other terminal devices. When using a terminal device, users can use the headphones connected to the terminal device to play audio, make calls, record audio, etc.

[0003] Among them, the microphone on the headset is easily affected by factors such as environmental noise when picking up the user's voice, resulting in poor quality of the picked up audio signal, thereby affecting the user's voice pickup effect during calls, recordings, etc. Summary of the Invention

[0004] In view of this, the present application provides an audio processing method, headphones and terminal device for improving the sound pickup effect when a user uses headphones.

[0005] In order to achieve the above-mentioned objectives, in a first aspect, an embodiment of the present application provides an audio processing method, which is applied to headphones, wherein the headphones include multiple microphones, and the method can be executed by the headphones or by a component in the headphones (such as a chip or module).

[0006] The method includes: after establishing a wireless communication connection with a terminal device, collecting multi-channel audio signals through multiple microphones; then transmitting the collected multi-channel audio signals to the terminal device through the wireless communication connection, so that the terminal device performs noise reduction processing on the multi-channel audio signals.

[0007] Among them, the terminal device can be a mobile phone, tablet computer, laptop computer, computer, smart TV, smart watch, car-mounted equipment, etc.

[0008] Noise reduction is an important method for improving sound pickup. It primarily involves identifying and separating speech from background noise, and suppressing background noise. The effectiveness of noise reduction is closely related to the noise reduction algorithm used. An inefficient noise reduction algorithm has limited processing power and identifies speech and background noise with low accuracy. This results in poor noise suppression (i.e., noise reduction) and speech impairment, both of which affect the final sound pickup. For example, the output speech (i.e., the target audio signal) may contain noise, be unclear, or be distorted. An efficient noise reduction algorithm can more accurately identify speech and background noise, resulting in better noise reduction and speech quality, and ultimately, better sound pickup. For example, the output speech may be clearer and have higher fidelity.

[0009] Efficient noise reduction algorithms are often complex and require significant computing resources to process audio signals in real time. Due to factors such as size and cost, headphones have limited computing power, making it difficult to support complex and efficient noise reduction algorithms. Consequently, the processing power of the noise reduction algorithms they employ is limited. Compared to headphones, terminal devices typically have significantly higher computing power, allowing them to run efficient and complex noise reduction algorithms.

[0010] In this method, after the headphones collect multi-channel audio signals through multiple microphones, the collected multi-channel audio signals are transmitted to the terminal device. In this way, the multi-channel audio signals can be noise-reduced by the terminal device to obtain the target audio signal. Compared with noise reduction of the collected multi-channel audio signals through headphones, noise reduction processing of the multi-channel audio signals by the terminal device can utilize the large computing power of the terminal device and adopt an efficient and complex noise reduction algorithm for noise reduction processing, thereby effectively improving the noise reduction effect and voice quality, and further effectively improving the sound pickup effect when the user uses headphones.

[0011] In a possible implementation of the first aspect, the method further includes: when the signal quality of the wireless communication connection between the headset and the terminal device does not meet the target condition, performing noise reduction processing on the collected multi-channel audio signals and then transmitting them to the terminal device via the wireless communication connection.

[0012] Based on the above solution, the transmission quality of the audio signal and the noise reduction effect can be taken into account at the same time, thereby improving the final sound pickup effect.

[0013] In a possible implementation of the first aspect, the method further includes: when the transmission quality of the audio signal is lower than a first threshold quality, reducing the number of channels for transmitting the audio signal to the terminal device.

[0014] Based on the above solution, the headset adaptively reduces the number of audio signal transmission paths based on the signal transmission quality, which can better adapt to the different wireless communication bandwidth capabilities of different headsets, and can also better adapt to the situation where the wireless communication quality changes due to user movement or different surrounding interference, thereby expanding the applicable scenarios, that is, improving adaptability.

[0015] In a possible implementation of the first aspect, the method further includes: when the number of transmission paths of the audio signal is reduced to one, performing noise reduction processing on each channel of collected audio signals and then transmitting the collected audio signals to the terminal device.

[0016] Based on the above solution, when the amount of audio signal data transmitted by the earphones to the terminal device is small, the earphones can use each audio signal to perform audio noise reduction, thereby improving the clarity of the audio signal transmitted to the terminal device. Furthermore, when the terminal device determines the target audio signal based on the audio signal, it can obtain a better sound pickup effect.

[0017] In a possible implementation of the first aspect, the method further includes: when the number of transmission paths of the audio signal is less than the maximum number of transmission paths and the transmission quality is higher than a second threshold quality, increasing the number of paths of the audio signal transmitted to the terminal device; wherein the second threshold quality is higher than the first threshold quality.

[0018] Based on the above solution, the audio signal received by the mobile phone can be made richer in data while meeting the signal quality requirements, thereby achieving a better noise reduction effect and a better sound pickup effect.

[0019] In a possible implementation of the first aspect, the earphone includes a first earphone and a second earphone, and the multi-channel audio signal transmitted to the terminal device includes: the multi-channel audio signal collected by the first earphone.

[0020] In a possible implementation of the first aspect, the earphone includes a first earphone and a second earphone, and the multi-channel audio signal transmitted to the terminal device includes: the multi-channel audio signal collected by the first earphone and the multi-channel audio signal collected by the second earphone.

[0021] In a possible implementation of the first aspect, transmitting the collected multi-channel audio signals to the terminal device includes: encoding and compressing the collected multi-channel audio signals into one composite audio signal and then transmitting the composite audio signal to the terminal device.

[0022] Based on the above solution, the headset encodes and compresses the collected multi-channel audio signals into a composite audio signal and then sends it to the terminal device. This can effectively reduce the amount of transmitted data, so that the terminal device can obtain rich audio data even when the wireless communication bandwidth is limited or the signal is poor, thereby improving the audio noise reduction effect on the terminal device side, and thus improving the sound pickup effect; on the other hand, this solution reduces the requirements for wireless communication bandwidth capabilities and wireless communication signal quality, so that it can be applied to more headsets and usage scenarios, that is, it has higher adaptability.

[0023] In a possible implementation of the first aspect, transmitting a collected multi-channel audio signal to a terminal device includes: performing audio enhancement processing on the collected multi-channel audio signal, and transmitting the processed multi-channel audio signal to the terminal device; wherein the audio enhancement processing includes acoustic echo cancellation and / or automatic gain control.

[0024] Based on the above solution, the quality of the audio signal transmitted from the earphone to the terminal device can be improved, thereby improving the final sound pickup effect.

[0025] In a second aspect, an embodiment of the present application provides an audio processing method, which is applied to a terminal device, wherein the method can be executed by the terminal device or a component (such as a chip or module) in the terminal device.

[0026] The method comprises: after establishing a wireless communication connection with an earphone, receiving multi-channel audio signals transmitted by the earphone; and then performing noise reduction processing on the received audio signals to obtain a target audio signal.

[0027] In a possible implementation of the second aspect, performing noise reduction processing on a received audio signal to obtain a target audio signal includes:

[0028] Noise reduction processing is performed on the received audio signal and the audio signal collected by the terminal device to obtain the target audio signal.

[0029] In a possible implementation of the second aspect, the number of audio signal paths transmitted by the earphone is reduced when the number is greater than one and the transmission quality is lower than a first threshold quality; and / or,

[0030] The number of channels of audio signals transmitted by the earphone is increased when the number of channels is less than the maximum number of channels and the transmission quality is higher than a second threshold quality.

[0031] In a possible implementation of the second aspect, multiple audio signals transmitted by the earphone are encoded and compressed into a composite audio signal, and noise reduction processing is performed on the received audio signals to obtain a target audio signal, including:

[0032] The received composite audio signal is decoded, and noise reduction processing is performed on the decoded multi-channel audio signals to obtain the target audio signal.

[0033] In a possible implementation of the second aspect, the terminal device performs noise reduction processing on the received audio signal when a target noise reduction function is turned on; and turns off the target noise reduction function when the battery level is lower than a target battery level.

[0034] Based on the above solution, the terminal device can reduce power consumption and save power by turning off the target noise reduction function when the power is low.

[0035] In a third aspect, an embodiment of the present application provides an audio processing method, which is applied to a system consisting of headphones and a terminal device, wherein the method includes: after the headphones establish a wireless communication connection with the terminal device, the headphones collect multi-channel audio signals through multiple microphones; then the headphones transmit the collected multi-channel audio signals to the terminal device; the terminal device receives the audio signals transmitted by the headphones; then the terminal device performs noise reduction processing on the received audio signals to obtain the target audio signals.

[0036] In a possible implementation of the third aspect, the method further includes: when the transmission quality of the audio signal is lower than a first threshold quality, the headset reduces the number of audio signal transmission paths to the terminal device; when the number of audio signal transmission paths is less than the maximum number of transmission paths and the transmission quality is higher than a second threshold quality, increasing the number of audio signal transmission paths to the terminal device; wherein the second threshold quality is higher than the first threshold quality.

[0037] In a possible implementation of the third aspect, the headset transmits the collected multi-channel audio signals to the terminal device, including: the headset encodes and compresses the collected multi-channel audio signals into one composite audio signal and then transmits the composite audio signal to the terminal device.

[0038] The terminal device performs noise reduction processing on the received audio signal to obtain the target audio signal, including: the terminal device decodes the received composite audio signal, and performs noise reduction processing on the decoded multi-channel audio signals to obtain the target audio signal.

[0039] In a fourth aspect, an embodiment of the present application provides a headset, comprising: a memory and a processor, the memory being used to store a program; the processor being used to execute the method described in the first aspect or any embodiment of the first aspect when calling the program.

[0040] In a fifth aspect, an embodiment of the present application provides a terminal device, comprising: a memory and a processor, the memory being used to store a program; the processor being used to execute the method described in the second aspect or any embodiment of the second aspect when calling the program.

[0041] In a sixth aspect, an embodiment of the present application provides a communication system, comprising the headset described in the fourth aspect and the terminal device described in the fifth aspect.

[0042] In a seventh aspect, an embodiment of the present application provides a readable storage medium having a program stored thereon, which, when executed by a processor, implements the method described in the first or second aspect above.

[0043] In an eighth aspect, an embodiment of the present application provides a program product, which, when executed on a device, enables the device to execute the method described in the first or second aspect above.

[0044] In a ninth aspect, an embodiment of the present application provides a chip system, comprising a processor coupled to a memory, wherein the processor executes a program stored in the memory to implement the method described in the first or second aspect above. The chip system may be a single chip or a chip module composed of multiple chips.

[0045] It can be understood that the beneficial effects of the second to ninth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the architecture of a communication system provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of the structure of the earphones provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the audio processing process for single-ended sound pickup provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of the audio processing process for double-ended pickup provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of another audio processing process provided in an embodiment of the present application;

[0052] Figure 7 Some interface diagrams of the switch intelligent noise reduction mode provided in the embodiments of the present application;

[0053] Figure 8 Schematic diagrams of other interfaces for switching the intelligent noise reduction mode provided in the embodiments of the present application;

[0054] Figure 9 Schematic diagrams of some interfaces for selecting sound pickup methods provided in embodiments of the present application;

[0055] Figure 10 A flowchart of an audio processing method provided in an embodiment of the present application;

[0056] Figure 11 A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application;

[0057] Figure 12 A schematic structural diagram of another audio processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following describes the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. The terms used in the implementation methods of the embodiments of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0059] When assisting terminal devices in picking up sound, headphones are easily affected by factors such as ambient noise, resulting in poor quality of the picked-up audio signal, which in turn affects the user's sound pickup during calls, recordings, and video recordings. To improve sound pickup, a common solution is to install multiple microphones on the headphones to simultaneously pick up multiple audio signals. These signals are then processed by noise reduction and other methods to obtain the target signal and transmit it to the terminal device. However, due to limitations such as size and cost, the computing power of headphones is limited, making it difficult for them to support complex audio processing algorithms. This results in limited ability for headphones to improve sound pickup. For example, noise reduction is an important audio processing process used by headphones to improve sound pickup. The quality of the noise reduction effect is closely related to the noise reduction algorithm used. Efficient noise reduction algorithms are usually complex and require a large amount of computing resources to process audio signals in real time, identify and separate speech (human voice) from background noise, and thus suppress background noise. Due to the limited computing power of headphones, it is difficult for them to support complex and efficient noise reduction algorithms. Therefore, the processing capacity of the noise reduction algorithms they use is limited, resulting in low accuracy of recognized voice and background noise. This will, on the one hand, lead to poor noise suppression effect (i.e. noise reduction effect), and on the other hand, it will also cause voice damage, all of which will affect the final sound pickup effect.

[0060] Based on this, an embodiment of the present application provides an audio processing solution, in which the headset can transmit the multi-channel audio signals collected by the microphone to the terminal device, and use the large computing power of the terminal device to perform audio enhancement processing such as noise reduction on the audio signals to improve the sound pickup effect; wherein, compared with the headset, the computing power of the terminal device is usually much higher, and it can run efficient and complex noise reduction algorithms, so that when performing noise reduction, it can more accurately identify voice and background noise, thereby effectively improving the noise reduction effect and voice quality, and thus effectively improving the sound pickup effect. In addition, in this solution, the headset can adaptively adjust the number of channels of audio signals transmitted to the terminal device according to the communication capability to improve the transmission quality of the audio signal, thereby obtaining a better sound pickup effect.

[0061] The following first introduces the communication system involved in this solution.

[0062] Figure 1 The architecture diagram of the communication system provided in the embodiment of the present application is as follows: Figure 1 As shown, the communication system may include a terminal device 100 and an earphone 200, and the terminal device 100 and the earphone 200 may establish a connection through wireless communication technology.

[0063] The wireless communication technology may be, for example, Bluetooth technology, which may be classic Bluetooth (BT), low energy Bluetooth (BLE), or other Bluetooth technologies to be released in the future.

[0064] Optionally, the terminal device 100 and the headset 200 can also establish a connection through wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Zigbee, frequency modulation (FM), near field communication (NFC) and other wireless communication technologies. For ease of description, the following embodiments of the present application are introduced by taking the terminal device 100 and the headset 200 as an example of establishing a Bluetooth connection through Bluetooth technology.

[0065] The terminal device 100 can be a mobile phone, a tablet computer, a laptop computer, a computer, a smart TV, a smart watch, a car-mounted device, etc. The embodiment of the present application does not specifically limit the specific type of the terminal device 100.

[0066] The headset 200 can be an earbud, in-ear, head-mounted, neck-mounted, or ear-hook wireless headset. A wireless headset can include two earphones worn on the user's left and right ears, respectively, connected by a cable, such as a neckband wireless headset; or it can be two independent parts, such as a true wireless stereo (TWS) headset. The specific type of headset 200 is not particularly limited in this embodiment of the application.

[0067] For earbud-type, in-ear, neck-worn or ear-hook wireless headphones, such as TWS headphones, users can use them in monaural mode or binaural mode. In monaural mode, users can wear only one earphone, such as only the left earphone or the right earphone; in binaural mode, users can wear both earphones at the same time. Among them, in binaural mode, the two earphones can be divided into primary and secondary, that is, one earphone is the primary earphone and the other is the secondary earphone; when the user uses the TWS headphones, the primary and secondary roles of the two earphones can be switched. For other types of wireless headphones, the two earphones can also be divided into primary and secondary.

[0068] After the terminal device 100 establishes a wireless connection with the headset 200, the user can make calls through the headset 200, for example, making a phone call, making a voice call or a video call in an application, etc. The user can also use the headset 200 to record video, audio, or input voice, etc. Of course, the user can also use the headset 200 to play audio signals such as music.

[0069] In some embodiments, the primary earphone and the secondary earphone of the earphone 200 can respectively establish wireless connections with the terminal device 100. The two earphones communicate with the terminal device 100 through their respective wireless connections, for example, exchanging audio data and control information. The control information may include connection control information for controlling the connection status, service control information for performing service control such as pause / play, etc. A wireless connection can also be established between the primary earphone and the secondary earphone to communicate through the wireless connection.

[0070] For example, the primary earphone and the secondary earphone can each establish a Bluetooth connection with the terminal device 100. After the terminal device 100 obtains the audio signal to be played, it can send the audio signal to the primary earphone and the secondary earphone respectively through the Bluetooth connection between the primary and secondary earphones. After the primary earphone and the secondary earphone pick up the audio signal, they can send the picked-up audio signal to the terminal device 100 through their respective corresponding Bluetooth connections. A Bluetooth connection can also be established between the primary earphone and the secondary earphone to exchange data, such as synchronization information and control information, through the Bluetooth connection.

[0071] In some embodiments, the main earphone of the earphone 200 can establish a wireless connection with the terminal device 100 and communicate through the wireless connection; the secondary earphone can communicate with the terminal device 100 through the main earphone.

[0072] For example, a Bluetooth connection can be established between the primary headset and the terminal device 100, through which the primary headset can exchange audio data and control information with the terminal device 100. A Bluetooth connection can also be established between the secondary headset and the primary headset, allowing the secondary headset to receive audio signals sent by the terminal device 100 by forwarding them through the primary headset, or by intercepting the Bluetooth link between the primary headset and the terminal device 100. The secondary headset can then send the audio signals picked up by the secondary headset to the primary headset for processing or forwarding to the terminal device 100. It will be understood that if the primary headset and the secondary headset are connected via a wired connection, data can be transmitted between the primary and secondary headsets via this wired connection.

[0073] The structures of the terminal device 100 and the headset 200 are described below respectively.

[0074] Figure 2 A schematic diagram of the structure of the terminal device provided in the embodiment of the present application is shown as follows: Figure 2 As shown, the terminal device 100 may include: a processor 110, a memory 120, a display screen 130, a power module 140, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a microphone 170B, a sensor module 180, etc.

[0075] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The functional characteristics of the illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0076] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. A memory can also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0077] The memory 120 can be used to store executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the terminal device 100 by running the instructions stored in the memory 120. The memory 120 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function, etc. The data storage area can store data (such as image data, etc.) created during the use of the terminal device 100. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (Universal Flash Storage, UFS), etc.

[0078] The display screen 130 is used to display human-computer interaction interfaces, images, videos, etc.

[0079] The power module 140 may include a battery, a power management unit, etc. The power management unit is used to manage the charging of the battery and the power supply of the battery to other modules.

[0080] The mobile communication module 150 can provide solutions for wireless communication technologies including 2G / 3G / 4G / 5G applied to the terminal device 100. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division-synchronous code division multiple access (TD-SCDMA), long term evolution (LTE), etc. In some embodiments, at least some functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 can be set in the same device as at least some modules of the processor 110.

[0081] The wireless communication module 160 can provide solutions for wireless communication technologies such as WLAN (such as Wi-Fi networks), Bluetooth, FM, NFC, global navigation satellite system (GNSS), infrared technology (IR), etc. applied to the terminal device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The terminal device 100 can communicate with the network and other devices (such as the headset 200) through wireless communication technologies.

[0082] The terminal device 100 can implement audio functions such as music playback, calls, and recording through the audio module 170, the speaker 170A, the microphone 170B, and the application processor.

[0083] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0084] The speaker 170A, also called a "horn", is used to convert audio electrical signals into sound signals.

[0085] Microphone 170B, also known as "microphone" or "microphone", is used to convert sound signals into audio electrical signals (i.e., audio signals). One or more microphones 170B can be set on the terminal device 100. For example, in some embodiments, two microphones 170B can be set on the terminal device 100: a main microphone for picking up (i.e., collecting) the user's voice; and an auxiliary microphone for noise reduction. In some embodiments, three or more microphones 170B can be set on the terminal device 100 to achieve sound collection and noise reduction, as well as to identify the source of sound and achieve directional pickup, etc. Among them, multiple microphones 170B can be independent components, or they can be at least partially integrated together. For example, multiple microphones 170B are integrated into a microphone array.

[0086] The sensor module 180 may include one or more sensors, for example, a touch sensor, which can be set on the display screen 130, and the touch sensor and the display screen 130 form a touch screen, also known as a "touch screen", to detect touch operations acting on or near it; for example, a distance sensor, which can be used to measure distance; or some other sensors.

[0087] Figure 3 A schematic diagram of the structure of the earphones provided in the embodiment of the present application is shown in FIG. Figure 3 As shown, the headset 200 may include: a processor 210, a memory 220, a power module 230, a wireless communication module 240, an audio module 250, a speaker 250A, a microphone 250B and a sensor module 260, etc.

[0088] It should be understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the headset 200. In other embodiments of this application, the headset 200 may include more or fewer components than illustrated, or may combine or separate certain components, or have a different component arrangement. The functional characteristics of the illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0089] The processor 210 may include one or more processing units. For example, the processor 210 may include a controller, a DSP, a micro-programmed control unit (MCU), a field programmable gate array (FPGA), etc. The different processing units may be independent devices or integrated into one or more processors. The processor 210 may be provided with a storage unit for storing instructions and data. In some embodiments, the storage unit in the processor 210 is a cache memory.

[0090] The memory 220 can be used to store executable program code, including instructions. The processor 210 executes the instructions stored in the memory 220 to execute various functional applications and data processing for the headset 200. The memory 220 may include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function, and the like. The data storage area may store data generated during the use of the headset 200, and the like. Furthermore, the memory 220 may include high-speed random access memory and non-volatile memory.

[0091] The power module 230 may include a battery, a power management unit, etc. The power management unit is used to manage the charging of the battery and the power supply of the battery to other modules.

[0092] The wireless communication module 240 can provide solutions for wireless communication technologies such as WLAN (such as Wi-Fi networks), Bluetooth, FM, and NFC for use in the headset 200. The wireless communication module 240 can be one or more devices that integrate at least one communication processing module. The headset 200 can communicate with the network and other devices (such as the terminal device 100) via wireless communication technologies. In some embodiments of the present application, the wireless communication module 240 can be a Bluetooth chip, and the communication method is Bluetooth communication.

[0093] The audio module 250 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 250 can also be used to encode and decode audio signals. In some embodiments, the audio module 250 can be provided in the processor 210, or some functional modules of the audio module 250 can be provided in the processor 210.

[0094] The speaker 250A is used to convert the audio electrical signal into a sound signal and output it. The user can listen to music, etc. through the speaker 250A.

[0095] The microphone 250B is used to pick up sound signals and convert the sound signals into audio electrical signals (ie, audio signals).

[0096] Each earphone 200 may be provided with one microphone 250B, or multiple microphones 250B, such as a main microphone, a feedforward microphone, a feedback microphone, a beamforming microphone, etc. The main microphone may be a condenser microphone or a bone conduction microphone, etc. The multiple microphones 250B may be independent components or at least partially integrated, for example, multiple microphones 250B are integrated into a microphone array.

[0097] The sensor module 260 may include one or more sensors, for example, a touch sensor for detecting a touch operation applied thereto or thereabout, a distance sensor for measuring distance, or other sensors.

[0098] In the embodiment of the present application, when a user uses the earphone 200 in conjunction with the terminal device 100 to make a call, record, videotape, or perform other sound pickup processes, the earphone 200 can transmit the multi-channel audio signal collected by a single earphone to the terminal device 100, or can transmit the multi-channel audio signal collected by both earphones to the terminal device 100. After receiving the audio signal transmitted by the earphone 200, the terminal device 100 can perform audio enhancement processing such as noise reduction on it to obtain a target audio signal; the terminal device 100 can also collect audio signals through its own microphone while the earphone 200 collects audio signals, and then, after receiving the audio signal transmitted by the earphone 200, it can combine the audio signal transmitted by the earphone 200 and the audio signal collected by itself to perform audio enhancement processing such as noise reduction to obtain the target audio signal.

[0099] Among them, the above-mentioned method of transmitting the audio signal collected by the headset to the terminal device can be called "single-ended sound pickup"; the method of simultaneously picking up sound by the terminal device and the headset can be called "dual-ended sound pickup". The following takes the example of the terminal device being a mobile phone and the headset and the mobile phone transmitting data via Bluetooth connection to describe in detail the audio processing processes corresponding to these two sound pickup methods. The audio processing solution provided in the embodiment of the present application can be applied to scenarios that require sound pickup, such as calls, recordings, or videos (referred to as sound pickup scenarios). For the sake of convenience, the following example will be described using the call scenario as an example.

[0100] Figure 4 The audio processing process diagram of the single-ended pickup provided in the embodiment of the present application is as follows: Figure 4 As shown, the audio processing process corresponding to the single-ended pickup method may include the following steps:

[0101] S101. The earphone collects audio signals through multiple microphones.

[0102] Taking a call scenario as an example, after the headset establishes a Bluetooth connection with the terminal device, the user makes a call through the terminal device and the headset. During the call, the headset can pick up the surrounding sounds in real time through the microphone and obtain audio signals.

[0103] As mentioned above, multiple microphones can be set on the headset, each microphone can collect one audio signal, and the headset can collect multiple audio signals through multiple microphones.

[0104] Headphones can include a primary headphone and a secondary headphone. In some implementations, multiple channels of audio signals can be captured through either the primary or secondary headphone. This approach is also known as monaural sound pickup. For example, in monaural mode, multiple channels of audio signals can be captured through the primary headphone; in binaural mode, multiple channels of audio signals can be captured through either the primary or secondary headphone. For example, if each headphone is equipped with four microphones, four channels of audio signals can be captured through the microphones on either the primary or secondary headphone.

[0105] In some implementations, multiple audio channels can be captured simultaneously through two earphones (i.e., a primary earphone and a secondary earphone). This is also known as binaural audio pickup. For example, in binaural mode, the primary earphone and the secondary earphone can each capture 4 channels of audio signals, resulting in a total of 8 channels of audio signals being captured simultaneously through the primary and secondary earphones.

[0106] In some embodiments, after collecting multiple audio signals, the headset can transmit the collected audio signals to the mobile phone and use the computing power of the mobile phone to perform noise reduction and other processing.

[0107] Taking into account that when the user is using the headset, due to factors such as distance, the Bluetooth signal between the headset and the mobile phone may have poor signal quality, in some embodiments, before the headset transmits multiple audio signals to the mobile phone, it can execute the following step S102 to detect whether the Bluetooth signal quality between the headset and the mobile phone meets the target conditions. If the Bluetooth signal quality meets the target conditions, the multiple audio signals are transmitted to the mobile phone to better ensure the quality of the audio signal received by the mobile phone.

[0108] S102: The headset determines whether the Bluetooth signal quality meets the target condition. If so, execute step S103; otherwise, execute step S107.

[0109] In some implementations, the Bluetooth signal quality may be measured using parameters such as received signal strength indicator (RSSI) or distance.

[0110] For example, the headset can detect the RSSI between the headset and the mobile phone. The larger the RSSI value, the better the Bluetooth signal quality; conversely, the smaller the RSSI value, the worse the Bluetooth signal quality.

[0111] Correspondingly, an RSSI threshold can be set. The headset can determine whether the Bluetooth signal quality meets the target conditions based on the relationship between the detected RSSI value and the RSSI threshold. If the detected RSSI value is greater than or equal to the RSSI threshold, it can be determined that the Bluetooth signal quality between the headset and the phone meets the target conditions. If the detected RSSI value is less than the RSSI threshold, it can be determined that the Bluetooth signal quality between the headset and the phone does not meet the target conditions.

[0112] The RSSI threshold value may be set according to actual needs and is not particularly limited in this embodiment.

[0113] For another example, the headset can measure the distance between the headset and the mobile phone, wherein the smaller the distance, the better the Bluetooth signal quality; conversely, the larger the distance, the worse the Bluetooth signal quality.

[0114] Correspondingly, a distance threshold can be set, and the headset can determine whether the Bluetooth signal quality meets the target condition based on the relationship between the measured distance and the distance threshold. If the measured distance is less than or equal to the distance threshold, it can be determined that the Bluetooth signal quality between the headset and the phone meets the target condition; if the measured distance is greater than the distance threshold, it can be determined that the Bluetooth signal quality between the headset and the phone does not meet the target condition.

[0115] The headset can measure the distance between the headset and the mobile phone through a Bluetooth ranging algorithm or other indoor positioning algorithms; the size of the distance threshold can be set according to actual needs, and this embodiment does not specifically limit this.

[0116] It is understandable that in some implementations, the Bluetooth signal quality may also be measured using other parameters, as long as the parameter can reflect the quality of the Bluetooth signal.

[0117] If the Bluetooth signal quality between the headset and the mobile phone meets the target condition, it means that the Bluetooth signal between the two is good. In this case, the headset can execute step S103 to transmit the multi-channel audio signal to the mobile phone.

[0118] If the Bluetooth signal quality between the headset and the mobile phone does not meet the target conditions, it means that the Bluetooth signal between the two is poor. In this case, the headset can execute step S107, perform noise reduction processing on the audio signal at the headset end, and transmit the audio signal obtained after noise reduction to the mobile phone. In this way, both the transmission quality of the audio signal and the noise reduction effect can be taken into account at the same time, thereby improving the final sound pickup effect.

[0119] S103: The earphone transmits the collected multi-channel audio signals to the mobile phone.

[0120] As mentioned above, multiple audio signals can be collected through one or two headphones. In some implementations, the headphones can transmit all the collected audio signals to the mobile phone; in some implementations, the multiple audio signals transmitted by the headphones to the mobile phone can also be part of the audio signals collected by the headphones.

[0121] For example, the headset can collect 4 audio signals. In some examples, the headset determines that the Bluetooth signal quality meets the target conditions through the above step S102, and it can transmit all 4 collected audio signals to the mobile phone; in some examples, in addition to determining that the Bluetooth signal quality meets the target conditions, the headset can further transmit some or all of the collected audio signals to the mobile phone based on the Bluetooth signal quality.

[0122] In some examples, the headset does not perform the above step S102, and can transmit all four channels of collected audio signals to the mobile phone; or it can transmit three or two channels of audio signals to the mobile phone to improve the quality of the audio signal received by the mobile phone.

[0123] In some embodiments, before transmitting multiple audio signals to the mobile phone, the headset may also perform some simple audio enhancement processing on each audio signal, and then transmit the processed audio signals to the mobile phone to improve the final sound pickup effect. The audio enhancement processing may include but is not limited to: acoustic echo cancellation (AEC) and automatic gain control (AGC).

[0124] S104: The headset detects the transmission quality of the audio signal. If the number of transmission paths for the audio signal is greater than 1 and the transmission quality is lower than a first threshold value Th1, step S105 is executed. If the number of transmission paths for the audio signal is less than the maximum number of transmission paths Nmax and the transmission quality is higher than a second threshold value Th2, step S106 is executed.

[0125] In the process of transmitting audio signals from the headset to the mobile phone, the transmission quality of the audio signal may be affected due to poor signal or limited bandwidth. Taking this situation into consideration, the headset can detect the transmission quality of the audio signal in the process of transmitting audio signals to the mobile phone, and adjust the number of audio signal transmission paths to the mobile phone (i.e., the number of audio signal transmission paths) according to the transmission quality of the audio signal. In this way, the audio signal received by the mobile phone can be richer in data while the signal quality meets the requirements, thereby obtaining a better noise reduction effect and a better sound pickup effect. On the other hand, the headset adaptively adjusts the number of audio signal transmission paths based on the signal transmission quality, which can better adapt to the different Bluetooth bandwidth capabilities of different headsets, and can also better adapt to the situation where the Bluetooth signal changes due to user movement or different surrounding interference, thereby expanding the applicable scenarios, that is, improving adaptability.

[0126] In some embodiments, the headset may reduce the number of transmission paths when the transmission quality of the audio signal is lower than the first threshold quality Th1 and the number of transmission paths of the audio signal is greater than 1, so as to improve the quality of the audio signal received by the mobile phone.

[0127] For example, initially, the headset transmits 4 channels of collected audio signals to the mobile phone. During the process of transmitting the audio signals, if the headset detects that the transmission quality of the audio signals is lower than the first threshold quality Th1, the number of transmission channels can be reduced, for example, by reducing 1 channel of audio signal. That is, after the subsequent audio signals are collected, 3 channels of audio signals are transmitted to the mobile phone. It can be understood that during the process of transmitting the audio signals, the headset can detect the transmission quality of the audio signals multiple times (for example, periodically). During the process of transmitting 3 channels of audio signals, if the transmission quality of the audio signals is detected to be lower than the first threshold quality Th1, 1 channel of audio signal can be reduced again; similarly, during the process of transmitting 2 channels of audio signals, if the transmission quality of the audio signals is detected to be lower than the first threshold quality Th1, the number of transmission channels of the audio signals can be reduced to 1 channel.

[0128] In some implementations, the transmission quality of the audio signal may be measured using a packet loss rate (PLR), wherein a higher PLR indicates worse transmission quality, and a lower PLR indicates better transmission quality.

[0129] Correspondingly, the first threshold quality Th1 can be represented by a packet loss threshold (herein referred to as the first packet loss threshold). The headset can determine the relationship between the audio signal transmission quality and the first threshold quality Th1 based on the relationship between the detected packet loss rate and the first packet loss threshold. When the packet loss rate detected by the headset is higher than the first packet loss threshold, it can be determined that the audio signal transmission quality is lower than the first threshold quality Th1.

[0130] Among them, the headset can determine the packet loss rate based on the response mechanism. Specifically, when the headset transmits an audio signal to the mobile phone, the audio signal is transmitted to the mobile phone in the form of a data packet; in the process of transmitting the audio signal, the headset can use a response mechanism to confirm whether the data packet is correctly received. In the response mechanism, after the sending end (such as the headset) sends a data packet to the receiving end (such as the mobile phone), if the receiving end receives the data packet, it can return a response message to the sending end, and the sending end can determine whether the data packet is successfully sent based on whether the response message is received. Based on the response mechanism, the headset can determine the number of data packets lost within a period of time, and then determine the packet loss rate, which is the ratio of the number of lost data packets to the number of sent data packets. It can be understood that the headset can also use other methods to determine the packet loss rate, and the embodiments of the present application are not particularly limited to this.

[0131] In some embodiments, the headset can increase the number of transmission paths when the transmission quality of the audio signal is higher than the second threshold quality Th2 and the number of transmission paths of the audio signal is less than the maximum number of transmission paths Nmax to improve the data richness of the audio signal received by the mobile phone.

[0132] The maximum number of transmission paths Nmax may be the maximum number of paths of audio signals collected by the earphone, for example, 4 paths or 8 paths, etc. The second threshold quality Th2 may be higher than the first threshold quality Th1.

[0133] For example, initially, the headset transmits fewer than four channels of audio signals to the mobile phone, or, during the process of transmitting the audio signals, due to poor transmission quality, the headset reduces the number of transmission channels, resulting in fewer than four channels. Taking the current audio signal transmission channel number as 2 as an example, during the process of transmitting the audio signal, if the headset detects that the transmission quality of the audio signal is higher than the second threshold quality Th2, the number of transmission channels can be increased, for example, by adding one channel of audio signal, that is, after subsequently collecting the audio signal, three channels of audio signal are transmitted to the mobile phone. Similar to the above embodiment, during the process of transmitting three channels of audio signals, if the headset detects that the transmission quality of the audio signal is higher than the second threshold quality Th2, one more channel of audio signal can be added, that is, the number of transmission channels of the audio signal is increased to four.

[0134] Similarly, the audio signal transmission quality can be measured by the packet loss rate, and the second threshold quality Th2 can be represented by the second packet loss threshold. The headset can determine the relationship between the audio signal transmission quality and the second threshold quality Th2 based on the relationship between the detected packet loss rate and the second packet loss threshold. When the packet loss rate detected by the headset is lower than the second packet loss threshold, it can be determined that the audio signal transmission quality exceeds the second threshold quality Th2.

[0135] The first quality threshold Th1, the second quality threshold Th2, and the corresponding packet loss thresholds can be set according to actual needs, and are not particularly limited in this embodiment.

[0136] It is understood that in some embodiments, the transmission quality of the audio signal may also be measured using other parameters, such as a signal-to-noise ratio, etc., as long as the parameter can reflect the transmission quality of the audio signal.

[0137] S105. The headset reduces the number of channels for transmitting audio signals to the mobile phone. When the number of channels for transmitting audio signals is reduced to one, step S107 is executed.

[0138] As described above, when the earphone is transmitting multiple audio channels to the mobile phone, if the transmission quality of the audio channels is poor, the number of transmission channels of the audio channels can be reduced.

[0139] In some implementations, as described in step S104 above, when the audio signal transmission quality is lower than the first threshold quality Th1, the number of audio signal transmission paths may be reduced by one. In some implementations, multiple paths may be reduced, or the number of audio signal transmission paths may be directly reduced to one. For ease of explanation, the following description uses reducing one path as an example.

[0140] Among them, in the case of transmitting an audio signal collected by a single earphone to a mobile phone, when reducing the audio signal, in some examples, one audio signal can be randomly selected from the audio signal collected by the currently transmitted secondary microphone as the audio signal to be subtracted, where all microphones except the main microphone are called secondary microphones.

[0141] In some examples, it is possible to determine which audio signal to reduce according to a certain priority. For example, a priority can be set for each microphone on the headset according to its importance, wherein the priority also represents the priority of the audio signal collected by the microphone. The higher the priority, the more important the microphone, that is, the more important the audio signal collected by the microphone; when reducing the number of transmission paths of the audio signal, the audio signal with the lowest priority among the currently transmitted audio signals can be subtracted. For example, the microphones on the main headset include: main microphone Mic1, feedforward microphone Mic2 and feedback microphone Mic3, and the priorities of the three decrease in sequence; the headset currently transmits 3 audio signals collected by these three microphones to the mobile phone, then when reducing the audio signal, the audio signal collected by the feedback microphone Mic3 can be preferentially subtracted. It can be understood that the priority relationship between the microphones here is only an example, and it is not used to limit the present application. The embodiments of the present application do not specifically limit this.

[0142] For the case of transmitting audio signals collected by two earphones to a mobile phone, in some examples, when reducing the audio signal, the audio signal to be subtracted can be preferentially selected from the audio signal collected by the secondary earphone. When the currently transmitted audio signal does not contain the audio signal collected by the secondary earphone, the audio signal to be subtracted can be selected from the audio signal collected by the main earphone. That is, the priority corresponding to each microphone on the main earphone is higher than the priority corresponding to each microphone on the secondary earphone as a whole. For any earphone between the main earphone and the secondary earphone, when reducing the audio signal collected by the earphone, the audio signal to be reduced can be determined in accordance with the above-mentioned method corresponding to a single earphone.

[0143] In some examples, the audio signals collected by the primary and secondary earphones can also be subtracted alternately or synchronously. For example, assuming each earphone includes the three microphones mentioned above, the primary microphone Mic1 of the primary earphone, the primary microphone Mic1 of the secondary earphone, the feedforward microphone Mic2 of the primary earphone, the feedforward microphone Mic2 of the secondary earphone, the feedback microphone Mic3 of the primary earphone, and the feedback microphone Mic3 of the secondary earphone, with priority gradually decreasing. The earphones transmit six channels of audio signals collected by these six microphones to the mobile phone. When reducing the audio signal, the audio signal collected by the feedback microphone Mic3 on the secondary earphone can be preferentially subtracted. The next time the audio signal is reduced, the audio signal collected by the feedback microphone Mic3 on the primary earphone can be subtracted. Similarly, when the signal transmission quality is poor, the audio signal collected by the microphone with the lowest priority can be subtracted each time. Alternatively, each time the number of audio signal transmission channels is reduced, the audio signal collected by the microphone with the lowest priority currently being transmitted on both earphones can be subtracted simultaneously. For example, the audio signals collected by the feedback microphone Mic3 on both the primary and secondary earphones can be subtracted simultaneously.

[0144] In some embodiments, when the number of audio signal transmission paths is reduced to one, the headset can transmit the audio signal collected by the main microphone to the mobile phone.

[0145] In some embodiments, when the number of audio signal transmission paths is reduced to one, the headset can execute the following step S107, perform noise reduction processing on each collected audio signal, and then transmit the audio signal obtained after noise reduction to the mobile phone. In this way, when the amount of audio signal data transmitted by the headset to the mobile phone is small, the headset can use each audio signal to perform audio noise reduction, thereby improving the clarity of the audio signal transmitted to the mobile phone. As a result, when the mobile phone determines the target audio signal based on the audio signal, it can obtain a better sound pickup effect.

[0146] S106: The headset increases the number of channels for transmitting audio signals to the mobile phone.

[0147] As described in step S104 , when the earphone is transmitting the audio signal to the mobile phone, if the transmission quality of the audio signal is good, the number of transmission paths for the audio signal may be increased.

[0148] Corresponding to the method of reducing the number of transmission paths in step S105, when increasing the number of audio signal transmission paths, the headset can increase the number of audio signal transmission paths by one or more channels if it detects that the transmission quality of the audio signal exceeds the second threshold quality Th2. For ease of explanation, the following description uses the method of increasing one channel as an example.

[0149] When determining which audio signal to add, in some examples, an audio signal can be randomly selected from the audio signals that are not transmitted as the audio signal to be added. In some examples, the priority order to be added can be used to determine the audio signal to be added, that is, the audio signal collected by the microphone with a higher priority is given priority. For example, continuing with the example of each earphone including three microphones, the earphone transmits the audio signal collected by a single earphone to the mobile phone, wherein the earphone currently transmits the audio signal collected by the main microphone Mic1 to the mobile phone. When adding the audio signal, the audio signal collected by the feedforward microphone Mic2 can be given priority. For another example, the earphone transmits the audio signals collected by two earphones to the mobile phone. When adding the audio signal, the audio signal collected by the microphone with the highest priority among the audio signals that are not currently transmitted can be added, or the audio signals collected by the microphone with the highest priority that are not currently transmitted on both earphones can be added at the same time.

[0150] S107: The earphone performs noise reduction processing on the collected multi-channel audio signals.

[0151] When the Bluetooth signal quality between the headset and the mobile phone does not meet the target conditions, or the number of audio signal transmission channels is reduced to one, the headset can perform noise reduction processing on the local end and then transmit the noise-reduced audio signal to the mobile phone to improve the sound pickup effect.

[0152] When performing noise reduction processing, the headset may use a noise reduction algorithm based on deep learning, such as a noise reduction algorithm based on convolutional neural networks (CNN) or a noise reduction algorithm based on recurrent neural networks (RNN). The headset may also use other noise reduction algorithms, such as a traditional differential noise reduction algorithm. This embodiment of the present application is not particularly limited to this.

[0153] In some embodiments, the headset may also perform other audio enhancement processing (such as AEC and AGC, etc.) on the audio signals before or after performing noise reduction processing on each audio signal, and then transmit the processed audio signals to the mobile phone.

[0154] S108. The earphone transmits the noise-reduced audio signal to the mobile phone.

[0155] After the earphones perform noise reduction processing on the collected audio signals, a clearer audio signal can be obtained, which can then be transmitted to the mobile phone.

[0156] S109: The mobile phone performs noise reduction processing on the received audio signal to obtain a target audio signal.

[0157] The mobile phone can receive one or more audio signals transmitted by the headphones. After receiving the audio signals transmitted by the headphones, the mobile phone can use the large computing power of the mobile phone to perform noise reduction processing on the audio signals to obtain a better target audio signal.

[0158] Compared to headphones, mobile phones typically have much higher computing power and can run efficient and complex noise reduction algorithms. When the phone receives multi-channel audio signals that have not undergone headphone noise reduction, compared to noise reduction on the headphone side, noise reduction based on each audio signal on the phone can achieve better noise reduction and thus better sound pickup. When the phone receives audio signals that have undergone headphone noise reduction, the phone further reduces the noise of this audio signal, which can also further improve the noise reduction effect of the audio signal, thereby achieving better sound pickup.

[0159] Among them, when performing noise reduction processing, the mobile phone can adopt a noise reduction algorithm based on deep learning, such as a noise reduction algorithm based on CNN or RNN; or, the mobile phone can also adopt other noise reduction algorithms, which are not particularly limited in this embodiment.

[0160] In some embodiments, in addition to performing noise reduction processing on the received audio signal, the mobile phone may also perform other audio enhancement processing on the audio signal, such as AEC and AGC, to further improve the sound pickup effect.

[0161] It should be noted that, as mentioned above, the secondary earphone can communicate with the terminal device (such as a mobile phone) through the primary earphone, or it can communicate with the terminal device independently. Correspondingly, for the binaural sound pickup in the above step S101, in one implementation, the audio signal collected by the secondary earphone can be transmitted to the primary earphone, and then the primary earphone can transmit its own collected audio signal and the audio signal received from the secondary earphone to the mobile phone together, that is, communicating with the mobile phone through a single earphone; in another implementation, the primary earphone and the secondary earphone can transmit their own collected audio signals to the mobile phone through the Bluetooth connection between each of them and the mobile phone, that is, the two earphones communicate with the mobile phone separately.

[0162] The above embodiments are mainly described by taking a single earphone communicating with a mobile phone as an example. It can be understood that in some embodiments, two earphones can also transmit audio signals to the mobile phone respectively. Correspondingly, the main earphone and the auxiliary earphone can respectively execute the above steps S102 to S108 to transmit audio signals to the mobile phone.

[0163] In addition, those skilled in the art will understand that the above embodiments are exemplary and are not intended to limit the present application. Where possible, the execution order of one or more of the above steps may be adjusted, or may be selectively combined to obtain one or more other embodiments. For example, in some embodiments, there is no strict timing execution relationship between steps S101 and S102, and step S101 may be executed before or after step S102, or may be executed simultaneously with step S102; in some embodiments, one or more of the above steps S102, S104 to S108 may not be executed. Those skilled in the art may select and combine any of the above steps as needed, and anything that does not deviate from the essence of the present application falls within the scope of protection of the present application.

[0164] Figure 5 The audio processing process diagram of the double-ended pickup provided in the embodiment of the present application is as follows: Figure 5 As shown, the audio processing process corresponding to the double-ended pickup method may include the following steps:

[0165] S201. The earphone collects audio signals through multiple microphones.

[0166] This step can refer to the relevant description of step S101 in the above embodiment, and will not be repeated here.

[0167] S202: The headset transmits one or more audio signals to the mobile phone.

[0168] This step can refer to the relevant description of steps S102 to S108 in the above embodiment, and will not be repeated here.

[0169] S203: The mobile phone collects audio signals through at least one microphone.

[0170] In this embodiment, while the earphone end collects audio signals through the microphone, the mobile phone end can also turn on the microphone to collect audio signals to obtain richer audio data.

[0171] As mentioned above, one or more microphones can be set on the mobile phone (terminal device), each microphone can collect one audio signal, and the mobile phone can collect one or more audio signals through one or more microphones.

[0172] It is understandable that when a mobile phone is equipped with multiple microphones, the mobile phone can turn on all microphones to collect multiple audio signals, or turn on some microphones to collect one or more audio signals, for example, only turn on the main microphone to collect one audio signal.

[0173] S204: The mobile phone performs noise reduction processing according to the received audio signal and the collected audio signal to obtain a target audio signal.

[0174] After receiving one or more audio signals from the headset, the phone can synchronize and align the collected audio signals with the received audio signals, and then perform noise reduction on the synchronized audio signals to obtain the target audio signal. Taking the above call scenario as an example, after the headset receives the target audio signal, it can transmit it to the other end of the call device; in the recording scenario, after the headset receives the target audio signal, it can use it as the final recording signal.

[0175] Among them, when performing synchronization alignment processing, the mobile phone can use audio alignment algorithms such as cross-correlation method and dynamic time warping (DTW) to align the audio signals of each channel. The specific algorithm used is not particularly limited here.

[0176] The noise reduction algorithm used by the mobile phone when performing noise reduction processing is similar to that in the aforementioned embodiment, and can be a noise reduction algorithm based on CNN or RNN, etc.

[0177] In this embodiment, while the earphone end is collecting audio signals, the mobile phone end is also collecting audio signals, so that richer and more comprehensive audio data can be obtained. The mobile phone performs noise reduction processing based on the audio signals received from the earphone and the audio signals collected by itself, so that a better noise reduction effect can be obtained, thereby obtaining a better sound pickup effect.

[0178] Figure 6 Another audio processing process diagram provided in the embodiment of the present application is as follows: Figure 6 As shown, the audio processing process provided in this embodiment may include the following steps:

[0179] S301 : The earphone collects audio signals through multiple microphones.

[0180] This step can refer to the relevant description of step S101 in the above embodiment, and will not be repeated here.

[0181] S302: The earphone encodes and compresses the collected multi-channel audio signals into one composite audio signal.

[0182] After collecting multiple audio signals, the headset can encode and compress the collected audio signals into one audio signal (ie, a composite audio signal) to reduce the amount of transmitted data.

[0183] Among them, when encoding and compressing the collected multi-channel audio signals, the headphones can adopt some more efficient encoding and compression algorithms. In some examples, the headphones can adopt a lossless encoding and compression algorithm to improve the data integrity of the audio signal obtained by the mobile phone.

[0184] In some embodiments, the headset may perform some simple audio enhancement processing (such as AEC and AGC, etc.) on each audio signal before encoding and compressing the audio signal, and then encode and compress the processed audio signal into a composite audio signal.

[0185] S303: The earphone transmits the composite audio signal to the mobile phone.

[0186] After the earphones encode and compress the collected audio signals into a composite audio signal, they can transmit it to the mobile phone and use the computing power of the mobile phone to perform noise reduction on the audio signal to obtain better sound pickup effect.

[0187] S304: The mobile phone decodes the received composite audio signal to obtain multi-channel audio signals.

[0188] After receiving the composite audio signal transmitted by the headset, the mobile phone can decode the composite audio signal to restore the original audio signals of each channel.

[0189] S305 : The mobile phone performs noise reduction processing on the multi-channel audio signals obtained by decoding to obtain a target audio signal.

[0190] After the mobile phone decodes the composite audio signal to obtain multiple audio signals, it can perform noise reduction processing on each decoded audio signal to obtain the target audio signal; it can also be similar to the above Figure 5 In the illustrated embodiment, noise reduction processing is performed based on the audio signal synchronously collected by the mobile phone and the various audio signals obtained by decoding to obtain the target audio signal. The specific description of the mobile phone collecting audio signals and performing noise reduction processing can be found in the relevant description of step S204 in the aforementioned embodiment and will not be repeated here.

[0191] In this embodiment, the headset encodes and compresses the collected multi-channel audio signals into a composite audio signal and then sends it to the mobile phone. This can effectively reduce the amount of transmitted data, so that the mobile phone can obtain rich audio data even when the Bluetooth bandwidth is limited or the signal is poor, thereby improving the audio noise reduction effect on the mobile phone and further improving the sound pickup effect; on the other hand, this solution reduces the requirements for Bluetooth bandwidth capability and Bluetooth signal quality, so that it can be applied to more headsets and usage scenarios, that is, it has higher adaptability.

[0192] For the above-mentioned audio processing method for noise reduction through terminal devices provided in the embodiments of the present application, in some implementations, the terminal device can adopt this audio processing method (hereinafter referred to as the intelligent noise reduction mode) by default in the sound pickup scenario, that is, when the terminal device picks up sound through headphones, it adopts the intelligent noise reduction mode to perform noise reduction and other processing on the audio signal transmitted by the headphones.

[0193] It can be understood that the above-mentioned naming method of "intelligent noise reduction mode" is only an example and should not be understood as a limitation to the embodiments of the present application. In some embodiments, naming methods may also be used, such as target noise reduction function, first mode or first function, etc.

[0194] In some implementations, to save power consumption and processing resources of the terminal device, the terminal device may not enable the intelligent noise reduction mode by default, and may only enable the intelligent noise reduction mode when the user manually enables it. This can also improve user flexibility.

[0195] Manual activation by the user can include a variety of possible activation methods. For example, the terminal device's settings interface may provide an option for intelligent noise reduction mode, through which the user can turn intelligent noise reduction mode on or off. Another example is that the terminal device may display a pop-up window or control in a sound pickup scenario for the user to turn intelligent noise reduction mode on or off. The following describes the functional implementation process corresponding to the above audio processing method, taking a mobile phone as an example terminal device.

[0196] Figure 7 Some interface diagrams of the switch intelligent noise reduction mode provided in the embodiment of the present application, such as Figure 7 As shown in (a) of FIG, the mobile phone displays a video call prompt interface 01 to prompt the user whether to connect the video call; the video call prompt interface 01 may display a "reject" option 011 and an "answer" option 012. When the user clicks the "answer" option 012 to connect the video call, the mobile phone may display the video call interface. Figure 7 As shown in (b), on the video call interface 02, the mobile phone can display a pop-up window 03 indicating whether to turn on the smart noise reduction mode. The pop-up window 03 can display an option 031 of "Turn on smart noise reduction mode" and can display a prompt message 032 to prompt the user to turn on the smart noise reduction mode. Here, for example, the prompt message 032 includes: "After turning it on, the microphone reception quality in noisy environments can be optimized."

[0197] In some examples, such as Figure 7 As shown in (b) in the figure, the user can click the "Turn on smart noise reduction mode" option 031, and the mobile phone can respond to the operation, turn on the smart noise reduction mode, and close the pop-up window 03, showing the following Figure 7The video call interface 02 shown in (c) of the figure is shown in Figure 02. At this point, the phone has enabled intelligent noise reduction mode. When the phone receives the audio signal transmitted by the earphones, it can perform noise reduction processing on the audio signal transmitted by the earphones to improve the sound pickup effect. After the call ends, the phone can automatically turn off intelligent noise reduction mode.

[0198] In some examples, such as Figure 8 As shown in (a) in the figure, the user may not choose to turn on the intelligent noise reduction mode. At this time, the user can slide up on the pop-up window 03 to trigger the mobile phone to close the pop-up window 03 and display the following Figure 8 Alternatively, the user may not perform any operation, and after a few seconds (such as 5 seconds), the mobile phone may automatically close the pop-up window 03 and display the following Figure 8 The video call interface 02 shown in (b) in FIG.

[0199] If the user wants to turn on the intelligent noise reduction mode during a video call, he can Figure 8 As shown in (b) of the figure, slide down from the top left of the screen to trigger the phone to open the notification center interface, as shown in Figure 8 As shown in (c) in the figure, the notification center interface 04 may display a notification message of the intelligent noise reduction mode, which may include the above-mentioned Figure 7 Similar information in the pop-up window 03 is shown: "Turn on smart noise reduction mode" option 041 and prompt information 042 for prompting the user to turn on the smart noise reduction mode. The user can click on the "Turn on smart noise reduction mode" option 041, as shown in FIG. Figure 8 As shown in (d), the mobile phone can respond to the user's operation, turn on the intelligent noise reduction mode, and update the intelligent noise reduction mode notification message displayed in the notification center interface 04. Here, for example, the updated intelligent noise reduction mode notification message displays the "Turn off intelligent noise reduction mode" option 043 and the prompt message 044: "It is turned on and the microphone reception quality in noisy environments is being optimized."

[0200] Afterwards, the user can close the notification center interface 04 and return to the video call interface 02; subsequently, during a video call, if the user wants to turn on the smart noise reduction mode, he can open the notification center interface 04 again and click the "Turn off smart noise reduction mode" option 043 displayed in the smart noise reduction mode notification message to trigger the phone to turn off the smart noise reduction mode.

[0201] In some examples, the above Figure 7 In the example shown, after the user clicks the "Turn on smart noise reduction mode" option 031, the mobile phone may not close the pop-up window 03 when responding to the operation, but may update the content of the pop-up window 03 to something similar to Figure 8The content displayed in the smart noise reduction mode notification message shown in (d) in FIG. The user can click the "Turn off smart noise reduction mode" option displayed in the updated pop-up window 03 to trigger the phone to turn off smart noise reduction mode. Alternatively, the user can trigger the phone to close pop-up window 03 by swiping up on pop-up window 03.

[0202] In some examples, the mobile phone may also provide a switch control corresponding to the smart noise reduction mode in the above-mentioned video call interface 02, and the user can turn on or off the smart noise reduction mode by clicking the control.

[0203] In some examples, the mobile phone may also provide a switch control corresponding to the smart noise reduction mode in the control center interface, and the user can turn on or off the smart noise reduction mode by clicking the control.

[0204] In some examples, the phone may also disable the intelligent noise reduction mode when the battery level is low (e.g., below a first battery level) to save power. At this point, the phone may also notify the user through a notification message that the intelligent noise reduction mode is disabled due to the low battery level. The first battery level may be 20% of the total battery level or another value, and the specific level is not specifically limited herein.

[0205] It is understood that when the mobile phone turns on or off the intelligent noise reduction mode, it can send a notification message to the headset to inform the headset of the switch status of the intelligent noise reduction mode. Correspondingly, after receiving the notification message, the headset can transmit one or more audio signals to the mobile phone through the audio processing method provided in the embodiment of the present application when the intelligent noise reduction mode is turned on; when the intelligent noise reduction mode is turned off, the collected audio signals are processed by the headset before being transmitted to the mobile phone after noise reduction.

[0206] For the several sound pickup methods corresponding to the aforementioned "single-ended sound pickup" and "double-ended sound pickup", in some embodiments, the mobile phone and the headset can support one of the sound pickup methods. That is, after the mobile phone turns on the intelligent noise reduction mode, the mobile phone and the headset use the supported sound pickup method to pick up the audio signal and perform corresponding processing. For example, using the single-ear sound pickup method in the above-mentioned single-ended sound pickup method, the headset transmits the multi-channel audio signals collected by the main headset to the mobile phone, and the mobile phone performs noise reduction and other processing on the received audio signals to obtain the target audio signal.

[0207] In some embodiments, the mobile phone and the headset can support the above-mentioned multiple sound pickup methods. After the mobile phone turns on the intelligent noise reduction mode, the mobile phone and the headset can select any supported sound pickup method to pick up and process the audio signal.

[0208] In some implementations, the mobile phone and the headset can adaptively select the sound pickup method to be used.

[0209] For example, a mobile phone can determine whether to use a dual-ended audio pickup mode based on the current battery level. When the battery level of the mobile phone is high (e.g., greater than or equal to a second battery level), the dual-ended audio pickup mode is used, activating its own microphone to collect audio signals. When the battery level of the mobile phone is lower than the second battery level, the single-ended audio pickup mode is used. The second battery level can be higher than the first battery level, and the specific level can be set as needed, and this embodiment of the application is not particularly limited to this.

[0210] The headset can determine whether to adopt monaural sound pickup or binaural sound pickup according to the current headset mode. For example, it can adopt monaural sound pickup in monaural mode and binaural sound pickup in binaural mode.

[0211] It will be understood that the above is only an example and is not intended to limit the present application. In other examples, the mobile phone and the headset may also determine the pickup method to be adopted based on other parameters. For example, the mobile phone may also determine whether to adopt a dual-end pickup method based on parameters such as the current processing capability, and the headset may also determine whether to adopt a single-ear pickup or a dual-ear pickup method based on parameters such as battery power, Bluetooth bandwidth capability, or Bluetooth signal quality.

[0212] In other implementations, the mobile phone and the headset may determine the sound pickup method to be adopted according to user operations.

[0213] Figure 9 Some interface diagrams for selecting the pickup method provided in the embodiments of this application, such as Figure 9 As shown in (a) of FIG, the mobile phone can provide a quick switch option for the smart noise reduction mode in the control center interface 05. The quick switch option may include a control 051 and a control 052. The user can turn the smart noise reduction mode on or off by clicking the control 051 and select the sound pickup mode by clicking the control 052. The control 051 can indicate the on / off state of the smart noise reduction mode by different colors. Here, for example, the smart noise reduction mode is on, and the control 051 is displayed in gray.

[0214] After the user clicks control 052, Figure 9 As shown in (b) of FIG, the mobile phone can display the setting interface 06 of the intelligent noise reduction mode, which may include a monaural sound pickup option 061 and a binaural sound pickup option 062 corresponding to the headset, as well as a single-ended sound pickup option 063 and a dual-ended sound pickup option 064 corresponding to the mobile phone. The user can select the sound pickup method corresponding to the headset and the sound pickup method corresponding to the mobile phone from these options. Here, for example, the monaural sound pickup option 061 and the single-ended sound pickup option 063 are selected, correspondingly, the headset adopts the monaural sound pickup method and the mobile phone adopts the single-ended sound pickup method.

[0215] In some examples, the user can select the binaural pickup option 062 to control the headset to adopt a binaural pickup mode; in some examples, the user can select the double-ended pickup option 064 to control the mobile phone to adopt a double-ended pickup mode.

[0216] In some examples, when the smart noise reduction mode is off, control 052 may not be displayed in the quick switch option. That is, when the smart noise reduction mode is on, control 052 is displayed in the quick switch option for the user to open the setting interface 06 to select the sound pickup method.

[0217] It is understandable that after the user sets the sound pickup mode, the mobile phone can send a control message to the headset to control the headset to adopt the corresponding sound pickup mode.

[0218] For the above Figure 6 The codec processing method corresponding to the embodiment shown can also be implemented in a similar manner as above. For example, in some examples, the headset and the mobile phone can use the default codec method when they have the corresponding codec capability. Figure 6 The codec processing method in the embodiment shown processes the audio signal; if the corresponding codec capability is not available, the audio signal is not processed. Figure 6 The codec processing method in the illustrated embodiment processes the audio signal. In some examples, if the headset and the mobile phone have corresponding codec capabilities, the mobile phone may also provide a corresponding option in the above-mentioned setting interface 06 for the user to select whether to use the codec processing method to process the audio signal. It is understood that after the mobile phone and the headset establish a wireless connection, the two can exchange their codec capabilities.

[0219] The above is explained by taking the control of the switch and sound pickup method of the intelligent noise reduction mode on the mobile phone as an example. In some implementation methods, relevant touch operation methods can also be designed on the headset, and the user can control the switch and sound pickup method of the intelligent noise reduction mode by inputting touch operations on the headset. For example, in the sound pickup scenario, the user performs a long press operation (or a continuous click operation, etc.) on the touch area of ​​the headset to turn on or off the intelligent noise reduction mode; when the intelligent noise reduction mode is on, the user performs a continuous click operation (such as a double-click operation, etc.) on the touch area of ​​the headset to switch the sound pickup method. After the user sets the sound pickup method, the headset can send a notification message to the mobile phone to notify the mobile phone to adopt the corresponding sound pickup method. It can be understood that this is just an example, and it is not used to limit this application. The embodiments of this application do not specifically limit the specific implementation method of the touch operation.

[0220] The above description is based on a video call scenario as an example. Similar implementation methods can be used in other sound pickup scenarios, and no further examples will be given here.

[0221] It is understandable that the operations input by the user on the terminal device are not limited to click operations, but may also be other touch operations, voice input operations or gesture operations, etc. The embodiments of the present application do not specifically limit this. The other user operations described in the embodiments of the present application are similar. For the sake of convenience, the embodiments of the present application use click operations as an example for illustrative explanation.

[0222] In addition, the user interface shown in the embodiment of the present application is only an example and is not intended to limit the present application. In some embodiments, the user interface displayed by the electronic device may include more or fewer interface elements than shown in the figure to achieve more or fewer functions; the position of each interface element can be adjusted as needed; each function can also be implemented using other interface elements, or can also be implemented in other user interfaces. This embodiment does not specifically limit this.

[0223] It should also be understood that any one of the above examples or the solution shown in any one of the figures can be an independent solution, or a solution composed of any multiple examples or a solution composed of any multiple figures can also be an independent solution, and this application does not impose any special restrictions on this.

[0224] Based on the above examples, the following describes the steps of the audio processing method provided by this application.

[0225] Figure 10 A flowchart of the audio processing method provided in the embodiment of the present application is shown in FIG. Figure 10 As shown, the audio processing method provided in the embodiment of the present application may include the following steps:

[0226] S100: The headset establishes a wireless communication connection with a terminal device.

[0227] The wireless communication connection established between the headset and the terminal device can be a Bluetooth connection, or the aforementioned Wi-Fi connection, NFC connection, etc.

[0228] The earphones may include a first earphone and a second earphone. In some examples, the first earphone is a primary earphone and the second earphone is a secondary earphone; in some examples, the first earphone is a secondary earphone and the second earphone is a primary earphone.

[0229] In some implementations, one of the first earphone and the second earphone (such as the main earphone) establishes a wireless communication connection with the terminal device; in some implementations, both the first earphone and the second earphone establish a wireless communication connection with the terminal device.

[0230] S200, the headset collects multi-channel audio signals through multiple microphones.

[0231] The headset may be provided with multiple microphones, and the headset may use all or some of the multiple microphones to collect audio signals.

[0232] In some implementations, the earphone can collect multi-channel audio signals through one of the first earphone and the second earphone, that is, the aforementioned monaural sound pickup method.

[0233] In some implementations, the earphones can simultaneously collect multiple audio signals through the first earphone and the second earphone, that is, the aforementioned binaural sound pickup method.

[0234] For other instructions on this step, please refer to the relevant description in the above embodiment and will not be repeated here.

[0235] S300: The earphone transmits the collected multi-channel audio signals to the terminal device.

[0236] In some implementations, the headset may transmit all channels of collected audio signals to the terminal device; in some implementations, the multi-channel audio signals transmitted by the headset to the terminal device may also be part of the audio signals collected by the headset.

[0237] In some embodiments, the multi-channel audio signal transmitted by the earphone to the terminal device may include: the multi-channel audio signal collected by the first earphone.

[0238] In some embodiments, the multi-channel audio signal transmitted by the earphone to the terminal device may include: a multi-channel audio signal collected by the first earphone and a multi-channel audio signal collected by the second earphone.

[0239] In some embodiments, before transmitting multi-channel audio information to a terminal device, the headset can detect whether the signal quality of the wireless communication connection between the headset and the terminal device meets the target conditions. If the signal quality of the wireless communication connection between the headset and the terminal device meets the target conditions, the collected multi-channel audio signals are transmitted to the terminal device through the wireless communication connection; if the signal quality of the wireless communication connection between the headset and the terminal device does not meet the target conditions, the collected multi-channel audio signals are subjected to noise reduction processing and then transmitted to the terminal device through the wireless communication connection to better ensure the quality of the audio signal received by the mobile phone.

[0240] The signal quality of the wireless communication connection may be measured using parameters such as RSSI or distance. The specific detection process may refer to the relevant description in the aforementioned embodiment and will not be described in detail here.

[0241] In some implementations, when transmitting the collected multi-channel audio signals to the terminal device, the headset can encode and compress the collected multi-channel audio signals into one composite audio signal and then transmit it to the terminal device to reduce the amount of transmitted data.

[0242] In some implementations, when transmitting the collected multi-channel audio signals to the terminal device, the headset may perform audio enhancement processing on the collected multi-channel audio signals and then transmit the processed multi-channel audio signals to the terminal device to improve the final sound pickup effect. The audio enhancement processing may include at least one of AGC and AEC.

[0243] In some embodiments, when the headset transmits an audio signal to the mobile phone, the headset can detect the transmission quality of the audio signal and adjust the number of channels of the audio signal transmitted to the mobile phone according to the transmission quality of the audio signal.

[0244] The transmission quality of the audio signal can be measured by parameters such as packet loss rate and signal-to-noise ratio.

[0245] In some implementations, the headset may reduce the number of paths for transmitting the audio signal to the terminal device when the transmission quality of the audio signal is lower than a first threshold quality.

[0246] Optionally, when the number of transmission paths for audio signals is reduced to one, the headset may perform noise reduction processing on each channel of collected audio signals and then transmit the collected audio signals to the terminal device.

[0247] In some implementations, the headset may increase the number of audio signal transmission paths to the terminal device when the number of audio signal transmission paths is less than the maximum number of transmission paths and the transmission quality is higher than a second threshold quality; wherein the second threshold quality is higher than the first threshold quality.

[0248] S400: The terminal device receives multi-channel audio signals transmitted by headphones.

[0249] As described above, in some embodiments, when transmitting audio signals to a mobile phone, the headset can adjust the number of channels of audio signals transmitted to the mobile phone based on the transmission quality of the audio signals. Correspondingly, the number of channels of audio signals received by the terminal device can be reduced or increased accordingly.

[0250] In some embodiments, the terminal device receives an audio signal after the headset performs noise reduction processing on multiple audio signals.

[0251] In some embodiments, the multi-channel audio signals transmitted by the earphone are encoded and compressed into one composite audio signal, and correspondingly, the terminal device receives the composite audio signal.

[0252] S500: The terminal device performs noise reduction processing on the received audio signal to obtain a target audio signal.

[0253] In some implementations, after receiving the audio signal transmitted by the earphone, the terminal device performs noise reduction processing based only on the audio signal to obtain a target audio signal.

[0254] In some implementations, while the headset collects audio signals through the microphone, the terminal device can also turn on the microphone to collect audio signals to obtain richer audio data; after the terminal device receives the audio signal transmitted by the headset, it performs noise reduction processing based on the received audio signal and the audio signal collected by the terminal device to obtain the target audio signal to improve the sound pickup effect.

[0255] As described above, in some embodiments, the multi-channel audio signals transmitted by the headset are encoded and compressed into a composite audio signal, which is received by the terminal device. Correspondingly, the terminal device can first decode the received composite audio signal and then perform noise reduction processing on the decoded multi-channel audio signal to obtain the target audio signal.

[0256] In some embodiments, the terminal device performs noise reduction processing on the received audio signal when the target noise reduction function (such as the aforementioned intelligent noise reduction mode) is turned on; when the battery level of the terminal device is lower than the target battery level, the target noise reduction function is turned off to reduce the power consumption of the terminal device and save power.

[0257] For other instructions in the above steps, please refer to the relevant description in the above embodiments and will not be repeated here.

[0258] Based on the same concept, as an implementation of the above method, an embodiment of the present application provides an audio processing device, which can be applied to headphones. Figure 11 A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application is shown in FIG. Figure 11 As shown, the audio processing device 300 provided in this embodiment may include: an input module 310 , a processing module 320 and a communication module 330 .

[0259] The input module 310 is configured to receive user input on the headset's display interface, such as touch input, voice input, gesture input, etc. The input module is configured to support the headset in executing the steps for receiving user input in the aforementioned embodiments and / or other processes used in the technology described herein. The input module may be a touch screen or other hardware or a combination of hardware and software.

[0260] The processing module 320 is used to support the headset to execute the processing operations in the steps of the methods in the above embodiments and / or other processes used in the technology described herein.

[0261] The communication module 330 is used to support the headset to perform operations related to the communication process between the terminal device and other devices in the above embodiments and / or other processes used for the technology described herein.

[0262] The audio processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be repeated here.

[0263] An embodiment of the present application also provides an audio processing device, which can be applied to a terminal device. Figure 12 This is a structural diagram of another audio processing device provided in an embodiment of the present application, such as Figure 12 As shown, the audio processing device 400 provided in this embodiment may include: a display module 410 , an input module 420 , a processing module 430 and a communication module 440 .

[0264] The display module 410 is used to support the terminal device in performing the interface display operations in the above embodiments and / or other processes for the technology described herein. The display module can be a touch screen or other hardware or a combination of hardware and software.

[0265] The input module 420 is used to receive user input on the display interface of the terminal device, such as touch input, voice input, gesture input, etc. The input module is used to support the terminal device in executing the steps of receiving user operations in the above embodiments and / or other processes used in the technology described herein. The input module can be a touch screen or other hardware or a combination of hardware and software.

[0266] The processing module 430 is used to support the terminal device in executing the processing operations in the steps of the methods in the above embodiments and / or other processes used in the technology described herein.

[0267] The communication module 440 is used to support the terminal device to perform operations related to the communication process between the headset and other devices in the above embodiments and / or other processes used for the technology described herein.

[0268] The audio processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be repeated here.

[0269] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The functional characteristics of the above-mentioned integrated units can be implemented in the form of hardware, or in the form of software, or in the form of a combination of hardware and software. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0270] An embodiment of the present application further provides a headset, comprising: a memory and a processor, wherein the memory is used to store a program; and the processor is used to execute the method described in the above method embodiment when calling the program.

[0271] An embodiment of the present application further provides a terminal device, comprising: a memory and a processor, wherein the memory is used to store a program; and the processor is used to execute the method described in the above method embodiment when calling the program.

[0272] An embodiment of the present application further provides a communication system, which includes the headset and terminal device in the above embodiment, and is used to execute the method in the above method embodiment.

[0273] An embodiment of the present application further provides a readable storage medium (also referred to as a computer-readable storage medium) on which a program is stored. When the program is executed by a processor, the method described in the above method embodiment is implemented.

[0274] The embodiment of the present application further provides a program product (also referred to as a computer program product), which, when executed on a device, enables the device to implement the method described in the above method embodiment.

[0275] The present application also provides a chip system, including a processor coupled to a memory, wherein the processor executes a program stored in the memory to implement the method described in the above method embodiment. The chip system can be a single chip or a chip module composed of multiple chips.

[0276] In the above embodiments, each processing step or functional feature can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a program product. The program product includes one or more instructions. When the instructions are loaded and executed on a device, the process or function described in the embodiments of the present application is generated in whole or in part. The instructions can be stored in a readable storage medium or transmitted via the readable storage medium.

[0277] The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0278] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0279] In the embodiments provided in this application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0280] It should be understood that in the description of this application and the appended claims, the terms "comprises," "includes," "has," and any variations thereof are intended to cover non-exclusive inclusions and mean "including but not limited to," unless otherwise specifically emphasized. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to the process, method, product, or apparatus.

[0281] In the description of this application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in this application is used to describe the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural.

[0282] Furthermore, in the description of this application, unless otherwise specified, "plurality" refers to two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0283] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0284] In addition, in the description of this application specification and the appended claims, the terms "first," "second," etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or precedence, nor should they be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein; and features specified as "first" or "second" may explicitly or implicitly include at least one of such features.

[0285] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0286] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized.

[0287] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An audio processing method, characterized in that: Applied to a headset, the headset includes a plurality of microphones, and the method includes: Establishing a wireless communication connection with a terminal device; Collecting multiple audio signals through the multiple microphones; The collected multi-channel audio signals are transmitted to the terminal device through the wireless communication connection, so that the terminal device performs noise reduction processing on the multi-channel audio signals.

2. The method according to claim 1, characterized in that The method further comprises: When the signal quality of the wireless communication connection between the headset and the terminal device does not meet the target condition, the collected multi-channel audio signals are subjected to noise reduction processing and then transmitted to the terminal device through the wireless communication connection.

3. The method according to claim 2, characterized in that The method further comprises: When the transmission quality of the audio signal is lower than a first threshold quality, the number of channels of the audio signal transmitted to the terminal device is reduced.

4. The method according to claim 3, characterized in that: The method further comprises: When the number of transmission paths of the audio signal is reduced to one, each channel of the collected audio signal is subjected to noise reduction processing and then transmitted to the terminal device.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: When the number of transmission paths of the audio signal is less than the maximum number of transmission paths and the transmission quality is higher than a second threshold quality, the number of paths of the audio signal transmitted to the terminal device is increased; and the second threshold quality is higher than the first threshold quality.

6. The method according to any one of claims 1 to 5, characterized in that: The earphone includes a first earphone and a second earphone, and the multi-channel audio signal transmitted to the terminal device includes: the multi-channel audio signal collected by the first earphone.

7. The method according to any one of claims 1 to 5, characterized in that: The earphone comprises a first earphone and a second earphone, and the multi-channel audio signal transmitted to the terminal device comprises: the multi-channel audio signal collected by the first earphone and the multi-channel audio signal collected by the second earphone.

8. The method according to any one of claims 1 to 7, characterized in that: The transmitting the collected multi-channel audio signal to the terminal device includes: The collected multi-channel audio signals are encoded and compressed into a composite audio signal and then transmitted to the terminal device.

9. The method according to any one of claims 1 to 8, characterized in that: The transmitting the collected multi-channel audio signal to the terminal device includes: After performing audio enhancement processing on the collected multi-channel audio signals, the processed multi-channel audio signals are transmitted to the terminal device; the audio enhancement processing includes acoustic echo cancellation and / or automatic gain control.

10. An audio processing method, characterized in that: Applied to a terminal device, the method comprises: Establishing a wireless communication connection with the headset; Receiving multi-channel audio signals transmitted by the earphone; The received audio signal is subjected to noise reduction processing to obtain a target audio signal.

11. The method according to claim 10, characterized in that The step of performing noise reduction processing on the received audio signal to obtain a target audio signal includes: Noise reduction processing is performed according to the received audio signal and the audio signal collected by the terminal device to obtain a target audio signal.

12. The method according to claim 10 or 11, characterized in that: When the number of paths of the audio signal transmitted by the earphone is greater than one and the transmission quality is lower than a first threshold quality, the number of paths of the audio signal transmitted by the earphone is reduced; and / or, The number of channels of the audio signal transmitted by the earphone increases when the number of channels is less than the maximum number of channels and the transmission quality is higher than a second threshold quality.

13. The method according to any one of claims 10 to 12, characterized in that: The multi-channel audio signals transmitted by the earphone are encoded and compressed into one composite audio signal, and the received audio signals are subjected to noise reduction processing to obtain a target audio signal, including: The received composite audio signal is decoded, and the multi-channel audio signals obtained by decoding are subjected to noise reduction processing to obtain the target audio signal.

14. The method according to any one of claims 10 to 13, characterized in that: When the target noise reduction function is turned on, the terminal device performs noise reduction processing on the received audio signal; when the power level is lower than the target power level, the target noise reduction function is turned off.

15. An audio processing method, applied to a system consisting of headphones and terminal equipment, characterized in that: include: The headset establishes a wireless communication connection with the terminal device; The earphone collects multi-channel audio signals through multiple microphones; The earphone transmits the collected multi-channel audio signals to the terminal device; The terminal device receives the audio signal transmitted by the headset; The terminal device performs noise reduction processing on the received audio signal to obtain a target audio signal.

16. The method according to claim 15, characterized in that The method further comprises: When the transmission quality of the audio signal is lower than a first threshold quality, the headset reduces the number of paths of the audio signal transmitted to the terminal device; When the number of transmission paths of the audio signal is less than the maximum number of transmission paths and the transmission quality is higher than a second threshold quality, the number of paths of the audio signal transmitted to the terminal device is increased; and the second threshold quality is higher than the first threshold quality.

17. The method according to claim 15 or 16, characterized in that The headset transmits the collected multi-channel audio signals to the terminal device, including: The headset encodes and compresses the collected multi-channel audio signals into one composite audio signal and transmits the composite audio signal to the terminal device; The terminal device performs noise reduction processing on the received audio signal to obtain a target audio signal, including: The terminal device decodes the received composite audio signal, performs noise reduction processing on the decoded multi-channel audio signals, and obtains the target audio signal.

18. An earphone, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method according to any one of claims 1 to 9 when calling the computer program.

19. A terminal device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method according to any one of claims 10 to 14 when calling the computer program.

20. A readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

21. A program product, characterized in that When the program product is run on a device, the device is caused to execute the method according to any one of claims 1 to 14.

22. A chip system, characterized in that: The chip system includes a processor, which is coupled to a memory, and the processor executes a program stored in the memory to implement the method according to any one of claims 1-14.

23. A communication system comprising: The headset as claimed in claim 18 and the terminal device as claimed in claim 19.

Images