Noise reduction method and device, equipment, storage medium and program product

Abstract

The invention discloses a noise reduction method and device, equipment, a storage medium and a program product, and relates to the technical field of audio processing, and the method comprises the steps: collecting a bone conduction signal and at least two paths of environment air conduction signals of an earphone wearer through an earphone, based on the bone conduction signal, performing first noise reduction processing on a target air conduction signal of a target direction determined based on the at least two paths of environment air conduction signals to remove a first air conduction component of a relatively low frequency band of an earphone wearer, and obtaining an initial noise-reduced signal; and performing second noise reduction processing on the initial noise-reduced signal based on the bone conduction signal, the at least two paths of environmental air conduction signals and the phase difference of the at least two paths of environmental air conduction signals, so as to remove a second air conduction signal of a higher frequency band of the earphone wearer, and obtain a target noise-reduced signal. According to the method and the device, the noise generated by the earphone wearer in the audio recorded by the earphone is effectively removed.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to a noise reduction method, apparatus, device, storage medium, and program product. Background Technology

[0002] Currently, some headphones have a live recording function, which records the sounds in the environment surrounding the headphone wearer (such as the speaker's voice). However, the recorded sound will contain sounds generated by the headphone wearer (such as speech, chewing sounds, breathing sounds, etc.), which are noise relative to the speaker's voice. Existing noise reduction methods cannot filter out the noise generated by the headphone wearer.

[0003] Therefore, how to filter out the noise generated by the headphone wearer in the audio recorded by the headphones has become an urgent technical problem to be solved. Summary of the Invention

[0004] In view of the above problems, this application provides a noise reduction method, apparatus, device, storage medium, and program product to filter out noise generated by the headphone wearer in audio recorded by headphones. The specific solution is as follows:

[0005] The first aspect of this application provides a noise reduction method, including:

[0006] Obtain the bone conduction signal of the headphone wearer collected by the headphone, as well as at least two ambient air conduction signals;

[0007] Based on the bone conduction signal, the target air conduction signal in the target direction determined by the at least two environmental air conduction signals is subjected to a first noise reduction process to remove the first air conduction component in the target air conduction signal corresponding to the bone conduction signal, so as to obtain the initial noise-reduced signal.

[0008] The initial noise-reduced signal is subjected to a second noise reduction process based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals, in order to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal, thereby obtaining the target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the frequency band to which the first air conduction component belongs.

[0009] In one possible implementation, a first noise reduction process is performed on the target air conduction signal based on the bone conduction signal, including:

[0010] The bone conduction signal is converted into a first air conduction signal for the headphone wearer;

[0011] The difference between the time-frequency representation of the target air conduction signal and the time-frequency representation of the first air conduction signal is determined as the initial denoised signal.

[0012] In one possible implementation, converting the bone conduction signal into a first air conduction signal for the headphone wearer includes:

[0013] The bone conduction signal is filtered by an adaptive filter to obtain the first air conduction signal.

[0014] The parameters of the adaptive filter are updated with the goal of minimizing the energy of the differential signal between the target air conduction signal and the air conduction signal output by the adaptive filter.

[0015] In one possible implementation, the second noise reduction processing of the initially noise-reduced signal based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals includes:

[0016] The time-frequency representations of the bone conduction signal, the time-frequency representations of each environmental air conduction signal, the phase difference, and the time-frequency representation of the initial denoised signal are input into the denoising model to obtain the denoised time-frequency sequence.

[0017] The denoised time-frequency sequence is subjected to inverse short-time Fourier transform to obtain the target denoised signal.

[0018] In one possible implementation, acquiring the bone conduction signal of the headphone wearer includes:

[0019] Bone conduction signals are collected from the helix or near the helix of the earphone wearer.

[0020] In one possible implementation, at least two ambient atmospheric conduction signals are acquired, including:

[0021] At least two ambient air currents are collected through at least two air conduction microphones located on the part of the earpiece positioned in front of the ear and in contact with the wearer's face.

[0022] A second aspect of this application provides a noise reduction device, comprising:

[0023] The acquisition module is used to acquire the bone conduction signal of the headphone wearer collected by the headphone, as well as at least two ambient air conduction signals;

[0024] A first-level noise reduction module is used to perform a first noise reduction process on the target air conduction signal determined based on the two environmental air conduction signals based on the bone conduction signal, so as to remove the first air conduction component corresponding to the bone conduction signal in the target air conduction signal and obtain the initial noise-reduced signal.

[0025] A secondary noise reduction module is used to perform a second noise reduction process on the initial noise-reduced signal based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals, so as to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal and obtain the target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the frequency band to which the first air conduction component belongs.

[0026] A third aspect of this application provides an earphone, comprising:

[0027] Bone conduction microphone is used to collect bone conduction signals from the headphone wearer;

[0028] At least two air conduction microphones, each used to acquire one ambient air conduction signal;

[0029] The noise reduction unit is configured to perform a first noise reduction process on a target air conduction signal determined based on at least two ambient air conduction signals in a target direction, based on the bone conduction signal, to remove the first air conduction component corresponding to the bone conduction signal in the target air conduction signal, thereby obtaining an initial noise-reduced signal; and to perform a second noise reduction process on the initial noise-reduced signal based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference between the at least two ambient air conduction signals, to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal, thereby obtaining a target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the frequency band to which the first air conduction component belongs.

[0030] In one possible implementation, the earphone is an ear-hook earphone, and when the wearer wears the earphone, the bone conduction microphone is located at or near the helix of the wearer's ear.

[0031] Alternatively, the headphones are eyeglass-style headphones, in which the bone conduction microphone is located on the temple of the glasses and in contact with the wearer when the wearer wears the headphones.

[0032] In one possible implementation, the at least two air conduction microphones are located on the ear-hook headphones in front of the ears and in contact with the wearer's face;

[0033] Alternatively, the at least two air conduction microphones are located on the temples of the glasses-style headphones.

[0034] A fourth aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the noise reduction method of the first aspect or any implementation thereof.

[0035] A fifth aspect of this application provides an electronic device, including at least one processor, a memory connected to the processor, and a communication device, wherein:

[0036] The communication device is used to connect and communicate with the headphones to receive bone conduction signals from the headphone wearer collected by the headphones, as well as at least two ambient air conduction signals.

[0037] The memory is used to store computer programs;

[0038] The processor is used to execute the computer program so that the electronic device can implement the noise reduction method of the first aspect or any implementation thereof.

[0039] The sixth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the noise reduction method described in the first aspect or any implementation thereof.

[0040] A seventh aspect of this application provides a noise reduction system, the noise reduction system comprising:

[0041] Headphones, and electronic devices connected to said headphones;

[0042] The earphone is used to collect the bone conduction signal of the earphone wearer and at least two ambient air conduction signals, and to send the bone conduction signal and the at least two ambient air conduction signals to the electronic device;

[0043] The electronic device is used to perform a first noise reduction process on a target air conduction signal determined based on the bone conduction signal and the target direction determined by the at least two ambient air conduction signals, to remove the first air conduction component corresponding to the bone conduction signal in the target air conduction signal, to obtain an initial noise-reduced signal; and to perform a second noise reduction process on the initial noise-reduced signal based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals, to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal, to obtain a target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the frequency band to which the first air conduction component belongs.

[0044] By employing the above technical solutions, the noise reduction method, apparatus, device, storage medium, and program product provided in this application acquire the bone conduction signal of the headphone wearer and at least two ambient air conduction signals through headphones. Based on the bone conduction signal, a first noise reduction process is performed on the target air conduction signal in the target direction determined by the at least two ambient air conduction signals to remove the lower-frequency first air conduction component of the headphone wearer, obtaining an initial noise-reduced signal. Based on the phase difference between the bone conduction signal, the at least two ambient air conduction signals, and the at least two ambient air conduction signals, a second noise reduction process is performed on the initial noise-reduced signal to remove the higher-frequency second air conduction signal of the headphone wearer, obtaining a target noise-reduced signal. Based on this application, effective removal of noise generated by the headphone wearer is achieved while ensuring the clarity of ambient sound in the audio recorded by the headphones. Attached Figure Description

[0045] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0046] Figure 1 A flowchart illustrating an implementation of the noise reduction method provided in this application;

[0047] Figure 2 This is a flowchart illustrating a first noise reduction process for a target air conduction signal based on a bone conduction signal, as provided in this application.

[0048] Figure 3 This application provides a flowchart for performing a second noise reduction process on the initially noise-reduced signal based on bone conduction signals, at least two ambient air conduction signals, and the phase difference of at least two ambient air conduction signals.

[0049] Figure 4 Example diagram of an ear provided for this application;

[0050] Figure 5 An example view of the appearance of the headphones provided in this application;

[0051] Figure 6 A schematic diagram of the noise reduction device provided in this application;

[0052] Figure 7 A schematic diagram of the structure of the earphone provided in this application;

[0053] Figure 8 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0054] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0055] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0056] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0057] The noise reduction method of this application can be used in headphones (e.g., ear-hook headphones, smart glasses headphones, etc.), or it can be implemented in combination with headphones and electronic devices (e.g., mobile phones and other electronic devices that can establish a communication connection with headphones).

[0058] like Figure 1 The diagram shown is a flowchart of one implementation of the noise reduction method provided in this application, which may include:

[0059] Step S101: Obtain the bone conduction signal of the headphone wearer collected by the headphone, as well as at least two ambient air conduction signals.

[0060] The bone conduction signal (also known as bone conduction signal) of the headphone wearer can be collected by the bone conduction microphone (also known as bone conduction microphone) on the headphone, and at least two air conduction signals (also known as air conduction signals) can be collected by at least two air conduction microphones (also known as air conduction microphones) on the headphone. Since the air conduction microphone collects the sound in the environment, it is referred to as environmental air conduction signal in this application, and each air conduction microphone collects one environmental air conduction signal.

[0061] Bone conduction microphones are used to collect mechanical vibration signals transmitted through the bones when the wearer speaks. These mechanical vibration signals are converted into vibration-electrical signals to reflect the low-frequency characteristics (mainly 50-1000Hz) of vocal cord vibration.

[0062] The air conduction signal collected by an air conduction microphone includes the following components: the air conduction component of the wearer's own voice, environmental noise, and the target sound source signal (such as the speaker's air conduction signal).

[0063] Optionally, to further improve the noise reduction effect, the bone conduction signal and the ambient air conduction signal can be preprocessed, and the subsequent noise reduction process can be carried out based on the preprocessed bone conduction signal and at least two preprocessed ambient air conduction signals.

[0064] As an example, the bone conduction signal can be bandpass filtered to retain mid-to-low frequency signals (signals in the frequency range of 50Hz-2000Hz) while removing DC and other frequency band signals. The filtered bone conduction signal and at least two ambient air conduction signals are then synchronously sampled and gain-calibrated to obtain a preprocessed bone conduction signal and at least two preprocessed ambient air conduction signals.

[0065] Step S102: Perform a first noise reduction process on the target air conduction signal based on the bone conduction signal and the target direction determined by at least two environmental air conduction signals to remove the first air conduction component corresponding to the bone conduction signal in the target air conduction signal, and obtain the initial noise-reduced signal.

[0066] Differential beamforming can be performed on at least two ambient air conduction signals to obtain the air conduction signal in the target direction (denoted as the target air conduction signal), which is the direction in which the speaker (e.g., conference speaker, speaker, etc.) is located.

[0067] Differential beamforming utilizes the spatial differences between air-conducting microphones to calculate the time difference of arrival (TDOA) and phase difference of sound, and then enhances the signal in the target direction through delay-and-sum or minimum variance distortionless response (MVDR) algorithms.

[0068] The first air conduction component refers to the low-frequency air conduction component (signal component with a frequency less than 2kHz) carried in the target air conduction signal of the headphone wearer.

[0069] With the bone conduction signal and the ambient air conduction signal preprocessed, differential beamforming can be performed on at least two preprocessed ambient air conduction signals to obtain the target air conduction signal in the target direction; and the target air conduction signal in the target direction can be subjected to a first noise reduction process based on the preprocessed bone conduction signal.

[0070] Step S103: Perform a second noise reduction process on the initial noise-reduced signal based on the bone conduction signal, at least two ambient air conduction signals, and the phase difference of at least two ambient air conduction signals, so as to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal and obtain the target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the frequency band to which the first air conduction component belongs.

[0071] A neural network model can be used to perform a second noise reduction process on the initially noise-reduced signal based on the bone conduction signal, at least two ambient air conduction signals, and the phase difference of at least two ambient air conduction signals, in order to remove the second air conduction component of the headphone wearer carried in the initially noise-reduced signal. The second air conduction component is the high-frequency air conduction component of the headphone wearer (signal component with a frequency greater than or equal to 2kHz) carried in the target air conduction signal.

[0072] With the bone conduction signal and the ambient air conduction signal preprocessed, a second noise reduction process can be performed on the initially noise-reduced signal based on the preprocessed bone conduction signal, at least two preprocessed ambient air conduction signals, and the phase difference of at least two preprocessed ambient air conduction signals.

[0073] The above steps S102 and S103 can be performed by the headset or by an electronic device that establishes a communication connection with the headset.

[0074] Optionally, when acquiring at least three ambient air conduction signals, the phase difference of these at least three ambient air conduction signals can be determined in the following way:

[0075] Calculate the phase difference between each pair of ambient air conduction signals;

[0076] The average value of each calculated phase difference is determined as the phase difference of the at least three ambient air conduction signals.

[0077] The noise reduction method provided in this application involves acquiring the bone conduction signal of the headphone wearer and at least two ambient air conduction signals through headphones. Based on the bone conduction signal, a first noise reduction process is performed on the target air conduction signal in the target direction determined by the at least two ambient air conduction signals to remove the lower frequency first air conduction component of the headphone wearer, resulting in an initial noise-reduced signal. Based on the phase difference between the bone conduction signal, the at least two ambient air conduction signals, and the at least two ambient air conduction signals, a second noise reduction process is performed on the initial noise-reduced signal to remove the higher frequency second air conduction signal of the headphone wearer, resulting in a target noise-reduced signal. This method effectively removes noise generated by the headphone wearer in the recorded audio while ensuring the clarity of ambient sound in the audio recorded by the headphones.

[0078] In an optional embodiment, a flowchart of the above-described implementation of the first noise reduction process for the target air conduction signal based on the bone conduction signal is shown below. Figure 2 As shown, it may include:

[0079] Step S201: Convert the bone conduction signal into the first air conduction signal of the headphone wearer.

[0080] An adaptive filter can be used to convert the bone conduction signal into the air conduction signal of the headphone wearer (i.e., the first air conduction component). The parameters of this adaptive filter are updated with the goal of minimizing the energy of the difference signal between the target air conduction signal and the air conduction signal output by the adaptive filter.

[0081] When updating the parameters of an adaptive filter, either the Least Mean Square (LMS) algorithm or the Recursive Least Squares (RLS) algorithm can be used. Compared to the LMS algorithm, the RLS algorithm results in faster convergence of the adaptive filter.

[0082] The differential signal between the target air conduction signal and the air conduction signal output by the adaptive filter is obtained by performing a difference operation between the two signals. The energy of the differential signal can be the mean square error of the amplitudes at each sampling point that constitutes the differential signal.

[0083] Assume the bone conduction signal is denoted as V bonc (t), the adaptive filter is denoted as w(t), and the first air conduction signal is denoted as The bone conduction signal can then be converted into a first air conduction signal using the following formula:

[0084] (1)

[0085] Assuming the target air conduction signal is Then, the difference signal e(t) between the target air conduction signal and the air conduction signal output by the adaptive filter is:

[0086] (2)

[0087] When updating the parameters of the adaptive filter, the goal is to minimize the energy of e(t).

[0088] The bone conduction signal and air conduction signal in formulas (1) and (2) can be signals before sampling or signals after sampling.

[0089] Step S202: The difference between the time-frequency representation of the target air conduction signal and the time-frequency representation of the first air conduction signal is determined as the initial denoised signal.

[0090] A short-time Fourier transform (STFT) can be performed on the target air conduction signal to obtain its time-frequency representation. Perform a short-time Fourier transform on the first air conduction signal to obtain its time-frequency representation. Correspondingly, the initial noise reduction signal It can be:

[0091] (3)

[0092] In an optional embodiment, a flowchart illustrating the second noise reduction process performed on the initially noise-reduced signal based on the bone conduction signal, at least two ambient air conduction signals, and the phase difference between the at least two ambient air conduction signals is shown below. Figure 3 As shown, it may include:

[0093] Step S301: Input the time-frequency representation of the bone conduction signal, the time-frequency representation of each environmental air conduction signal, the phase difference, and the time-frequency representation of the initial denoised signal into the denoising model to obtain the denoised time-frequency sequence.

[0094] The denoising model can be a pre-trained neural network model. The denoising model can be obtained through supervised training of the neural network model.

[0095] In the training dataset used to train the noise reduction model, each training sample may include: a bone conduction signal collected by the headphones and at least two ambient air conduction signals, the phase difference of each ambient air conduction signal, and the initial noise-reduced signal. The initial noise-reduced signal in the training sample is obtained by performing a first noise reduction process on the target air conduction signal, determined based on the target direction based on the at least two ambient air conduction signals in the training sample, using the bone conduction signal from the training sample. The label of the training sample (denoted as the sample label) is the air conduction signal that does not carry the sound signal generated by the headphone wearer. As an example, the sample label can be collected by the headphones when the headphone wearer is silent in an acoustic environment (e.g., a speech environment or other possible recording scenarios).

[0096] As an example, each ambient air conduction signal in the training sample can be obtained by fusing an air conduction signal that does not carry the sound signal generated by the headphone wearer (collected by the headphone when the headphone wearer is silent in a sound environment) with an air conduction signal that only contains the sound signal generated by the headphone wearer (collected by the headphone when the headphone wearer makes a sound in a quiet environment).

[0097] When training the denoising model, the time-frequency representation of the bone conduction signal in the training samples, the phase difference of the time-frequency representation of each environmental air conduction signal, and the time-frequency representation of the initial denoised signal can be input into the denoising model to obtain the denoised time-frequency sequence output by the denoising model (i.e., the time-frequency representation of the denoised signal). The parameters of the denoising model are updated with the goal of the denoised time-frequency sequence output by the denoising model approaching the time-frequency representation of the sample label.

[0098] Step S302: Perform inverse short-time Fourier transform (ISTFT) on the denoised time-frequency sequence to obtain the target denoised signal.

[0099] The target noise-reduced signal is the pure ambient sound signal after removing the wearer's own noise.

[0100] In an optional embodiment, one way to collect the bone conduction signal of the headphone wearer described above is as follows:

[0101] It collects bone conduction signals from the helix or its vicinity near the user's earlobe. In other words, when a user wears headphones, the bone conduction microphone on the headphones is located at or near the user's helix.

[0102] like Figure 4 The image shown is an example diagram of an ear provided in an embodiment of this application. In this example diagram, the red circular area represents the helix and its surrounding area.

[0103] In an optional embodiment, one way to implement the above-mentioned acquisition of at least two ambient air conduction signals is as follows:

[0104] At least two ambient air currents are collected using at least two air-conduction microphones located on the part of the earpiece positioned in front of the ear and in contact with the wearer's face.

[0105] like Figure 5 The image shown is an example of the appearance of an earphone provided in an embodiment of this application. In this example image, the two red circular areas are the areas where the two air conduction microphones are located, and the area within the red rectangular frame is the area where the bone conduction microphone is located. When the user wears... Figure 5 As shown, the main part within the dashed box is located in front of the ear, the area within the red rectangle is near the helix or the base of the helix, and the area where the two air conduction microphones are located is in contact with the user's face, with the microphone holes facing upwards. Figure 5 The part outside the dotted frame of the earphone shown is an auxiliary component, which assists the main body in hanging the earphone on the user's ear. When the user wears the earphone, the auxiliary component is located behind the ear.

[0106] In an optional embodiment, the two air conduction microphones may be located at Figure 5 The microphone hole is facing downwards at the position shown.

[0107] Corresponding to the method embodiments, this application also provides a noise reduction device. A schematic diagram of a noise reduction device provided in the embodiments of this application is shown below. Figure 6 As shown, it may include:

[0108] Obtain module 601, primary noise reduction module 602 and secondary noise reduction module 603;

[0109] The acquisition module 601 is used to acquire the bone conduction signal of the headphone wearer collected by the headphone, as well as at least two ambient air conduction signals.

[0110] The first-level noise reduction module 602 is used to perform a first noise reduction process on the target air conduction signal determined based on the two environmental air conduction signals based on the bone conduction signal, so as to remove the first air conduction component corresponding to the bone conduction signal in the target air conduction signal and obtain the initial noise-reduced signal.

[0111] The secondary noise reduction module 603 is used to perform a second noise reduction process on the initial noise-reduced signal based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals, so as to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal and obtain the target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the frequency band to which the first air conduction component belongs.

[0112] The noise reduction device provided in this application acquires the bone conduction signal of the headphone wearer and at least two ambient air conduction signals through headphones. Based on the bone conduction signal, a first noise reduction process is performed on the target air conduction signal in the target direction determined by the at least two ambient air conduction signals to remove the lower frequency first air conduction component of the headphone wearer, resulting in an initial noise-reduced signal. Based on the phase difference between the bone conduction signal, the at least two ambient air conduction signals, and the at least two ambient air conduction signals, a second noise reduction process is performed on the initial noise-reduced signal to remove the higher frequency second air conduction signal of the headphone wearer, resulting in a target noise-reduced signal. This achieves effective removal of noise generated by the headphone wearer in the recorded audio while ensuring the clarity of ambient sound in the audio recorded by the headphones.

[0113] Optionally, the detailed and extended functions of each module can be found in the description above, and will not be repeated here.

[0114] Each module in the aforementioned noise reduction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0115] Corresponding to the method embodiments, this application also provides an earphone, such as Figure 7 The diagram shown is a structural schematic of an earphone provided in an embodiment of this application, which may include:

[0116] Bone conduction microphone 701, at least two air conduction microphones 702 and noise reduction unit 703;

[0117] Among them, the bone conduction microphone 701 is used to collect bone conduction signals from the headphone wearer;

[0118] Each air conduction microphone 702 is used to acquire one ambient air conduction signal;

[0119] The noise reduction unit 703 is used to perform a first noise reduction process on the target air conduction signal determined based on the bone conduction signal and the target direction determined based on the at least two ambient air conduction signals, so as to remove the first air conduction component corresponding to the bone conduction signal in the target air conduction signal to obtain an initial noise-reduced signal; and to perform a second noise reduction process on the initial noise-reduced signal based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals, so as to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal to obtain a target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the component to which the first air conduction component belongs.

[0120] In an optional embodiment, the earphone is an ear-hook type, and when the wearer wears the earphone, the bone conduction microphone is located at or near the helix of the wearer's ear. Correspondingly, at least two air conduction microphones may be located on the portion of the ear-hook earphone positioned in front of the ear and conforming to the wearer's face.

[0121] In one optional embodiment, the headphones are eyeglass-style headphones (shaped like eyeglasses, with headphone functions integrated into the temples), and when the wearer wears the headphones, the bone conduction microphone is located on the temples of the eyeglasses in contact with the wearer. Correspondingly, at least two air conduction microphones are located on the temples of the eyeglass-style headphones.

[0122] This application also provides an electronic device in its embodiments. (See reference...) Figure 8 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic device in the embodiments of this application can be a terminal device (e.g., an in-vehicle infotainment system, a large-screen device, a smart home device, a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.). Figure 8 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0123] like Figure 8 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. When the electronic device is powered on, the RAM 803 also stores various programs and data required for the operation of the electronic device. The processing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0124] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, memory cards, hard drives, etc.; and communication devices 809. Communication device 809 allows electronic devices to communicate wirelessly or wiredly with other devices (e.g., headphones) to exchange data. Although Figure 8 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0125] Corresponding to the method embodiments, this application also provides a noise reduction system, which includes headphones and an electronic device connected to the headphones;

[0126] The earphone is used to collect the bone conduction signal of the earphone wearer and at least two ambient air conduction signals, and to send the bone conduction signal and the at least two ambient air conduction signals to the electronic device.

[0127] The electronic device is used to perform a first noise reduction process on a target air conduction signal determined based on the bone conduction signal and the target direction determined by the at least two ambient air conduction signals, to remove the first air conduction component corresponding to the bone conduction signal in the target air conduction signal, to obtain an initial noise-reduced signal; and to perform a second noise reduction process on the initial noise-reduced signal based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals, to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal, to obtain a target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the frequency band to which the first air conduction component belongs.

[0128] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the noise reduction methods provided in this application.

[0129] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the noise reduction methods provided in this application.

[0130] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0132] In the above embodiments, the functionality can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented entirely or partially as a computer program product. Those skilled in the art can use different methods to implement the described functions for each specific solution, but such implementation should not be considered beyond the scope of this application.

[0133] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0134] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0135] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A noise reduction method, characterized in that, include: Obtain the bone conduction signal of the headphone wearer collected by the headphone, as well as at least two ambient air conduction signals; Based on the bone conduction signal, the target air conduction signal in the target direction determined by the at least two environmental air conduction signals is subjected to a first noise reduction process to remove the first air conduction component in the target air conduction signal corresponding to the bone conduction signal, so as to obtain the initial noise-reduced signal. The initial noise-reduced signal is subjected to a second noise reduction process based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals, in order to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal, thereby obtaining the target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the frequency band to which the first air conduction component belongs.

2. The method according to claim 1, characterized in that, The first noise reduction process for the target air conduction signal based on the bone conduction signal includes: The bone conduction signal is converted into a first air conduction signal for the headphone wearer; The difference between the time-frequency representation of the target air conduction signal and the time-frequency representation of the first air conduction signal is determined as the initial denoised signal.

3. The method according to claim 2, characterized in that, The step of converting the bone conduction signal into a first air conduction signal for the headphone wearer includes: The bone conduction signal is filtered by an adaptive filter to obtain the first air conduction signal. The parameters of the adaptive filter are updated with the goal of minimizing the energy of the differential signal between the target air conduction signal and the air conduction signal output by the adaptive filter.

4. The method according to claim 1, characterized in that, The second noise reduction process, based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals, includes: The time-frequency representations of the bone conduction signal, the environmental air conduction signals, the phase difference, and the initial denoised signal are input into the denoising model to obtain the denoised time-frequency sequence. The denoised time-frequency sequence is subjected to inverse short-time Fourier transform to obtain the target denoised signal.

5. The method according to claim 1, characterized in that, The acquisition of bone conduction signals from the headphone wearer includes: Bone conduction signals are collected from the helix or near the helix of the earphone wearer.

6. The method according to claim 1, characterized in that, Collect at least two ambient atmospheric conduction signals, including: At least two ambient air currents are collected through at least two air conduction microphones located on the part of the earpiece positioned in front of the ear and in contact with the wearer's face.

7. An earphone, characterized in that, include: Bone conduction microphone is used to collect bone conduction signals from the headphone wearer; At least two air conduction microphones, each used to acquire one ambient air conduction signal; The noise reduction unit is used to perform a first noise reduction process on the target air conduction signal determined based on at least two environmental air conduction signals based on the bone conduction signal, so as to remove the first air conduction component corresponding to the bone conduction signal in the target air conduction signal and obtain the initial noise-reduced signal. The initial noise-reduced signal is subjected to a second noise reduction process based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals, in order to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal, thereby obtaining the target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the frequency band to which the first air conduction component belongs.

8. The earphone according to claim 7, characterized in that, The earphone is an ear-hook type earphone. When the wearer wears the earphone, the bone conduction microphone is located at or near the helix of the wearer's ear. Alternatively, the headphones are eyeglass-style headphones, in which the bone conduction microphone is located on the temple of the glasses and in contact with the wearer when the wearer wears the headphones.

9. The headphones according to claim 8, characterized in that, The at least two air conduction microphones are located on the ear-hook headphones, in front of the ears and in contact with the wearer's face; Alternatively, the at least two air conduction microphones are located on the temples of the glasses-style headphones.

10. An electronic device, characterized in that, The electronic device includes at least one processor, a memory connected to the processor, and a communication device; wherein: The communication device is used to establish a communication connection with the headphones to receive bone conduction signals from the headphone wearer collected by the headphones, as well as at least two ambient air conduction signals. The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the noise reduction method as described in any one of claims 1 to 6.

11. A noise reduction system, characterized in that, include: Headphones, and electronic devices connected to the headphones; The earphone is used to collect the bone conduction signal of the earphone wearer and at least two ambient air conduction signals, and to send the bone conduction signal and the at least two ambient air conduction signals to the electronic device; The electronic device is used to perform a first noise reduction process on the target air conduction signal based on the bone conduction signal and the target direction determined by the at least two environmental air conduction signals, so as to remove the first air conduction component in the target air conduction signal corresponding to the bone conduction signal and obtain the initial noise-reduced signal. The initial noise-reduced signal is subjected to a second noise reduction process based on the bone conduction signal, the at least two ambient air conduction signals, and the phase difference of the at least two ambient air conduction signals, in order to remove the second air conduction component of the headphone wearer carried in the initial noise-reduced signal, thereby obtaining the target noise-reduced signal; the frequency band to which the second air conduction component belongs is higher than the frequency band to which the first air conduction component belongs.

Images