Audio signal processing methods and apparatus, headphone devices, storage media

Abstract

An audio signal processing method, apparatus, headphone device, and storage medium are disclosed. The method, applied to a headphone device, includes: acquiring ambient sound signals via a feedforward microphone and acquiring in-ear audio signals via a feedback microphone when the headphone device outputs a target audio signal; determining a target filter group corresponding to the current scene of the headphone device based on the ambient sound signals, the target filter group including one or more noise reduction filters; calculating noise reduction parameters based on the ambient sound signals and the in-ear audio signals; and determining a target filter from the target filter group based on the noise reduction parameters, the target filter being used to perform noise reduction processing on the output target audio signal. Implementing this embodiment allows for targeted noise reduction processing of the output audio signal based on the current scene of the headphone device, which is beneficial for improving the accuracy and reliability of active noise cancellation in headphone devices.

Description

Technical Field

[0001] This application relates to the field of electronic equipment technology, and in particular to an audio signal processing method and apparatus, headphone device, and storage medium. Background Technology

[0002] Currently, when users use headphones to listen to music, watch videos (i.e., the audio signal corresponding to the video output by the headphone device), and make calls, these headphones typically offer some noise cancellation to reduce the impact of ambient noise on the audio signal output, providing users with better sound quality. However, in practice, it has been found that traditional noise cancellation solutions are often not very accurate and struggle to precisely reduce ambient noise, thus reducing the reliability of active noise cancellation in headphones. Summary of the Invention

[0003] This application discloses an audio signal processing method and apparatus, an earphone device, and a storage medium, which can perform targeted noise reduction processing on the output audio signal of the earphone device based on the current scene of the earphone device, thereby improving the accuracy and reliability of the earphone device's active noise reduction.

[0004] The first aspect of this application discloses an audio signal processing method applied to a headphone device, the headphone device including a feedforward microphone and a feedback microphone, the method comprising:

[0005] When the headphone device outputs a target audio signal, ambient sound signals are acquired through the feedforward microphone, and in-ear audio signals are acquired through the feedback microphone.

[0006] Based on the ambient sound signal, a target filter group corresponding to the current scene of the headphone device is determined, and the target filter group includes one or more noise reduction filters;

[0007] Based on the ambient sound signal and the in-ear audio signal, the noise reduction parameters are calculated;

[0008] Based on the noise reduction parameters, a target filter is determined from the target filter bank, and the target filter is used to perform noise reduction processing on the target audio signal to be output.

[0009] A second aspect of this application discloses an audio signal processing method applied to a headphone device, the headphone device including a feedforward microphone, the method comprising:

[0010] Ambient sound signals are acquired through the feedforward microphone;

[0011] Calculate the power spectral density corresponding to the ambient sound signal;

[0012] Based on the power spectral density, determine the scene noise type corresponding to the current scene of the headphone device;

[0013] A target filter is determined based on the noise type of the scene, and the target filter is used to perform noise reduction processing on the target audio signal to be output.

[0014] The third aspect of this application discloses a method for calculating power spectral density, including:

[0015] The ambient sound signal is subjected to a first audio preprocessing to obtain the target ambient sound signal, wherein the first audio preprocessing includes at least analog-to-digital conversion and downsampling;

[0016] The target ambient sound signal is windowed and segmented according to the unit window length to obtain at least one frame of ambient sound sub-signal;

[0017] Each frame of ambient sound signal is subjected to a Fourier transform, and the power spectral density corresponding to the ambient sound signal is calculated based on the transformed ambient sound signal of each frame.

[0018] A fourth aspect of this application discloses an audio signal processing method applied to a headphone device, the headphone device including a feedforward microphone, the method comprising:

[0019] The ambient sound signal is acquired through the feedforward microphone, and the power spectral density corresponding to the ambient sound signal is calculated.

[0020] The power spectral density is quantized to determine the power spectral density gradient of the ambient sound signal in each frequency sub-band.

[0021] If the power spectral density gradient corresponding to each frequency sub-band meets the target scene noise condition, then the scene noise type corresponding to the target scene noise condition is determined to be the scene noise type corresponding to the scene where the headphone device is currently located.

[0022] A fifth aspect of this application discloses an audio signal processing method applied to a headphone device, the headphone device including a feedforward microphone and a feedback microphone, the method comprising:

[0023] When the headphone device outputs a target audio signal, ambient sound signals are acquired through the feedforward microphone, and in-ear audio signals are acquired through the feedback microphone.

[0024] Based on the target audio signal, the in-ear audio signal is subjected to signal cancellation processing to obtain the residual in-ear audio signal;

[0025] Based on the ambient sound signal and the residual audio signal in the ear, calculate the frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear, as well as the subband energy corresponding to the residual audio signal in the ear;

[0026] The target filter is determined based on the frequency domain coherence coefficient and the subband energy, and the target filter is used to perform noise reduction processing on the target audio signal to be output.

[0027] A sixth aspect of this application discloses an audio signal processing method applied to a headphone device, the headphone device including a feedback microphone, the method comprising:

[0028] When the headphone device outputs a target audio signal, the in-ear audio signal is acquired through the feedback microphone;

[0029] The target audio signal is filtered by a transfer function filter to obtain the transmitted audio signal corresponding to the target audio signal. The transfer function filter is used to characterize the influence of the audio transmission system in which the headphone device is located on the transmission of the target audio signal.

[0030] The error signal between the in-ear audio signal and the transmitted audio signal is calculated as the residual in-ear audio signal.

[0031] A seventh aspect of this application discloses an audio signal compensation device applied to an earphone device, the earphone device including a feedforward microphone and a feedback microphone, the audio signal processing device including:

[0032] The signal acquisition unit is used to acquire ambient sound signals through the feedforward microphone and in-ear audio signals through the feedback microphone when the headphone device outputs a target audio signal.

[0033] The first determining unit is configured to determine a target filter group corresponding to the current scene of the headphone device based on the ambient sound signal, wherein the target filter group includes one or more noise reduction filters;

[0034] The parameter calculation unit is used to calculate the noise reduction parameters based on the ambient sound signal and the in-ear audio signal;

[0035] The second determining unit is used to determine a target filter from the target filter bank according to the noise reduction parameters, and the target filter is used to perform noise reduction processing on the target audio signal to be output.

[0036] An eighth aspect of this application discloses an audio signal processing apparatus applied to a headphone device, the headphone device including a feedforward microphone, the audio signal processing apparatus comprising:

[0037] An ambient sound signal acquisition unit is used to acquire ambient sound signals through the feedforward microphone;

[0038] A power spectral density calculation unit is used to calculate the power spectral density corresponding to the ambient sound signal;

[0039] The noise determination unit is used to determine the scene noise type corresponding to the current scene of the headphone device based on the power spectral density.

[0040] A filter determination unit is used to determine a target filter based on the scene noise type, and the target filter is used to perform noise reduction processing on the target audio signal to be output.

[0041] The ninth aspect of this application discloses a power spectral density calculation device, comprising:

[0042] A preprocessing unit is used to perform a first audio preprocessing on the ambient sound signal to obtain a target ambient sound signal, wherein the first audio preprocessing includes at least analog-to-digital conversion and downsampling;

[0043] A windowing segmentation unit is used to window and segment the target ambient sound signal according to a unit window length to obtain at least one frame of ambient sound sub-signal;

[0044] The transformation calculation unit is used to perform Fourier transform on each frame of ambient sound sub-signal and calculate the power spectral density corresponding to the ambient sound signal based on the transformed ambient sound sub-signal of each frame.

[0045] A tenth aspect of this application discloses an audio signal processing apparatus applied to an earphone device, the earphone device including a feedforward microphone, the audio signal processing apparatus comprising:

[0046] An ambient sound signal acquisition unit is used to acquire ambient sound signals through the feedforward microphone;

[0047] A power spectral density calculation unit is used to calculate the power spectral density corresponding to the ambient sound signal;

[0048] A quantization calculation unit is used to quantize the power spectral density and determine the power spectral density gradient corresponding to each frequency sub-band of the ambient sound signal;

[0049] The noise determination unit is used to determine the scene noise type corresponding to the target scene noise condition as the scene noise type corresponding to the current scene of the headphone device if the power spectral density gradient corresponding to each frequency domain sub-band meets the target scene noise condition.

[0050] The eleventh aspect of this application discloses an audio signal processing device applied to a headphone device, the headphone device including a feedforward microphone and a feedback microphone, the audio signal processing device comprising:

[0051] The signal acquisition unit is used to acquire ambient sound signals through the feedforward microphone and in-ear audio signals through the feedback microphone when the headphone device outputs a target audio signal.

[0052] The cancellation processing unit is used to perform signal cancellation processing on the in-ear audio signal based on the target audio signal to obtain a residual in-ear audio signal;

[0053] The parameter calculation unit is used to calculate the frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear, and the subband energy corresponding to the residual audio signal in the ear, based on the ambient sound signal and the residual audio signal in the ear.

[0054] A filter determination unit is used to determine a target filter based on the frequency domain coherence coefficient and the subband energy. The target filter is used to perform noise reduction processing on the target audio signal to be output.

[0055] The twelfth aspect of this application discloses an audio signal processing apparatus applied to an earphone device, the earphone device including a feedback microphone, the audio signal processing apparatus comprising:

[0056] An in-ear audio signal acquisition unit is used to acquire in-ear audio signals through the feedback microphone when the headphone device outputs a target audio signal.

[0057] A transfer filtering unit is used to filter the target audio signal through a transfer function filter to obtain the transfer audio signal corresponding to the target audio signal, wherein the transfer function filter is used to characterize the influence of the audio transmission system in which the headphone device is located on the transmission of the target audio signal;

[0058] An error calculation unit is used to calculate the error signal between the in-ear audio signal and the transmitted audio signal, as the residual in-ear audio signal.

[0059] The thirteenth aspect of this application discloses an earphone device, including a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform all or part of the steps in any of the audio signal processing methods disclosed in the first to sixth aspects of this application.

[0060] The fourteenth aspect of this application discloses a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements all or part of the steps in any of the audio signal processing methods disclosed in the first to sixth aspects of this application.

[0061] Compared with related technologies, the embodiments of this application have the following beneficial effects:

[0062] In this embodiment, the headphone device applying the audio signal processing method may include a feedforward microphone and a feedback microphone. When the headphone device outputs a target audio signal, it can acquire ambient sound signals through its feedforward microphone and simultaneously acquire in-ear audio signals through its feedback microphone. Based on the ambient sound signals, the headphone device can determine a target filter group corresponding to the current scene of the headphone device. This target filter group may include one or more noise reduction filters. Based on this, the headphone device can calculate noise reduction parameters based on the ambient sound signals and the in-ear audio signals, and determine a target filter from the target filter group according to these noise reduction parameters. The target filter is then used to perform noise reduction processing on the target audio signal to be output by the headphone device. Therefore, by implementing this embodiment, the headphone device can first determine the current scene of the headphone device based on the acquired ambient sound signals, and determine a target filter group consisting of a set of noise reduction filters corresponding to the ambient noise in that scene. Then, it can further select a suitable target filter from the target filter group to perform targeted noise reduction processing on the target audio signal to be output by the headphone device. The above audio signal processing method comprehensively considers the influence of external noise represented by ambient sound signals and the influence of the internal structure of the headphone device represented by in-ear audio signals. It effectively avoids the problem of low accuracy when judging the noise type based solely on ambient sound signals, so that the headphone device can achieve more accurate noise reduction processing for the noise type of its current scene, which is conducive to improving the accuracy and reliability of active noise reduction of the headphone device. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1A This is a schematic diagram illustrating an application scenario of the audio signal processing method disclosed in the embodiments of this application;

[0065] Figure 1BThis is a schematic diagram illustrating another application scenario of the audio signal processing method disclosed in the embodiments of this application;

[0066] Figure 2 This is a schematic diagram of the structure of a headphone device disclosed in an embodiment of this application;

[0067] Figure 3 This is a schematic flowchart of an audio signal processing method disclosed in an embodiment of this application;

[0068] Figure 4 This is a noise classification spectrum diagram disclosed in an embodiment of this application;

[0069] Figure 5 This is a flowchart illustrating another audio signal processing method disclosed in an embodiment of this application;

[0070] Figure 6 This is a schematic diagram of a scene noise type determination process disclosed in an embodiment of this application;

[0071] Figure 7 This is a schematic diagram of a power spectral density gradient disclosed in an embodiment of this application;

[0072] Figure 8 This is a flowchart illustrating another audio signal processing method disclosed in the embodiments of this application;

[0073] Figure 9 This is a schematic diagram of a process for signal cancellation processing of audio signals in the ear, as disclosed in an embodiment of this application.

[0074] Figure 10 This is a schematic diagram of the overall signal flow of an audio signal processing method disclosed in an embodiment of this application;

[0075] Figure 11 This is a flowchart illustrating the fourth audio signal processing method disclosed in the embodiments of this application;

[0076] Figure 12 This is a schematic flowchart of a power spectral density calculation method disclosed in an embodiment of this application;

[0077] Figure 13 This is a flowchart illustrating the fifth audio signal processing method disclosed in the embodiments of this application;

[0078] Figure 14 This is a flowchart illustrating the sixth audio signal processing method disclosed in the embodiments of this application;

[0079] Figure 15 This is a flowchart illustrating the seventh audio signal processing method disclosed in the embodiments of this application;

[0080] Figure 16This is a modular schematic diagram of an audio signal processing device disclosed in an embodiment of this application;

[0081] Figure 17 This is a modular schematic diagram of another audio signal processing device disclosed in the embodiments of this application;

[0082] Figure 18 This is a modular schematic diagram of the power spectral density calculation device disclosed in the embodiments of this application;

[0083] Figure 19 This is a modular schematic diagram of another audio signal processing device disclosed in the embodiments of this application;

[0084] Figure 20 This is a modular schematic diagram of the fourth audio signal processing device disclosed in the embodiments of this application;

[0085] Figure 21 This is a modular schematic diagram of the fifth audio signal processing device disclosed in the embodiments of this application;

[0086] Figure 22 This is a modular schematic diagram of a headphone device disclosed in an embodiment of this application. Detailed Implementation

[0087] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0088] It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0089] This application discloses an audio signal processing method and apparatus, an earphone device, and a storage medium, which can perform targeted noise reduction processing on the output audio signal of the earphone device based on the current scene of the earphone device, thereby improving the accuracy and reliability of the earphone device's active noise reduction.

[0090] The following will be described in detail with reference to the accompanying drawings.

[0091] Please refer to the following: Figure 1A and Figure 1B , Figure 1AThis is a schematic diagram illustrating an application scenario of the audio signal processing method disclosed in the embodiments of this application. Figure 1B This is a schematic diagram illustrating another application scenario of the audio signal processing method disclosed in the embodiments of this application. For example... Figure 1A As shown, this application scenario may include headphone device 10, meaning that headphone device 10 can independently implement the audio signal processing method disclosed in the embodiments of this application. Wherein, as Figure 2 As shown, the headphone device 10 may include a speaker 11, a feedback microphone 12, and a feedforward microphone 13. When a user wears the headphone device 10, the feedback microphone 12 can be positioned between the speaker 11 and the user, in front of the speaker 11, so that it can be used to collect the in-ear audio signal output by the speaker 11 and transmitted through the audio transmission system of the headphone device 10 (i.e., the path through which the target audio signal output by the headphone device 10 is transmitted between the headphone and the user, which can characterize the influence of factors such as the device structure of the headphone device 10, the user's ear shape characteristics, and the wearing leakage when the user wears the headphone device 10 on the transmission of the target audio signal). The feedforward microphone 13 can be positioned between the speaker 11 and the external environment, in front of the speaker 11, so that it can be used to collect the ambient sound signal of the external environment. It can be understood that... Figure 2 The structure of the headphone device 10 shown is merely an example. Provided that its feedback microphone 12 and feedforward microphone 13 can properly acquire the corresponding audio signals, the headphone device 10 may also adopt other layout structures. No specific limitations are made in this embodiment.

[0092] In some embodiments, such as Figure 1B As shown, the headphone device 10 can be worn by the user 20 and can establish a communication connection with the terminal device 30. Based on the above communication connection, the headphone device 10 can obtain audio data (such as music files, recording files, chat voices, etc. to be output) from the terminal device 30 and convert the obtained audio data into a target audio signal for output to the user 20 through the speaker 11 of the headphone device 10.

[0093] It is understood that user 10 can also indirectly control the output of the target audio signal by the headphone device 10 (e.g., start output, pause output, stop output, etc.) by interacting with the terminal device 30. In some embodiments, when the terminal device 30 detects an interactive operation by user 10 (such as a touch operation like clicking or swiping an interactive button on the terminal device 30, issuing a voice operation containing a specified keyword such as "play music" to the terminal device 30, or a movement operation such as moving the terminal device 30 according to a preset trajectory), it can issue a corresponding control command to the headphone device 10 to trigger the headphone device 10 to output the target audio signal. On this basis, the headphone device 10 can simultaneously collect ambient sound signals through its feedforward microphone 13 and collect in-ear audio signals through its feedback microphone 12, so as to further realize the active noise cancellation (ANC) function of the headphone device 10 based on the aforementioned ambient sound signals and in-ear audio signals.

[0094] In this embodiment, when the headphone device 10 needs to implement active noise cancellation, it can perform corresponding parameter calculations based on the ambient sound signal and the in-ear audio signal collected when it outputs the target audio signal to determine a suitable noise reduction filter. Specifically, based on the ambient sound signal, the headphone device 10 can determine a target filter group corresponding to the current scene of the headphone device 10, and the target filter group includes one or more noise reduction filters. On this basis, the headphone device 10 can calculate noise reduction parameters based on the ambient sound signal and the in-ear audio signal, and determine a target filter from the target filter group based on the noise reduction parameters, so as to perform noise reduction processing on the target audio signal to be output by the headphone device 10 through the target filter.

[0095] As can be seen, by implementing the above audio signal processing method, the headphone device 10 can first determine the current scene of the headphone device 10 based on the ambient sound signal it has collected, and determine a target filter group consisting of a set of noise reduction filters corresponding to the ambient noise in the scene. Then, it can further select a suitable target filter from the target filter group to perform targeted noise reduction processing on the target audio signal to be output by the headphone device 10. The above audio signal processing method comprehensively considers the influence of external noise represented by the ambient sound signal and the influence of the internal structure of the headphone device represented by the in-ear audio signal. It effectively avoids the problem of low accuracy that may occur when judging the noise type based solely on the ambient sound signal, so that the headphone device 10 can achieve more accurate noise reduction processing for the noise type of its current scene, which is conducive to improving the accuracy and reliability of the headphone device 10's active noise reduction.

[0096] The aforementioned headphone device 10 may include various types of headphones with active noise cancellation, particularly TWS (True Wireless Stereo) headphones. In some embodiments, the headphone device 10 may include a first earphone and a second earphone (for example, the first earphone and the second earphone may be paired left and right earphones, respectively). The first earphone and the second earphone may have the same layout structure to simultaneously achieve the corresponding active noise cancellation function through the aforementioned audio signal processing method. The aforementioned terminal device 30 may include various devices or systems with wireless communication capabilities, such as mobile phones, smart wearable devices, in-vehicle terminals, tablet computers, PCs (Personal Computers), PDAs (Personal Digital Assistants), etc., which are not specifically limited in this embodiment.

[0097] Please see Figure 3 , Figure 3 This is a flowchart illustrating an audio signal processing method disclosed in an embodiment of this application. This method can be applied to the aforementioned headphone device, which may include a feedforward microphone and a feedback microphone. Figure 3 As shown, the audio signal processing method may include the following steps:

[0098] 302. When the headphone device outputs the target audio signal, the ambient sound signal is collected through the feedforward microphone, and the in-ear audio signal is collected through the feedback microphone.

[0099] In this embodiment, when a user uses a headphone device to output a target audio signal (e.g., playing music files, recording files, or voice chat), in order to reduce interference from ambient sound signals from the external environment and thus improve the user's listening experience, the headphone device can employ active noise reduction measures in addition to passive noise reduction based on its own structure. Specifically, when the headphone device outputs the target audio signal through a speaker, it can collect ambient sound signals from the external environment through its built-in feedforward microphone, and simultaneously collect in-ear audio signals through its built-in feedback microphone. In subsequent steps, the headphone device can determine a suitable noise reduction filter based on the ambient sound signal and the in-ear audio signal to perform noise reduction processing on the target audio signal to be output by the headphone device.

[0100] In some embodiments, the feedforward microphone can remain on to continuously collect ambient sound signals of the environment in which the headphone device is currently located. This allows the acquisition of the ambient sound signals corresponding to the output of the target audio signal when the headphone device outputs the target audio signal. For example, the headphone device can obtain the timestamp of the target audio signal output by its speaker, and based on this timestamp, extract the ambient sound signals collected by the feedforward microphone at times near that timestamp (e.g., delayed by 0.01 milliseconds, delayed by 0.1 milliseconds, etc.), thereby accurately obtaining an ambient sound signal synchronized with the target audio signal.

[0101] In other embodiments, the headphone device can also continuously collect ambient sound signals through a feedforward microphone and directly apply the collected ambient sound signals to subsequent noise reduction processing to achieve real-time, continuous pipeline processing and ensure the real-time performance of active noise reduction by the headphone device.

[0102] In some embodiments, the feedforward microphone of the headphone device may not be continuously on, but may be triggered to turn on when its speaker outputs the target audio signal, and the audio signal collected after the feedforward microphone is turned on is used as the ambient sound signal corresponding to the target audio signal. Based on this, when the headphone device pauses or stops outputting the target audio signal, the feedforward microphone can be controlled to stop collecting the ambient sound signal.

[0103] While acquiring ambient sound signals through the aforementioned feedforward microphone, the headphone device can also acquire in-ear audio signals through its feedback microphone. The target audio signal output by the headphone device through its speaker can be transmitted within the audio system in which the headphone device operates. After being received by the feedback microphone, the resulting in-ear audio signal can be used to assess the impact on the target audio signal during transmission within the audio system, particularly interference from ambient sound signals, in order to determine appropriate noise reduction parameters in subsequent steps. It can be understood that, since the feedback microphone is located between the speaker and the user, the aforementioned audio system can be approximated by the path through which the target audio signal is transmitted between the speaker and the feedback microphone.

[0104] In some embodiments, the feedback microphone of the headphone device may also remain on, thereby continuously acquiring in-ear audio signals in a manner similar to how a feedforward microphone acquires ambient sound signals. For example, the headphone device may also extract in-ear audio signals acquired by the feedback microphone at times close to the target audio signal output by its speaker (e.g., delayed by 0.01 milliseconds, delayed by 0.1 milliseconds, etc.), thereby accurately obtaining in-ear audio signals that are time-synchronized with both the target audio signal and the ambient sound signal.

[0105] In other embodiments, the feedback microphone of the headphone device may not be continuously on, but may be triggered to turn on when the speaker outputs the target audio signal, and the audio signal collected after the feedback microphone is turned on may be taken as the in-ear audio signal corresponding to the target audio signal. Optionally, for the in-ear audio signal collected by the feedback microphone, the headphone device may also use its built-in signal processing module to compare the waveform of the target audio signal output by the speaker with the in-ear audio signal. When the comparison result indicates that the waveform similarity between the target audio signal and the in-ear audio signal meets the similarity threshold (such as 50%, 80%, etc.), the in-ear audio signal can be identified as the in-ear audio signal corresponding to the target audio signal.

[0106] 304. Based on the above ambient sound signals, determine the target filter group corresponding to the current scene of the headphone device. The target filter group includes one or more noise reduction filters.

[0107] In this embodiment, to accurately determine the current scene of the headphone device, the scene noise type corresponding to different scenes can be classified based on the spectral characteristics of the ambient sound signals in different scenes. These different scenes may include low-frequency scenes with a large proportion of low-frequency ambient sound signals, such as airplanes and high-speed trains; mid-to-high-frequency scenes with a large proportion of mid-to-high-frequency ambient sound signals, such as restaurants and shopping malls; and quiet scenes with uniformly distributed ambient sound signals of different frequency bands and low signal energy, such as libraries and bedrooms, but are not limited to these. For example, please refer to... Figure 4 , Figure 4 This is a noise classification spectrum diagram disclosed in an embodiment of this application. For example... Figure 4 As shown, the airport ambient sound signal indicated by solid line A, the shopping mall ambient sound signal indicated by dotted line B, and the bedroom ambient sound signal indicated by dashed line C can have significantly different spectral characteristics. The specific division method can be found in the following example.

[0108] For example, the headphone device can classify scene noise types corresponding to different scenarios based on certain spectral characteristic parameters. In some embodiments, the aforementioned spectral characteristic parameters may include signal energy. For instance, if the proportion of signal energy in the 0–200Hz frequency band of the ambient sound signal exceeds a first threshold and the signal energy is greater than T1, the corresponding scene can be classified as a low-frequency scene; if the proportion of signal energy in the 0–1000Hz frequency band of the ambient sound signal exceeds a second threshold (the second threshold may be equal to or unequal to the first threshold), and the signal energy is between T1 and T2 (T1 > T2), the corresponding scene can be classified as a mid-high frequency scene; if the signal energy distribution of the ambient sound signal in the 0–2500Hz frequency band is uniform (e.g., the variance is less than a third threshold) and the signal energy is less than T2, the corresponding scene can be classified as a quiet scene. Optionally, the aforementioned low-frequency scene and mid-high frequency scene may also be supplemented with a criterion for judging whether the signal energy distribution is uniform, which is not specifically limited in this embodiment.

[0109] In other embodiments, the aforementioned spectral characteristic parameters may also include power spectral density. For example, the power spectral density of ambient sound signals in the 0–200 Hz frequency band in low-frequency scenarios may be greater than P1, the power spectral density of ambient sound signals in the 0–1000 Hz frequency band in mid-to-high-frequency scenarios may be between P1 and P2 (P1 > P2), and the power spectral density of ambient sound signals in the 0–2500 Hz frequency band in quiet scenarios may be less than P2.

[0110] Based on this, the candidate noise reduction filters can be divided into several groups, each group corresponding to different scene noise types, to implement scene-adaptive active noise reduction for the headphone device in the corresponding scene. For example, the center frequency band, gain, and other filter parameters of each group of noise reduction filters can have certain differences, thus adapting to the spectral characteristics of ambient sound signals in different scenes. In some embodiments, the headphone device can determine its current scene based on the spectral characteristics of the ambient sound signal collected by its feedforward microphone, and then determine the target filter group corresponding to the current scene from at least one candidate filter group. Each group of noise reduction filters can include one or more noise reduction filters with different center frequency band, noise reduction peak value, gain, and other filter parameters, so as to facilitate the selection of a suitable target filter for a specific scene in subsequent steps.

[0111] 306. Based on the above ambient sound signal and in-ear audio signal, the noise reduction parameters are calculated.

[0112] In this embodiment, the noise reduction parameters may include a first noise reduction parameter and a second noise reduction parameter, allowing the headphone device to select a suitable target filter in subsequent steps based on the two calculated noise reduction parameters. The calculation based on the ambient sound signal and the in-ear audio signal comprehensively considers the external noise influence represented by the ambient sound signal and the internal structural influence of the headphone device represented by the in-ear audio signal. This effectively avoids the potential for low accuracy when judging noise type solely based on the ambient sound signal, improving the accuracy and reliability of the headphone device's active noise reduction based on the noise type of its current environment.

[0113] For example, the first noise reduction parameter mentioned above may include correlation parameters, such as time-domain correlation coefficient and frequency-domain coherence coefficient. Taking the frequency-domain coherence coefficient as an example, when the headphone device calculates the frequency-domain coherence coefficient based on the ambient sound signal and the in-ear audio signal, the frequency-domain coherence coefficient can be used to represent the degree of correlation between the ambient sound signal and the in-ear audio signal in the frequency domain, thereby helping to determine the interference of the target audio signal with the ambient sound signal during the transmission of the target audio signal in the audio system where the headphone device is located. Specifically, the headphone device can first calculate the residual in-ear audio signal based on the ambient sound signal and the in-ear audio signal. The residual in-ear audio signal can be used to represent the audio signal closely related to the ambient sound signal obtained after removing the target audio signal from the in-ear audio signal. Based on this, the headphone device can calculate the frequency-domain coherence coefficient between the ambient sound signal and the residual in-ear audio signal, and then apply the frequency-domain coherence coefficient to the subsequent step of selecting a specific target filter from the target filter bank.

[0114] For example, the second noise reduction parameter mentioned above may include a noise reduction depth parameter, such as signal energy and a filter gain further determined based on the signal energy. Taking signal energy as an example, the signal energy may specifically refer to the signal energy of the ambient sound signal or the residual audio signal in the ear, which can be used to determine the specific degree of interference of the ambient sound signal on the target audio signal. For example, the headphone device can first acquire the residual audio signal in the ear, and then calculate the signal energy (i.e., sub-band energy) of the residual audio signal in a specific frequency domain sub-band. The specific frequency domain sub-band may be determined by the scene noise type corresponding to the current scene of the headphone device, so as to calculate the most representative frequency band in the current scene and improve the targeting of the headphone device's active noise reduction. On this basis, the headphone device can apply the signal energy to the subsequent step of determining the target filter, or further calculate the corresponding filter gain based on the signal energy and then apply the filter gain to the subsequent steps to determine the noise reduction depth of the target filter, so as to accurately configure the target filter to actively reduce noise of the target audio signal to be output by the headphone device.

[0115] 308. Based on the noise reduction parameters, determine the target filter from the above target filter group. The target filter is used to perform noise reduction processing on the target audio signal to be output.

[0116] In this embodiment, after calculating the aforementioned noise reduction parameters, the headphone device can further determine a target filter matching the noise reduction parameters from the determined target filter group. The target noise reduction filter group may include one or more noise reduction filters. The center frequency band, noise reduction peak value, gain, and other filter parameters of these filters may have minor differences (relatively minor compared to different groups of noise reduction filters), allowing the headphone device to select a suitable target filter for its specific scenario. Furthermore, the headphone device can configure the target filter based on the aforementioned noise reduction parameters, such as configuring the gain of the target filter, to obtain a target filter that can be directly used, facilitating timely noise reduction processing of the target audio signal to be output by the headphone device.

[0117] In some embodiments, the target filter may also consist of multiple filters. Specifically, when a unique target filter is determined based on the noise reduction parameters, the target filter can be configured to perform noise reduction processing on the target audio signal to be output by the headphone device; when the noise type of the scene corresponding to the current scene of the headphone device is relatively complex, and multiple target filters need to be determined based on the noise reduction parameters, corresponding noise reduction processing can also be performed by configuring cascaded FIR (Finite Impulse Response) filters or IIR (Infinite Impulse Response) filters.

[0118] As an optional implementation, while determining and configuring the target filter according to the above noise reduction parameters to achieve active noise reduction, the headphone device can also be further configured with a matching equalization filter, thereby optimizing the noise reduction performance of the headphone device while achieving sound quality equalization, so as to optimize the user's sound quality experience.

[0119] As can be seen, by implementing the audio signal processing method described in the above embodiments, the headphone device can first determine the current scene of the headphone device based on the ambient sound signal it has collected, and determine a target filter group consisting of a set of noise reduction filters corresponding to the ambient noise in the scene. Then, it can further select a suitable target filter from the target filter group to perform targeted noise reduction processing on the target audio signal to be output by the headphone device. The above audio signal processing method comprehensively considers the influence of external noise represented by the ambient sound signal and the influence of the internal structure of the headphone device represented by the in-ear audio signal. It effectively avoids the problem of low accuracy that may occur when judging the noise type based solely on the ambient sound signal, so that the headphone device can achieve more accurate noise reduction processing for the noise type of its current scene, which is conducive to improving the accuracy and reliability of the headphone device's active noise reduction.

[0120] Please see Figure 5 , Figure 5 This is a schematic flowchart of another audio signal processing method disclosed in an embodiment of this application. This method can be applied to the aforementioned headphone device, which may include a feedforward microphone and a feedback microphone. Figure 5 As shown, the audio signal processing method may include the following steps:

[0121] 502. When the headphone device outputs the target audio signal, the ambient sound signal is collected through the feedforward microphone, and the in-ear audio signal is collected through the feedback microphone.

[0122] Step 502 is similar to step 302 above, and will not be described again here.

[0123] 504. Calculate the power spectral density corresponding to the ambient sound signal.

[0124] In this embodiment, the headphone device can analyze the spectral characteristics of an ambient sound signal by calculating its power spectral density, thereby determining the scene corresponding to the ambient sound signal, i.e., the scene in which the headphone device is currently located. For example, after calculating the power spectral density of the ambient sound signal, the headphone device can determine the scene noise type corresponding to the current scene in a subsequent step based on the power spectral density, and further determine a target filter group matching the scene noise type from at least one candidate filter group.

[0125] In some embodiments, before specifically calculating the power spectral density, the headphone device may first perform a first audio preprocessing on the ambient sound signal to obtain a target ambient sound signal. This first audio preprocessing may include at least analog-to-digital conversion (ADC) and downsampling. Specifically, to reduce the computational load and power consumption during noise reduction processing, the headphone device may reduce the sampling rate of the ambient sound digital signal obtained after ADC to a certain value (e.g., 8kHz, 16kHz) to obtain the target ambient sound signal. Based on this, the headphone device may window the target ambient sound signal according to a unit window length to obtain at least one frame of ambient sound sub-signal, then perform a Fourier transform on each frame of ambient sound sub-signal, and calculate the power spectral density corresponding to the ambient sound signal based on the transformed frames of ambient sound sub-signal.

[0126] Specifically, the headphone device can use its built-in signal processing module (such as a DSP module, i.e., a digital signal processor) to perform frame-by-frame windowing processing on the aforementioned target ambient sound signal. That is, the macroscopically unstable audio signal is divided into multiple audio signal frames with short-term stationarity (e.g., audio signal frames with a frame length of 10–30 milliseconds), and then windowed and truncated according to a specified window function to obtain each frame of ambient sound sub-signal. For example, windowing and truncating can be implemented using a window function as shown in Formula 1:

[0127] Formula 1:

[0128] w(n) = 1, 0 ≤ n ≤ N-1;

[0129] w(n) = 0, otherwise

[0130] Here, the piecewise function w(n) is the window function, and N is the unit window length. By convolving the target ambient sound signal with this window function in the time domain, the windowing truncation effect can be achieved.

[0131] Furthermore, the ambient sound signal of a certain frame obtained after frame segmentation and windowing can be subjected to short-time Fourier transform using algorithms such as FFT (Fast Fourier Transform), as shown in Equation 2 below (specific form not shown):

[0132] Formula 2:

[0133]

[0134] Where x(n) is the target ambient sound signal, which can represent the nth frame of the target ambient sound signal; m can represent the time series in the corresponding Fourier transform X(k,m), and k can represent the frequency domain sub-band sequence. Based on this, the process by which the headphone device calculates the power spectral density based on the transformed ambient sound sub-signals of each frame can be shown in the following formula 3:

[0135] Formula 3:

[0136] PS(k,m)=(1-α)*PS(k,m-1)+α*|X(k,m) 2

[0137] Here, PS(k,m) represents the power spectral density of the ambient sound signal in the m-th frame corresponding to the k-th frequency sub-band, and α represents the iteration factor, i.e., the weighting factor of the modulus of the current frame's sub-band spectrum signal. Therefore, the headphone device can calculate the power spectral density of the ambient sound signal in each frequency sub-band based on the transformed ambient sound signals of each frame, where each frequency sub-band represents the frequency components of the ambient sound signal within its corresponding frequency range.

[0138] It is understandable that if m equals 1, the headphone device can actually calculate the power spectral density corresponding to the m-th frame ambient sound signal based on the transformed m-th frame ambient sound signal (i.e., the 1st frame); if m is greater than 1 and less than or equal to M (M is the total number of frames and M is a positive integer), the headphone device can calculate the power spectral density corresponding to the m-th frame ambient sound signal based on the transformed m-th frame ambient sound signal and the power spectral density corresponding to the (m-1)-th frame ambient sound signal.

[0139] 506. Based on the power spectral density above, determine the scene noise type corresponding to the current scene of the headphone device.

[0140] In this embodiment, after calculating the power spectral density corresponding to the ambient sound signal, the headphone device can analyze the spectral characteristics of the ambient sound signal based on the power spectral density, thereby determining the type of scene noise corresponding to the current scene of the headphone device. For example, the process of determining the scene noise type can be as follows: Figure 6 As shown, after the headphone device acquires the ambient sound signal through its feedforward microphone, it can sequentially perform analog-to-digital conversion, downsampling, and FFT on the ambient sound signal. Then, based on the ambient sound sub-signals obtained after FFT, the power spectral density of the ambient sound signal in each frequency domain sub-band can be calculated. Furthermore, the scene noise type corresponding to the current scene of the headphone device can be determined, so as to determine the filter type to be used for noise reduction in subsequent steps (i.e., determine the target filter group).

[0141] In some embodiments, when determining the scene noise type corresponding to the current scene, the headphone device can first quantize the aforementioned power spectral density to determine the power spectral density gradient corresponding to each frequency sub-band of the ambient sound signal. Based on this, if the power spectral density gradient corresponding to each frequency sub-band meets the target scene noise condition, the headphone device can determine the scene noise type corresponding to the target scene noise condition as the scene noise type corresponding to the current scene of the headphone device.

[0142] The aforementioned target scene noise conditions may include the power spectral density values ​​that the ambient sound signals in different scenarios, such as low-frequency scenarios, mid-to-high-frequency scenarios, and quiet scenarios, should meet. For example, the power spectral density of the ambient sound signal in the 0-200Hz frequency band in a low-frequency scenario can be greater than P1, the power spectral density of the ambient sound signal in the 0-1000Hz frequency band in a mid-to-high-frequency scenario can be between P1 and P2 (P1 > P2), and the power spectral density of the ambient sound signal in the 0-2500Hz frequency band in a quiet scenario can be less than P2.

[0143] For example, after quantizing the power spectral density, the power spectral density gradient obtained by the headphone device can be as follows: Figure 7 As shown (taking an airport as an example), the headphone device can directly compare the quantized power spectral density gradient with the aforementioned thresholds P1 and P2 to determine whether the power spectral density gradient corresponding to each frequency sub-band within the target frequency range (i.e., the aforementioned 0–200Hz band, or 0–1000Hz band, or 0–2500Hz band) meets the aforementioned noise conditions for each target scene. Specifically, the headphone device can determine whether the power spectral density within the target frequency range conforms to the power spectral density range corresponding to each target scene (i.e., greater than P1, or between P1 and P2, or less than P2) based on the power spectral density gradient corresponding to each frequency sub-band. If it conforms to the power spectral density range corresponding to a certain target scene, the scene noise type corresponding to that target scene can be determined as the scene noise type corresponding to the current scene of the headphone device.

[0144] The target frequency range mentioned above corresponds to the target scenario and may include frequency ranges corresponding to one or more frequency domain sub-bands. The target scenario can be any scenario from one or more preset scenarios, such as low-frequency scenarios like airplanes and high-speed trains, medium- and high-frequency scenarios like restaurants and shopping malls, and quiet scenarios like libraries and bedrooms, but is not limited to these.

[0145] As an alternative implementation, the headphone device can also obtain its current location and compare it with the scene locations recorded by the headphone device to determine the corresponding scene noise type. Then, it can directly call the noise reduction filter used by the headphone device at the current location for noise reduction, which greatly reduces the amount of computation of the headphone device, and also helps to reduce the power consumption of the headphone device and extend the battery life.

[0146] As another alternative implementation, the headphone device can also use a trained model to identify the aforementioned ambient sound signals, thereby directly determining the type of scene noise corresponding to the current scene of the headphone device, which can effectively improve the efficiency of noise reduction processing of the headphone device.

[0147] 508. From at least one candidate filter group, determine a target filter group that matches the noise type of the scene, the target filter group comprising one or more noise reduction filters.

[0148] In this embodiment of the application, the candidate noise reduction filters can be divided into several groups, and each group of candidate filters corresponds to the different scene noise types mentioned above, so as to achieve scene-adaptive noise reduction processing for the target audio signal to be output by the headphone device for the corresponding scene.

[0149] In some embodiments, the headphone device can pre-mark each group of candidate filters, for example, by encoding or labeling the scene noise type targeted by each group of candidate filters. Based on this, after obtaining the scene noise type corresponding to its current scene, the headphone device can sequentially compare the marking content of each group of candidate filters to determine the target filter group that matches the scene noise type.

[0150] In other embodiments, the headphone device can also determine the noise reduction frequency range targeted by each group of candidate filters based on the filter parameters (e.g., filtering range, center frequency, etc.) corresponding to each group of candidate filters. Based on this, the headphone device can initially determine multiple target filter groups from each group of candidate filters whose noise reduction frequency range can cover the corresponding scene, and then further select the target filter group whose noise reduction frequency range best matches from these multiple target filter groups, thereby improving the flexibility of the headphone device in determining the target filter group.

[0151] 510. Based on the above ambient sound signal and in-ear audio signal, the noise reduction parameters are calculated.

[0152] 512. Based on the noise reduction parameters, determine the target filter from the above target filter group. The target filter is used to perform noise reduction processing on the target audio signal to be output.

[0153] Steps 510 and 512 are similar to steps 306 and 308 described above, and will not be repeated here. For specific parameters including frequency domain coherence coefficient and / or sub-band energy, please refer to the description in the next embodiment.

[0154] As can be seen, the audio signal processing method described in the above embodiments can comprehensively consider the influence of external noise represented by ambient sound signals and the influence of the internal structure of the headphone device represented by in-ear audio signals. This effectively avoids the problem of low accuracy that may occur when judging the noise type solely based on ambient sound signals. This allows the headphone device to achieve more precise noise reduction processing for the noise type of its current environment, thus improving the accuracy and reliability of active noise cancellation. Furthermore, by calculating the power spectral density corresponding to the ambient sound signal, the headphone device can analyze the spectral characteristics of the ambient sound signal, thereby accurately determining the current environment of the headphone device and further improving the accuracy of active noise cancellation for that environment.

[0155] Please see Figure 8 , Figure 8 This is a flowchart illustrating another audio signal processing method disclosed in an embodiment of this application. This method can be applied to the aforementioned headphone device, which may include a feedforward microphone and a feedback microphone. Figure 8 As shown, the audio signal processing method may include the following steps:

[0156] 802. When the headphone device outputs the target audio signal, the ambient sound signal is collected through the feedforward microphone, and the in-ear audio signal is collected through the feedback microphone.

[0157] Step 802 is similar to step 302 above, and will not be described again here.

[0158] 804. Calculate the power spectral density corresponding to the ambient sound signal.

[0159] 806. Based on the power spectral density above, determine the scene noise type corresponding to the current scene of the headphone device.

[0160] 808. From at least one candidate filter group, determine a target filter group that matches the noise type of the scene, the target filter group comprising one or more noise reduction filters.

[0161] Steps 804, 806, and 808 are similar to steps 504, 506, and 508 described above, and will not be repeated here.

[0162] 810. Based on the above target audio signal, perform signal cancellation processing on the audio signal in the ear to obtain the residual audio signal in the ear.

[0163] In this embodiment, the in-ear audio signal collected by the headphone device through its feedback microphone is affected by ambient sound interference, but the main audio component of the in-ear audio signal is still the transmitted target audio signal. To highlight the correlation between the ambient sound signal and the in-ear audio signal, the headphone device can first remove the transmitted target audio signal from the in-ear audio signal to obtain a residual in-ear audio signal closely related to the ambient sound signal. Based on this, subsequent calculations of the in-ear audio signal by the headphone device can be performed on this residual in-ear audio signal, thereby completing the subsequent noise reduction process more efficiently.

[0164] In some embodiments, the headphone device can filter the target audio signal using a transfer function filter to obtain the transmitted audio signal corresponding to the target audio signal. The transfer function filter can be used to characterize the influence of the audio transmission system in which the headphone device operates (i.e., the transmission path of the target audio signal between the headphone and the user, including the combined effects of the headphone device 10's device structure, the user's ear shape characteristics, and wearing leakage when the user wears the headphone device 10) on the transmission of the target audio signal. Optionally, the transfer function filter can be an FIR filter.

[0165] In one embodiment, the aforementioned transfer function filter can be pre-programmed and stored in the headphone device's storage module before it leaves the factory. For example, the transfer function filter can be implemented based on the ear-shaped transfer function measured when the headphone device is placed in a standard ear-shaped fixture (such as IEC 711), i.e., in an anechoic chamber environment, the headphone device is placed in a well-sealed standard ear-shaped fixture, and the ear-shaped transfer function at this time is measured; alternatively, it can be implemented based on a statistically obtained ear-shaped transfer function, for example, in an anechoic chamber environment, the transfer functions of a large number of users wearing the headphone device normally are obtained, and their average value is calculated to configure the corresponding transfer function filter; furthermore, if the transfer functions of a large number of users wearing the headphone device normally are represented in the form of function curves, then an average curve can be obtained from the function curves, and the function corresponding to the average curve can be determined as the required ear-shaped transfer function to implement the corresponding transfer function filter.

[0166] In another embodiment, the aforementioned transfer function filter can also be obtained by the user actively triggering a test in a quiet environment while the user is wearing the headphone device. For example, for the target audio signal x(n) output by the headphone device through the speaker, the headphone device can first define it as follows:

[0167] Formula 4:

[0168] x(n)=[x(n),x(n-1),...,x(n-N+1)]T

[0169] Where N is the number of coefficients of the transfer function filter to be determined, that is, the transfer function filter can be determined by N coefficients (initially 0). Let w(n) represent the transfer function filter, then after filtering the target audio signal x(n) using the transfer function filter w(n), the resulting transferred audio signal y(n) can be expressed by the following formula 5:

[0170] Formula 5:

[0171] y(n)=w T (n)x(n)

[0172] Before calculating the transmitted audio signal y(n) using Formula 5 above, the target audio signal x(n) needs to undergo a second audio preprocessing step, which may include downsampling. Based on this, the headphone device can calculate the error signal e(n) between the transmitted audio signal y(n) and the in-ear audio signal d(n) collected through its feedback microphone.

[0173] Formula 6:

[0174] e(n) = d(n) - y(n)

[0175] The aforementioned in-ear audio signal d(n) also needs to undergo a third audio preprocessing step, which may include analog-to-digital conversion and downsampling.

[0176] It should be noted that when the headphone device calculates the aforementioned error signal e(n), it can use this error signal e(n) as the desired residual audio signal in the ear. At this point, the aforementioned process of signal cancellation processing of the in-ear audio signal d(n) can be performed as follows: Figure 9 As shown, after the headphone device acquires the in-ear audio signal d(n) through its feedback microphone, it can sequentially perform analog-to-digital conversion, downsampling, and other steps on the in-ear audio signal. Then, it can subtract the d(n) from the target audio signal (i.e., the transmitted audio signal y(n)) that has been downsampled and filtered by the transfer function filter to obtain the error signal e(n). This error signal e(n) is then used as the residual in-ear audio signal for FFT (the residual in-ear audio signals before and after FFT can be represented by Se and Sef, respectively) for subsequent calculation of noise reduction parameters.

[0177] Alternatively, the headphone device can also use the LMS (Least Mean Square) algorithm to iterate the above signal cancellation process to eliminate the errors caused by the acoustic device itself, obtain a more accurate transfer function filter w(n), and at the same time obtain a more accurate residual audio signal in the ear.

[0178] For example, after the headphone device subtracts the in-ear audio signal d(n) from the transmitted audio signal y(n) to obtain the error signal e(n), it can further calculate the mean square error between the in-ear audio signal d(n) and the transmitted audio signal y(n) based on the error signal e(n). The mean square error J can be calculated as shown in Formula 7 below:

[0179] Formula 7:

[0180] J = E[e 2 (n)]=E[d 2 (n)]+2E[d(n)w T (n)x(n)]+E[w T (n)x(n)x T (n)w(n)]

[0181] Here, E can represent the mathematical expectation. Furthermore, based on the mean square error J mentioned above, the headphone device can update the transfer function filter w(n), as shown in Equation 8 below:

[0182] Formula 8:

[0183] w(n+1)=w(n)+2ue(n)x(n)

[0184] Wherein, u is the step size factor used for updating. After updating the transfer function filter w(n) to obtain w(n+1), the headphone device can re-execute the above signal cancellation process, filter the target audio signal x(n) through the new transfer function filter w(n+1), obtain the transfer audio signal y(n) corresponding to the target audio signal x(n), and repeat the calculation steps shown in formulas 6, 7, and 8 until the update stopping condition is met, and use the error signal e(n) obtained when the update stopping condition is met as the residual audio signal in the ear. Wherein, the above update stopping condition may include the iteration number condition (e.g., the update number reaches the upper limit) and / or the iteration parameter condition (e.g., the transfer function filter w(n) or the step size factor u meets certain numerical conditions), which are not specifically limited in this embodiment.

[0185] 812. Calculate the frequency domain coherence coefficient between the above ambient sound signal and the residual audio signal in the ear.

[0186] In this embodiment of the application, by calculating the frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear, the degree of correlation between the two in the frequency domain can be determined, which helps to quantify the interference of the ambient sound signal on the target audio signal during the transmission of the audio system in which the headphone device is located, so as to facilitate targeted noise reduction processing.

[0187] For example, the headphone device can calculate the frequency domain coherence coefficient Rf between the ambient sound signal and the residual audio signal in the ear based on the first sub-signals corresponding to the ambient sound signal in each target frequency domain sub-band and the second sub-signals corresponding to the residual audio signal in the ear in each target frequency domain sub-band. The frequency range corresponding to each target frequency domain sub-band can be determined by the target filter bank.

[0188] Specifically, the above calculation process can be shown in Formula 9 below:

[0189] Formula 9:

[0190]

[0191] Wherein, Sef is the residual audio signal in the ear, and Sof is the ambient sound signal. Both the residual audio signal Sef and the ambient sound signal Sof can be preprocessed through analog-to-digital conversion, downsampling, etc. k can represent the frequency domain sub-band sequence, that is, Sef(k) can represent the k-th sub-band (first sub-signal) of the residual audio signal in the ear, and Sof(k) can represent the k-th sub-band (second sub-signal) of the ambient sound signal. The above i and j can represent the starting sub-band sequence and the ending sub-band sequence, respectively. Their specific values ​​can be determined by the target filter bank, thereby limiting the frequency range corresponding to each target frequency domain sub-band involved in the above calculation process.

[0192] 814. Based on the above residual audio signal in the ear, calculate the subband energy corresponding to the residual audio signal in the ear.

[0193] In this embodiment of the application, by calculating the subband energy corresponding to the residual audio signal in the ear, the specific degree of interference of the ambient sound signal to the target audio signal can be quantitatively determined, so as to specifically limit the noise reduction depth of the headphone device in subsequent steps.

[0194] For example, the headphone device can calculate the sub-band energy E corresponding to the residual audio signal in the ear based on the third sub-signals corresponding to each target frequency domain sub-band of the residual audio signal in the ear. Sef The frequency range corresponding to each of the target frequency domain sub-bands can also be determined by the target filter bank.

[0195] Specifically, the above calculation process can be shown in the following formula 10:

[0196] Formula 10:

[0197]

[0198] Where Sef is the residual audio signal in the ear, and k can represent the frequency domain sub-band sequence, that is, Sef(k) can represent the k-th sub-band (third sub-signal) of the residual audio signal in the ear. Similarly, i and j can also represent the starting sub-band sequence and the ending sub-band sequence, respectively, and their specific values ​​can be determined by the target filter bank, thereby limiting the frequency range corresponding to each target frequency domain sub-band involved in the above calculation process.

[0199] It is understood that step 814 can be executed after step 810, that is, step 814 can be executed in parallel with step 812. Thus, the headphone device can independently and simultaneously calculate the frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear, as well as the subband energy corresponding to the residual audio signal in the ear. These two noise reduction parameters can then be applied together in the subsequent process of determining the target filter.

[0200] 816. Based on the frequency domain coherence coefficient, the noise reduction filter with the highest coherence is determined from the target filter bank as the target filter, and the gain coefficient corresponding to the target filter is determined based on the subband energy. The target filter is configured according to the above gain coefficient to perform noise reduction processing on the target audio signal to be output.

[0201] In this embodiment, after calculating the frequency domain coherence coefficient and subband energy, the headphone device can determine a suitable target filter from one or more noise reduction filters included in the target filter bank based on these two noise reduction parameters. The frequency domain coherence coefficient characterizes the interference of ambient sound signals on the target audio signal during transmission through the audio system where the headphone device is located. Therefore, after determining the noise type corresponding to its current scene, the headphone device can further determine the subdivided scene it is in, so as to select appropriate filter parameters such as center frequency band (or bandwidth) and noise reduction peak value to achieve noise reduction, thereby determining a suitable target filter from the target filter bank.

[0202] In some embodiments, when determining the target filter from the target filter bank based on the aforementioned frequency domain coherence coefficients, the headphone device can first obtain the frequency domain coherence coefficients of each noise reduction filter in the target filter bank when applied to the current scene. That is, it can configure each noise reduction filter to reduce the noise of the target audio signal, and calculate the frequency domain coherence coefficients between the residual audio signal in the ear and the ambient sound signal using the methods shown in Formulas 6 and 9, based on the in-ear audio signal collected by the headphone device after noise reduction. On this basis, the headphone device can compare the frequency domain coherence coefficients corresponding to each noise reduction filter with the frequency domain coherence coefficients between the ambient sound signal and the residual audio signal in the ear, and determine the noise reduction filter with the highest coherence (e.g., the closest frequency domain coherence coefficient) from the target filter bank as the target filter based on the comparison results.

[0203] In other embodiments, the headphone device may also directly determine the noise reduction filter with the smallest frequency domain coherence coefficient (i.e., the target audio signal is least affected by the ambient sound signal after noise reduction) based on the frequency domain coherence coefficient of each of the above noise reduction filters, and use it as the target filter.

[0204] In some embodiments, the headphone device can also calculate the coherence parameters corresponding to each of the above-mentioned noise reduction filters. If the noise reduction effect of a certain noise reduction filter is better in the current scene of the headphone device, the larger the coherence parameter is, so that the headphone device can directly determine the noise reduction filter with the best noise reduction effect as the target filter based on the coherence parameter.

[0205] Furthermore, the subband energy can characterize the specific degree of interference of the ambient sound signal on the target audio signal. Therefore, after the headphone device determines the target filter, it can determine the noise reduction depth that the target filter should be configured according to the subband energy (for example, by determining it through filter parameters such as gain). Thus, the target filter can be configured reasonably to obtain a completely determined target filter, which can then be applied to the subsequent noise reduction processing of the target audio signal to be output.

[0206] In some embodiments, the headphone device can determine the corresponding noise reduction level based on the subband energy, and obtain the gain coefficient corresponding to the target filter based on the noise reduction level. The target filter is then configured according to the gain coefficient to perform noise reduction processing on the target audio signal to be output.

[0207] In other embodiments, the headphone device may first acquire the sub-band energy of each noise reduction filter in the target filter group when applied to the current scene. That is, each noise reduction filter is configured to reduce the noise of the target audio signal, and the sub-band energy corresponding to the residual audio signal in the ear is calculated based on the in-ear audio signal collected by the headphone device after noise reduction, using the methods shown in Formulas 6 and 10 above. On this basis, the headphone device can directly determine the noise reduction filter with the smallest sub-band energy (i.e., the sub-band energy corresponding to the residual audio signal in the ear after noise reduction) based on the sub-band energy of each noise reduction filter, and use it as the target filter.

[0208] As an optional implementation, after determining the target filter, the headphone device can update the target filter according to certain rules to ensure that the headphone device can flexibly and promptly adjust the target filter used for noise reduction processing in response to changes in its current environment. In some embodiments, when the target filter group is updated, the headphone device can traverse each noise reduction filter in the updated target filter group to redetermine the most suitable target filter based on filter parameters such as the frequency domain coherence coefficient and subband energy corresponding to each noise reduction filter. In other embodiments, if the target filter group has not been updated after a first time period (e.g., 10 minutes, 2 hours, etc.), the headphone device can also traverse each noise reduction filter in the target filter group to similarly redetermine the target filter based on the filter parameters corresponding to each noise reduction filter. By implementing the above method, the headphone device can perform real-time fine-tuning according to subtle changes in ambient noise and ensure smooth switching of the target filter used by the headphone device. This improves the accuracy and flexibility of the headphone device's active noise cancellation while minimizing disturbance to the user, thus enhancing the user experience.

[0209] In this embodiment, the earphone device collects ambient sound signals through its feedforward microphone and in-ear audio signals through its feedback microphone. After a series of processing steps, primarily by the signal processing module built into the earphone device, a target filter for noise reduction of the target audio signal to be output can be determined. Exemplarily, the target filter can be specifically divided into a target feedforward filter and a target feedback filter, which can be jointly applied to the noise reduction of the target audio signal.

[0210] Please see Figure 10 , Figure 10 This is a schematic diagram of the overall signal flow of an audio signal processing method disclosed in an embodiment of this application. For example... Figure 10As shown, the ambient sound signal acquired by the feedforward microphone, after analog-to-digital conversion, downsampling, and FFT processing, can be used to calculate the power spectral density and the frequency domain coherence coefficient between the ambient sound signal and the in-ear audio signal. The in-ear audio signal acquired by the feedback microphone, after analog-to-digital conversion and downsampling, can be subtracted from the target audio signal after downsampling and transfer function filtering. The resulting error signal (residual in-ear audio signal), after FFT processing, can be used to calculate the subband energy and the frequency domain coherence coefficient between the ambient sound signal and the ambient sound signal. Based on this, the headphone device can determine the target filter bank based on the power spectral density corresponding to the ambient sound signal, and further determine the target filter from the target filter bank based on the frequency domain coherence coefficient between the ambient sound signal and the residual in-ear audio signal. Simultaneously, the headphone device can also calculate the gain coefficient used to configure the target filter based on the subband energy corresponding to the residual in-ear audio signal.

[0211] The target filter and corresponding gain coefficients identified above can be applied to both the target feedforward filter and the target feedback filter. Optionally, the ambient sound signal, after analog-to-digital conversion, can also be directly applied to the construction of the feedforward filter; similarly, the in-ear audio signal, after analog-to-digital conversion, can also be directly applied to the construction of the feedback filter. Based on this, by mixing the target audio signal output from the headphone device with the outputs of the target feedforward filter and the target feedback filter respectively after necessary equalization steps, a balanced and noise-reduced target audio signal for output can be obtained. The headphone device then performs digital-to-analog conversion on the target audio signal through its speaker to obtain the target audio signal for output.

[0212] As can be seen, the audio signal processing method described in the above embodiments can comprehensively consider the influence of external noise represented by ambient sound signals and the influence of the internal structure of the headphone device represented by in-ear audio signals. This effectively avoids the problem of low accuracy that may occur when judging the noise type solely based on ambient sound signals. This allows the headphone device to achieve more precise noise reduction processing for the noise type of its current environment, improving the accuracy and reliability of active noise cancellation. Furthermore, calculating noise reduction parameters iteratively helps eliminate errors introduced by the acoustic components themselves, further improving the accuracy of noise reduction processing. In addition, by updating the target filter in a timely manner, the headphone device can perform real-time fine-tuning based on subtle changes in ambient noise and ensure smooth switching of the target filter used by the headphone device. This improves the accuracy and flexibility of active noise cancellation while minimizing user disturbance, thus enhancing the user experience.

[0213] Please see Figure 11 , Figure 11 This is a flowchart illustrating the fourth audio signal processing method disclosed in this application. This method can be applied to the aforementioned headphone device, which may include a feedforward microphone. Figure 11 As shown, the audio signal processing method may include the following steps:

[0214] 1102. Acquire ambient sound signals using a feedforward microphone;

[0215] Step 1102 is similar to step 302 above, and will not be repeated here.

[0216] 1104. Calculate the power spectral density corresponding to the above ambient sound signal;

[0217] 1106. Based on the power spectral density above, determine the scene noise type corresponding to the current scene of the headphone device;

[0218] Steps 1104 and 1106 are similar to steps 504 and 506 above, and will not be repeated here.

[0219] 1108. Based on the above scene noise type, determine the target filter, which is used to perform noise reduction processing on the target audio signal to be output.

[0220] In some embodiments, the headphone device may further determine, based on the aforementioned scene noise type, a target filter group matching the scene noise type from at least one candidate filter group, and then determine a target filter from the target filter group. The target filter group may include one or more noise reduction filters.

[0221] As can be seen, by implementing the audio signal processing method described in the above embodiments, the headphone device can analyze the spectral characteristics of the ambient sound signal by calculating the power spectral density corresponding to the ambient sound signal, thereby accurately determining the current scene of the headphone device and further improving the accuracy of the headphone device in actively reducing noise for its current scene.

[0222] Please see Figure 12 , Figure 12 This is a flowchart illustrating a power spectral density calculation method disclosed in an embodiment of this application. This method can be applied to the aforementioned headphone device. Figure 12 As shown, the power spectral density calculation method may include the following steps:

[0223] 1202. Perform a first audio preprocessing on the ambient sound signal to obtain a target ambient sound signal, wherein the first audio preprocessing includes at least analog-to-digital conversion and downsampling;

[0224] 1204. Window the target ambient sound signal according to the unit window length to obtain at least one frame of ambient sound sub-signal;

[0225] 1206. Perform Fourier transform on each frame of ambient sound sub-signal, and calculate the power spectral density of the ambient sound signal based on the transformed ambient sound sub-signal of each frame.

[0226] In some embodiments, the aforementioned ambient sound sub-signals may comprise a total of M frames, where M is a positive integer. If m equals 1, the headphone device can calculate the power spectral density corresponding to the m-th frame ambient sound sub-signal based on the transformed m-th frame ambient sound sub-signal; if m is greater than 1 and less than or equal to M, the headphone device can calculate the power spectral density corresponding to the m-th frame ambient sound sub-signal based on the transformed m-th frame ambient sound sub-signal and the power spectral density corresponding to the (m-1)-th frame ambient sound sub-signal.

[0227] In other embodiments, the headphone device may also calculate the power spectral density of the ambient sound signal in each frequency domain sub-band based on the transformed ambient sound sub-signals of each frame, wherein each frequency domain sub-band is a frequency domain component of the ambient sound signal in each corresponding frequency range.

[0228] As can be seen, the audio signal processing method described in the above embodiments can segment macroscopically unstable audio signals into multiple audio signal frames with short-term stationarity through windowing, thereby improving the convenience of calculating power spectral density in headphone devices. Furthermore, by dividing the frequency domain into sub-bands, the flexibility of calculating power spectral density within a specific frequency range can be further enhanced, significantly reducing the computational load and effectively improving computational efficiency.

[0229] Please see Figure 13 , Figure 13 This is a flowchart illustrating the fifth audio signal processing method disclosed in this application. This method can be applied to the aforementioned headphone device, which may include a feedforward microphone. Figure 13 As shown, the audio signal processing method may include the following steps:

[0230] 1302. Acquire ambient sound signals using a feedforward microphone;

[0231] Step 1302 is similar to step 302 above, and will not be described again here.

[0232] 1304. Calculate the power spectral density corresponding to the above ambient sound signal;

[0233] Step 1304 is similar to step 504 above, and will not be described again here.

[0234] 1306. Quantize the power spectral density to determine the power spectral density gradient corresponding to each frequency sub-band of the above ambient sound signal;

[0235] 1308. If the power spectral density gradient corresponding to each frequency domain sub-band meets the target scene noise conditions, then the scene noise type corresponding to the target scene noise conditions is determined to be the scene noise type corresponding to the current scene of the headphone device.

[0236] In some embodiments, the headphone device can determine whether the power spectral density within the target frequency range conforms to the power spectral density range corresponding to the target scene based on the power spectral density gradient corresponding to each frequency domain sub-band. If it conforms to the power spectral density range corresponding to the target scene, the scene noise type corresponding to the target scene can be determined as the scene noise type corresponding to the scene currently in which the headphone device is located. The aforementioned target frequency range may correspond to a target scene and includes frequency ranges corresponding to one or more frequency domain sub-bands. The target scene is any one of one or more preset scenes.

[0237] As can be seen, by implementing the audio signal processing method described in the above embodiments, it is possible to conveniently and accurately determine the type of scene noise corresponding to the current scene of the headphone device, thereby improving the accuracy of noise reduction processing of the headphone device.

[0238] Please see Figure 14 , Figure 14 This is a flowchart illustrating the sixth audio signal processing method disclosed in this application. This method can be applied to the aforementioned headphone device, which may include a feedforward microphone and a feedback microphone. Figure 14 As shown, the audio signal processing method may include the following steps:

[0239] 1402. When the headphone device outputs the target audio signal, ambient sound signals are collected through a feedforward microphone, and in-ear audio signals are collected through a feedback microphone.

[0240] Step 1402 is similar to step 302 above, and will not be described again here.

[0241] 1404. Based on the above target audio signal, perform signal cancellation processing on the audio signal in the ear to obtain the residual audio signal in the ear;

[0242] 1406. Based on the above ambient sound signal and the residual audio signal in the ear, calculate the frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear, as well as the subband energy corresponding to the residual audio signal in the ear.

[0243] Steps 1404 and 1406 are similar to steps 810, 812 and 814 as described above, and will not be repeated here.

[0244] 1408. Based on the frequency domain coherence coefficient and subband energy, determine the target filter, which is used to perform noise reduction processing on the target audio signal to be output.

[0245] In some embodiments, the headphone device can determine the noise reduction filter with the highest coherence from the target filter bank as the target filter based on the aforementioned frequency domain coherence coefficient. This target filter bank corresponds to the current scene of the headphone device and includes one or more noise reduction filters. Furthermore, the headphone device can determine the gain coefficient corresponding to the target filter based on the aforementioned subband energy. This target filter is configured according to the aforementioned gain coefficient to perform noise reduction processing on the target audio signal to be output.

[0246] As can be seen, the audio signal processing method described in the above embodiments can comprehensively consider the influence of external noise represented by ambient sound signals and the influence of the internal structure of the headphone device represented by in-ear audio signals. This effectively avoids the problem of low accuracy when judging the noise type based solely on ambient sound signals, so that the headphone device can achieve more accurate noise reduction processing for the noise type of its current scene, which is conducive to improving the accuracy and reliability of the headphone device's active noise reduction.

[0247] Please see Figure 15 , Figure 15 This is a flowchart illustrating the seventh audio signal processing method disclosed in this application. This method can be applied to the aforementioned headphone device, which may include a feedback microphone. Figure 15 As shown, the audio signal processing method may include the following steps:

[0248] 1502. When the headphone device outputs the target audio signal, the audio signal inside the ear is collected through the feedback microphone;

[0249] Step 1502 is similar to step 302 above, and will not be described again here.

[0250] 1504. The target audio signal is filtered by a transfer function filter to obtain the transmitted audio signal corresponding to the target audio signal. The transfer function filter is used to characterize the influence of the audio transmission system of the headphone device on the transmission of the target audio signal.

[0251] 1506. Calculate the error signal between the audio signal inside the ear and the transmitted audio signal, and use it as the residual audio signal inside the ear.

[0252] In some embodiments, when calculating the aforementioned error signal, the headphone device can subtract the in-ear audio signal from the transmitted audio signal to obtain the error signal, and then calculate the mean square error between the in-ear audio signal and the transmitted audio signal based on the error signal. Based on this, the headphone device can update the transfer function filter according to the mean square error, and re-execute the steps of filtering the target audio signal using the transfer function filter to obtain the transmitted audio signal corresponding to the target audio signal, until the update stop condition is met. The error signal obtained when the update stop condition is met is taken as the residual in-ear audio signal.

[0253] As can be seen, implementing the audio signal processing method described in the above embodiments, which calculates noise reduction parameters iteratively, helps eliminate errors introduced by the acoustic devices themselves, further improving the accuracy of noise reduction processing in headphone devices. Furthermore, by updating the target filter in a timely manner, the headphone device can perform real-time fine-tuning based on subtle changes in ambient noise and ensure a smooth switching of the target filter used by the headphone device. This improves the accuracy and flexibility of active noise cancellation in headphone devices while minimizing disturbance to the user, thus enhancing the user experience.

[0254] Please see Figure 16 , Figure 16 This is a modular schematic diagram of an audio signal processing device disclosed in an embodiment of this application. This audio signal processing device can be applied to the aforementioned headphone device, which may include a feedforward microphone and a feedback microphone. Figure 16 As shown, the audio signal processing device may include a signal acquisition unit 1601, a first determination unit 1602, a parameter calculation unit 1603, and a second determination unit 1604, wherein:

[0255] The signal acquisition unit 1601 is used to acquire ambient sound signals through a feedforward microphone and in-ear audio signals through a feedback microphone when the headphone device outputs a target audio signal.

[0256] The first determining unit 1602 is used to determine a target filter group corresponding to the current scene of the headphone device based on the above-mentioned ambient sound signal. The target filter group includes one or more noise reduction filters.

[0257] The parameter calculation unit 1603 is used to calculate the noise reduction parameters based on the above-mentioned ambient sound signal and the in-ear audio signal;

[0258] The second determining unit 1604 is used to determine a target filter from the target filter bank according to the noise reduction parameters. The target filter is used to perform noise reduction processing on the target audio signal to be output.

[0259] As can be seen, using the audio signal processing device described in the above embodiments, the headphone device can first determine the current scene of the headphone device based on the ambient sound signal it has collected, and determine a target filter group consisting of a set of noise reduction filters corresponding to the ambient noise in the scene. Then, it can further select a suitable target filter from the target filter group to perform targeted noise reduction processing on the target audio signal to be output by the headphone device. The above audio signal processing method comprehensively considers the influence of external noise represented by the ambient sound signal and the influence of the internal structure of the headphone device represented by the in-ear audio signal, effectively avoiding the problem of low accuracy that may occur when judging the noise type based solely on the ambient sound signal. This allows the headphone device to achieve more accurate noise reduction processing for the noise type of its current scene, which is beneficial to improving the accuracy and reliability of the headphone device's active noise reduction.

[0260] In one embodiment, the first determining unit 1602 described above may include a power spectral density calculation subunit (not shown), a noise determining subunit, and a filter determining subunit, wherein:

[0261] The power spectral density calculation subunit is used to calculate the power spectral density corresponding to the ambient sound signal;

[0262] The noise determination subunit is used to determine the type of scene noise corresponding to the current scene of the headphone device based on the power spectral density.

[0263] The filter determination subunit is used to determine a target filter group that matches the scene noise type from at least one candidate filter group.

[0264] In one embodiment, the power spectral density calculation subunit described above can be specifically used for:

[0265] The ambient sound signal is subjected to a first audio preprocessing to obtain the target ambient sound signal, wherein the first audio preprocessing includes at least analog-to-digital conversion and downsampling;

[0266] The target ambient sound signal is windowed and segmented according to the unit window length to obtain at least one frame of ambient sound sub-signal;

[0267] Fourier transform is performed on each frame of ambient sound signal, and the power spectral density of the ambient sound signal is calculated based on the transformed ambient sound signal of each frame.

[0268] As an optional implementation, the aforementioned ambient sound sub-signal may comprise a total of M frames, where M is a positive integer. When the power spectral density calculation subunit calculates the power spectral density corresponding to the ambient sound signal based on the transformed ambient sound sub-signals of each frame, it may specifically include:

[0269] If m equals 1, then calculate the power spectral density corresponding to the m-th frame ambient phonon signal based on the transformed m-th frame ambient phonon signal;

[0270] If m is greater than 1 and less than or equal to M, then the power spectral density of the environmental phonon signal in the m-th frame is calculated based on the transformed environmental phonon signal in the m-th frame and the power spectral density of the environmental phonon signal in the (m-1)-th frame.

[0271] As another optional implementation, when the power spectral density calculation subunit calculates the power spectral density corresponding to the ambient sound signal based on the transformed ambient sound sub-signals of each frame, it may specifically include:

[0272] Based on the transformed ambient sound sub-signals of each frame, the power spectral density of the ambient sound signal in each frequency domain sub-band is calculated, where each frequency domain sub-band is the frequency domain component of the ambient sound signal in each corresponding frequency range.

[0273] In one embodiment, the noise determination subunit described above can be specifically used for:

[0274] The power spectral density is quantized to determine the power spectral density gradient of the ambient sound signal in each frequency sub-band.

[0275] If the power spectral density gradient corresponding to each frequency sub-band meets the target scene noise conditions, then the scene noise type corresponding to the target scene noise conditions is determined to be the scene noise type corresponding to the current scene of the headphone device.

[0276] Specifically, when determining the type of scene noise corresponding to the current scene of the headphone device, the noise determination subunit can also:

[0277] Based on the power spectral density gradient corresponding to each frequency sub-band, determine whether the power spectral density in the target frequency range conforms to the power spectral density range corresponding to the target scene. If it conforms to the power spectral density range corresponding to the target scene, then determine the scene noise type corresponding to the target scene as the scene noise type corresponding to the current scene of the headphone device.

[0278] The target frequency range corresponds to the target scenario and includes the frequency range corresponding to one or more frequency domain sub-bands. The target scenario is any one of one or more preset scenarios.

[0279] As can be seen, by using the audio signal processing device described in the above embodiments, the headphone device can analyze the spectral characteristics of the ambient sound signal by calculating the power spectral density corresponding to the ambient sound signal, thereby accurately determining the current scene of the headphone device and further improving the accuracy of the headphone device in actively reducing noise for its current scene.

[0280] In one embodiment, the noise reduction parameters may include frequency domain coherence coefficients, and the parameter calculation unit 1603 may include a cancellation processing subunit (not shown) and a parameter calculation subunit, wherein:

[0281] The cancellation processing subunit is used to perform signal cancellation processing on the in-ear audio signal based on the target audio signal to obtain the residual audio signal in the in-ear.

[0282] The parameter calculation subunit is used to calculate the frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear.

[0283] In one embodiment, the above-described cancellation processing subunit can be specifically used for:

[0284] The target audio signal is filtered by a transfer function filter to obtain the transmitted audio signal corresponding to the target audio signal. The transfer function filter is used to characterize the influence of the audio transmission system in which the headphone device is located on the transmission of the target audio signal.

[0285] The error signal between the in-ear audio signal and the transmitted audio signal is calculated and used as the residual in-ear audio signal.

[0286] As an optional implementation, the above-mentioned cancellation processing subunit may specifically include the following when calculating the error signal between the in-ear audio signal and the transmitted audio signal:

[0287] The error signal is obtained by subtracting the audio signal inside the ear from the transmitted audio signal.

[0288] Based on the error signal, calculate the mean square error between the in-ear audio signal and the transmitted audio signal;

[0289] Based on the mean square error, update the transfer function filter and re-execute the filtering of the target audio signal through the transfer function filter to obtain the transfer audio signal corresponding to the target audio signal until the update stopping condition is met. The error signal obtained when the update stopping condition is met is taken as the residual audio signal in the ear.

[0290] In one embodiment, the above parameter calculation subunit can be specifically used for:

[0291] Based on the first sub-signals corresponding to each target frequency domain sub-band of the ambient sound signal and the second sub-signals corresponding to each target frequency domain sub-band of the residual audio signal in the ear, the frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear is calculated. The frequency range corresponding to each target frequency domain sub-band is determined by the target filter bank.

[0292] Based on this, the second determining unit 1604 mentioned above can be specifically used for:

[0293] The frequency domain coherence coefficients of one or more noise reduction filters in the target filter bank are compared with the frequency domain coherence coefficients between the ambient sound signal and the residual audio signal in the ear. Based on the comparison results, the noise reduction filter with the highest coherence is determined from the target filter bank and used as the target filter.

[0294] In one embodiment, the noise reduction parameters may further include subband energy, and the parameter calculation subunit of the parameter calculation unit 1603 may further be used for:

[0295] Calculate the subband energy corresponding to the residual audio signal in the ear based on the residual audio signal in the ear;

[0296] Based on this, the second determining unit 1604 mentioned above can be specifically used for:

[0297] Based on the frequency domain coherence coefficient, the noise reduction filter with the highest coherence is selected from the target filter bank as the target filter. Based on the subband energy, the gain coefficient corresponding to the target filter is determined. The target filter is configured according to the gain coefficient to perform noise reduction processing on the target audio signal to be output.

[0298] As an optional implementation, the parameter calculation subunit, when calculating the subband energy corresponding to the residual audio signal in the ear, may specifically include:

[0299] The energy of the sub-band corresponding to the residual audio signal in the ear is calculated based on the third sub-signal corresponding to each target frequency domain sub-band. The frequency range corresponding to each target frequency domain sub-band is determined by the target filter bank.

[0300] In one embodiment, the audio signal processing apparatus may further include an update unit (not shown), which may be used to:

[0301] If the target filter bank is updated, or if the target filter bank is not updated within the first time period, the target filter is re-determined from the above target filter bank based on the noise reduction parameters.

[0302] As can be seen, the audio signal processing device described in the above embodiments can comprehensively consider the influence of external noise represented by ambient sound signals and the influence of the internal structure of the headphone device represented by in-ear audio signals. This effectively avoids the problem of low accuracy that may occur when judging the noise type solely based on ambient sound signals. This allows the headphone device to achieve more precise noise reduction processing for the noise type of its current environment, improving the accuracy and reliability of active noise cancellation. Furthermore, calculating noise reduction parameters iteratively helps eliminate errors introduced by the acoustic components themselves, further improving the accuracy of noise reduction processing. In addition, by updating the target filter in a timely manner, the headphone device can perform real-time fine-tuning based on subtle changes in ambient noise and ensure smooth switching of the target filter used by the headphone device. This improves the accuracy and flexibility of active noise cancellation while minimizing user disturbance, thus enhancing the user experience.

[0303] Please see Figure 17 , Figure 17 This is a modular schematic diagram of another audio signal processing device disclosed in an embodiment of this application. This audio signal processing device can be applied to the aforementioned headphone device, which may include a feedforward microphone. For example... Figure 17 As shown, the audio signal processing device may include an ambient sound signal acquisition unit 1701, a power spectral density calculation unit 1702, a noise determination unit 1703, and a filter determination unit 1704, wherein:

[0304] The ambient sound signal acquisition unit 1701 is used to acquire ambient sound signals through a feedforward microphone;

[0305] The power spectral density calculation unit 1702 is used to calculate the power spectral density corresponding to the ambient sound signal.

[0306] The noise determination unit 1703 is used to determine the scene noise type corresponding to the current scene of the headphone device based on the power spectral density mentioned above.

[0307] The filter determination unit 1704 is used to determine a target filter based on the above-mentioned scene noise type. The target filter is used to perform noise reduction processing on the target audio signal to be output.

[0308] In one embodiment, the filter determination unit 1704 described above can be specifically used for:

[0309] Based on the scene noise type, a target filter group matching the scene noise type is determined from at least one candidate filter group, and the target filter group includes one or more noise reduction filters.

[0310] Determine the target filter from the target filter bank.

[0311] By using the audio signal processing device described in the above embodiments, the headphone device can analyze the spectral characteristics of the ambient sound signal by calculating the power spectral density corresponding to the ambient sound signal, thereby accurately determining the current scene of the headphone device and further improving the accuracy of the headphone device in actively reducing noise for its current scene.

[0312] Please see Figure 18 , Figure 18 This is a modular schematic diagram of a power spectral density calculation device disclosed in an embodiment of this application. This power spectral density calculation device can be applied to the aforementioned headphone device. Figure 18 As shown, the power spectral density calculation device may include a preprocessing unit 1801, a windowing and segmentation unit 1802, and a transformation calculation unit 1803, wherein:

[0313] The preprocessing unit 1801 is used to perform a first audio preprocessing on the ambient sound signal to obtain a target ambient sound signal, wherein the first audio preprocessing includes at least analog-to-digital conversion and downsampling;

[0314] The windowing segmentation unit 1802 is used to window and segment the target ambient sound signal according to the unit window length to obtain at least one frame of ambient sound sub-signal.

[0315] The transformation calculation unit 1803 is used to perform Fourier transform on each frame of ambient sound sub-signal and calculate the power spectral density corresponding to the ambient sound signal based on the transformed ambient sound sub-signal of each frame.

[0316] As an optional implementation, the aforementioned environmental sound sub-signal may include a total of M frames, where M is a positive integer, and the aforementioned transformation calculation unit 1803 may specifically be used for:

[0317] If m equals 1, then calculate the power spectral density corresponding to the m-th frame ambient phonon signal based on the transformed m-th frame ambient phonon signal;

[0318] If m is greater than 1 and less than or equal to M, then the power spectral density of the environmental phonon signal in the m-th frame is calculated based on the transformed environmental phonon signal in the m-th frame and the power spectral density of the environmental phonon signal in the (m-1)-th frame.

[0319] As another optional implementation, the transformation calculation unit 1803 described above can specifically be used for:

[0320] Based on the transformed ambient sound sub-signals of each frame, the power spectral density of the ambient sound signal in each frequency domain sub-band is calculated, where each frequency domain sub-band is the frequency domain component of the ambient sound signal in each corresponding frequency range.

[0321] The audio signal processing apparatus described in the above embodiments can segment macroscopically unstable audio signals into multiple audio signal frames with short-term stationarity by windowing, thereby improving the convenience of calculating power spectral density in headphone devices. Furthermore, by dividing the frequency domain into sub-bands, the flexibility of calculating power spectral density within a specific frequency range can be further enhanced, significantly reducing the computational load and effectively improving computational efficiency.

[0322] Please see Figure 19 , Figure 19 This is a modular schematic diagram of another audio signal processing device disclosed in the embodiments of this application. This audio signal processing device can be applied to the aforementioned headphone device, which may include a feedforward microphone. Figure 19 As shown, the audio signal processing device may include an ambient sound signal acquisition unit 1901, a power spectral density calculation unit 1902, a quantization calculation unit 1903, and a noise determination unit 1904, wherein:

[0323] The ambient sound signal acquisition unit 1901 is used to acquire ambient sound signals through a feedforward microphone;

[0324] The power spectral density calculation unit 1902 is used to calculate the power spectral density corresponding to the ambient sound signal.

[0325] The quantization calculation unit 1903 is used to quantize the power spectral density and determine the power spectral density gradient corresponding to each frequency sub-band of the above ambient sound signal.

[0326] The noise determination unit 1904 is used to determine the scene noise type corresponding to the target scene noise condition as the scene noise type corresponding to the current scene of the headphone device, provided that the power spectral density gradient corresponding to each frequency sub-band meets the target scene noise condition.

[0327] In one embodiment, the noise determination unit 1904 described above can be specifically used for:

[0328] Based on the power spectral density gradient corresponding to each frequency sub-band, determine whether the power spectral density in the target frequency range conforms to the power spectral density range corresponding to the target scene. If it conforms to the power spectral density range corresponding to the target scene, then determine the scene noise type corresponding to the target scene as the scene noise type corresponding to the current scene of the headphone device.

[0329] The target frequency range corresponds to the target scenario and includes the frequency range corresponding to one or more frequency domain sub-bands. The target scenario is any one of one or more preset scenarios.

[0330] The audio signal processing device described in the above embodiments can conveniently and accurately determine the type of scene noise corresponding to the current scene of the headphone device, thereby improving the accuracy of noise reduction processing of the headphone device.

[0331] Please see Figure 20 , Figure 20 This is a modular schematic diagram of the fourth audio signal processing device disclosed in the embodiments of this application. This audio signal processing device can be applied to the aforementioned headphone device, which may include a feedforward microphone and a feedback microphone. For example... Figure 20 As shown, the audio signal processing device may include a signal acquisition unit 2001, a cancellation processing unit 2002, a parameter calculation unit 2003, and a filter determination unit 2004, wherein:

[0332] The signal acquisition unit 2001 is used to acquire ambient sound signals through a feedforward microphone and in-ear audio signals through a feedback microphone when the headphone device outputs a target audio signal.

[0333] The cancellation processing unit 2002 is used to perform signal cancellation processing on the in-ear audio signal based on the target audio signal to obtain the residual audio signal in the in-ear.

[0334] The parameter calculation unit 2003 is used to calculate the frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear, as well as the subband energy corresponding to the residual audio signal in the ear, based on the above-mentioned ambient sound signal and the residual audio signal in the ear.

[0335] The filter determination unit 2004 is used to determine the target filter based on the frequency domain coherence coefficient and subband energy, and the target filter is used to perform noise reduction processing on the target audio signal to be output.

[0336] In one embodiment, the parameter calculation unit 2003 described above can be specifically used for:

[0337] Based on the frequency domain coherence coefficient, the noise reduction filter with the highest coherence is determined from the target filter bank as the target filter. The target filter bank corresponds to the current scene of the headphone device and includes one or more noise reduction filters.

[0338] Based on the subband energy, the gain coefficient corresponding to the target filter is determined. The target filter is configured according to the gain coefficient to perform noise reduction processing on the target audio signal to be output.

[0339] The audio signal processing device described in the above embodiments can comprehensively consider the influence of external noise represented by ambient sound signals and the influence of the internal structure of the headphone device represented by in-ear audio signals. This effectively avoids the problem of low accuracy when judging the noise type based solely on ambient sound signals, so that the headphone device can achieve more accurate noise reduction processing for the noise type of its current scene, which is conducive to improving the accuracy and reliability of the headphone device's active noise reduction.

[0340] Please see Figure 21 , Figure 21 This is a modular schematic diagram of the fifth audio signal processing device disclosed in the embodiments of this application. This audio signal processing device can be applied to the aforementioned headphone device, which may include a feedback microphone. For example... Figure 21 As shown, the audio signal processing device may include an in-ear audio signal acquisition unit 2101, a transmission filtering unit 2102, and an error calculation unit 2103, wherein:

[0341] The in-ear audio signal acquisition unit 2101 is used to acquire in-ear audio signals through a feedback microphone when the headphone device outputs a target audio signal;

[0342] The transfer filtering unit 2102 is used to filter the target audio signal through a transfer function filter to obtain the transfer audio signal corresponding to the target audio signal. The transfer function filter is used to characterize the influence of the audio transmission system in which the headphone device is located on the transmission of the target audio signal.

[0343] Error calculation unit 2103 is used to calculate the error signal between the above-mentioned in-ear audio signal and the transmitted audio signal, as the residual in-ear audio signal.

[0344] In one embodiment, the error calculation unit 2103 described above can be specifically used for:

[0345] The error signal is obtained by subtracting the audio signal inside the ear from the transmitted audio signal.

[0346] Based on the error signal, calculate the mean square error between the in-ear audio signal and the transmitted audio signal;

[0347] Based on the mean square error, update the transfer function filter and re-execute the filtering of the target audio signal through the transfer function filter to obtain the transfer audio signal corresponding to the target audio signal until the update stopping condition is met. The error signal obtained when the update stopping condition is met is taken as the residual audio signal in the ear.

[0348] The audio signal processing apparatus described in the above embodiments calculates noise reduction parameters iteratively, which helps eliminate errors introduced by the acoustic components themselves and further improves the accuracy of noise reduction processing in headphone devices. Furthermore, by updating the target filter in a timely manner, the headphone device can perform real-time fine-tuning based on subtle changes in ambient noise and ensure a smooth switching of the target filter used by the headphone device. This improves the accuracy and flexibility of active noise cancellation in headphone devices while minimizing disturbance to the user, thus enhancing the user experience.

[0349] Please see Figure 22 , Figure 22 This is a modular schematic diagram of an earphone disclosed in an embodiment of this application. Figure 22 As shown, the headphones may include:

[0350] Memory 2201 storing executable program code;

[0351] Processor 2202 coupled to memory 2201;

[0352] The processor 2202 can call the executable program code stored in the memory 2201 to execute all or part of the steps in any of the audio signal processing methods described in the above embodiments.

[0353] Furthermore, embodiments of this application disclose a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program enables a computer to perform all or part of the steps in any of the audio signal processing methods described in the above embodiments.

[0354] Furthermore, this application further discloses a computer program product that, when run on a computer, enables the computer to execute all or part of the steps in any of the audio signal processing methods described in the above embodiments.

[0355] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0356] The above provides a detailed description of an audio signal processing method and apparatus, headphone device, and storage medium disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An audio signal processing method, characterized in that, Applied to an earphone device, the earphone device including a feedforward microphone and a feedback microphone, the method includes: When the headphone device outputs a target audio signal, ambient sound signals are acquired through the feedforward microphone, and in-ear audio signals are acquired through the feedback microphone. Based on the power spectral density corresponding to the ambient sound signal, a target filter group corresponding to the current scene of the headphone device is determined, and the target filter group includes one or more noise reduction filters. Based on the ambient sound signal and the in-ear audio signal, the noise reduction parameters are calculated; Based on the noise reduction parameters, a target filter is determined from the target filter bank, and the target filter is used to perform noise reduction processing on the target audio signal to be output.

2. The method according to claim 1, characterized in that, The step of determining the target filter group corresponding to the current scene of the headphone device based on the ambient sound signal includes: Calculate the power spectral density corresponding to the ambient sound signal; Based on the power spectral density, determine the scene noise type corresponding to the current scene of the headphone device; From at least one candidate filter group, a target filter group that matches the noise type of the scene is determined.

3. The method according to claim 2, characterized in that, The calculation of the power spectral density corresponding to the ambient sound signal includes: The ambient sound signal is subjected to a first audio preprocessing to obtain a target ambient sound signal, wherein the first audio preprocessing includes at least analog-to-digital conversion and downsampling; The target ambient sound signal is windowed and segmented according to the unit window length to obtain at least one frame of ambient sound sub-signal; Each frame of ambient sound signal is subjected to a Fourier transform, and the power spectral density corresponding to the ambient sound signal is calculated based on the transformed ambient sound signal of each frame.

4. The method according to claim 3, characterized in that, The ambient sound sub-signal comprises M frames, where M is a positive integer. The calculation of the power spectral density corresponding to the ambient sound signal based on each transformed frame of ambient sound sub-signal includes: If m equals 1, then the power spectral density corresponding to the m-th frame ambient phonon signal is calculated based on the transformed m-th frame ambient phonon signal. If m is greater than 1 and less than or equal to M, then the power spectral density corresponding to the m-th frame ambient phonon signal is calculated based on the transformed m-th frame ambient phonon signal and the power spectral density corresponding to the (m-1)-th frame ambient phonon signal.

5. The method according to claim 3, characterized in that, The step of calculating the power spectral density corresponding to the ambient sound signal based on the transformed ambient sound sub-signals of each frame includes: Based on the transformed ambient sound sub-signals of each frame, the power spectral density of the ambient sound signal in each frequency domain sub-band is calculated, wherein each frequency domain sub-band is the frequency domain component of the ambient sound signal in each corresponding frequency range.

6. The method according to claim 2, characterized in that, The step of determining the scene noise type corresponding to the current scene of the headphone device based on the power spectral density includes: The power spectral density is quantized to determine the power spectral density gradient of the ambient sound signal in each frequency sub-band. If the power spectral density gradient corresponding to each frequency sub-band meets the target scene noise condition, then the scene noise type corresponding to the target scene noise condition is determined to be the scene noise type corresponding to the scene where the headphone device is currently located.

7. The method according to claim 6, characterized in that, If the power spectral density gradient corresponding to each frequency sub-band meets the target scene noise condition, then the scene noise type corresponding to the target scene noise condition is determined as the scene noise type corresponding to the current scene of the headphone device, including: Based on the power spectral density gradient corresponding to each frequency sub-band, it is determined whether the power spectral density in the target frequency range conforms to the power spectral density range corresponding to the target scene. If it conforms to the power spectral density range corresponding to the target scene, the scene noise type corresponding to the target scene is determined to be the scene noise type corresponding to the scene where the headphone device is currently located. The target frequency range corresponds to the target scene and includes frequency ranges corresponding to one or more frequency domain sub-bands. The target scene is any one of one or more preset scenes.

8. The method according to any one of claims 1 to 7, characterized in that, The noise reduction parameters include frequency domain coherence coefficients. The noise reduction parameters, calculated based on the ambient sound signal and the in-ear audio signal, include: Based on the target audio signal, the in-ear audio signal is subjected to signal cancellation processing to obtain the residual in-ear audio signal; Calculate the frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear.

9. The method according to claim 8, characterized in that, The step of performing signal cancellation processing on the in-ear audio signal based on the target audio signal to obtain the residual in-ear audio signal includes: The target audio signal is filtered by a transfer function filter to obtain the transmitted audio signal corresponding to the target audio signal. The transfer function filter is used to characterize the influence of the audio transmission system in which the headphone device is located on the transmission of the target audio signal. The error signal between the in-ear audio signal and the transmitted audio signal is calculated as the residual in-ear audio signal.

10. The method according to claim 9, characterized in that, The calculation of the error signal between the in-ear audio signal and the transmitted audio signal, as the residual in-ear audio signal, includes: The error signal is obtained by subtracting the in-ear audio signal from the transmitted audio signal. Based on the error signal, calculate the mean square error between the in-ear audio signal and the transmitted audio signal; Based on the mean square error, the transfer function filter is updated, and the filtering of the target audio signal through the transfer function filter is re-executed to obtain the transfer audio signal corresponding to the target audio signal, until the update stop condition is met, and the error signal obtained when the update stop condition is met is taken as the residual audio signal in the ear.

11. The method according to claim 8, characterized in that, The calculation of the frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear includes: The frequency domain coherence coefficient between the ambient sound signal and the residual audio signal in the ear is calculated based on the first sub-signals corresponding to the ambient sound signal in each target frequency domain sub-band and the second sub-signals corresponding to the residual audio signal in the ear in each target frequency domain sub-band. The frequency range corresponding to each target frequency domain sub-band is determined by the target filter bank.

12. The method according to claim 8, characterized in that, The step of determining the target filter from the target filter bank based on the noise reduction parameters includes: The frequency domain coherence coefficients of one or more noise reduction filters included in the target filter group are compared with the frequency domain coherence coefficients between the ambient sound signal and the residual audio signal in the ear. Based on the comparison results, the noise reduction filter with the highest coherence is determined from the target filter group and used as the target filter.

13. The method according to claim 8, characterized in that, The noise reduction parameters also include subband energy. After performing signal cancellation processing on the in-ear audio signal based on the target audio signal to obtain the residual in-ear audio signal, the method further includes: Calculate the subband energy corresponding to the residual audio signal in the ear based on the residual audio signal in the ear; The step of determining the target filter from the target filter bank based on the noise reduction parameters includes: Based on the frequency domain coherence coefficient, the noise reduction filter with the highest coherence is determined from the target filter bank as the target filter, and the gain coefficient corresponding to the target filter is determined based on the subband energy. The target filter is configured according to the gain coefficient to perform noise reduction processing on the target audio signal to be output.

14. The method according to claim 13, characterized in that, The step of calculating the sub-band energy corresponding to the residual audio signal in the ear based on the residual audio signal in the ear includes: The energy of the sub-band corresponding to the residual audio signal in the ear is calculated based on the third sub-signal corresponding to each target frequency domain sub-band, wherein the frequency range corresponding to each target frequency domain sub-band is determined by the target filter bank.

15. The method according to any one of claims 1 to 7, characterized in that, After determining the target filter from the target filter bank based on the noise reduction parameters, the method further includes: If the target filter group is updated, or if the target filter group is not updated within a first time period, the target filter is re-determined from the target filter group based on the noise reduction parameters.

16. An audio signal processing device, characterized in that, Applied to headphone devices, the headphone devices including a feedforward microphone and a feedback microphone, the audio signal processing device includes: The signal acquisition unit is used to acquire ambient sound signals through the feedforward microphone and in-ear audio signals through the feedback microphone when the headphone device outputs a target audio signal. The first determining unit is configured to determine a target filter group corresponding to the current scene of the headphone device based on the power spectral density corresponding to the ambient sound signal, wherein the target filter group includes one or more noise reduction filters; The parameter calculation unit is used to calculate the noise reduction parameters based on the ambient sound signal and the in-ear audio signal. The second determining unit is used to determine a target filter from the target filter bank according to the noise reduction parameters, and the target filter is used to perform noise reduction processing on the target audio signal to be output.

17. A headphone device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 15.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 15.

Images