Audio processing method and apparatus, earphone, and storage medium

Abstract

The present disclosure relates to an audio processing method, device, earphone and storage medium. The audio processing method comprises: obtaining a target frequency response curve; determining a first sound signal based on the target frequency response curve; determining a second sound signal based on the target frequency response curve; determining a first sound energy value according to the first sound signal; determining a second sound energy value according to the second sound signal; determining a filter parameter to be updated according to the first sound energy value and the second sound energy value; adjusting a preset filter based on the filter parameter to be updated, and processing a sound signal played by a loudspeaker using the adjusted filter. In the present disclosure, the sound signal in the ear canal can mask the environmental sound signal entering the ear canal by adjusting the sound signal played by the loudspeaker, and the subjective effect of not hearing the environmental sound signal is achieved by using the psychoacoustic masking effect, thereby improving the listening experience of the user.

Description

Technical Field

[0001] This disclosure relates to the field of audio signal processing technology, and in particular to an audio processing method, apparatus, headphones, and storage medium. Background Technology

[0002] When users wear semi-in-ear headphones, the lack of ear loops allows a significant amount of ambient noise to leak into the ears, affecting the listening experience of the headphone speakers. While active noise cancellation algorithms are typically used to address external noise, semi-in-ear headphones, unlike in-ear headphones which have a sealed acoustic chamber, present greater challenges in designing noise cancellation algorithms, resulting in less stable noise cancellation performance. Furthermore, they place higher demands on the headphone speakers, leading to poor noise cancellation performance for speakers with weak low-frequency capabilities. Therefore, external noise still negatively impacts the listening experience of the headphone speakers.

[0003] Therefore, how to achieve noise cancellation in semi-in-ear headphones is an urgent problem to be solved. Summary of the Invention

[0004] To overcome the problems existing in the related technologies, this disclosure provides an audio processing method, apparatus, headphones, and storage medium.

[0005] According to a first aspect of the present disclosure, an audio processing method is provided, applied to headphones, the processing method comprising:

[0006] Obtain the target frequency response curve, which is the frequency response curve received by the ear canal of the wearer of the headphones;

[0007] Based on the target frequency response curve, a first sound signal is determined; wherein, the first sound signal represents the ambient sound that penetrates the headphones and is heard by the wearer;

[0008] Based on the target frequency response curve, a second sound signal is determined; wherein the second sound signal represents the sound played by the speaker heard by the wearer;

[0009] A first sound energy value is determined based on the first sound signal, wherein the first sound energy value is the energy of each frequency point contained in the first sound signal;

[0010] A second sound energy value is determined based on the second sound signal, wherein the second sound energy value is the energy of each frequency point contained in the second sound signal;

[0011] Based on the first sound energy value and the second sound energy value, determine the filter parameters to be updated;

[0012] The preset filter is adjusted based on the filter parameters to be updated, and the adjusted filter is used to process the sound signal played by the speaker.

[0013] In an exemplary embodiment, determining the first sound signal based on the target frequency response curve includes:

[0014] Acquire ambient sound signals around the headphones;

[0015] The first sound signal is determined based on the ambient sound signal and the pre-stored first mapping relationship; wherein, the first mapping relationship is used to represent the mapping relationship between the target frequency response curve and the first sound transmission path, and the first sound transmission path represents the sound transmission path from the feedforward microphone of the earphone to the ear canal.

[0016] In one exemplary embodiment, determining the second sound signal based on the target frequency response curve includes:

[0017] Obtain the speaker sound signal played by the headphone's speaker.

[0018] A second sound signal is determined based on the speaker sound signal and a pre-stored second mapping relationship; wherein, the second mapping relationship is used to represent the mapping relationship between the target frequency response curve and the second sound transmission path, and the second sound transmission path represents the sound transmission path from the speaker of the earphone to the ear canal.

[0019] In one exemplary embodiment, obtaining the target frequency response curve includes:

[0020] Acquire a test signal, wherein the test signal is the sound signal collected by the feedback microphone after the initial test signal emitted by the speaker propagates in the wearer's ear canal;

[0021] The target frequency response curve is determined based on the initial test signal and the test signal, wherein the target frequency response curve is used to characterize the tightness of the user's wearing and the user's ear canal information.

[0022] In an exemplary embodiment, determining the target frequency response curve based on the initial test signal and the test signal includes:

[0023] A third initial sound energy value is determined based on the initial test signal, wherein the third initial sound energy value is the energy of each frequency point contained in the initial test signal;

[0024] A third sound energy value is determined based on the test signal, wherein the third sound energy value is the energy of each frequency point contained in the test signal;

[0025] The target frequency response curve is determined based on the third sound energy value and the third initial sound energy value.

[0026] In an exemplary embodiment, determining the target frequency response curve based on the third sound energy value and the third initial sound energy value includes:

[0027] Based on the third sound energy value and the third initial sound energy value, a first reference parameter is determined, wherein the first reference parameter is the cross power spectrum of the third sound energy value and the third initial sound energy value at each frequency point;

[0028] Based on the third initial sound energy value, a second reference parameter is determined, wherein the second reference parameter is the autopower spectrum of the third initial sound energy value at each frequency point;

[0029] The target frequency response curve is determined based on the first reference parameter and the second reference parameter.

[0030] In an exemplary embodiment, determining the target frequency response curve based on the first reference parameter and the second reference parameter includes:

[0031] The ratio of the first reference parameter to the second reference parameter at each frequency point is used as the frequency response parameter of the target frequency response curve, and the target frequency response curve is formed.

[0032] In an exemplary embodiment, determining the filter parameters to be updated based on the first sound energy value and the second sound energy value includes:

[0033] When the first sound energy value is less than or equal to the second sound energy value, the filter parameters to be updated are determined.

[0034] In one exemplary embodiment, acquiring the ambient sound signal includes:

[0035] Acquire environmental time-domain signals, which are the representation of environmental sound in the time domain;

[0036] The environmental time-domain signal is transformed by time-frequency transformation to obtain an environmental sound signal, wherein the environmental sound signal is the frequency domain representation of the environmental sound and includes multiple frequency points.

[0037] In one exemplary embodiment, acquiring the speaker sound signal played by the headphone's speaker includes:

[0038] Acquire the speaker time-domain signal, which is the way the speaker sound is expressed in the time domain;

[0039] The speaker's time-domain signal is transformed by time-frequency conversion to obtain the speaker's sound signal, wherein the speaker's sound signal is the expression of the speaker's sound in the frequency domain and includes multiple frequency points.

[0040] According to a second aspect of the present disclosure, an audio processing apparatus is provided for use in headphones, the processing apparatus comprising:

[0041] The acquisition module is configured to acquire a target frequency response curve, which is the frequency response curve received by the ear canal of the wearer of the headphones;

[0042] The first determining module is configured to determine a first sound signal based on the target frequency response curve; wherein the first sound signal represents the ambient sound that penetrates the headphones and is heard by the wearer;

[0043] The second determining module is configured to determine a second sound signal based on the target frequency response curve; wherein the second sound signal represents the sound played by the speaker heard by the wearer;

[0044] The third determining module is configured to determine a first sound energy value based on the first sound signal, wherein the first sound energy value is the energy of each frequency point contained in the first sound signal;

[0045] The fourth determining module is configured to determine a second sound energy value based on the second sound signal, wherein the second sound energy value is the energy of each frequency point contained in the second sound signal;

[0046] The fifth determining module is configured to determine the filter parameters to be updated based on the first sound energy value and the second sound energy value;

[0047] The processing module is configured to adjust the parameters of a preset filter based on the filter parameters to be updated, and to process the sound signal played by the speaker using the adjusted filter.

[0048] In one exemplary embodiment, the first determining module is further configured to:

[0049] Acquire ambient sound signals around the headphones;

[0050] The first sound signal is determined based on the ambient sound signal and the pre-stored first mapping relationship; wherein, the first mapping relationship is used to represent the mapping relationship between the target frequency response curve and the first sound transmission path, and the first sound transmission path represents the sound transmission path from the feedforward microphone of the earphone to the ear canal.

[0051] In one exemplary embodiment, the second determining module is further configured to:

[0052] Obtain the speaker sound signal played by the headphone's speaker.

[0053] A second sound signal is determined based on the speaker sound signal and a pre-stored second mapping relationship; wherein, the second mapping relationship is used to represent the mapping relationship between the target frequency response curve and the second sound transmission path, and the second sound transmission path represents the sound transmission path from the speaker of the earphone to the ear canal.

[0054] In one exemplary embodiment, the acquisition module is further configured to:

[0055] Acquire a test signal, wherein the test signal is the sound signal collected by the feedback microphone after the initial test signal emitted by the speaker propagates in the wearer's ear canal;

[0056] The target frequency response curve is determined based on the initial test signal and the test signal, wherein the target frequency response curve is used to characterize the tightness of the user's wearing and the user's ear canal information.

[0057] In one exemplary embodiment, the acquisition module is further configured to:

[0058] A third initial sound energy value is determined based on the initial test signal, wherein the third initial sound energy value is the energy of each frequency point contained in the initial test signal;

[0059] A third sound energy value is determined based on the test signal, wherein the third sound energy value is the energy of each frequency point contained in the test signal;

[0060] The target frequency response curve is determined based on the third sound energy value and the third initial sound energy value.

[0061] In one exemplary embodiment, the acquisition module is further configured to:

[0062] Based on the third sound energy value and the third initial sound energy value, a first reference parameter is determined, wherein the first reference parameter is the cross power spectrum of the third sound energy value and the third initial sound energy value at each frequency point;

[0063] Based on the third initial sound energy value, a second reference parameter is determined, wherein the second reference parameter is the autopower spectrum of the third initial sound energy value at each frequency point;

[0064] The target frequency response curve is determined based on the first reference parameter and the second reference parameter.

[0065] In one exemplary embodiment, the acquisition module is further configured to:

[0066] The ratio of the first reference parameter to the second reference parameter at each frequency point is used as the frequency response parameter of the target frequency response curve, and the target frequency response curve is formed.

[0067] In one exemplary embodiment, the fifth determining module is further configured to:

[0068] When the first sound energy value is less than or equal to the second sound energy value, the filter parameters to be updated are determined.

[0069] In one exemplary embodiment, the first determining module is further configured to:

[0070] Acquire environmental time-domain signals, which are the representation of environmental sound in the time domain;

[0071] The environmental time-domain signal is transformed by time-frequency transformation to obtain an environmental sound signal, wherein the environmental sound signal is the frequency domain representation of the environmental sound and includes multiple frequency points.

[0072] In one exemplary embodiment, the second determining module is further configured to

[0073] Acquire the speaker time-domain signal, which is the way the speaker sound is expressed in the time domain;

[0074] The speaker's time-domain signal is transformed by time-frequency conversion to obtain the speaker's sound signal, wherein the speaker's sound signal is the expression of the speaker's sound in the frequency domain and includes multiple frequency points.

[0075] According to a third aspect of the present disclosure, an earphone is provided, the earphone including a housing and a feedforward microphone, a feedback microphone, a speaker, and a controller disposed on the housing, the controller being communicatively connected to the feedforward microphone, the feedback microphone, and the speaker respectively;

[0076] The feedforward microphone is used to collect ambient sound signals around the headphones;

[0077] The feedback microphone is used to collect sound signals from inside the ear canal of the earphone wearer;

[0078] The speaker is used to play speaker sound signals and initial test signals;

[0079] The controller includes a processor and a memory, the memory storing computer program instructions executable by the processor, the processor being configured to invoke the computer program instructions to perform an audio processing method as described in the first aspect of the present disclosure.

[0080] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which computer program instructions are stored, which, when invoked by a processor, execute the audio processing method as described in the first aspect of the present disclosure.

[0081] The above-described method of this disclosure has the following beneficial effects: The method of this disclosure does not require high requirements for the hardware structure of the headphones, nor does it require complex noise reduction algorithm design. It utilizes the psychoacoustic masking effect to allow the sound played by the speaker heard by the wearer to mask the sound of ambient sound penetrating the headphones and being heard by the wearer, thereby achieving the effect that the wearer cannot subjectively hear the ambient sound signal, thus improving the user's listening experience.

[0082] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0083] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0084] Figure 1 This is a flowchart illustrating an audio processing method according to an exemplary embodiment;

[0085] Figure 2 This is a schematic diagram of the structure of an earphone according to an exemplary embodiment;

[0086] Figure 3 This is a flowchart illustrating a method for determining a first sound signal based on a target frequency response curve in step S102 according to an exemplary embodiment.

[0087] Figure 4 This is a flowchart illustrating a method for determining a second sound signal based on a target frequency response curve in step S103 according to an exemplary embodiment.

[0088] Figure 5 This is a flowchart illustrating the method for obtaining the target frequency response curve in step S101 according to an exemplary embodiment;

[0089] Figure 6 This is a flowchart illustrating the method for determining the target frequency response curve based on the initial test signal and the test signal in step S502 according to an exemplary embodiment;

[0090] Figure 7 This is a flowchart illustrating a method for determining a target frequency response curve based on a third sound energy value and a third initial sound energy value in step S603, according to an exemplary embodiment.

[0091] Figure 8 This is a block diagram illustrating an audio processing apparatus according to an exemplary embodiment;

[0092] Figure 9 This is a block diagram illustrating an earphone device according to an exemplary embodiment. Detailed Implementation

[0093] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0094] In an exemplary embodiment of this disclosure, an audio processing method is provided, applied to headphones, including semi-in-ear Bluetooth headphones, semi-in-ear wired headphones, and other headphones that do not have a closed acoustic cavity. Figure 1 This is a flowchart illustrating an audio processing method according to an exemplary embodiment, such as... Figure 1 As shown, the audio processing method includes the following steps:

[0095] Step S101: Obtain the target frequency response curve, which is the frequency response curve received by the ear canal of the wearer of the headphones.

[0096] Step S102: Based on the target frequency response curve, determine the first sound signal; wherein, the first sound signal represents the ambient sound that is heard by the wearer after penetrating the headphones;

[0097] Step S103: Determine the second sound signal based on the target frequency response curve; wherein the second sound signal represents the sound played by the speaker heard by the wearer;

[0098] Step S104: Determine the first sound energy value based on the first sound signal, wherein the first sound energy value is the energy of each frequency point contained in the first sound signal;

[0099] Step S105: Determine the second sound energy value based on the second sound signal, wherein the second sound energy value is the energy of each frequency point contained in the second sound signal;

[0100] Step S106: Determine the filter parameters to be updated based on the first sound energy value and the second sound energy value;

[0101] Step S107: Adjust the parameters of the preset filter based on the filter parameters to be updated, and use the adjusted filter to process the sound signal played by the speaker.

[0102] In an exemplary embodiment of this disclosure, an audio processing method is provided to achieve noise reduction effect in headphones. When a user wears headphones, a target frequency response curve is acquired. This target frequency response curve is the frequency response curve received by the wearer's ear canal. Based on the target frequency response curve, a first sound signal and a second sound signal are determined. The first sound signal represents the ambient sound that penetrates the headphones and is heard by the wearer, while the second sound signal represents the sound played by the speaker that the wearer hears. A first sound energy value is determined based on the first sound signal, where the first sound energy value is the energy of each frequency point contained in the first sound signal. A second sound energy value is determined based on the second sound signal, where the second sound energy value is the energy of each frequency point contained in the second sound signal. Based on the first and second sound energy values, filter parameters to be updated are determined. Based on the filter parameters to be updated, the preset filter parameters are adjusted, and the sound signal played by the speaker is processed using the adjusted filter. After processing, the second sound energy value is made greater than the first sound energy value. According to the psychoacoustic masking effect, the sound signal played by the speaker heard by the wearer at this time will mask the ambient sound that penetrates the headphones and is heard by the wearer, thereby achieving the effect of subjectively not hearing the ambient sound signal and improving the user's listening experience.

[0103] The audio processing method disclosed herein can be used as the default noise cancellation mode for headphones, which is automatically triggered every time the user wears the headphones without requiring the user to manually turn it on; or it can be used as an alternative noise cancellation mode, which is displayed on the headphone control interface corresponding to the headphones every time the user wears the headphones, requiring the user to manually select to turn it on, and then the audio processing method disclosed herein is executed after the noise cancellation mode is turned on.

[0104] In step S101, Figure 2 This is a schematic diagram of the structure of an earphone according to an exemplary embodiment, such as... Figure 2 As shown, the acoustic components of the headphones mainly include: a feedforward microphone 201, a feedback microphone 202, and a speaker 203. The feedforward microphone 201 is placed outside the ear after wearing the headphones and is used to monitor ambient sound signals around the headphones; the feedback microphone 202 is placed inside the ear canal after wearing the headphones and near the speaker, and is used to monitor sound signals inside the ear canal in real time; the speaker 203 is connected to an audio playback source and is used to play sound signals from the audio playback source.

[0105] Because different users have varying abilities to perceive sound signals through their ear canals, and because the tightness of the headphones varies from user to user, the sound signals heard through the headphones will also differ. Therefore, a target frequency response curve needs to be determined each time a user wears headphones. The target frequency response curve is determined by factors such as ear canal information and the tightness of the fit that cause different auditory experiences. The target frequency response curve characterizes the frequency response curve received by the wearer's ear canal under the current wearing condition; that is, the mapping relationship between the actual sound signal played and the sound signal heard by the user.

[0106] In step S102, when the user wears the headphones, because the semi-in-ear headphones are not a closed acoustic cavity and different users have different ear canal information, part of the ambient sound will penetrate the headphones and enter the wearer's ear canal and be heard by the user, while part will remain outside the wearer's ear and the user cannot hear the ambient sound outside the ear. Based on the target frequency response curve, the current wearing state of the user can be determined, and the sound signal of the ambient sound that penetrates the headphones and is heard by the user is the first sound signal.

[0107] In step S103, when the user wears the headphones, if an electronic device connected to the headphones plays an audio signal, the audio source can be an application on the electronic device that has the function of playing audio signals, such as music software or video software. The audio signal from the audio source will be played through the headphone's speaker. Since semi-in-ear headphones are not closed acoustic chambers and different users have different ear canal information, part of the sound signal played by the speaker will be transmitted into the wearer's ear canal and heard by the wearer, while part will diffuse outside the wearer's ear canal. The part of the speaker sound signal diffused outside the ear canal cannot be heard by the user. Based on the target frequency response curve, the current wearing state of the user can be determined, and the sound signal played by the speaker heard by the wearer is the second sound signal.

[0108] In steps S104 and S105, both the first and second sound signals are frequency domain signals. The energy of each frequency point contained in the first and second sound signals is calculated, resulting in the first sound energy value and the second sound energy value. The sound energy value can be calculated using any method that reflects the energy of the sound signal at each frequency point. The first sound energy value is the energy of each frequency point contained in the first sound signal; therefore, the first sound energy value reflects the energy of ambient sound penetrating the headphones and heard by the wearer at each frequency point. The second sound energy value is the energy of each frequency point contained in the second sound signal; therefore, the second sound energy value reflects the energy of the sound played by the speaker heard by the wearer at each frequency point.

[0109] In steps S106 and S107, based on the numerical relationship between the first sound energy value and the second sound energy value, the relationship between the ambient sound heard by the wearer and the sound played by the speaker at each frequency point can be determined. When the first sound energy value is less than the second sound energy value at the same frequency point, it indicates that the sound played by the speaker heard by the wearer is greater than the ambient sound heard. According to the psychoacoustic masking effect, at this frequency point, the sound played by the speaker heard by the wearer can mask the ambient sound heard, that is, the wearer cannot subjectively hear the ambient sound. Therefore, there is no need to update the filter coefficient at this frequency point, and no need to process the sound signal played by the speaker at this frequency point. When the first sound energy value is greater than or equal to the second sound energy value at the same frequency, it means that the ambient sound heard by the wearer is greater than or equal to the sound played by the speaker. According to the psychoacoustic masking effect, at this frequency, the ambient sound heard by the wearer can mask the sound played by the speaker, meaning the wearer cannot subjectively hear the sound played by the speaker. Therefore, the filter parameters to be updated are determined, and the preset filter is adjusted based on these parameters. The adjusted filter is then used to process the sound signal played by the speaker to increase the energy value of the sound played by the speaker. The preset filter is a filter that increases the gain of the sound played by the speaker. After adjusting the parameters of the preset filter, the adjusted filter is used to process the sound signal played by the speaker, enabling the wearer to hear the sound played by the speaker.

[0110] In an exemplary embodiment of this disclosure, when a user wears headphones, a target frequency response curve is acquired. Based on the target frequency response curve, a first sound signal of ambient sound penetrating the headphones and heard by the wearer, and a second sound signal played by the speaker and heard by the wearer are determined. A first sound energy value is determined based on the first sound signal, and a second sound energy value is determined based on the second sound signal. Based on the first and second sound energy values, filter parameters to be updated are determined. Based on the filter parameters to be updated, a preset filter is adjusted, and the adjusted filter is used to process the sound signal played by the speaker. After processing the sound signal played by the speaker, the second sound energy value is made greater than the first sound energy value. According to the psychoacoustic masking effect, the sound signal played by the speaker and heard by the wearer at this time will mask the ambient sound penetrating the headphones and heard by the wearer, thereby achieving the effect of subjectively not hearing the ambient sound signal and improving the user's listening experience.

[0111] In one exemplary embodiment, Figure 3 This is a flowchart illustrating a method for determining a first sound signal based on a target frequency response curve in step S102, according to an exemplary embodiment, as shown below. Figure 3 As shown, it includes the following steps:

[0112] Step S301: Acquire ambient sound signals around the headphones;

[0113] Step S302: Determine the first sound signal based on the ambient sound signal and the pre-stored first mapping relationship; wherein, the first mapping relationship is used to represent the mapping relationship between the target frequency response curve and the first sound transmission path, and the first sound transmission path represents the sound transmission path from the headphone's feedforward microphone to the ear canal.

[0114] In step S301, the ambient sound signal around the headphones can be acquired through the feedforward microphone in the headphones. When acquiring the ambient sound signal, based on a preset sampling duration, the ambient time-domain signal is acquired through the feedforward microphone to obtain the time-domain representation of the ambient sound. The preset sampling duration can be set according to actual needs, for example, 8 seconds. After acquiring the ambient time-domain signal, a time-frequency transformation of the ambient sound signal is performed using a Fourier function. The ambient sound signal is the frequency-domain representation of the ambient sound and includes multiple frequency points.

[0115] For example, if the environmental time-domain signal is denoted as s1(m,n) and the environmental sound signal is denoted as S1(m,h), then:

[0116]

[0117] Where S1(m,h) represents the frequency domain signal of the h-th frequency point corresponding to the m-th frame in the time domain, m is the time domain signal of the m-th frame, and h is the frequency domain signal of the h-th frame in the time domain. For Fourier transform operations, h is the parameter of the Fourier transform. After performing the Fourier transform, the spectrum of h frequency points in the frequency domain is obtained. The value of h can be set according to actual needs. For example, when generating frequency points based on each signal sampling point in the time domain, h is the number of signal sampling points.

[0118] In step S302, the first sound transmission path represents the sound transmission path from the headphone's feedforward microphone to the ear canal. Since the feedforward microphone can acquire ambient sound signals from around the headphone, the mapping relationship of the first sound transmission path represents the mapping relationship of the sound propagation path of the ambient sound signal into the ear canal. The target frequency response curve reflects the proportion of the ambient sound signal heard by the wearer through the headphone in the current wearing state. The first mapping relationship, i.e., the mapping relationship between the target frequency response curve and the first sound transmission path at different frequency points, is pre-stored in the headphone. After obtaining the target frequency response curve, the mapping relationship of the first sound transmission path can be obtained according to the first mapping relationship. Based on the mapping relationship between the ambient sound signal and the first sound transmission path, the first sound signal can be determined.

[0119] In one example, the ambient sound signal at frequency k is denoted as A. kThe mapping relationship of the first sound transmission path is denoted as F. 1，k At that time, the first sound signal B k For: B k =A k *F 1，k .

[0120] In one exemplary embodiment, Figure 4 This is a flowchart illustrating a method for determining a second sound signal based on a target frequency response curve in step S103, according to an exemplary embodiment, as shown below. Figure 4 As shown, it includes the following steps:

[0121] Step S401: Obtain the speaker sound signal played by the headphone speaker.

[0122] Step S402: Determine the second sound signal based on the speaker sound signal and the pre-stored second mapping relationship; wherein, the second mapping relationship is used to represent the mapping relationship between the target frequency response curve and the second sound transmission path, and the second sound transmission path represents the sound transmission path from the speaker of the earphone to the ear canal.

[0123] In step S401, the electrical signal acquisition module in the headphones acquires the speaker sound signal played by the audio playback source. When acquiring the speaker sound signal, the electrical signal acquisition module obtains the speaker time-domain signal based on the same sampling duration as the ambient time-domain signal, thus obtaining the time-domain representation of the sound played by the speaker. After acquiring the speaker time-domain signal, a time-frequency transformation is performed using a Fourier function. The speaker sound signal is the frequency-domain representation of the sound played by the speaker, and it includes multiple frequency points.

[0124] For example, if the loudspeaker's time-domain signal is denoted as s2(m,n) and the loudspeaker's sound signal is denoted as S2(m,h), then:

[0125]

[0126] Where S2(m,h) represents the frequency domain signal of the h-th frequency point corresponding to the m-th frame in the time domain, m is the time domain signal of the m-th frame, and h is the frequency domain signal of the h-th frame in the time domain. For Fourier transform operations, h is the parameter of the Fourier transform. After performing the Fourier transform, the spectrum of h frequency points in the frequency domain is obtained. The value of h can be set according to actual needs. For example, when generating frequency points based on each signal sampling point in the time domain, h is the number of signal sampling points.

[0127] In step S402, the second sound transmission path represents the sound transmission path from the headphone's speaker to the ear canal. Since the speaker's sound signal can be obtained through the speaker, the mapping relationship of the second sound transmission path represents the mapping relationship of the sound propagation path of the speaker's sound signal into the ear canal. The target frequency response curve reflects the proportion of the speaker's sound signal heard by the wearer in the current wearing state. The second mapping relationship, i.e., the mapping relationship between the target frequency response curve and the second sound transmission path at different frequency points, is pre-stored in the headphones. After obtaining the target frequency response curve, the mapping relationship of the second sound transmission path can be obtained according to the second mapping relationship. Based on the mapping relationship between the speaker's sound signal and the second sound transmission path, the second sound signal can be determined.

[0128] In one example, the loudspeaker sound signal at frequency k is denoted as C. k The mapping relationship of the second sound transmission path is denoted as F. 2，k At that time, the second sound signal D k For: D k =C k *F 2，k .

[0129] Understandably, when headphones are worn loosely, more of the sound signal from the headphone speaker diffuses out of the ear, resulting in a smaller second sound signal heard by the wearer. Therefore, the mapping relationship of the second sound transmission path is weaker. Conversely, more ambient sound signal enters the ear canal, resulting in a larger first sound signal heard by the wearer. Therefore, the mapping relationship of the first sound transmission path is stronger. Thus, the mapping relationship of the first and second sound transmission paths is inversely related.

[0130] It should be noted that the first and second mapping relationships pre-stored in the headphones in steps S302 and S402 are obtained in advance through experimental data. The headphones are worn on an artificial head in the laboratory, and the target frequency response curve under the current wearing state is determined. A microphone is placed in the artificial head, and the first and second sound signals are acquired through the microphone in the artificial head. The speaker sound signal is acquired through the headphone's speaker, and the ambient sound signal is acquired through the headphone's feedforward microphone. The proportional relationship between the ambient sound signal and the first sound signal at different frequency points is calculated to determine the mapping relationship of the first sound transmission path, and the first mapping relationship is formed based on the correspondence between the mapping relationship of the first sound transmission path and the target frequency response curve. The proportional relationship between the speaker sound signal and the second sound signal at different frequency points is calculated to determine the mapping relationship of the second sound transmission path, and the second mapping relationship is formed based on the correspondence between the mapping relationship of the second sound transmission path and the target frequency response curve. It can be understood that the first and second mapping relationships can also be combined into one mapping relationship, that is, the mapping relationship of the first sound transmission path corresponding to the target frequency response curve at different frequency points and the mapping relationship of the first sound transmission path.

[0131] In one exemplary embodiment, Figure 5 This is a flowchart illustrating the method for obtaining the target frequency response curve in step S101 according to an exemplary embodiment, as shown below. Figure 5 As shown, it includes the following steps:

[0132] Step S501: Acquire a test signal, wherein the test signal is the sound signal collected by the feedback microphone after the initial test signal emitted by the speaker propagates in the wearer's ear canal;

[0133] Step S502: Determine the target frequency response curve based on the initial test signal and the test signal, wherein the target frequency response curve is used to characterize the tightness of the user's wearing and the user's ear canal information.

[0134] The initial test signal is the headphone's wearing prompt tone, which plays every time the user wears the headphones. The headphone's electrical signal acquisition module acquires the time-domain sound signal generated by the prompt tone played by the speaker, and performs a time-frequency transformation to form the initial test signal. The sampling duration of the initial test signal is the same as the duration of the wearing prompt tone. The test signal inside the wearer's ear canal is acquired through the headphone's feedback microphone, including the initial test signal played by the speaker and noise signals entering from outside the ear. When acquiring the test signal inside the wearer's ear canal, based on the same sampling duration as the initial test signal, the feedback microphone acquires the time-domain signal inside the wearer's ear canal, and performs a time-frequency transformation to form the test signal.

[0135] For example, if the test signal inside the wearer's ear canal is denoted as s3(m,n) in the time domain and S3(m,h) in the frequency domain, and the initial test signal is denoted as s4(m,n) in the time domain and S4(m,h) in the frequency domain, then:

[0136]

[0137]

[0138] Where m is the time-domain signal of the m-th frame in the time domain, and the... For Fourier transform operations, h is the parameter of the Fourier transform. After the Fourier transform, the spectrum of h frequency points in the frequency domain is obtained. The value of h can be set according to actual needs. For example, when generating frequency points based on each signal sampling point in the time domain, h is the number of signal sampling points. S3(m,h) and S4(m,h) represent the frequency domain signal of the h-th frequency point corresponding to the m-th frame in the time domain.

[0139] Since the test signal is related to the tightness of the user's earphones and the user's ear canal information, the target frequency response curve determined based on the initial test signal and the test signal can reflect the transmission of speaker sound signals and ambient sound signals under the current user's current wearing condition.

[0140] In one exemplary embodiment, Figure 6 This is a flowchart illustrating the method for determining the target frequency response curve based on the initial test signal and the test signal in step S502 according to an exemplary embodiment. Figure 6 As shown, it includes the following steps:

[0141] Step S601: Determine the third initial sound energy value based on the initial test signal. The third initial sound energy value is the energy of each frequency point contained in the initial test signal.

[0142] Step S602: Determine the third sound energy value based on the test signal. The third sound energy value is the energy of each frequency point contained in the test signal.

[0143] Step S603: Determine the target frequency response curve based on the third sound energy value and the third initial sound energy value.

[0144] Both the initial test signal and the test signal are sound signals in the frequency domain, each containing multiple frequency points. The energy of each frequency point in the initial test signal is calculated to obtain the third initial sound energy value, and the energy of each frequency point in the test signal is calculated to obtain the third sound energy value. The sound energy value at each frequency point can be calculated using any method that reflects sound energy.

[0145] In one exemplary embodiment, Figure 7This is a flowchart illustrating a method for determining a target frequency response curve based on a third sound energy value and a third initial sound energy value in step S603, according to an exemplary embodiment. Figure 7 As shown, it includes the following steps:

[0146] Step S701: Determine the first reference parameter based on the third sound energy value and the third initial sound energy value. The first reference parameter is the cross power spectrum of the third sound energy value and the third initial sound energy value at each frequency point.

[0147] Step S702: Determine the second reference parameter based on the third initial sound energy value. The second reference parameter is the autopower spectrum of the third initial sound energy value at each frequency point.

[0148] Step S703: Determine the target frequency response curve based on the first reference parameter and the second reference parameter.

[0149] In step S701, the cross power spectrum can represent the correlation between two sound signals. Therefore, the first reference parameter can be the cross power spectrum of the third sound energy value and the third initial sound energy value. The first reference parameter is determined by calculating the cross power spectrum of the third sound energy value and the third initial sound energy value at each frequency point. Specifically, the cross power spectrum of the third sound energy value and the third initial sound energy value is the product of the third sound energy value and the third initial sound energy value at the same frequency point.

[0150] For example, let the third sound energy value at the k-th frequency be denoted as A(k), the third initial sound energy value at the k-th frequency be denoted as B(k), and the first reference parameter be denoted as G1. When the first reference parameter is the cross-power spectrum of the third sound energy value and the third initial sound energy value, then:

[0151] G1 = A(k) * B(k)

[0152] In step S702, the autopower spectrum can represent the autocorrelation of the sound signal. Therefore, the second reference parameter can be the autopower spectrum of the third initial sound energy value. The second reference parameter is determined by calculating the autopower spectrum of the third initial sound energy value at each frequency point. The autopower spectrum of the third initial sound energy value is the square of the third initial sound energy value at each frequency point.

[0153] For example, let A(k) denote the third initial sound energy value at the k-th frequency point, and let G2 denote the second reference parameter. When the second reference parameter is the power spectrum of the third initial sound energy value, then:

[0154] G2=A(k)*A(k)

[0155] In step S703, after obtaining the first reference parameter and the second reference parameter, the target frequency response curve is determined based on the first and second reference parameters. The target frequency response curve represents parameters related to headphone wearing. With the same initial test signal, the test signal heard by the user differs under different wearing conditions. Since the first reference parameter reflects the correlation between the initial test signal and the test signal within the ear canal, and the second reference parameter reflects the autocorrelation of the initial test signal, when determining the target frequency response curve, the ratio of the first reference parameter to the second reference parameter at the same frequency point can be used as the frequency response parameter of that frequency point on the target frequency response curve. The target frequency response curve is formed by the frequency response parameters of each frequency point. For example, at frequency point k, when the first reference parameter is G1 and the second reference parameter is G2, the frequency response parameter H on the target frequency response curve at that frequency point is... k for: Connect the frequency response parameters of each frequency point to form the target frequency response curve.

[0156] Since each user's ear canal information and wearing method are different, the frequency response of the acoustic path will also be different, resulting in different users hearing the same initial test signal differently. In this embodiment, after each user wears the headphones, the target frequency response curve is obtained in real time and accurately through the initial test signal to characterize the frequency response curve of the acoustic path and reflect the mapping relationship between the initial test signal and the test signal actually heard by the human ear in the current wearing state.

[0157] In an exemplary embodiment, step S106, which determines the filter parameter to be updated based on the first sound energy value and the second sound energy value, includes: determining the filter parameter to be updated when the first sound energy value is greater than the second sound energy value.

[0158] After determining the first and second sound energy values, the filter parameters to be updated are determined based on the numerical relationship between the first and second sound energy values ​​at corresponding frequency points. At frequency point k, the first sound energy value is denoted as S. k The energy value of the second sound is denoted as N. k When S k ≥N kAt this frequency point, the ambient sound passing through the headphones and heard by the wearer can mask the sound played by the speaker. In this case, the ambient sound heard by the user interferes with the speaker sound, making it impossible to hear the speaker clearly. Therefore, the filter coefficients to be updated are determined. Based on the filter parameters to be updated, the preset filter parameters are adjusted, and the adjusted filter is used to process the sound signal played by the speaker to make the first sound energy value less than the second sound energy value. The filter coefficients to be updated can be the gain of the sound signal or the volume of the sound signal. At frequency point k, the first sound energy value is denoted as S. k The energy value of the second sound is denoted as N. k When S k <N k At this frequency, the sound played by the speaker can mask ambient sounds that penetrate the headphones and are heard by the wearer. At this time, the user can hear the speaker sound clearly, so there is no need to adjust the filter coefficient.

[0159] In an exemplary embodiment of this disclosure, when a user wears headphones, the mapping relationship between the initial sound signal and the sound signal actually heard by the human ear in the current wearing state is obtained by determining the target frequency response curve. Based on feature parameters, the first sound energy information corresponding to the sound signal in the wearer's ear canal and the second sound energy information corresponding to the noise signal in the wearer's ear canal are determined. According to the numerical relationship between the first and second sound energy information, when it is determined that the sound signal in the ear canal will be masked by the noise signal in the ear canal, the initial sound signal is adjusted so that the first sound energy information is greater than the second sound energy information. The method in this disclosure has low requirements for headphone hardware structure and does not require complex noise reduction algorithm design. It utilizes the psychoacoustic masking effect to allow the sound signal in the user's ear canal to mask the noise signal in the ear canal, thereby achieving the effect that the wearer cannot subjectively hear the noise signal and improving the user's listening experience.

[0160] In an exemplary embodiment of this disclosure, an audio processing apparatus is provided for use with headphones. Figure 8 This is a block diagram illustrating an audio processing apparatus according to an exemplary embodiment, such as... Figure 8 As shown, the processing device includes:

[0161] The acquisition module 801 is configured to acquire a target frequency response curve, wherein the target frequency response curve is the frequency response curve received by the ear canal of the wearer of the headphones;

[0162] The first determining module 802 is configured to determine a first sound signal based on the target frequency response curve; wherein the first sound signal represents the ambient sound that penetrates the headphones and is heard by the wearer;

[0163] The second determining module 803 is configured to determine a second sound signal based on the target frequency response curve; wherein the second sound signal represents the sound played by the speaker heard by the wearer;

[0164] The third determining module 804 is configured to determine a first sound energy value based on the first sound signal, wherein the first sound energy value is the energy of each frequency point contained in the first sound signal;

[0165] The fourth determining module 805 is configured to determine a second sound energy value based on the second sound signal, wherein the second sound energy value is the energy of each frequency point contained in the second sound signal;

[0166] The fifth determining module 806 is configured to determine the filter parameters to be updated based on the first sound energy value and the second sound energy value;

[0167] The processing module 807 is configured to adjust the parameters of a preset filter based on the filter parameters to be updated, and to process the sound signal played by the speaker using the adjusted filter.

[0168] In an exemplary embodiment, the first determining module 802 is further configured to:

[0169] Acquire ambient sound signals around the headphones;

[0170] The first sound signal is determined based on the ambient sound signal and the pre-stored first mapping relationship; wherein, the first mapping relationship is used to represent the mapping relationship between the target frequency response curve and the first sound transmission path, and the first sound transmission path represents the sound transmission path from the feedforward microphone of the earphone to the ear canal.

[0171] In one exemplary embodiment, the second determining module 803 is further configured to:

[0172] Obtain the speaker sound signal played by the headphone's speaker.

[0173] A second sound signal is determined based on the speaker sound signal and a pre-stored second mapping relationship; wherein, the second mapping relationship is used to represent the mapping relationship between the target frequency response curve and the second sound transmission path, and the second sound transmission path represents the sound transmission path from the speaker of the earphone to the ear canal.

[0174] In one exemplary embodiment, the acquisition module 801 is further configured to:

[0175] Acquire a test signal, wherein the test signal is the sound signal collected by the feedback microphone after the initial test signal emitted by the speaker propagates in the wearer's ear canal;

[0176] The target frequency response curve is determined based on the initial test signal and the test signal, wherein the target frequency response curve is used to characterize the tightness of the user's wearing and the user's ear canal information.

[0177] In one exemplary embodiment, the acquisition module 801 is further configured to:

[0178] A third initial sound energy value is determined based on the initial test signal, wherein the third initial sound energy value is the energy of each frequency point contained in the initial test signal;

[0179] A third sound energy value is determined based on the test signal, wherein the third sound energy value is the energy of each frequency point contained in the test signal;

[0180] The target frequency response curve is determined based on the third sound energy value and the third initial sound energy value.

[0181] In one exemplary embodiment, the acquisition module 801 is further configured to:

[0182] Based on the third sound energy value and the third initial sound energy value, a first reference parameter is determined, wherein the first reference parameter is the cross power spectrum of the third sound energy value and the third initial sound energy value at each frequency point;

[0183] Based on the third initial sound energy value, a second reference parameter is determined, wherein the second reference parameter is the autopower spectrum of the third initial sound energy value at each frequency point;

[0184] The target frequency response curve is determined based on the first reference parameter and the second reference parameter.

[0185] In one exemplary embodiment, the acquisition module 801 is further configured to:

[0186] The ratio of the first reference parameter to the second reference parameter at each frequency point is used as the frequency response parameter of the target frequency response curve, and the target frequency response curve is formed.

[0187] In one exemplary embodiment, the fifth determining module 806 is further configured to:

[0188] When the first sound energy value is less than or equal to the second sound energy value, the filter parameters to be updated are determined.

[0189] In an exemplary embodiment, the first determining module 802 is further configured to:

[0190] Acquire environmental time-domain signals, which are the representation of environmental sound in the time domain;

[0191] The environmental time-domain signal is transformed by time-frequency transformation to obtain an environmental sound signal, wherein the environmental sound signal is the frequency domain representation of the environmental sound and includes multiple frequency points.

[0192] In one exemplary embodiment, the second determining module 802 is further configured to

[0193] Acquire the speaker time-domain signal, which is the way the speaker sound is expressed in the time domain;

[0194] The speaker's time-domain signal is transformed by time-frequency conversion to obtain the speaker's sound signal, wherein the speaker's sound signal is the expression of the speaker's sound in the frequency domain and includes multiple frequency points.

[0195] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0196] In an exemplary embodiment of this disclosure, an earphone is provided, the earphone including a housing and a feedforward microphone, a feedback microphone, a speaker and a controller disposed on the housing, the controller being communicatively connected to the feedforward microphone, the feedback microphone and the speaker respectively;

[0197] The feedforward microphone is used to collect ambient sound signals around the headphones;

[0198] The feedback microphone is used to collect sound signals from inside the ear canal of the earphone wearer;

[0199] The speaker is used to play speaker sound signals and initial test signals;

[0200] The controller includes a processor and a memory, the memory storing computer program instructions executable by the processor, the processor being configured to invoke the computer program instructions to perform an audio processing method as described in this disclosure.

[0201] Figure 9 This is a block diagram illustrating an earphone device 900 according to an exemplary embodiment.

[0202] Reference Figure 9The device 900 may include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.

[0203] Processing component 902 typically controls the overall operation of device 900, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 902 may include one or more processors 920 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 902 may include one or more modules to facilitate interaction between processing component 902 and other components. For example, processing component 902 may include a multimedia module to facilitate interaction between multimedia component 908 and processing component 902.

[0204] Memory 904 is configured to store various types of data to support the operation of device 900. Examples of this data include instructions for any application or method operating on device 900, contact data, phonebook data, messages, pictures, videos, etc. Memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0205] Power supply component 906 provides power to various components of device 900. Power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 900.

[0206] Multimedia component 908 includes a screen that provides an output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 908 includes a front-facing camera and / or a rear-facing camera. When the device 900 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0207] Audio component 910 is configured to output and / or input audio signals. For example, audio component 910 includes a microphone (MIC) configured to receive external audio signals when device 900 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 904 or transmitted via communication component 916. In some embodiments, audio component 910 also includes a speaker for outputting audio signals.

[0208] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0209] Sensor assembly 914 includes one or more sensors for providing status assessments of various aspects of device 900. For example, sensor assembly 914 may detect the on / off state of device 900, the relative positioning of components such as the display and keypad of device 900, changes in position of device 900 or a component of device 900, the presence or absence of user contact with device 900, orientation or acceleration / deceleration of device 900, and temperature changes of device 900. Sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 914 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0210] Communication component 916 is configured to facilitate wired or wireless communication between device 900 and other devices. Device 900 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 916 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0211] In an exemplary embodiment, the apparatus 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0212] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, which can be executed by a processor 920 of the device 900 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0213] A non-transitory computer-readable storage medium having stored thereon computer program instructions that, when invoked by a processor, execute an audio processing method as described in this disclosure.

[0214] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0215] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An audio processing method, characterized in that, Applied to headphones, the processing method includes: Obtain a target frequency response curve, which is the frequency response curve received by the ear canal of the wearer of the headphones, and the headphones include a feedforward microphone and a feedback microphone disposed on the housing; Based on the target frequency response curve, a first sound signal is determined; wherein, the first sound signal represents the ambient sound that penetrates the headphones and is heard by the wearer; Based on the target frequency response curve, a second sound signal is determined; wherein the second sound signal represents the sound played by the speaker heard by the wearer; A first sound energy value is determined based on the first sound signal, wherein the first sound energy value is the energy of each frequency point contained in the first sound signal; A second sound energy value is determined based on the second sound signal, wherein the second sound energy value is the energy of each frequency point contained in the second sound signal; Based on the first sound energy value and the second sound energy value, determine the filter parameters to be updated; The preset filter is adjusted based on the filter parameters to be updated, and the sound signal played by the speaker is processed using the adjusted filter. The step of determining the first sound signal based on the target frequency response curve includes: The ambient sound signal around the headphones is acquired through the feedforward microphone, and the ambient sound signal includes multiple frequency points; The first sound signal is determined based on the ambient sound signal and the pre-stored first mapping relationship; wherein, the first mapping relationship is used to represent the mapping relationship between the target frequency response curve at different frequency points and the first sound transmission path, and the first sound transmission path represents the sound transmission path from the feedforward microphone of the earphone to the ear canal.

2. The audio processing method according to claim 1, characterized in that, Determining the second sound signal based on the target frequency response curve includes: Obtain the speaker sound signal played by the headphone's speaker. A second sound signal is determined based on the speaker sound signal and a pre-stored second mapping relationship; wherein, the second mapping relationship is used to represent the mapping relationship between the target frequency response curve and the second sound transmission path, and the second sound transmission path represents the sound transmission path from the speaker of the earphone to the ear canal.

3. The audio processing method according to claim 1, characterized in that, The acquisition of the target frequency response curve includes: Acquire a test signal, wherein the test signal is the sound signal collected by the feedback microphone after the initial test signal emitted by the speaker propagates in the wearer's ear canal; The target frequency response curve is determined based on the initial test signal and the test signal, wherein the target frequency response curve is used to characterize the tightness of the user's wearing and the user's ear canal information.

4. The audio processing method according to claim 3, characterized in that, The step of determining the target frequency response curve based on the initial test signal and the test signal includes: A third initial sound energy value is determined based on the initial test signal, wherein the third initial sound energy value is the energy of each frequency point contained in the initial test signal; A third sound energy value is determined based on the test signal, wherein the third sound energy value is the energy of each frequency point contained in the test signal; The target frequency response curve is determined based on the third sound energy value and the third initial sound energy value.

5. The audio processing method according to claim 4, characterized in that, Determining the target frequency response curve based on the third sound energy value and the third initial sound energy value includes: Based on the third sound energy value and the third initial sound energy value, a first reference parameter is determined, wherein the first reference parameter is the cross power spectrum of the third sound energy value and the third initial sound energy value at each frequency point; Based on the third initial sound energy value, a second reference parameter is determined, wherein the second reference parameter is the autopower spectrum of the third initial sound energy value at each frequency point; The target frequency response curve is determined based on the first reference parameter and the second reference parameter.

6. The audio processing method according to claim 5, characterized in that, Determining the target frequency response curve based on the first reference parameter and the second reference parameter includes: The ratio of the first reference parameter to the second reference parameter at each frequency point is used as the frequency response parameter of the target frequency response curve, and the target frequency response curve is formed.

7. The audio processing method according to claim 1, characterized in that, The step of determining the filter parameters to be updated based on the first sound energy value and the second sound energy value includes: When the first sound energy value is greater than or equal to the second sound energy value, the filter parameters to be updated are determined.

8. The audio processing method according to claim 1, characterized in that, The acquisition of ambient sound signals around the headphones includes: Acquire environmental time-domain signals, which are the representation of environmental sound in the time domain; The environmental time-domain signal is subjected to time-frequency transformation to obtain an environmental sound signal, wherein the environmental sound signal is the frequency domain representation of the environmental sound.

9. The audio processing method according to claim 2, characterized in that, The acquisition of the speaker sound signal played by the headphone's speaker includes: Acquire the speaker time-domain signal, which is the way the speaker sound is expressed in the time domain; The speaker's time-domain signal is transformed by time-frequency conversion to obtain the speaker's sound signal, wherein the speaker's sound signal is the expression of the speaker's sound in the frequency domain and includes multiple frequency points.

10. An audio processing apparatus, characterized in that, Applied to headphones, the processing device includes: The acquisition module is configured to acquire a target frequency response curve, which is the frequency response curve received by the ear canal of the wearer of the headphones. The headphones include a feedforward microphone and a feedback microphone disposed on the housing. The first determining module is configured to determine a first sound signal based on the target frequency response curve; wherein the first sound signal represents the ambient sound that penetrates the headphones and is heard by the wearer; The second determining module is configured to determine a second sound signal based on the target frequency response curve; wherein the second sound signal represents the sound played by the speaker heard by the wearer; The third determining module is configured to determine a first sound energy value based on the first sound signal, wherein the first sound energy value is the energy of each frequency point contained in the first sound signal; The fourth determining module is configured to determine a second sound energy value based on the second sound signal, wherein the second sound energy value is the energy of each frequency point contained in the second sound signal; The fifth determining module is configured to determine the filter parameters to be updated based on the first sound energy value and the second sound energy value; The processing module is configured to adjust the parameters of a preset filter based on the filter parameters to be updated, and to process the sound signal played by the speaker using the adjusted filter. The first determining module is further configured to acquire ambient sound signals around the headphones via the feedforward microphone, the ambient sound signals including multiple frequency points; The first sound signal is determined based on the ambient sound signal and the pre-stored first mapping relationship; wherein, the first mapping relationship is used to represent the mapping relationship between the target frequency response curve at different frequency points and the first sound transmission path, and the first sound transmission path represents the sound transmission path from the feedforward microphone of the earphone to the ear canal.

11. An earphone, characterized in that, The earphone includes a housing and a feedforward microphone, a feedback microphone, a speaker, and a controller disposed on the housing. The controller is communicatively connected to the feedforward microphone, the feedback microphone, and the speaker, respectively. The feedforward microphone is used to collect ambient sound signals around the headphones; The feedback microphone is used to collect sound signals from inside the ear canal of the earphone wearer; The speaker is used to play speaker sound signals and initial test signals; The controller includes a processor and a memory, the memory storing computer program instructions executable by the processor, the processor being configured to invoke the computer program instructions to perform the method as described in any one of claims 1-9.

12. A non-transitory computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are invoked by the processor, the method described in any one of claims 1-9 is executed.

Images