Audio simulation method, device, electronic device and storage medium

Abstract

The present application discloses an audio simulation method, device, electronic device and storage medium, the method comprising: obtaining at least two filters, wherein different filters represent different influencing factors on the audio signal collected by the feedback microphone of the headset when wearing the headset; obtaining the original audio signal; using at least two filters to perform at least two filtering processes on the original audio signal in sequence, and generating a simulated audio signal collected by the feedback microphone when wearing the headset. In the above manner, the present application can make the simulated audio signal more consistent with the real situation.

Description

Technical Field

[0001] The present application relates to the field of signal processing technology, and in particular to an audio simulation method, device, electronic device and storage medium. Background Art

[0002] Currently, a large amount of training data is required to train the noise reduction model at the headphone feedback microphone. There are generally two approaches to the existing training data collection. One is to use different real people wearing headphones to record a large amount of speech and noise signals, and the other is to use data augmentation methods to simulate a large amount of speech signals. For the former, a large amount of data recording costs are required and the recorded data can hardly be reused, while for the latter, it is impossible to cover the signal changes under different usage conditions of the headphones. Summary of the invention

[0003] The main technical problem solved by the present application is to provide an audio simulation method, device, electronic device and storage medium, which can make the simulated audio signal more consistent with the actual situation.

[0004] To solve the above technical problems, a technical solution adopted in the present application is: to provide an audio simulation method, the method comprising: obtaining at least two filters, wherein different filters represent different influencing factors on the audio signal collected by the feedback microphone of the headset when the headset is worn; obtaining the original audio signal; using at least two filters to perform at least two filtering processes on the original audio signal in sequence to generate a simulated audio signal collected by the feedback microphone when the headset is worn.

[0005] To solve the above technical problems, another technical solution adopted in the present application is: to provide an audio simulation device, which includes: a first acquisition module, used to obtain at least two filters, wherein different filters represent different influencing factors on the audio signal collected by the feedback microphone of the headset when wearing the headset; a first acquisition module, used to obtain the original audio signal; a generation module, used to use at least two filters to perform at least two filtering processes on the original audio signal in sequence, to generate a simulated audio signal collected by the feedback microphone when wearing the headset.

[0006] To solve the above technical problems, another technical solution adopted in the present application is: to provide an electronic device, comprising a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the above audio simulation method.

[0007] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a computer-readable storage medium storing program instructions that can be executed by a processor, and the program instructions are used to implement the above audio simulation method.

[0008] The above scheme obtains at least two filters and the original audio signal, and uses at least two filters to perform at least two filtering processes on the original audio signal in sequence, so as to generate a simulated audio signal collected by the feedback microphone when wearing headphones. Since different filters represent different influencing factors on the audio signal collected by the feedback microphone of the headset when wearing headphones, the simulated audio signal collected by the feedback microphone when wearing headphones can be obtained by filtering the original audio signal at least twice. Compared with the method of using different real people wearing headphones to record a large number of voice and noise signals, the present application can generate a large number of simulated audio signals without real people recording, which greatly reduces the manpower and time costs; and compared with the method of using data augmentation methods to generate simulated audio signals, the simulated audio signal is more realistic by covering the influence of factors such as the headset structure, the human ear structure, or the wearing and use of the headset on the generated simulated audio signal.

[0009] Furthermore, the simulated audio signal collected by the feedback microphone when wearing headphones is used for training the noise reduction model, which can improve the noise reduction effect and generalization ability of the noise reduction model and save a lot of training data recording costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flowchart of an embodiment of an audio simulation method provided by the present application;

[0011] Figure 2 yes Figure 1 The flowchart of step S13 is shown as an embodiment;

[0012] Figure 3 is a schematic diagram of an embodiment of a target frequency response curve provided by the present application;

[0013] Figure 4 is a schematic diagram of another embodiment of the target frequency response curve provided by the present application;

[0014] Figure 5 yes Figure 2 The flowchart of step S21 of an embodiment is shown;

[0015] Figure 6 is a schematic diagram of an embodiment of a disturbance filter provided by the present application;

[0016] Figure 7 is a schematic diagram of an embodiment of the amplitude of a disturbance filter provided by the present application;

[0017] Figure 8 yes Figure 2 The flowchart of step S22 of an embodiment is shown;

[0018] Fig. 9 yes Figure 1The flowchart of another embodiment of step S13 is shown;

[0019] Fig.10 yes Fig. 9 The flowchart of step S91 is shown as an embodiment;

[0020] Fig.11 This application provides a low-shelf filter H LS (ω) and the high shelf filter H HS (ω) Amplitude-frequency response diagram of an embodiment;

[0021] Fig.12 It is a flow chart of an embodiment of filtering a disturbed audio signal to obtain a simulated audio signal provided by the present application;

[0022] Fig.13 It is a schematic diagram of the framework of an embodiment of an audio simulation device provided by the present application;

[0023] Fig.14 It is a schematic diagram of the framework of an embodiment of an electronic device provided by the present application;

[0024] Fig.15 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0025] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0026] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0027] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0028] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of the audio simulation method provided by the present application. It should be noted that if there are substantially the same results, this embodiment does not Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment includes:

[0029] Step S11: Obtain at least two filters.

[0030] The method of this embodiment is used to filter the original audio signal at least twice using at least two filters to obtain a simulated audio signal. The filters described herein represent different factors affecting the audio signal collected by the feedback microphone of the headset when the headset is worn. In other words, the setting of the filter will affect the audio signal collected by the feedback microphone of the headset when the headset is worn, thereby facilitating the subsequent generation of the desired simulated audio signal based on the influence of the filter on the audio signal.

[0031] There is no limitation on the type, model or quality factor of the filter, and it can be set according to actual use needs. For example, the filter is a recursive filter (Infinite Impulse Response, referred to as IIR filter).

[0032] Optionally, the number of filters is not limited and can be set according to actual use needs, for example, 2, 3 or more.

[0033] In one embodiment, when what needs to be generated later is a simulated audio signal collected by the feedback microphone when the human ear wears the earphone, the filter that needs to be obtained can be determined based on whether the simulated audio signal actually generated is a simulated noise signal or a simulated voice signal. In a specific embodiment, the simulated audio signal is a simulated noise signal. Since the noise signal collected at the feedback microphone is affected by factors such as the earphone structure, the ear structure of the person wearing the earphone, or the tightness of the earphone, the filter that needs to be obtained at this time includes at least two of a noise reduction filter, a disturbance filter, and a noise filter. Among them, the noise reduction filter represents the influence of the noise reduction processing of the earphone on the audio signal collected by the feedback microphone, the disturbance filter represents the influence of different ear structures on the audio signal collected by the feedback microphone, and the noise transfer filter represents the influence of the tightness of the earphone on the audio signal collected by the feedback microphone.

[0034] In other specific implementations, the simulated audio signal is a simulated speech signal. Since the speech signal collected by the feedback microphone is affected by factors such as the ear structure of the wearer and the tightness of the earphone, the filters that need to be obtained at this time include a disturbance filter and a speech transfer filter. Among them, the speech transfer filter represents the influence of the tightness of the earphone on the audio signal collected by the feedback microphone.

[0035] Step S12: Acquire the original audio signal.

[0036] The method of this embodiment is used to perform at least two filtering processes on the original audio signal using at least two filters to obtain a simulated audio signal. The original audio signal described herein includes but is not limited to a speech signal (e.g., a natural human speech signal, etc.), a noise signal, etc. The noise signal can be traffic noise (e.g., the sound emitted by a motor vehicle, an airplane, or a train, etc.), industrial noise, construction noise, social life noise (e.g., the sound generated by a television, a washing machine, or social activities), engine noise (e.g., combustion noise or mechanical noise, etc.), etc., which are not specifically limited here.

[0037] In one embodiment, the original audio signal can be obtained from local storage or cloud storage. It is understandable that in other embodiments, the original audio signal can also be collected by a voice collection device, which is not specifically limited here. For example, if the original audio signal is a noise signal, the noise signal in the current environment is collected by a voice collection device.

[0038] Step S13: using at least two filters to perform at least two filtering processes on the original audio signal in sequence, so as to generate a simulated audio signal collected by the feedback microphone when the earphone is worn.

[0039] In this embodiment, at least two filters are used to process the original audio signal at least twice in sequence to generate a simulated audio signal collected by the feedback microphone when wearing headphones. In other words, taking into account the factors affecting the simulated audio signal collected at the feedback microphone, a filter that can have a corresponding impact on the original audio signal is set, so that the original audio signal can be filtered by the filter to obtain the simulated audio signal collected by the feedback microphone. Compared to the method of using different real people wearing headphones to record a large number of voice and noise signals, the present application can generate a large number of simulated audio signals without real people recording, which greatly reduces manpower and time costs; and compared to the method of using data augmentation methods to generate simulated audio signals, it can cover the impact of factors such as headphone structure, human ear structure, or headphone wearing and use conditions on the generated audio signal.

[0040] Furthermore, the simulated audio signal collected by the feedback microphone when wearing headphones is used for training the noise reduction model, which can improve the noise reduction effect and generalization ability of the noise reduction model and save a lot of training data recording costs.

[0041] In one embodiment, if the simulated audio signal is a simulated noise signal, the original audio signal is the original noise signal, and at least two of the noise reduction filter, the disturbance filter, and the noise filter are sequentially used to process the original noise signal at least twice to generate a simulated noise signal collected by the feedback microphone when the headset is worn. In a specific embodiment, since the noise signal collected at the feedback microphone is affected by the headset structure, the ear structure of the wearer, and the tightness of the headset, the noise reduction filter, the disturbance filter, and the noise filter are sequentially used to process the original noise signal three times to generate a simulated noise signal collected by the feedback microphone when the headset is worn.

[0042] In one embodiment, if the simulated audio signal is a simulated speech signal, and the original audio signal is the original speech signal, the original speech signal is processed twice in sequence using a perturbation filter and a speech transfer filter to generate a simulated speech signal collected by a feedback microphone when wearing headphones.

[0043] Since the audio signal collected by the feedback microphone when wearing the headset includes both a voice signal and a noise signal, in one embodiment, the simulated audio signal collected by the feedback microphone when wearing the headset is a simulated mixed audio signal. Specifically, the original audio signal includes an original noise signal and an original voice signal, and the original noise signal and the original voice signal are respectively subjected to at least two filtering processes in sequence on the original audio signal using at least two filters to generate the simulated audio signal collected by the feedback microphone when wearing the headset, and a simulated noise signal corresponding to the original noise signal and a simulated voice signal corresponding to the original voice signal are obtained; then, the simulated noise signal and the simulated voice signal are synthesized to obtain a simulated mixed audio signal collected by the feedback microphone when wearing the headset.

[0044] There is no limitation on the algorithm involved in the synthesis of the simulated noise signal and the simulated speech signal, and it can be specifically set according to actual use needs.

[0045] In the above implementation, at least two filters and the original audio signal are obtained, and the original audio signal is filtered at least twice in sequence using at least two filters to generate a simulated audio signal collected by the feedback microphone when the headset is worn. Since different filters represent different influencing factors on the audio signal collected by the feedback microphone of the headset when the headset is worn, the simulated audio signal collected by the feedback microphone when the headset is worn can be obtained by filtering the original audio signal at least twice. Compared with the method of using different real people wearing headphones to record a large number of voice and noise signals, the present application can generate a large number of simulated audio signals without real people recording, which greatly reduces manpower and time costs; and compared with the method of using data augmentation methods to generate simulated audio signals, the simulated audio signal is more realistic because it covers the influence of factors such as the headset structure, the human ear structure, or the wearing and use of the headset on the generated simulated audio signal.

[0046] Furthermore, the simulated audio signal collected by the feedback microphone when wearing headphones is used for training the noise reduction model, which can improve the noise reduction effect and generalization ability of the noise reduction model and save a lot of training data recording costs.

[0047] See also Figure 2 , Figure 2 yes Figure 1 It should be noted that if there is substantially the same result, this embodiment does not use Figure 2 The process sequence shown is limited. Figure 2 As shown, in this embodiment, the simulated audio signal is a simulated noise signal, and the original audio signal is an original noise signal, specifically including:

[0048] Step S21: using a noise reduction filter to perform noise reduction filtering on the original audio signal to obtain a noise reduction audio signal having a noise reduction gain corresponding to the frequency of the original audio signal.

[0049] In this embodiment, the original audio signal is subjected to noise reduction filtering processing using a noise reduction filter to obtain a noise reduction audio signal having a noise reduction gain corresponding to the frequency of the original audio signal. In other words, by processing the original audio signal using a noise reduction filter, a portion of the noise signal can be filtered out, thereby obtaining a noise reduction audio signal having a noise reduction gain corresponding to the original audio signal. For example, there is a frequency of 5Hz in the original audio signal, and its corresponding gain is 5dB. After the original audio signal is subjected to noise reduction filtering processing using a noise reduction filter, the noise reduction gain corresponding to the frequency of 5Hz is -10dB. Among them, the gain change of the noise reduction audio signal is within the preset gain change range, and the preset gain change range is not limited, and can be specifically set according to actual use needs.

[0050] In one embodiment, if Figure 3and Figure 4 As shown, Figure 3 is a schematic diagram of an embodiment of a target frequency response curve provided by the present application, Figure 4 It is a schematic diagram of another embodiment of the target frequency response curve provided by the present application. The noise reduction filter is determined based on the target frequency response curve. The target frequency response curve is determined by collecting at least one set of test noise signals using the feedback microphone of at least one test headset. The noise reduction gain corresponding to the frequency of the original audio signal represents the gain corresponding to the frequency in the target frequency response curve. The number of test headsets is not limited and can be specifically set according to actual use needs.

[0051] When the user wears the earphones, the physical structure of the earphones can produce a passive noise reduction effect to isolate a portion of the mid- and high-frequency noise; and when the earphones turn on the overall noise reduction function, the low-frequency noise at the human ear can be eliminated, working together with the passive noise reduction. Therefore, in one embodiment, if Figure 5 As shown, Figure 5 yes Figure 2 The flowchart of step S21 of an embodiment shown in the figure needs to first determine whether the noise reduction mode of the earphone is passive noise reduction or overall noise reduction, so as to determine what kind of noise reduction filtering process the noise reduction filter performs on the original audio signal, which specifically includes the following sub-steps:

[0052] Step S211: Based on the noise reduction control parameter, determine whether the noise reduction mode of the noise reduction filter is passive noise reduction or overall noise reduction.

[0053] In this implementation, the noise reduction mode of the noise reduction filter is determined to be passive noise reduction or overall noise reduction according to the noise reduction control parameter. Figure 3 As shown, according to H PNC (ω) Determine that the noise reduction mode of the noise reduction filter is passive noise reduction; Figure 4 As shown, according to H NC (ω) Determine that the noise reduction method of the noise reduction filter is overall noise reduction.

[0054] Step S212: using a noise reduction filter to perform noise reduction filtering on the original audio signal in a determined noise reduction manner to obtain a noise-reduced audio signal.

[0055] In this embodiment, the original audio signal is subjected to noise reduction filtering processing using a noise reduction filter in a determined noise reduction manner, thereby obtaining a noise-reduced audio signal. Specifically, when the noise reduction manner of the noise reduction filter is determined to be passive noise reduction, the original audio signal is subjected to noise reduction filtering processing using the passive noise reduction noise reduction manner to filter out the mid- and high-frequency noise in the original audio signal; when the noise reduction manner of the noise reduction filter is determined to be overall noise reduction, the original audio signal is subjected to noise reduction filtering processing using the overall noise reduction noise reduction manner to simultaneously filter out the mid- and high-frequency noise and low-frequency noise in the original audio signal.

[0056] In a specific implementation, the time-frequency transfer function of the noise when the overall noise reduction function is turned on or off is defined as G n (g,ω):

[0057]

[0058] Among them, g is a random number between 0 and 1. In 70% of cases, the user turns on the overall noise reduction function during the call, and the overall noise reduction of the headset works at this time; in 30% of cases, the user does not turn on the overall noise reduction function during the call, and only the passive noise reduction of the headset works at this time.

[0059] In one embodiment, a target frequency response curve is obtained according to at least one measured frequency response curve, and a noise reduction filter expressing the target frequency response curve is generated. The number of measured frequency response curves used to obtain the target frequency response curve according to the measured frequency response curve is not limited, and can be specifically set according to actual use needs. In a specific embodiment, at least one measured frequency response curve is data-fitted using an IIR filter to obtain the target frequency response curve, and a noise reduction filter expressing the target frequency response curve is generated. In other specific embodiments, at least one measured frequency response curve may also be subjected to other processing to obtain the target frequency response curve, which is not specifically limited here.

[0060] In a specific implementation, a first target frequency response curve is obtained based on at least one measured frequency response curve corresponding to the passive noise reduction process, and a passive noise reduction filter expressing the first target frequency response curve is generated in the noise reduction filter. Also, a second target frequency response curve is obtained based on at least one measured frequency response curve corresponding to the overall noise reduction process, and an overall noise reduction filter expressing the second target frequency response curve is generated in the noise reduction filter. For example, Figure 3 As shown, data fitting is performed on the measured frequency response curve 1 corresponding to the passive noise reduction process to obtain a first target frequency response curve, and a passive noise reduction filter expressing the first target frequency response curve is generated in the noise reduction filter.

[0061] Among them, each measured frequency response curve is determined by a set of test noise signals of different frequencies collected by the feedback microphone of a test headset when responding to noise reduction processing. Specifically, the headset is worn on a real human ear or a simulated ear, the overall noise reduction function of the headset is turned on, the external speaker plays a swept frequency signal, and the frequency response curve H1(ω) from the speaker to the headset feedback microphone is measured; the overall noise reduction function of the headset is turned off, the external speaker plays a swept frequency signal, and the frequency response curve H2(ω) from the speaker to the headset feedback microphone is measured; the headset is removed, the external speaker plays a swept frequency signal, and the frequency response curve H3(ω) from the speaker to the headset feedback microphone is measured; therefore, H2(ω) and H3(ω) are used to obtain the target frequency response curve H under the passive noise reduction mode PNC(ω)=H2(ω) / H3(ω), and use H1(ω) and H3(ω) to get the target frequency response curve H under the overall noise reduction method. NC (ω) = H2(ω) / H3(ω).

[0062] Step S22: using a disturbance filter to perform disturbance filtering on the noise reduction audio signal to obtain a second disturbance audio signal.

[0063] like Figure 3 and Figure 4 As shown, when different human ears wear headphones, there are differences within 10dB in different frequency bands, so some random disturbances need to be added to the target frequency response curve obtained after data fitting to simulate the influence of different human ear structures on the generation of simulated audio signals collected by the feedback microphone when wearing headphones.

[0064] Therefore, in this embodiment, the disturbance filter is used to perform disturbance filtering on the noise reduction audio signal to obtain a second disturbance audio signal. The gain change of the second disturbance audio signal is within a preset gain change range, so that random frequency response disturbances within the preset gain change range are added to different frequency points of the noise reduction audio signal. For example, the gain change of the second disturbance audio signal is within a 10 dB change range, for example, [-5dB 5dB] or [-4dB 6dB], etc.

[0065] In one embodiment, the disturbance filter is a second-order recursive filter. It is understandable that in other embodiments, the disturbance filter may also be other types of filters, such as high / low shelf filters, peak filters, notch filters, etc. Among them, the quality factor of the filter is not limited and can be specifically set according to actual use needs.

[0066] In one embodiment, the preset filter coefficient range of each filter coefficient in the disturbance filter is determined by using the preset gain variation range. That is, the preset filter coefficient range of each filter coefficient in the disturbance filter can be determined according to the preset gain variation range. In a specific embodiment, the expression of the disturbance filter is determined, and a target equation is constructed so that the expression satisfies the preset gain variation range. The target equation is solved to obtain the preset filter coefficient range of each filter coefficient. Specifically, the expression of the determined disturbance filter is as follows:

[0067]

[0068] Among them, H 2rd represents the disturbance filter; [a1, a2, b1, b2] represents the filter coefficients of the disturbance filter;

[0069] Assuming that the gain change of the second disturbance audio signal is within a 10 dB change range, that is, within a [-5dB 5dB] range, the target equation for making the expression satisfy the preset gain change range is specifically as follows:

[0070] -5≤20lg 10 |H 2rd |≤5

[0071] Additionally, the perturbation filter is stable, so the poles are inside the unit circle:

[0072] |p1|<1,|p2|<1

[0073] Therefore, the constraints on the perturbation filter are as follows:

[0074]

[0075] Furthermore, an optimization tool is used to search for a numerical solution. Let a1, a2, b1, and b2 be -1 respectively, and traverse to 1 with a step size of 0.001, and use the genetic algorithm to find the filter coefficient range of the disturbance filter that meets the constraints as shown below:

[0076] -0.153≤b1≤0.149, -0.1475≤b2≤0.154

[0077] -0.146≤a1≤0.141, -0.1502≤a2≤0.154

[0078] Randomly select filter coefficients to obtain disturbance filters within the specified range, and randomly draw 4 groups of disturbance filters H 2rd The amplitude response is as follows: Figure 6 As shown, Figure 6 is a schematic diagram of an embodiment of a disturbance filter provided by the present application, Figure 6 It can be seen that the filter shapes, quality factors and gains of the four groups of disturbance filters are different. In addition, Figure 7 As shown, Figure 7 is a schematic diagram of an embodiment of the amplitude of the disturbance filter provided by the present application. Taking the maximum and minimum values ​​in the above filter coefficient range, it can be obtained from Figure 7 It can be seen that the amplitude range of the disturbance filter is limited to [-5dB5dB].

[0079] In one embodiment, if Figure 8 As shown, Figure 8 yes Figure 2 The flowchart of step S22 of an embodiment is shown, wherein the disturbance filtering of the noise reduction audio signal by using the disturbance filter specifically includes the following sub-steps:

[0080] Step S221: within a preset filter coefficient range, select a value for the filter coefficient of the disturbance filter.

[0081] In this implementation, the filter coefficient of the disturbance filter is selected within the preset filter coefficient range. That is, the filter coefficient is selected within the preset filter coefficient range so that the gain change of the subsequent second disturbance audio signal is within the preset gain change range. The preset filter coefficient range is not limited and can be specifically set according to actual use needs.

[0082] Step S222: Based on the frequency of the original audio signal, a disturbance filter is used to perform disturbance filtering to obtain a second disturbance audio signal matching the frequency.

[0083] In this embodiment, based on the frequency of the original audio signal, the disturbance filter after the value is taken is used to perform disturbance filtering to obtain a second disturbance audio signal that matches the frequency. Since in this embodiment, the original audio signal is an original noise signal, the original audio signal mentioned at this time is a noise reduction audio signal after noise reduction filtering, so the disturbance filter after the value is taken is used to perform disturbance filtering to obtain a second disturbance audio signal that matches the frequency.

[0084] Step S23: Filter the second disturbance audio signal using a noise transfer filter to obtain a simulated audio signal.

[0085] When a user wears headphones, they may wear them normally or loosely. Compared with wearing them normally, when the headphones are worn loosely, the voice received by the feedback microphone will lose the bone conduction voice component, and more noise will leak into the headphones. Therefore, when generating simulated audio signals, the tightness of the user's headphones needs to be taken into account.

[0086] Therefore, in this embodiment, the second disturbance audio signal is filtered using a noise transfer filter to obtain a simulated audio signal, so that the simulated audio signal finally obtained takes into account the tightness of the user's earphone wearing.

[0087] In one embodiment, the noise transfer filter is a time-frequency function based on the probability of tightness or looseness of the earphone, wherein the noise transfer filter adopts a high-shelf filter.

[0088] See also Fig. 9 , Fig. 9 yes Figure 1 FIG. 1 is a flow chart of another embodiment of step S13. It should be noted that if there is substantially the same result, this embodiment does not necessarily use Fig. 9 The process sequence shown is limited. Fig. 9As shown, in this embodiment, the simulated audio signal is a simulated voice signal, and the original audio signal is an original voice signal, specifically including:

[0089] Step S91: using a disturbance filter to perform disturbance filtering on the original speech signal to obtain a first disturbance audio signal.

[0090] like Figure 3 and Figure 4 As shown, when different human ears wear headphones, there are differences within 10dB in different frequency bands, so some random disturbances need to be added to the target frequency response curve obtained after data fitting to simulate the influence of different human ear structures on the generation of simulated audio signals collected by the feedback microphone when wearing headphones.

[0091] Therefore, in this embodiment, the original speech signal is perturbed and filtered using a perturbation filter to obtain a first perturbation audio signal. The gain variation of the first perturbation audio signal is within a preset gain variation range, so that a random frequency response perturbation within the preset gain variation range is added to different frequency points of the original speech signal. The gain variations of the original speech signal and the first perturbation audio signal are within a preset gain variation range. For example, the gain variation of the first perturbation audio signal is within a 10 dB variation range, for example, [-5dB 5dB] or [-4dB 6dB], etc.

[0092] It should be noted that, for the relevant contents about the construction of the disturbance filter, determination of the filter coefficient range of the disturbance filter, etc., please refer to step S22, which will not be repeated here.

[0093] In one embodiment, if Fig.10 As shown, Fig.10 yes Fig. 9 The flowchart of step S91 of an embodiment shown in FIG. 1 includes the following sub-steps:

[0094] Step S911: within a preset filter coefficient range, select a value for the filter coefficient of the disturbance filter.

[0095] Step S911 is similar to step S221 and will not be described again here.

[0096] Step S912: Based on the frequency of the original audio signal, a disturbance filter is used to perform disturbance filtering to obtain a first disturbance audio signal matching the frequency.

[0097] Step S912 is similar to step S222 and will not be described again here.

[0098] Step S92: Utilize a speech transfer filter to filter the first disturbance audio signal to obtain a simulated audio signal.

[0099] When a user wears headphones, they may wear them normally or loosely. Compared with wearing them normally, when the headphones are worn loosely, the voice received by the feedback microphone will lose the bone conduction voice component, and more noise will leak into the headphones. Therefore, when generating simulated audio signals, the tightness of the user's headphones needs to be taken into account.

[0100] Therefore, in this embodiment, the first disturbance audio signal is filtered using a speech transfer filter to obtain a simulated audio signal, so that the simulated audio signal finally obtained takes into account the tightness of the user's earphone wearing.

[0101] In one embodiment, the speech transfer filter and the noise transfer filter are time-frequency functions based on the probability of tightness or looseness of the earphone, wherein the speech transfer filter adopts a low-shelf filter.

[0102] In one embodiment, a time-frequency transfer function based on the probability of tightness or looseness of the earphone is constructed to obtain a noise transfer filter and a speech transfer filter.

[0103] Specifically, the time-frequency transfer function of speech leakage when the earphone is worn normally is defined as F s (p,ω), the time-frequency transfer function of noise leakage when the earphone is worn loose is F n (p,ω), the specific formula is as follows:

[0104]

[0105]

[0106] Where p is a random number between 0 and 1, which is used to control the probability of wearing leakage; H LS (ω) is a low-shelf filter; H HS (ω) is a high-shelf filter; e p is a random value related to p, used as a disturbance term of the overall gain, and H LS (ω) and H HS (ω) Overall gain increase or decrease 0dB to 6dB.

[0107] Among them, the low shelf filter H LS The expression of (ω) can be:

[0108]

[0109] High shelf filter H HS The expression of (ω) can be:

[0110]

[0111] Also, see Fig.11 , Fig.11 This application provides a low-shelf filter H LS (ω) and the high shelf filter H HS (ω) Amplitude-frequency response diagram of an embodiment.

[0112] See also Fig.12 , Fig.12 1 is a flow chart of an embodiment of filtering a disturbance audio signal to obtain a simulated audio signal provided by the present application. It should be noted that if there are substantially the same results, this embodiment is not used. Fig.12 The process sequence shown is limited. Fig.12 As shown, this embodiment includes:

[0113] Step S1201: Determine the probability of tightness or looseness of the earphone of the speech transmission filter or the noise transmission filter based on the probability control parameter.

[0114] In this embodiment, the probability of the earphone being loose or tight of the speech transmission filter or the noise transmission filter can be determined according to the probability control parameter. Specifically, the probability of the earphone being loose or tight of the speech transmission filter or the noise transmission filter is determined according to the probability control parameter p.

[0115] Step S1202: using a noise reduction filter to perform corresponding filtering processing using the determined probability of tightness of the earphone.

[0116] In this embodiment, the noise reduction filter is used to perform corresponding filtering processing using the determined probability of tightness of the earphone. For example, the time-frequency transfer function of the voice leakage, that is, the earphone is worn normally, is F s Taking (p,ω) as an example, when simulating the voice signal received by the feedback microphone, it is considered that in 70% of cases it is normal for the user to wear headphones. At this time, due to the existence of bone conduction components, the low frequency of the voice is enhanced and the noise is weakened, which is manifested as a positive gain low shelf filter for voice and a negative gain high shelf filter for noise. In 30% of cases, the user's headphones are loosely worn, the low frequency of the voice is weakened, and the noise is enhanced, which is manifested as a negative gain low shelf filter for voice and a positive gain high shelf filter for noise.

[0117] See also Fig.13 , Fig.13It is a schematic diagram of the framework of an embodiment of an audio simulation device provided by the present application. The audio simulation device 130 includes a first acquisition module 131, a second acquisition module 132 and a generation module 133. The first acquisition module 131 is used to acquire at least two filters, wherein different filters represent different influencing factors on the audio signal collected by the feedback microphone of the headset when the headset is worn; the second acquisition module 132 is used to acquire the original audio signal; the generation module 133 is used to use at least two filters to perform at least two filtering processes on the original audio signal in sequence, and generate a simulated audio signal collected by the feedback microphone when the headset is worn.

[0118] Among them, if the simulated audio signal is a simulated noise signal, the at least two filters include at least two of a noise reduction filter, a disturbance filter, and a noise transfer filter; and / or, if the simulated audio signal is a simulated speech signal, the at least two filters include a disturbance filter and a speech transfer filter.

[0119] Among them, the above-mentioned simulated audio signal is a simulated speech signal, and the original audio signal is an original speech signal; the generation module 133 is used to use at least two filters to perform at least two filtering processes on the original audio signal in sequence to generate a simulated audio signal collected by the feedback microphone when wearing headphones, specifically including: using a perturbation filter to perform perturbation filtering on the original speech signal to obtain a first perturbation audio signal; using a speech transfer filter to filter the first perturbation audio signal to obtain a simulated audio signal.

[0120] Among them, the above-mentioned simulated audio signal is a simulated noise signal, and the original audio signal is an original noise signal; the generation module 133 is used to use at least two filters to perform at least two filtering processes on the original audio signal in sequence to generate a simulated audio signal collected by the feedback microphone when wearing headphones, specifically including: using a noise reduction filter to perform noise reduction filtering on the original audio signal to obtain a noise reduction audio signal with a noise reduction gain corresponding to the frequency of the original audio signal; using a disturbance filter to perform disturbance filtering on the noise reduction audio signal to obtain a second disturbance audio signal; using a noise transfer filter to filter the second disturbance audio signal to obtain a simulated audio signal.

[0121] The above-mentioned noise reduction filter is determined based on a target frequency response curve, and the target frequency response curve is determined by collecting at least one group of test noise signals using a feedback microphone of at least one test headset, and the noise reduction gain corresponding to the frequency of the original audio signal represents the gain corresponding to the frequency in the target frequency response curve; and / or, the generation module 133 is used to use the noise reduction filter to perform noise reduction filtering on the original audio signal to obtain a noise reduction audio signal with a noise reduction gain corresponding to the frequency of the original audio signal, specifically including: based on the noise reduction control parameter, determining whether the noise reduction method of the noise reduction filter is passive noise reduction or overall noise reduction; using the noise reduction filter to perform noise reduction filtering on the original audio signal using the determined noise reduction method to obtain a noise reduction audio signal.

[0122] Among them, the above-mentioned disturbance filter is a second-order recursive filter; and / or, the generation module 133 is used to use the disturbance filter to perform disturbance filtering on the original speech signal to obtain a first disturbance audio signal, or the generation module 133 is used to use the disturbance filter to perform disturbance filtering on the noise reduction audio signal to obtain a second disturbance audio signal, specifically including: within a preset filter coefficient range, the filter coefficient of the disturbance filter is valued; based on the frequency of the original audio signal, the disturbance filter is used to perform disturbance filtering to obtain a first disturbance audio signal or a second disturbance audio signal matching the frequency.

[0123] Among them, the above-mentioned speech transfer filter or noise transfer filter is a time-frequency function based on the probability of earphone tightness; and / or, the generation module 133 is used to use the speech transfer filter to filter the first disturbance audio signal to obtain a simulated audio signal, or the generation module 133 is used to use the noise transfer filter to filter the second disturbance audio signal to obtain a simulated audio signal, specifically including: based on the probability control parameter, determining the earphone tightness probability of the speech transfer filter or the noise transfer filter; using the noise reduction filter to perform corresponding filtering processing using the determined earphone tightness probability.

[0124] The audio simulation device 130 further includes a construction module 134, which is used to obtain at least two filters, including any at least two steps below: based on at least one measured frequency response curve, a target frequency response curve is obtained, and a noise reduction filter expressing the target frequency response curve is generated, wherein each measured frequency response curve is determined by a set of test noise signals of different frequencies collected by a feedback microphone of a test headset during corresponding noise reduction processing; using a preset gain variation range, a preset filter coefficient range of each filter coefficient in the disturbance filter is determined. A time-frequency transfer function based on the tightness probability of the headset is constructed to obtain a noise transfer filter and / or a speech transfer filter.

[0125] The construction module 134 is used to obtain a target frequency response curve based on at least one measured frequency response curve, and generate a noise reduction filter that expresses the target frequency response curve, specifically including: obtaining a first target frequency response curve based on at least one measured frequency response curve corresponding to the passive noise reduction process, and generating a passive noise reduction filter that expresses the first target frequency response curve in the noise reduction filter; and obtaining a second target frequency response curve based on at least one measured frequency response curve corresponding to the overall noise reduction process, and generating an overall noise reduction filter that expresses the second target frequency response curve in the noise reduction filter; the construction module 134 is used to determine each frequency response curve in the disturbance filter by using a preset gain variation range. The preset filter coefficient range of the filter coefficient specifically includes: determining the expression of the disturbance filter, constructing a target equation that makes the expression satisfy the preset gain variation range, solving the target equation, and obtaining the preset filter coefficient range of each filter coefficient; the construction module 134 is used to construct a time-frequency transfer function based on the probability of earphone tightness to obtain a noise transfer filter and / or a speech transfer filter, specifically including: using the probability of earphone tightness and a low shelf filter to construct a first time-frequency transfer function as a speech transfer filter, and using the probability of earphone tightness and a high shelf filter to construct a second time-frequency transfer function as a noise transfer filter.

[0126] Among them, the above-mentioned original audio signal includes an original noise signal and an original speech signal, and the steps of performing at least two filtering processes on the original audio signal in sequence using at least two filters to generate a simulated audio signal collected by the feedback microphone when wearing headphones, and obtaining a simulated noise signal corresponding to the original noise signal and a simulated speech signal corresponding to the original speech signal; the audio simulation device 130 also includes a synthesis module 135, and the synthesis module 135 is used to synthesize the simulated noise signal and the simulated speech signal to obtain a simulated mixed audio signal collected by the feedback microphone when wearing headphones.

[0127] See also Fig.14 , Fig.14 1 is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application. The electronic device 140 includes a memory 141 and a processor 142 coupled to each other, the memory 141 stores program instructions, and the processor 142 is used to execute the program instructions to implement the steps in any of the above-mentioned audio simulation method embodiments. Specifically, the electronic device 140 may include but is not limited to: a desktop computer, a laptop computer, a server, a mobile phone, a tablet computer, etc., which are not limited here.

[0128] Specifically, the processor 142 is used to control itself and the memory 141 to implement the steps in any of the above-mentioned audio simulation method embodiments. The processor 142 can also be called a CPU (Central Processing Unit). The processor 142 may be an integrated circuit chip with signal processing capabilities. The processor 142 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 142 can be implemented by an integrated circuit chip.

[0129] See also Fig.15 , Fig.15 The computer-readable storage medium 150 stores program instructions 151 that can be executed by a processor, and the program instructions 151 are used to implement the steps in any of the above audio simulation method embodiments.

[0130] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0131] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0132] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0134] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0136] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An audio simulation method, characterized in that: The method comprises: Acquire at least two filters, wherein different filters represent different influencing factors on the audio signal collected by the feedback microphone of the headset when the headset is worn; Get the original audio signal; The original audio signal is filtered at least twice in sequence by using the at least two filters to generate a simulated audio signal collected by the feedback microphone when the headset is worn, if the simulated audio signal is a simulated noise signal, the at least two filters include at least two of a noise reduction filter, a disturbance filter, and a noise transfer filter, and / or, if the simulated audio signal is a simulated speech signal, the at least two filters include a disturbance filter and a speech transfer filter; Wherein, obtaining at least two filters includes any at least two steps of the following to obtain at least two filters: Based on at least one measured frequency response curve, a target frequency response curve is obtained, and a noise reduction filter expressing the target frequency response curve is generated, wherein each of the measured frequency response curves is determined by using a set of test noise signals of different frequencies collected by a feedback microphone of a test headset during corresponding noise reduction processing; Determine a preset filter coefficient range of each filter coefficient in the disturbance filter by using the preset gain variation range; A time-frequency transfer function based on the probability of tightness or looseness of the earphone is constructed to obtain a noise transfer filter and / or a speech transfer filter.

2. The method according to claim 1, characterized in that The simulated audio signal is a simulated speech signal, and the original audio signal is an original speech signal; the at least two filters are used to perform at least two filtering processes on the original audio signal in sequence to generate a simulated audio signal collected by the feedback microphone when wearing headphones, including: Performing disturbance filtering on the original speech signal using the disturbance filter to obtain a first disturbance audio signal; The first disturbance audio signal is filtered using the speech transfer filter to obtain the simulated audio signal.

3. The method according to claim 1, characterized in that The simulated audio signal is a simulated noise signal, and the original audio signal is an original noise signal; the at least two filters are used to perform at least two filtering processes on the original audio signal in sequence to generate a simulated audio signal collected by the feedback microphone when wearing headphones, including: Performing noise reduction filtering processing on the original audio signal using the noise reduction filter to obtain a noise reduction audio signal having a noise reduction gain corresponding to the frequency of the original audio signal; Performing disturbance filtering on the noise reduction audio signal using the disturbance filter to obtain a second disturbance audio signal; The second disturbance audio signal is filtered using the noise transfer filter to obtain the simulated audio signal.

4. The method according to claim 3, characterized in that The noise reduction gain corresponding to the frequency of the original audio signal represents the gain corresponding to the frequency in the target frequency response curve; and / or, The step of performing noise reduction filtering on the original audio signal using the noise reduction filter to obtain a noise reduction audio signal having a noise reduction gain corresponding to the frequency of the original audio signal includes: Based on the noise reduction control parameter, determining whether the noise reduction mode of the noise reduction filter is passive noise reduction or overall noise reduction; The noise reduction filter is used to perform noise reduction filtering processing on the original audio signal in a determined noise reduction manner to obtain the noise-reduced audio signal.

5. The method according to claim 2 or 3, characterized in that: The disturbance filter is a second-order recursive filter; and / or, The method of using the disturbance filter to perform disturbance filtering on the original speech signal to obtain a first disturbance audio signal, or using the disturbance filter to perform disturbance filtering on the noise reduction audio signal to obtain a second disturbance audio signal, comprises: Within a preset filter coefficient range, selecting a value for the filter coefficient of the disturbance filter; Based on the frequency of the original audio signal, disturbance filtering is performed using the disturbance filter after the value is taken to obtain a first disturbance audio signal or a second disturbance audio signal matching the frequency.

6. The method according to claim 2, characterized in that The step of filtering the first disturbance audio signal by using the speech transfer filter to obtain the simulated audio signal comprises: Determining the probability of the earphone being loose or tight for the speech transmission filter or the noise transmission filter based on the probability control parameter; The noise reduction filter is used to perform corresponding filtering processing using the determined probability of tightness of the earphone.

7. The method according to claim 3, characterized in that The step of filtering the second disturbance audio signal by using the noise transfer filter to obtain the simulated audio signal comprises: Determining the probability of the earphone being loose or tight for the speech transmission filter or the noise transmission filter based on the probability control parameter; The noise reduction filter is used to perform corresponding filtering processing using the determined probability of tightness of the earphone.

8. The method according to claim 1, characterized in that The step of obtaining a target frequency response curve based on at least one measured frequency response curve and generating a noise reduction filter expressing the target frequency response curve comprises: Based on at least one measured frequency response curve corresponding to the passive noise reduction process, a first target frequency response curve is obtained, and a passive noise reduction filter expressing the first target frequency response curve is generated in the noise reduction filter; and based on at least one measured frequency response curve corresponding to the overall noise reduction process, a second target frequency response curve is obtained, and an overall noise reduction filter expressing the second target frequency response curve is generated in the noise reduction filter; The method of using the preset gain variation range to determine the preset filter coefficient range of each filter coefficient in the disturbance filter includes: Determine an expression of the disturbance filter, construct a target equation that makes the expression satisfy a preset gain variation range, solve the target equation, and obtain a preset filter coefficient range of each filter coefficient; The step of constructing a time-frequency transfer function based on the probability of tightness or looseness of the earphone to obtain a noise transfer filter and / or a speech transfer filter comprises: A first time-frequency transfer function is constructed using the earphone tightness probability and a low shelf filter to serve as a speech transfer filter, and a second time-frequency transfer function is constructed using the earphone tightness probability and a high shelf filter to serve as a noise transfer filter.

9. The method according to claim 1, characterized in that: The original audio signal includes an original noise signal and an original speech signal, and the steps of using the at least two filters to perform at least two filtering processes on the original audio signal in sequence to generate a simulated audio signal collected by the feedback microphone when wearing headphones are respectively performed to obtain a simulated noise signal corresponding to the original noise signal and a simulated speech signal corresponding to the original speech signal; The method further comprises: The simulated noise signal and the simulated speech signal are synthesized to obtain a simulated mixed audio signal collected by the feedback microphone when the headset is worn.

10. An audio simulation device, characterized in that: include: A first acquisition module is used to acquire at least two filters, wherein different filters represent different influencing factors on the audio signal collected by the feedback microphone of the headset when the headset is worn; A second acquisition module is used to acquire the original audio signal; A generating module, configured to perform at least two filtering processes on the original audio signal in sequence using the at least two filters to generate a simulated audio signal collected by the feedback microphone when the headset is worn, wherein if the simulated audio signal is a simulated noise signal, the at least two filters include at least two of a noise reduction filter, a disturbance filter, and a noise transfer filter, and / or, if the simulated audio signal is a simulated speech signal, the at least two filters include a disturbance filter and a speech transfer filter; Wherein, obtaining at least two filters includes any at least two steps of the following to obtain at least two filters: Based on at least one measured frequency response curve, a target frequency response curve is obtained, and a noise reduction filter expressing the target frequency response curve is generated, wherein each of the measured frequency response curves is determined by using a set of test noise signals of different frequencies collected by a feedback microphone of a test headset during corresponding noise reduction processing; Determine a preset filter coefficient range of each filter coefficient in the disturbance filter by using the preset gain variation range; A time-frequency transfer function based on the probability of tightness or looseness of the earphone is constructed to obtain a noise transfer filter and / or a speech transfer filter.

11. An electronic device, characterized in that: The invention comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the audio simulation method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the audio simulation method according to any one of claims 1 to 9.

Images