How audio signals are processed

Abstract

A method for processing an audio signal. The present invention relates to a computer-implemented method for processing an audio signal, where an audio signal containing speech is acquired, a smoothed difference signal is derived from the acquired audio signal and a smoothed spectrum thereof. At least one peak is detected in the acquired difference signal, and a peaking filter is generated based on the detected at least one peak. The peaking filter has at least a Q-factor and a gain parameter. The Q-factor is based on a geometrically acquired cutoff frequency around the detected at least one peak, and the gain parameter is derived from a gain interaction matrix having elements based on a magnitude response at the cutoff frequency around the detected at least one peak.

Description

[Technical field]

[0001] This application claims priority from Danish patent application PA202270098, filed March 11, 2022, the disclosure of which is incorporated herein by reference.

[0002] The present invention relates to a computer-implemented method for audio signal processing, a computer system for performing the method, and a non-transitory computer-readable storage medium. [Background technology]

[0003] Producing high quality audio is a key factor in conveying content, not only pure audio content, but also video content aimed at a wide audience, which is of paramount importance to content creators today. However, traditional audio recording methods often suffer from average or poor sound quality. There are many causes for poor quality of recorded sound, one of which is artifacts at certain audible frequencies. Such noises can be caused by ground loop noise (typically a 50Hz or 60Hz hum), light hum, room modes, etc.

[0004] Such artificial noises can be very distracting to the listener, but as an additional drawback, they require a lot of effort in post-processing the audio, especially since improving the audio quality often requires iterative work and the content provider has to listen to the processed audio signal until the desired result is achieved.

[0005] In response to the above problems, noise reduction processes have been developed that aim to specifically suppress the frequencies at which the above mentioned artificial noise occurs. These processes are used as part of or in addition to existing noise reduction processes to identify and remove noise from the recorded audio signal. This process often requires manual intervention and the setting of several parameters to obtain the desired quality and optimal results.

[0006] Some solutions have proposed predefined profiles to match noisy recorded audio signals. Although such methods can improve the sound, they are not flexible because they require many predefined profiles to process recorded audio signals under different environments. In general, such methods change the characteristics of recorded audio signals, which is usually undesirable. Other solutions include iterative methods and methods using deep learning networks to build models for artificial noise suppression. However, such methods are computationally expensive. Summary of the Invention [Problem to be solved by the invention]

[0007] It is therefore desirable to simplify or at least reduce the complexity of the noise removal process and to provide a noise removal process that can be flexibly implemented with less computational cost. [Means for solving the problem]

[0008] This and other objects are addressed by the independent claims. Characteristic features and further aspects of the proposed principle are set out in the dependent claims.

[0009] The inventors propose to use a simpler geometrical auxiliary model to obtain the filter elements used to suppress pre-specified artificial noise peaks. The proposal is based on the finding that an equalizing peaking filter cascade, also called peaking EQ with negative gain, can be used to precisely suppress frequencies that are unpleasant to the listener. In this cascade, the output of one filter element serves as the input for the subsequent element, and the transfer function of the entire cascade is the product of the transfer functions of the individual filter elements.

[0010] Unlike traditional models that require a standard profile to identify desired frequencies, the proposed method is adaptive, i.e., the center frequency of each filter element is adjusted to the actual sound signal, making it very flexible. In other words, the actual noise peaks are identified in the recorded audio signal without the need for a pre-recorded profile.

[0011] In one aspect, a computer-implemented method for audio signal processing is proposed, which obtains an audio signal, in particular an audio signal containing speech. The audio signal is pre-processed to generate a smoothed difference signal and its smoothed spectrum. According to the proposed principle, at least one peak, in particular originating from a noise source, is detected in the generated smoothed difference signal or its smoothed spectrum.

[0012] This aspect is an adaptive part of the proposed process since the detection and identification of peaks originating from noise sources is adjustable and not based on a pre-set profile. As an example, certain peaks detected below a first frequency threshold or above a certain second frequency threshold can be ignored. The first frequency threshold is useful for not attenuating parts of the sound (since the fundamental tone of the sound is often below 1 kHz and the harmonics are usually above 1 kHz), while the second frequency threshold is useful for reducing the calculation effort. The second frequency threshold is often above 15 kHz, which is in a barely audible frequency range and does not affect the listener's experience. Usually, the first frequency threshold is 150 Hz or 200 Hz, and generally below 300 Hz. On the other hand, peaks at frequencies above 20 kHz can often be ignored, especially if the magnitude of the peaks is not too large.

[0013] In a further step, after detecting at least one peak, a peaking filter is generated based on the detected at least one peak. The filter is then used to precisely suppress peaks of the acquired audio signal. For this purpose, the bandwidth of the respective filter as well as its negative gain must be defined. The bandwidth is also called Q-factor. For the purposes of this application, the Q-factor is used, but the proposed method can be implemented by setting a desired bandwidth as well. In this respect, Q-factor and bandwidth are to be understood as similar and included in the scope of the claims, unless otherwise stated.

[0014] It has been found that the Q-factor required to achieve good suppression is not a constant value for various assumed situations, but in fact varies depending on the characteristics of each peak. Therefore, a suitable Q-factor for peak suppression must be determined for each detected peak. The Q-factor is therefore determined geometrically based on a cutoff frequency determined around at least one detected peak.

[0015] However, when suppressing multiple peaks, adjacent negative peaking filters may affect each other. To reduce such undesirable interactions, the gain of each filter element is derived from a gain interaction matrix having elements based on the magnitude response at the cutoff frequency around at least one detected peak. In other words, the gain interaction matrix takes into account the position, Q factor or bandwidth of each filter element to identify a compromise between suppressing a peak and affecting adjacent filters.

[0016] As an example, the bandwidth of each filter element in a series of filter elements in a peaking filter is adjusted after the Q-factor of each element is determined. This ensures that the filter elements do not overlap and keeps interaction between elements to a minimum. This restriction may start with the filter elements that suppress peaks at low frequencies and continue to higher frequencies. As an example, the total bandwidth of the filter elements in a peaking filter with multiple elements may not exceed 1 / 3 octave.

[0017] The proposed method allows for the detection of noise peaks and their individual, flexible and efficient suppression. Such peak detection allows for flexible adjustments and allows the method to be applied to sounds and voices recorded in different environments. The method is largely independent of other audio processing, but can also be easily implemented in existing workflows. It also requires a low amount of computation, so that it can be used for pre-processing even on recording devices with low computational power. As mentioned above, the proposed method can be applied to multiple peaks as well as single peaks. In the latter example, multiple peaks are detected and peaking filters are generated individually based on the detected peaks. The peaking filter is composed of a cascade of filter elements, each associated with one of the detected peaks. Such an approach is suitable when the peaks are close together, since the gain and Q factor of each filter element affect the neighboring filter elements. Alternatively, individual peaks can be detected and a single-element peaking filter applied to the detected peak. A second peak is then detected and a new peaking filter is applied to the second peak. This process is repeated for all peaks and is computationally intensive, making it particularly useful when the various peaks are spaced apart and do not influence each other.

[0018] The peaking filter may be composed of a number of filter elements, each of which is defined by a Q-factor and a gain parameter. The gain parameter of each filter element is derived from a gain interaction matrix based on a magnitude response at a cutoff frequency at each of the detected number of peaks. The Q-factor is based on a geometrically derived cutoff frequency around an associated one of the detected number of peaks. In other words, the Q-factor and the gain parameter define each filter element of the peaking filter.

[0019] Some aspects relate to obtaining a smoothed difference signal. For example, the audio signal is processed to generate a smoothed spectrum of the obtained audio signal. Then, a difference between the obtained audio signal and the smoothed spectrum is determined and calculated to obtain the smoothed difference signal. A smoothing factor of the smoothed spectrum may be adjustable, as an example. Additionally, optionally, the difference signal is further smoothed by a smoothing factor different from the smoothing factor used to provide the smoothed spectrum. The latter optional step is not required, but further softens the spectral difference and simplifies the subsequent calculation of the Q-factor.

[0020] One aspect relates to the generation of a smoothed spectrum. It is generally known that a "good spectrum" is a smooth spectrum. That is, if some parts "stand out" in the frequency spectrum, such as sharp peaks or bumps, they are very likely to be spurious signals, noise or some other undesirable components. Depending on the characteristics of such "misses" in the smoothed spectrum, a peaking filter may be applied. Thus, first, the captured audio signal may be windowed with a certain window length. The window length may be in the range of 5 seconds to 30 seconds, in particular 10 seconds to 25 seconds, in particular 15 seconds to 20 seconds, in particular less than 22 seconds.

[0021] The short-time Fourier transform of the windowed signal is then calculated and averaged. Alternatively, the periodogram of the windowed signal can be calculated using, for example, Welch's method, and the square root can be taken from it. The spectrum thus calculated can optionally be converted to a dB scale and further smoothed.

[0022] As an example, the at least one peak may be detected by applying a threshold to the obtained difference signal. The peak or peaks are identified as exceeding the threshold. In another approach, the step of detecting the at least one peak includes applying a threshold to the obtained difference signal and identifying a largest first peak that exceeds the threshold. The location of this peak may be stored for further processing. Then, a largest second peak is identified that is centered on the largest first peak and is at least a minimum distance in frequency from the largest first peak on the low frequency side.

[0023] The latter step can be repeated, centered on the second largest peak, until all peaks above the low frequency threshold have been identified. Similarly, a third largest peak is identified on the high frequency side, at least the smallest distance in frequency from the first largest peak. The step can be repeated on the low frequency side, centered on the third largest peak. All identified peaks can be stored in memory. A peaking filter is then applied to each audio signal, with each filter element having its gain parameter and Q-factor calculated, associated with one of the peaks.

[0024] In yet another aspect, the step of applying a peaking filter to the at least one detected peak includes, after detecting the at least one peak, calculating a gain parameter at the at least one peak using a gain interaction matrix based on a magnitude response at a cutoff frequency around the at least one peak. For this purpose, a gain interaction matrix based on a magnitude response at a cutoff frequency around the at least one peak may be used. The gain interaction matrix takes into account possible interactions of gains of different adjacent filter elements. A Q-factor is then calculated based on the determined cutoff frequency around the at least one detected peak and its gain.

[0025] Some aspects relate to the calculation of the Q-factor. For example, the Q-factor may be determined based on a bandwidth given by the logarithm of the difference by the respective cutoff frequencies around at least one detected peak. As an example, the Q-factor can be derived by identifying two inflection points of the smoothed difference signal around at least one detected peak (left and right). Then, a virtual tangent line passing through the inflection point corresponding to the steepest slope is calculated. The expression "calculating the virtual tangent line" also includes approximating a function very close to the tangent line function. Then, the intersection point between the virtual tangent line and the average gain value is determined. The average gain value in this case is derived from the gain value of the target peak and is considered to be half of that value. Based on the determined intersection point, the bandwidth can be calculated (by mirroring the intersection point on the axis passing through the peak on the frequency axis). Based on the bandwidth, the Q-factor is determined, which also depends on the intersection point.

[0026] In another example, the Q-factor is determined by calculating a bandwidth given by the identified intersection point and a second intersection point having the same frequency distance from the identified intersection point and the at least one detected peak. In yet another example, the Q-factor is derived by calculating a second derivative of the smoothed difference signal. From this derivative, a frequency coordinate or respective pair (i.e., frequency and gain value) is identified in the smoothed difference signal around the at least one peak. The frequency coordinate corresponds to the location where the sign of the second derivative changes. Gain values ​​for the two coordinates are determined from the smoothed difference signal, and the Q-factor is determined corresponding to the two gain values.

[0027] In yet another aspect, the bandwidth is determined based on an average of the determined gain values ​​and a second derivative through one of two identified points, the one of the two identified points having a local maximum in the first derivative. Alternatively, the bandwidth is determined based on half the central gain, also referred to as the gain midpoint, and a second derivative through one of two identified points, the one of the two identified points having a local maximum in the first derivative.

[0028] Another aspect relates to a computer system having one or more processors and a memory coupled to the one or more processors. The memory comprises instructions that, when executed by the one or more processors, cause the one or more processors to perform a method as recited in any of the preceding claims. A non-transitory computer-readable storage medium comprises computer-executable instructions for performing a method as recited in any of the preceding claims. [Brief description of the drawings]

[0029] Further aspects and embodiments of the proposed principles will become apparent from the detailed description of the embodiments and examples which follow with reference to the accompanying drawings. [Figure 1] FIG. 1 is a frequency-gain diagram of a cascaded peaking EQ filter that can suppress noise peaks in an audio signal. [Diagram 2] FIG. 2 is a schematic diagram showing a flow for audio processing. [Figure 3A] FIG. 3A illustrates an embodiment of a method for processing an audio signal according to some aspects of the proposed principles. [Figure 3B] FIG. 3B illustrates another embodiment of a method for processing an audio signal according to some aspects of the proposed principles. [Figure 4A] FIG. 4A is a frequency-gain diagram illustrating the steps of obtaining a smoothed difference signal according to some principles of the proposed method. [Figure 4B] FIG. 4B is a frequency-gain diagram illustrating the steps of obtaining a smoothed difference signal according to some principles of the proposed method. [Figure 4C] FIG. 4C is a frequency-gain diagram illustrating the steps of obtaining a smoothed difference signal according to some principles of the proposed method. [Diagram 5] FIG. 5 is a frequency-gain diagram illustrating the results of a smoothed difference signal associated with a real speech signal having peaks to be suppressed and a smoothed target spectrum. [Figure 6A] FIG. 6A shows an example of a smoothed difference signal having a peak and a peaking filter obtained from the smoothed difference signal according to some aspects of the proposed principles. [Figure 6B] FIG. 6B illustrates an example of a smoothed difference signal having a peak and a peaking filter obtained from the smoothed difference signal according to some aspects of the proposed principles. [Figure 7A] FIG. 7A is a frequency-gain diagram of the location of the peak to be suppressed and the smoothed target response according to some aspects of the proposed principles. [Figure 7B] FIG. 7B shows a subsequent step, showing the frequencies of the "left" and "right" sides of the smoothed target response. [Figure 7C] FIG. 7C illustrates the first and second derivatives of the smoothed target spectrum of FIG. 7A. [Figure 7D] FIG. 7D illustrates a subsequent step of locating the inflection point of the second derivative, according to some aspects of the proposed principles. [Figure 7E] FIG. 7E is a frequency-gain diagram illustrating some further steps of the proposed method according to some embodiments. [Figure 7F] FIG. 7F is a frequency-gain diagram illustrating some further steps of the proposed method according to some embodiments. [Figure 7G] FIG. 7G shows the filter frequency response, the bandwidth definition points, and the resulting Q-factor. [Figure 8] FIG. 8 shows an example of a search tree for negative peaks illustrating some aspects of the proposed principles. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] The following embodiments and examples disclose various aspects and their combinations according to the principles of the present proposal. The embodiments and examples are not necessarily drawn to scale. Similarly, different elements may be displayed enlarged or reduced to emphasize individual aspects. It goes without saying that the individual aspects of the embodiments and examples shown in the drawings can be combined with each other without further modification without contradicting the principles according to the present invention. Some aspects show basic structures or forms. However, it should be noted that in practice, slight differences and deviations from the ideal form may occur without contradicting the concept of the present invention.

[0031] Recorded audio signals may contain continuous or periodic noise from various artificial sources. These sources include, for example, electrical humming from circuits, open or closed electrical loops, etc. Typically, such noise is at a single frequency within the audible range, i.e., narrow band, and is not considered broadband or wideband. These noise peaks should be suppressed during processing of the recorded signal to improve the listener's experience. Figures 4A and 5 show examples of recorded signals containing such narrowband noise peaks in the frequency spectrum.

[0032] To remove noise, a negative peaking EQ filter can be applied to the signal, which has a narrow but very large negative gain and selectively attenuates a selected narrow region of the frequency spectrum. Various types of filters can be used for such purposes. In the present embodiment, a second order IIR filter is utilized, but it should be understood that various other types of filters and combinations thereof can be used to achieve the desired result, i.e., suppressing spurious noise without significantly affecting the signal being used. The transfer function of a negative peaking EQ filter for suppressing a single peak at a center frequency fc with a gain gdb, a sampling rate fs, and a given Q factor is given by:

number

[0033] A recorded audio signal may consist of one or more noise peaks. FIG. 4A shows an example with multiple narrowband peaks to be removed. Thus, by stacking multiple filter elements with transfer functions as described above, a negative peaking EQ filter, also called a negative peaking EQ filter cascade, can be created. The output of one filter element in the negative peaking EQ filter cascade serves as the input for the next filter element. For N such filter elements, the transfer function of the entire negative peaking EQ filter cascade is:

number

[0034] 1 shows the transfer function of a negative peaking EQ filter with three filter elements P1, P2, and P3. When applied to an audio signal, the negative peaking EQ filter suppresses each frequency portion by a given negative gain.

[0035] The center frequency of filter element P1 is about 12 kHz, while the center frequencies of filter elements P2 and P3 are 15 kHz and 20 kHz, respectively. Each filter is characterized by a Q-factor and a gain value, the latter being -6 dB, -4 dB, and -3 dB, respectively. In the proposed negative peaking EQ filter and method in this application, the Q-factor based on the bandwidth of the filter elements can be adjusted to accommodate different bandwidths of noise peaks. This is shown in the bandwidth of the negative peaks in Figure 1. Peak P1 is composed of the highest Q-factor, while peaks P2 and P3 have decreasing Q-factors.

[0036] It has also been found that when the peaks to be suppressed are closely spaced, not only the Q-factor but also the gain parameters of adjacent filters influence each other. This behavior is observed between peaks P1 and P2, respectively. The gain of the entire negative peaking filter is therefore based on a gain interaction matrix that takes into account the various peaks and their bandwidths. The gain of each filter is calculated based on these parameters.

[0037] FIG. 2 shows the basic steps of a method for recording and processing recorded audio. Audio is recorded via a microphone device 1 and then stored as a digital signal in a memory storage device 2. The microphone device comprises one or more microphones, for example, to record audio and ambient sounds and provide a spatial sound environment. In some embodiments, the recorded sounds are simply audio and ambient sounds, for example, recorded in stereo or mono. The recorded audio is digitized by AD conversion and stored in memory 2 as a digital audio file or audio data. The resolution of the audio file is greater than 8 bits, for example 16 bits. The audio file can store raw data (for example, pcm with sampling rates of 44.1 kHz, 48 kHz, 96 kHz) as is or in a predefined desired lossless format.

[0038] After recording, the audio file is processed by step 3 or 4. As shown, the audio file can be processed by steps 3 and 4 in parallel or sequentially, such that the output of step 3 is used as the input to step 4. Figure 3A shows an exemplary embodiment of a method for processing an audio signal according to some embodiments of the proposed principles.

[0039] An audio signal is obtained in step S1 and stored in a memory as a digital signal. Then, in step S2, a smoothed difference signal is obtained from the stored audio signal and its smoothed spectrum. Such a process can be implemented according to various embodiments, as will be explained in more detail with reference to Figs. 4A-4C and 5. In general, the smoothed difference signal includes all peaks and is also called the target frequency response. The frequency response has one or more negative peaks that are approximated by a negative peaking filter. In other words, the method proposes to generate a negative peaking EQ filter that closely resembles the smoothed difference signal. The negative peaking EQ filter thus generated is used on the recorded audio signal to appropriately suppress noise.

[0040] The smoothed difference signal is shown as a frequency gain spectrum. In step S3, at least one peak is detected in the smoothed difference signal. A portion of the smoothed difference signal is identified as a peak if it exceeds a certain threshold. The threshold is adjustable and can be set according to the frequency at which the peak occurs. Such an approach allows noise peaks in the frequency band of interest, i.e. the frequency range in which the ear is most sensitive, to be suppressed more than other peaks. For example, since this method is employed to correct speech, peaks found at frequencies below 300 Hz are essentially ignored. This reduces the risk of inadvertently suppressing the fundamental frequency of the speaker. The location and "intensity" or volume of the detected peaks are stored in a memory.

[0041] In a subsequent step S4, a negative peaking filter response is determined based on the detected peaks. More specifically, the calculated negative peaking filter response is defined as one or more filter elements, each filter element being centered on one or more of the detected peaks. Each filter element is given by a Q-factor and a gain parameter, which are determined from the respective detected peaks of the smoothed difference signal. Due to the above-mentioned possible interactions between adjacent filter elements, the gain parameter and the Q-factor can be calculated based on this behavior.

[0042] The gain parameters calculated in step S5 are derived from a gain interaction matrix having elements based on the magnitude response at the cutoff frequency around at least one detected peak. The gain interaction matrix is ​​used to adjust the interactions between each filter element in the negative peaking EQ filter. If there is only one filter element, i.e., target spectrum, and the cascade consists of one negative peaking EQ filter, the desired gain is simply the value of the magnitude spectrum at that peak. A similar approach can be used if the filter elements are spaced apart from each other, provided there is no interaction between the elements.

[0043] However, this is highly unlikely. Typically, one must assume a series of closely spaced filter elements in a negative peaking EQ filter, where each filter element interacts nonlinearly with the other filter elements. To calculate the gain of each filter element in the negative peaking EQ filter, one must account for the interactions between each filter element in the cascade. For this purpose, an interaction matrix MI is constructed as follows:

number

[0044] Matrix Elements

number

number

[0045] The filter elements of the negative peaking EQ filter used to calculate the interaction matrix are characterized by a Q-factor, the calculation of which is described in more detail with reference to Figures 7A-7F.

[0046] A prototype gain of gdb=17db was used as an initial starting point for each filter element. This starting point proved to be a good fit for the calculations. The individual gains for the filter elements were then calculated and stacked into a vector using the following iterative formula:

number

[0047] In equation (4), t corresponds to the value of the target spectrum at m points,

number

number

[0048] MI is the interaction matrix calculated from equation (3), but the filter gain at the i-th step is

number

[0049] Apart from the gain coefficients, a Q-factor is also calculated for each filter element using the methodology described in Figures 7A-7F. The Q-factor is geometrically obtained based on a determined cut-off frequency around at least one detected peak. The generated negative peaking EQ filter is then applied to the captured audio signal.

[0050] Fig. 3B shows another embodiment of a method for processing an audio signal according to some aspects of the proposed principles. The audio to be corrected is converted to a spectrum by calculating its periodogram in step S2. This step is similar to the previous example, but the periodogram is a different way to generate a frequency gain spectrum. The calculated periodogram is smoothed in step S32 to obtain a smoothed spectrum. The smoothed spectrum and the calculated periodogram are then used to derive a smoothed difference signal. This result is set as the target response for the subsequent processing ANGPEQ in step S35, similar to the embodiment of Fig. 3A, from which a negative peaking EQ filter is derived.

[0051] If only a single peak is detected the process is relatively straightforward, but if there is more than one peak the following steps are repeated until the filter parameters Q and gains have been determined for all identified peaks. For this process step S36, i.e. identifying the bandwidth of each filter element, branches to steps S360 to S367. These steps are then repeated.

[0052] In step S360, the most negative peak is selected as the first peak for determining the filter parameters. The expression "minimum gain" corresponds to the "most negative" peak in the target response. Once the first peak is selected, the process proceeds to identify the inflection point on the side with the steepest curve, where inflection point x is defined as f''(x)=0, where f''(x) is the second derivative of the target response. Similarly, the steepest rise corresponds to the extremum of the first derivative of the target response function, or (more precisely) the minimum of the first derivative of the target response function. In other words, starting from the peak, identify the position in frequency and gain where the first derivative has an extremum and where the second derivative is zero (basically there are two points, one on the left and one on the right, where the second derivative is zero, and the one with the smaller value of the first derivative is selected).

[0053] In step S362, the virtual line is linearly extended from the point identified in step S361. In particular, a tangent to the identified point is derived. The tangent is expressed as g at the intersection point ip. db It effectively crosses a constant corresponding to g / 2. db is the target gain value. The determined intersection point ip corresponds to half the bandwidth of the filter element centered on the peak, as shown in S364. The bandwidth is expressed in octaves and is stored in memory in step S265.

[0054] The above steps S361-S365 are then repeated for the next peak on the low frequency side of the target response spectrum, first a predetermined distance away from the current peak. This newly identified peak is set as the new current peak, and the steps are repeated. Once all peaks on the low frequency side have been processed, the flow is repeated again in step S367 for all identified peaks on the next higher frequency side. Of course, the direction (first low frequency side, then high frequency side) can be changed.

[0055] In this regard, Figure 8 shows an example of such a search and identification process carried out in steps S361-S365. As can be seen, the first path goes towards lower frequencies until the lowest peak (number 3) is identified. Then the process goes towards the next highest peak on the low frequency side, 4. Once all peaks have been identified, the process continues for the peaks on the high frequency side, again moving from the low peak to the next highest one. In the example shown in Figure 8, for the peak [1234567], the peaks are checked in the following order: 4→2→1→3→6→5→7.

[0056] Once the final bandwidths are calculated, the process continues by converting the obtained bandwidths for each filter element into corresponding Q-factors, for example using the equations above in step S40. A gain interaction matrix is ​​constructed from the respective cutoff frequencies in step S41, as in the above example. Finally, in step S42, the gains of each of the individual filter elements are calculated based on the gain interaction matrix and the filter type. The Q-factors and gain parameters of each filter element are combined to generate a full peaking filter for the audio signal. The generated filter is then applied to the audio signal to suppress noise peaks.

[0057] 4A-4C show the principle of generating a smoothed difference signal, also called target spectrum, from a stored original audio signal. An example spectrum of an original audio signal is shown in FIG. 4A. The audio signal contains several regularly spaced peaks resulting from recording, recoding or initial pre-processing. The peaks are in the range of about 4 kHz to approximately 20 kHz, i.e. in the audible range. Their respective values ​​are 10 dB to 30 dB louder than the other sounds. These peaks are the targets for suppression. FIG. 4B shows the result of smoothing in the case of the audio spectrum of FIG. 4A. The smoothed spectrum of FIG. 4B is derived using the following equation:

number

number

number

[0058] where Δf is the bandwidth of a single spectrum, and two smoothing functions are applied in sequence starting from equation (6).

[0059] Next, Figure 4C shows the result of the next step: the difference between the audio spectrum in Figure 4A and the smoothed spectrum in Figure 4B is calculated. This gives a target response with a negative gain peaking at about -20db. From there, a peaking filter is generated.

[0060] Figure 5 shows another example of the spectrum of an audio signal C1 with a single peak around 6.5 kHz. As can be seen from the curve, the overall gain is smaller with increasing frequency. This is due to the recorded signal (i.e. recorded speech has energy mainly at low frequencies and only small energy at higher frequencies above 1.5 kHz). Curve C2 is the smoothed spectrum of C1, which steadily decreases up to about 6 kHz, at which point there is a plateau and a small rise mainly originating from the peak of curve C1. The resulting target response signal is given by curve C3. The target response is also shown in detail in Figure 6A. The negative peak with a center frequency of 6.5 kHz constitutes a negative gain of about -30 db.

[0061] According to the proposed method, the target response is matched by a negative peaking EQ filter with an identical or nearly identical response with a negative peak at 6.5 kHz. The proposed approach of the present disclosure produces such a response. The result is shown in FIG. 6B. The figure shows the target response C3, overlaid with the filter response curve C4. The filter response includes a very high Q factor to suppress noise at the center frequency, while other and adjacent portions of the noise peak are substantially unaffected.

[0062] The estimation or determination of the Q-factor for each filter element in the negative peaking EQ filter is a key element to obtain a filter that accurately suppresses noise peaks while not affecting other parts of the signal. The Q-factor needs to be not too large to suppress useful parts of the signal without affecting adjacent filters, and not too low to ensure sufficient suppression. It has been found that the Q-factor consists of widely varying values ​​(Q values ​​in the range of 10 to 100 have been found to be useful). The flexible and adjustable determination of the Q-factor is based on a geometric approach. Figures 7A-7G show an exemplary embodiment of such an approach. It should be understood that in this embodiment, the individual steps are visualized to outline the principles. However, the proposed method is performed in software implemented on a computer.

[0063] Figure 7A shows the target response curve and, more precisely, its smoothed spectrum (as mentioned above) for better visualization and simplification. This method is used for the difference signal itself and not necessarily for the smoothed version, but depending on the occurrence of peaks, it is also possible to utilize a smoothed spectrum. There is a detection peak at position P1 around 7,5 kHz, which is suppressed with a negative peaking EQ filter. To determine the value of the Q-factor, the following relationship between the Q-value of the filter and its bandwidth B (defined as the frequency range between two points where the peak center has the value gdb / 2 (also called the midpoint)) is used:

[0064]

number

number

[0065] When another filter element of the peaking EQ filter is nearby, the additional filter "pulls down" the magnitude response of the first filter element, making it less "steep." This behavior can be seen, for example, in Figure 1, and also in Figures 7A and 7B. In particular, the left side LS of the target response curve is steeper than the right side RS.

[0066] Now, the proposed principle is to define the bandwidth by first identifying the steepest inflection point in order to "guess" where the two midpoints should be. For this purpose, it is necessary to calculate the first derivative of the target response curve and then the second derivative from it. These two curves are shown in Figure 7C.

[0067] As shown, the first derivative F1 has an extremum to the left of the center frequency point P1, and the inflection point P2 is given by the zero crossing P2=0 on the steep side. If the inflection point can be identified, then there are two possibilities: (a) there is only a single peak, or there are other peaks widely spaced apart. In such a case, the inflection point is one of the midpoints, and due to the symmetrical nature of the peak (note that it is symmetrical), the bandwidth is found by the difference from this point to the center of the peak. (b) there is another peak nearby that will affect the Q-factor. In this case, it is necessary to predict where the midpoint will be and find the bandwidth from that midpoint in the same way as above.

[0068] Both cases (a) and (b) can be handled in a similar manner as shown in Figures 7D-7F. Figure 7D shows the result of finding an inflection point P1 on the steepest side of the slope and a deflection point P2' to the right of the center frequency point P1 toward the high frequency side. A tangent line is then generated that passes through the inflection point P2. In the digital domain, the tangent line can be generated by simply calculating the function between the two sample points immediately to the left and right of the inflection point.

[0069] The midpoint is the point P3 where the line intersects the constant at gdb / 2, as shown in Figure 7E. Thus, the straight line L1 at gdb / 2 is given by half the target gain, which is the gain at the central frequency point P1. Alternatively, the average of the gain values ​​from the determined gain values ​​at the two inflection points P3 and P3' can be used. Point P3 is the first point that defines the bandwidth. For the second point, we assume that the filter bandwidth is symmetrical around the central frequency. Thus, a second midpoint, located on the steeper lower side of the central peak, is generated by "mirroring" midpoint P3 along the central peak. This can be done by finding the difference in frequency df=f(P1)-f(P3) and simply adding this difference to the frequency of the central peak P1. The result is point P3 shown in Figure 7F.

[0070] The two midpoints P3 and P3' are part of the response curve of the filter elements, which can be calculated by using the bandwidth and determining the Q-factor from there. Figure 7G shows a filter response with a Q-factor of 3.28 and a bandwidth of 0.37. The bandwidth filter curve is slightly distorted to take into account further peaks that need to be suppressed with adjacent filter elements. In such cases, since further filter elements need to be determined for the additionally detected peaks, it is desirable to constrain the Q so that the bandwidth between each peaking filter and the next filter does not exceed a half octave. In view of this, the total bandwidth of each filter element is limited to about 0.3-0.4, in particular 0.33 octaves, after the respective Q-factors have been determined. This keeps the interactions at low values ​​and still gives good suppression. The Q-factor is calculated independently of the subsequent bandwidth limitation, since that parameter is used to determine the gain value.

[0071] Another aspect is the movement of the detected peak over time. The peak may only occur periodically, or may change slightly in frequency. For this reason, the filter elements of the negative peaking EQ are negatively extended to accommodate the change over time. In the case of the biquad second-order IIR filter used in this example, its time-independent structure can be expressed as a differential equation:

number

[0072] The parameters a k and b k are the same as in the above formula (1). Due to the above-mentioned shift of the detection peak over time, the parameters a k and b k become time-dependent in themselves and in their overall configuration, and as a result, they are expressed as follows:

number

number

[0073] The parameters a k and b k are functions of Q and gdb, and so can be parameterized over time. In addition, we impose limits on the Q-factor and gain parameters, e.g., Q does not change by more than 10% of its original value, and gdb always lies within the range 0 to -30. These limits ensure that Q and gdb do not diverge over time, but remain within well-defined boundaries. The Q-factor and gain may be further smoothed to reduce the chance of sudden spikes in Q-factor and gain peaks over time.

number

number

[0074] Parameters

number

[0075] The present application provides a simple and efficient method for suppressing noise peaks in recorded audio signals, in particular audio signals containing speech. The proposed method uses a geometric approach to estimate the Q-factor of the peaking filter or the filter elements of a peaking filter cascade, respectively. The use of a smoothed spectrum and the determination of a target response instead of a pre-prepared "profile" of a recorded audio target for matching makes the method flexible. This allows processing of all kinds of different audio signals with varying noise peaks and levels. Although the present exemplary embodiment of the principles proposed herein relates to a negative peaking EQ filter, i.e. a filter used for noise suppression, the method can be used to emphasize certain parts of the frequency spectrum in the same way. For this reason, the present application proposes a sound signal processing method in which a peaking EQ filter is determined based on a target response determined by the difference between the smoothed spectrum and the original sound signal. The gain itself can be negative to suppress or positive to emphasize center frequencies around the filter elements of the peaking EQ filter, and combinations thereof.

Claims

1. 1. A computer-implemented method for audio signal processing, the method comprising: Obtaining an audio signal, in particular an audio signal containing speech; obtaining a smoothed difference signal from the obtained audio signal and the smoothed spectrum thereof; detecting at least one peak in the acquired difference signal; generating a peaking filter based on the detected at least one peak; the peaking filter comprises at least a gain parameter and a Q factor; the gain parameters are derived from a gain interaction matrix having elements based on a magnitude response at a cutoff frequency around the at least one detected peak; The Q factor is geometrically obtained based on a cutoff frequency around the detected at least one peak. method.

2. detecting the at least one peak includes detecting a plurality of peaks; The peaking filter is generated based on the detected peaks. the peaking filter comprises a cascade of filter elements, each filter element being associated with one of the detected peaks; the gain parameters of each filter element are derived from a gain interaction matrix based on magnitude responses at cutoff frequencies around each of the detected peaks; a Q-factor of each filter element based on a geometrically obtained cutoff frequency around an associated one of the detected peaks; The method of claim 1.

3. applying the peaking filter to the captured audio signal and storing the processed audio signal. The method according to claim 1 or 2.

4. Obtaining the smoothed difference signal comprises: providing a smoothed spectrum of the captured audio signal; and evaluating a difference between the acquired audio signal and the smoothed spectrum to obtain the difference signal; and optionally smoothing the difference signal with a smoothing factor different from a smoothing factor used to provide a smoothed spectrum. The method according to any one of claims 1 to 3.

5. Providing the smoothed spectrum includes: performing windowing of the captured audio signal with a window length; calculating and averaging over time a short-time Fourier transform of the windowed signal; Calculating the periodogram of the windowed signal, in particular taking its square root using Welch's method; Optionally, converting the calculated spectrum to dB scale; smoothing the calculated spectrum; at least one of The method according to claim 4.

6. The window length is in the range of 5 seconds to 30 seconds, in particular in the range of 10 seconds to 25 seconds, in particular in the range of 15 seconds to 20 seconds, in particular shorter than 22 seconds; The method according to claim 5.

7. Detecting the at least one peak includes: applying a threshold to the obtained difference signal; and identifying peaks in the difference signal that exceed the threshold. The method according to any one of claims 1 to 6.

8. Detecting the at least one peak includes: applying a threshold to the obtained difference signal; identifying a first maximum peak that exceeds the threshold; and Identifying a maximum second peak that is centered around the maximum first peak and is at a minimum distance from the maximum first peak on a low frequency side, at least in terms of frequency; Optionally, repeating the previous step centered on said maximum second peak; identifying a maximum third peak that is a minimum distance, at least in frequency, from the maximum first peak centered at a high frequency; Optionally, repeating the previous step centered on said maximum third peak; applying the peaking filter to each of the maximum first, second, and third peaks; The method according to any one of claims 1 to 7.

9. Generating a peaking filter based on the detected at least one peak may include, after detecting the at least one peak, calculating the gain parameters at the at least one peak using a gain interaction matrix based on a magnitude response at a cutoff frequency around the at least one peak; and calculating the Q-factor based on a determined cutoff frequency and its gain around the at least one detected peak. The method according to any one of claims 1 to 8.

10. The Q factor is based on a bandwidth given by a logarithm of the difference between respective cutoff frequencies around the at least one detected peak. The method according to any one of claims 1 to 9.

11. The Q factor is identifying an inflection point of the smoothed difference signal around the detected at least one peak; calculating a virtual tangent line passing through the inflection point corresponding to the steepest slope; identifying an intersection between the imaginary tangent line and an average gain value derived from the detected gain value of at least one peak; and obtaining the Q-factor corresponding to the identified intersection point. The method according to any one of claims 1 to 10.

12. Obtaining the Q-factor comprises: determining a bandwidth given by the identified intersection point and a second intersection point that is the same frequency distance from the detected at least one peak as the identified intersection point; The method of claim 10.

13. The Q factor is calculating a second derivative of the smoothed difference signal; identifying two points around the at least one peak in the smoothed difference signal where the second derivative changes sign; determining a gain value from the smoothed difference signal corresponding to the detected at least one peak; and obtaining the Q-factor corresponding to two of the gain values. The method according to any one of claims 1 to 12.

14. Obtaining the Q-factor comprises: determining a bandwidth based on half the gain value and a second derivative passing through one of the two identified points having an extremum in the first derivative; The method of claim 13.

15. one or more processors; a memory coupled to said one or more processors and comprising instructions, when executed by said one or more processors, causing said one or more processors to perform a method according to any one of claims 1 to 14, Computer system.

16. A non-transitory computer readable storage medium comprising computer executable instructions for performing the method of any one of claims 1 to 14.

Images