Computer-implemented method and apparatus for processing an audio signal

Abstract

The invention relates to a computer-implemented method (100) for processing an audio signal, comprising: providing (110) the audio signal; determining (120) at least one frequency change expected in the audio signal, wherein the expected frequency change is characteristic of a class of audio signals and a window width of a window function of a Fourier analysis; applying (130) a Fourier analysis to the audio signal to generate a processed audio signal based on the expected frequency change; providing (140) the processed audio signal.

Description

[0001] The invention relates to a computer-implemented method and a device for processing an audio signal.

[0002] To classify audio signals, the associated data is usually preprocessed using various methods tailored to the specific application. For example, in a general audio data classifier such as Google's YamNet, the data undergoes a Fourier transform and then a Mel filter bank is calculated. The preprocessed data is then passed to a classifier, in this case a Convolution Neural Network, which is tasked with determining the class of the sounds.

[0003] In most cases, no preprocessing is performed, nor is a Fourier analysis used. A Fourier analysis translates the time signal into the frequency domain. A short-time FFT (STFT = Short Time Fourier Transformation) can perform this section by section, creating a two-dimensional spectrogram with one axis representing time and the other representing frequency. In this form, audio data is also readable by the human eye, and some sounds can be visually classified.

[0004] Fourier analysis assumes that a signal repeats itself infinitely, which is why the signal must be multiplied by a window function to avoid "jumps" at the edges of the window. Fourier analysis is particularly advantageous when a tone with a constant frequency is to be extracted. The longer the selected window, the better a tone can be differentiated from noise.

[0005] For signals that represent a type of oscillation, i.e., can be mathematically represented by a sine function, but have a frequency that changes over time (e.g., sweeps), an appropriate window must be selected for the Fourier analysis. For very rapidly changing frequencies, such as a phaser signal, a very small window must be selected to even identify a single frequency as a peak in the spectrogram. This has the disadvantage that the small window makes averaging over a longer period impossible, making it difficult to suppress the noise.

[0006] In Zhong, Jingang; Huang, Yu: Time-frequency representation based on an adaptive short-time Fourier transform. In: IEEE transactions on signal processing, Vol. 58, 2010, No. 10, pp. 5118-5128, ISSN 1053-587X, an algorithm for time-frequency representation based on an adaptive short-time Fourier transform is described. DE 10 2020 202 603 A1 describes a device and a method for detecting a characteristic signal in the environment of a vehicle.

[0007] It is an object of the present invention to provide a computer-implemented method and apparatus that at least ameliorate one or more of the aforementioned disadvantages. In particular, it is an object of the present invention to process the audio signal with varying frequencies in such a way that averaging over a larger time range is possible, thus reducing noise.

[0008] According to a first aspect, the object is achieved by a computer-implemented method for processing an audio signal. The method comprises: - Providing the audio signal; - determining at least one frequency change expected in the audio signal, wherein the expected frequency change is characteristic of a class of audio signals and a window width of a window function of a Fourier analysis; - Applying a Fourier analysis to the audio signal to generate a processed audio signal based on the expected frequency change; - Providing the processed audio signal.

[0009] Providing the audio signal may include receiving, storing, and / or recording the audio signal. Providing the processed audio signal may include storing and / or sending it to another unit for further processing of the processed audio signal.

[0010] The window width can be characteristic of the class of the audio signal. The window width can be based on the expected frequency change. The inventors recognized that a certain dynamic range is expected based on the frequency change, and this can be advantageously used in Fourier analysis.

[0011] The inventors recognized that a frequency change in the audio signal can be anticipated or expected, since certain classes are associated with specific frequency changes. For example, in the case of police or fire department warning sounds, a specific frequency change can be assumed that is characteristic of a police or fire department warning sound. This anticipation makes it possible to average over a larger time range in the Fourier analysis due to the larger window width, thus better suppressing noise.

[0012] Determining the expected frequency change may comprise determining the expected frequency change from a plurality of expected frequency changes, wherein frequency changes of the plurality of expected frequency changes are characteristic of a respective class of audio signals and a respective window width.

[0013] The class of audio signal may be one of a predetermined warning tone (e.g., a police vehicle), a sweep audio signal, a constant tone with a Doppler effect, a siren audio signal, a wail audio signal, a yelp audio signal, and / or a phaser audio signal.

[0014] The expected frequency change can be represented mathematically as a derivative of the frequency with respect to time.

[0015] The Fourier analysis may include or be a Fourier transform, in particular a short-time Fourier transform.

[0016] The method may further comprise identifying at least one frequency peak in the processed audio signal.

[0017] The method may further comprise classifying the audio signal based on the processed audio signal and / or the at least one frequency peak.

[0018] In one embodiment of the invention, applying the Fourier analysis comprises preliminarily stretching the audio signal by resampling the audio signal, wherein the audio signal has an increasing and / or decreasing frequency. The resampling can be performed at a frequency based on the expected frequency change. With a decreasing or falling frequency, the sampling points can be set closer together in time at the beginning of the interval and become larger towards the end of the interval.

[0019] The expected frequency change and / or the plurality of frequency changes can be provided in advance, for example stored.

[0020] In an alternative or additional embodiment of the invention, the Fourier analysis may be a discrete Fourier transform.

[0021] In the alternative or additional embodiment of the invention, applying the Fourier analysis further comprises extending the transformation matrix of the Fourier analysis by shape functions with quadratic phase responses and / or a frequency ramp, wherein the quadratic phase responses and / or the frequency ramp are based on the expected frequency change.

[0022] The expansion of the transformation matrix is explained in detail below. The transformation matrix or the twiddle factor can be adjusted. In the discrete Fourier transform, a vector x of length N is extracted from the time domain using the transformation matrix. W(k,j)=[1111w(2,2)..1w(3,2)..], w(k,j)=e−2πik jN mapped into the spectral domain by calculating the discrete Fourier coefficients X using the matrix product: X=W x=DFT(x)

[0023] Each row of the matrix W describes equidistant evaluations of a ansatz function in the form of a sinusoidal oscillation with a constant frequency. If this row is

[0024] If vector x is scalar-multiplied, their product yields the degree of agreement with the ansatz function and the phase shift as a complex Fourier coefficient. Instead of using a constant frequency as the ansatz, the transformation matrix can be defined using ansatz functions with quadratic phase responses, which is equivalent to a frequency ramp: Xα = W α where the quadratic part of j in the extended transformation matrix Wα is responsible for a linear frequency response in the ansatz, and the coefficient α determines the slope of the frequency change rate. The imaginary part φ of wα describes the instantaneous phase of the ansatz function in radians, so that the phase velocity is obtained by its derivative. dφdj=−2π(kN+2α(j−N2))

[0025] By formulating j−N2 ensures that the frequency ramp is in the middle of the interval, at j=N2, exactly corresponds to the center frequency, which kN Thus, the complex vector X α the degree of agreement and the phase shift of frequency ramps with a fixed frequency change rate specified by α at a series of support points at center frequencies specified by the discrete Fourier transform.

[0026] If multiple X α with different frequency change rates for the same input vector x, a matrix with matching values for a mean frequency / frequency change rate grid can be created and thus not only dominant frequencies but also tuples of frequency and associated frequency change rate can be determined.

[0027] In a further alternative or additional embodiment of the invention, the method further comprises: - Determining a phase of the audio signal; - Modifying the phase of the audio signal based on the determined expected frequency change such that the phase is compensated according to the expected frequency change, wherein applying the Fourier analysis to the audio signal is and / or includes applying the Fourier analysis to the audio signal with the modified phase. The frequency of the audio signal may be proportional to the time derivative of the phase, and the frequency change may correspond to the derivative of the frequency. Thus, the second derivative of the phase is proportional to the frequency change. Consequently, the phase of the shape function may be modified according to the expected frequency change.

[0028] The phase can be modified by resetting the phase to a starting tone. The starting tone can be a first temporal segment of the audio signal.

[0029] Modifying the phase is described in detail below.

[0030] The phase of the audio signal can be modified in such a way that, for example, with increasing frequency, i.e., a positive frequency change, the phase, which changes too rapidly compared to a constant tone, is compensated for and reset to the initial tone, thus compensating for the frequency change. Fourier analysis can be applied not only to real data but also, of course, to complex data, which is why such a phase change can also be applied to real data before Fourier analysis.

[0031] This can be implemented in such a way that the j-squared part of the ansatz function is extracted from the transformation matrix and multiplied element by element with the input data before the matrix multiplication with the transformation matrix: Xα=Wρα⋅x, ρα(j)=e−2πiα(j−N2)2N

[0032] This equivalent form determines the number of multiplications with the terms ρ α (j) is reduced by a factor of N and, in addition, the discrete Fourier transform can be significantly accelerated by using optimized fast Fourier transform algorithms when using vector lengths N that correspond to a power of two: Xα=FFT(ρα⋅x)

[0033] Determining the at least one expected frequency change may be and / or include determining a plurality of expected frequency changes. Applying the Fourier analysis to the audio signal may include applying a Fourier analysis to generate a respective processed audio signal based on a respective determined frequency change, wherein the method further comprises providing the processed audio signals. Two or more Fourier analyses based on different expected frequency changes may be performed in parallel.

[0034] According to a second aspect, the object is achieved by a device for processing an audio signal. The device comprises a memory for storing and / or providing the audio signal, and a processor. The processor is designed to: - determining at least one frequency change expected in the audio signal, wherein the expected frequency change is characteristic of a class of audio signals and a window width of a window function of a Fourier analysis; - Applying a Fourier analysis to the audio signal to generate a processed audio signal based on the expected frequency change; - Providing the processed audio signal.

[0035] The device may further comprise an audio sensor for recording the audio signal. The expected frequency change may be stored in the memory.

[0036] Method features described with respect to the method according to the first aspect may be embodied as device features according to the second aspect and vice versa.

[0037] The object is achieved according to a third aspect by a computer program product comprising instructions which, when the computer program product is executed by a device according to the second aspect, cause the device to carry out the method according to the first aspect.

[0038] The object is achieved according to a fourth aspect by a vehicle comprising a device according to the second aspect.

[0039] The vehicle may further comprise a controller configured to control the vehicle based at least in part on the processed audio signal.

[0040] Preferred embodiments are explained using the accompanying figures. They show: Fig. 1 a schematic representation of a computer-implemented method for processing an audio signal; Fig. 2 a discrete Fourier analysis of an audio signal with a Wail audio signal and a Yelp audio signal; Fig. 3 a phase portrait of the frequency from the Fourier analysis for Yelp and Wail; Fig. 4 a result of the Fourier analysis according to the method according to the invention; Fig. 5 is a schematic representation of an apparatus for processing an audio signal; and Fig. 6 a schematic representation of a vehicle with such a device.

[0041] Fig. 1 shows a computer-implemented method 100 for processing an audio signal. The method 100 can be stored in the form of a computer program product, for example, on a memory of a device 200. The method 100 can be used to classify audio signals. The method 100 includes providing 110 the audio signal. Furthermore, the method 100 includes determining 120 at least one frequency change expected in the audio signal. The expected frequency change is characteristic of a class of audio signals and a window width of a window function of a Fourier analysis.

[0042] The class of the audio signal can be one of a predetermined warning tone, such as a sweep audio signal, a constant tone with a Doppler effect, a siren audio signal, a wail audio signal, a yelp audio signal, and / or a phaser audio signal. Such audio signals exhibit characteristic frequency changes. Depending on the selected window width, these audio signals can be extracted relatively well using Fourier analysis. However, for very rapidly changing frequencies such as the phaser signal, a very small window must be selected in order to even find a single frequency as a peak in the spectrogram. This has the disadvantage that, due to the small window, averaging over a large time range is not possible, which means that the noise is hardly suppressed.

[0043] To achieve such improved averaging and to reduce noise, the present invention proposes determining the expected frequency change. Furthermore, the method 100 comprises applying 130 a Fourier analysis to the audio signal to generate a processed audio signal based on the expected frequency change and providing 140 the processed audio signal, for example, to a controller 310 of a vehicle 300.

[0044] The inventors have recognized that by predetermining the expected frequency change and applying Fourier analysis based on the expected frequency change, similar to beamforming, one or more theoretically possible arrival angles, here classified audio signals or frequency changes, can be tested. Consequently, the window width can be increased compared to a Fourier analysis that is not based on an expected frequency change.

[0045] The invention comprises at least three approaches.

[0046] A first approach is to stretch the audio signal by resampling, where the stretching and / or resampling is based on the expected frequency change. Stretching the audio signal can compensate for an expected increasing frequency change. This allows a larger window width to be used in the Fourier analysis.

[0047] A second approach is to adapt the transformation matrix for a discrete Fourier transform during Fourier analysis. In the discrete Fourier transform, a vector x of length N is extracted from the time domain using the transformation matrix W W(k,j)=[1111w(2,2)..1w(3,2)..], w(k,j)=e−2πik jN mapped into the spectral domain by calculating the discrete Fourier coefficients X using the matrix product: X=W x=DFT(x)

[0048] Each row of the matrix W describes equidistant evaluations of a ansatz function in the form of a sinusoidal oscillation with a constant frequency. If this row is scalar-multiplied by the vector x, their product yields the degree of

[0049] Agreement with the ansatz function and the phase shift as a complex Fourier coefficient.

[0050] Instead of taking a constant frequency as an approach, the transformation matrix can be defined by approach functions with quadratic phase responses, which is equivalent to a frequency ramp: Xα = W α

[0051] Due to the quadratic part of j in the extended transformation matrix, W αis responsible for a linear frequency response, and the coefficient α determines the slope of the frequency change or the rate of frequency change. The imaginary part φ of w α describes the instantaneous phase of the shape function in radians, so that the phase velocity can be obtained by its derivative: dφdj=−2π(kN+2α(j−N2))

[0052] By formulating j−N2 ensures that the frequency ramp is in the middle of the interval, at j=N2, exactly corresponds to the center frequency, which kN Thus, the complex vector X α the degree of agreement and the phase shift of frequency ramps with a fixed frequency change rate specified by α at a series of support points at center frequencies specified by the discrete Fourier transform. If several X αwith different frequency change rates for the same input vector x, a matrix with matching values for a mean frequency / frequency change rate grid can be created and thus not only dominant frequencies but also tuples of frequency and associated frequency change rate can be determined.

[0053] An exemplary so-called Fourier analysis in accordance with the method 100 is shown in Fig. 2. The processed audio signal exhibits two simultaneous warning tones: Wail and Yelp. Using the previously described method 100, the local peaks can be identified and classified.

[0054] Fig. Figure 3 shows a phase portrait of the frequency change of a wail audio signal and a yelp audio signal from the discrete Fourier analysis. Consequently, such frequency changes can be provided and / or determined as expected frequency changes, and the Fourier analysis can be performed accordingly.

[0055] Fig. Figure 4 shows a result of a short-time Fourier transform according to the proposed method 100. By anticipating the expected frequency changes of the Yelp audio signal and the Wail audio signal, it is possible to represent and identify the Fourier transform profiles of both signals. A classification can then be performed accordingly.

[0056] A third approach is to modify the phase of the audio signal. The phase of the audio signal is modified in such a way that, for example, with increasing frequency, i.e., a positive frequency change df / dt, the phase, which changes too rapidly compared to a constant tone, is compensated and reset to the initial tone, thus compensating for the frequency change df / dt. Fourier analysis can be applied not only to real data but also to complex data. Therefore, such a phase change can also be applied to real data before a Fourier analysis.

[0057] For example, since multiple warning signals may occur in urban traffic, the audio signal can be processed for several associated expected frequency changes. This processing can take place in parallel to quickly identify and / or classify the warning signal.

[0058] Fig. Figure 5 shows a schematic representation of a device 200 for processing an audio signal. The device 200 can be a computer, a mobile phone, an audio device, or an electronic device. The device 200 includes a memory 210 for storing and / or providing the audio signal. Furthermore, or alternatively, the memory 210 can store the computer program product.

[0059] The device 200 further comprises a processor 220. The processor 220 can be configured to execute the computer program. Furthermore, the processor 220 is configured to determine at least one frequency change expected in the audio signal. The expected frequency change is characteristic of a class of audio signals and a window width of a window function of a Fourier analysis. Furthermore, the processor 220 is configured to apply a Fourier analysis to the audio signal to generate a processed audio signal based on the expected frequency change. The processor 220 is further configured to provide the processed audio signal.

[0060] The provided audio signal can be stored in memory 210. Device 200 can include a communication unit for transmitting and / or receiving information, in particular the audio signal and the processed audio signal. Device 200 can further include an audio sensor for recording the audio signal.

[0061] Fig. 6 shows a schematic representation of a vehicle 300 with a controller 310 and the device 200. The controller 310 is designed to at least partially control the vehicle 300 based on the processed audio signal. Reference symbol 100 Computer-implemented method for processing an audio signal 110 Providing the audio signal 120 Determining at least one frequency change expected in the audio signal 130 Applying a Fourier analysis to the audio signal to generate a processed audio signal based on the expected frequency change 140 Providing the processed audio signal 200 Device for processing an audio signal 210 storage 220 processor 300 vehicles 310 Control

Claims

[1] A computer-implemented method (100) for processing an audio signal, comprising: Providing (110) the audio signal; Determining (120) at least one frequency change expected in the audio signal, wherein the expected frequency change is characteristic of a class of audio signals and a window width of a window function of a Fourier analysis; applying (130) a Fourier analysis to the audio signal to generate a processed audio signal based on the expected frequency change; Providing (140) the processed audio signal, characterized by that the application (130) of the Fourier analysis comprises a preliminary stretching of the audio signal by means of resampling the audio signal, wherein the audio signal has a rising and / or falling frequency and / or characterized by , that Fourier analysis is a discrete Fourier transform, wherein applying the Fourier analysis further comprises extending the transformation matrix of the Fourier analysis by shape functions with quadratic phase responses and / or a frequency ramp, wherein the quadratic phase response and / or the frequency ramp are based on the expected frequency change and / or characterized by Determining a phase of the audio signal; Modifying the phase of the audio signal based on the determined expected frequency change such that the phase is compensated according to the expected frequency change, wherein applying the Fourier analysis to the audio signal is and / or comprises applying the Fourier analysis to the audio signal with the modified phase. [2] Method (100) according to claim 1, wherein determining (120) the expected frequency change comprises determining the expected frequency change from a plurality of expected frequency changes, where frequency changes of the plurality of expected frequency changes are characteristic of a respective class of audio signals and a respective window width. [3] The method (100) of claim 1 or 2, wherein the class of the audio signal is one of a predetermined warning tone, a sweep audio signal, a constant tone with Doppler effect, a siren audio signal, a wail audio signal, a yelp audio signal and / or a phaser audio signal. [4] Method (100) according to one of the preceding claims, wherein the Fourier analysis comprises or is a short-time Fourier transform. [5] Method (100) according to one of the preceding claims, further comprising: Identifying at least one frequency peak in the processed audio signal. [6] Method (100) according to one of the preceding claims, further comprising: Classifying the audio signal based on the processed audio signal and / or at least one frequency peak. [7] The method (100) of any preceding claim, wherein if modifying the phase occurs, it occurs such that the phase is reset to an initial tone of the phase. [8] Method (100) according to one of the preceding claims, wherein determining (120) the at least one expected frequency change is and / or comprises determining a plurality of expected frequency changes, wherein applying the Fourier analysis to the audio signal comprises applying a Fourier analysis to generate a respective processed audio signal based on a respective determined frequency change, wherein the method (100) further comprises providing the processed audio signals. [9] Device (200) for processing an audio signal, comprising: a memory (210) for storing and / or providing the audio signal; a processor (220) configured to execute a computer program product; a memory on which the computer program product is stored, the computer program product comprising instructions which, when the computer program product is executed by the processor (220), cause the processor (220) to carry out the following steps: Determining at least one frequency change expected in the audio signal, wherein the expected frequency change is characteristic of a class of audio signals and a window width of a window function of a Fourier analysis; Applying a Fourier analysis to the audio signal to generate a processed audio signal based on the expected frequency change; Providing the processed audio signal, characterized bythat the application (130) of the Fourier analysis comprises a preliminary stretching of the audio signal by means of resampling the audio signal, wherein the audio signal has a rising and / or falling frequency and / or characterized by , that Fourier analysis is a discrete Fourier transform, wherein applying the Fourier analysis further comprises extending the transformation matrix of the Fourier analysis by shape functions with quadratic phase responses and / or a frequency ramp, wherein the quadratic phase response and / or the frequency ramp are based on the expected frequency change and / or characterized by Determining a phase of the audio signal; Modifying the phase of the audio signal based on the determined expected frequency change such that the phase is compensated according to the expected frequency change, wherein applying the Fourier analysis to the audio signal is and / or comprises applying the Fourier analysis to the audio signal with the changed phase. [10] A computer program product comprising instructions which, when the computer program product is executed by a device (200) according to claim 9, cause the device (200) according to claim 9 to carry out the method (100) according to any one of claims 1 to 8. [11] Vehicle (300) comprising a device (200) according to claim 9. [12] Vehicle (300) according to claim 11, further comprising: a controller (310) configured to control the vehicle (300) based at least partially on the processed audio signal.

Images