How to obtain the sound source location

Abstract

The invention relates to a method for determining the location of a sound source relative to a dedicated reference point, where a first audio signal and a plurality of second audio signals synchronized in time are recorded. The location can be determined by applying an estimated filter to a correlation signal derived by correlation in the frequency domain between the first audio signal and at least one of the plurality of second audio signals. Two timing values ​​are derived in the time domain in at least one filtered correlation signal, which exceed a dedicated threshold. The distance between the dedicated reference point and the sound source is then based on the obtained first and second timing values, respectively.

Description

[Technical field]

[0001] This application claims priority to Danish application PA202170662 filed on December 22, 2021, the disclosure of which is incorporated herein by reference in its entirety.

[0002] The present invention relates to a method for obtaining the position of a sound source relative to a dedicated reference point, and also to a computer system and a non-transitory computer readable storage medium. [Background technology]

[0003] Sound field or spatial audio systems and formats such as Ambisonics or Dolby Atmos provide encoded sound information related to a given sound scene. Such an approach allows for the assignment of positional information to sound sources within a sound scene. Such techniques are already known from computer games, where recorded sounds are given the positional information of game objects, and also from the live capture of events such as large orchestras or sporting events. As a result, the number of possible applications is enormous and ranges from the above-mentioned immersive effects, for example by giving the impression of participating in a sporting event, to virtual reality and augmented reality experiences.

[0004] Recording sound for such applications is often difficult using spatial audio microphones. While these are useful for capturing raw sound field information from specific points in space, they have technical limitations as they are based on beamforming technology and are generally expensive. For example, sound quality may be degraded for people located far away from the microphone. Also, in noisier or more reverberant situations, or when multiple people are speaking, it is difficult to identify and separate individual sound sources for the purposes of equalization and other processing techniques.

[0005] Meanwhile, audio content creators are also recognizing the need for high-quality audio that includes the use of spatial audio information to improve recording quality and add additional acoustic effects that enhance the listener's immersion. As a result, there is a need for a lower-cost solution that achieves the benefits and advantages of a high level of spatial audio microphones. This solution must work regardless of hardware and be flexible enough to be used in a variety of scenarios. Summary of the Invention [Means for solving the problem]

[0006] The present disclosure with the proposed principles provides a method to achieve several of the benefits and advantages mentioned above. The inventors have discovered a method that provides accurate position determination in both distance and angle of a sound source relative to a dedicated reference point. The proposed method is largely independent of the hardware used and is scalable to different quality levels. Furthermore, the method allows for offline and real-time processing. As a result, the proposed method can be included in a variety of applications, including but not limited to sound capture and processing for podcasts, movies, live or other events, audio and teleconferencing, virtual reality, video game applications, etc.

[0007] In one aspect, we propose a method for determining the position of a sound source relative to a dedicated reference point. In this context, the term "position" includes the distance from the sound source to the dedicated reference point, the angle based on one or two axes passing through the reference point, or a combination thereof. The method includes obtaining a first audio signal recorded at the sound source. Similarly, a number of second audio signals are recorded at positions in a known relationship to the dedicated reference point. The first audio signal and the number of second audio signals are synchronized in time.

[0008] Typically, it is assumed that the first audio signal is recorded in the vicinity of the audio source. This means that the distance is relatively small compared to the distance between the audio source and a dedicated reference point. However, the term "at the audio source" should not be understood in a very limited sense. Rather, this expression includes and allows for a certain distance between the actual audio source and the microphone. Similarly, the multiple second audio signals are recorded at different positions with known distances and angles to the reference point. Temporal synchronization is important for the method proposed in the following steps. In some cases, such temporal synchronization can be achieved by providing a common time base for the recorded audio signals. In some other examples, the recorded audio signals can be used to provide the time base, for example by correlating in time a dedicated start signal included in the first audio signal and the multiple second audio signals and recorded.

[0009] Here, a filter acting on the signal-to-noise ratio at each frequency bin of the first audio signal in the time-frequency domain is estimated for the first audio signal. The first audio signal is then correlated in the frequency domain with at least one of the plurality of second audio signals to obtain at least one correlation signal. In some cases, the first audio signal is correlated with each of the plurality of second audio signals to obtain multiple correlation signals.

[0010] A pre-estimated filter is applied to the at least one correlation signal to obtain at least one filtered correlation signal.

[0011] In a next step, a distance between the dedicated reference point and the sound source can be estimated. For this purpose, a first timing value in the at least one filtered correlation signal that exceeds a dedicated threshold in the time domain is obtained. Also, based on the first timing value, a second timing value corresponding to the threshold in the at least one filtered correlation signal is obtained.

[0012] A distance between the dedicated reference point and the sound source is derived based on the obtained first and second timing values, respectively. If one or more filtered correlation signals have been derived in the previous step, multiple distances can be obtained that allow improving the distance determination (e.g., by calculating an average value including an error margin, etc.).

[0013] Alternatively or additionally, the angle of the sound source relative to an axis passing through a dedicated reference point may also be calculated. For this purpose, at least two filtered correlated signals and an optional a priori estimate or knowledge of the position of the microphones providing the plurality of second audio signals are utilized. In a subsequent step after applying the above-mentioned filter and correlating the filtered first signal with at least two of the plurality of second audio signals in the frequency domain to obtain at least two correlated audio signals, the at least two filtered correlated audio signals are truncated around a specific time period. Then, a cross-correlation between a pair of the truncated filtered correlated audio signals is obtained. The angle of arrival of the filtered first audio signal is derived by projecting the obtained cross-correlation into a spherical space based on the a priori estimate or knowledge of the position of the microphones providing the plurality of second audio signals.

[0014] The proposed method allows obtaining distance and angle independently of each other. The correlation of the two different signals can be done offline on recorded and stored audio signals, or in real time if necessary, allowing the method to be used for various applications. Furthermore, it is possible to exchange the first and second audio signals as well as derive slowly moving sound sources. The accuracy of the proposed method can be further improved by correlating different first audio signals. The method is robust against sound reflections, making it useful for recording sessions in closed spaces.

[0015] The speed of sound through a material depends on its temperature. For accurate measurements, the air temperature is measured, especially in the vicinity of the multiple second sound sources. Such measurements can be repeated periodically to correct for temperature changes during the recording session. The distance is estimated depending on the derived air temperature.

[0016] In one embodiment, the specific time period is proportional to the time-of-flight between two different positions of the plurality of second audio signals. The window defined by the specific time period ensures that sound reflections caused by room walls and other large objects are omitted during the cross-correlation step. This also means that the distance between two different positions of the plurality of second audio signals can be adjusted depending on the recording session environment, thereby further improving the flexibility of the proposed method.

[0017] In one embodiment, the step of truncating the filtered correlated audio signals comprises a step of upsampling. The upsampling can be performed on a plurality of the filtered correlated audio signals or on the truncated filtered correlated audio signals. This step provides finer time resolution and thus higher resolution in both distance and angle determination. For the angle determination, a steer response can be applied to each of the truncated filtered correlated audio signals to obtain a cross correlation.

[0018] The position where the cross correlation (PHAT in SRP-PHAT) of a pair of second audio signals (associated with a particular position) is maximum corresponds to the arrival time difference between those positions. The arrival time difference can be mapped to the angle of incidence using knowledge of the positions where the second audio signals were recorded. The cross correlation can therefore be projected into a spherical space instead of the time domain. This is performed for all filtered, truncated cross correlation estimates for a set of suitable pairs of second positions (or respective associated second audio signals) to determine the angle of arrival appropriate for the application. For example, if the sound source is only in a horizontal plane, it is sufficient to use the set of positions that span this plane and no other sets, which reduces the computational complexity. The projected estimates are simply summed and a search for the maximum is performed. The position of the maximum corresponds to the arrival angle.

[0019] In other embodiments, the step of estimating a filter of the first audio signal consists of estimating a spectral mask. For such a filter of the spectral mask, a signal-to-noise ratio (SNR) can be determined based on the noise of the respective time slice corresponding to the previous time slice. The parameters of the spectral mask are set to 0 if the signal-to-noise ratio does not exceed a predetermined threshold for a given frequency. Similarly, the parameters of the spectral mask are set to 1 for a given frequency if the signal-to-noise ratio exceeds a predetermined threshold. In some other embodiments, estimating a filter acting on the signal-to-noise ratio in each frequency bin of the first audio signal comprises applying a quantile filter, in particular a median filter, for smoothing the power spectrum of each time slice of the power spectrum obtained from the one or more first recorded audio signals. Then, the noise of the respective time slice is estimated depending on the previous time slice. As in the previous approach, it is evaluated whether the signal-to-noise ratio for a given frequency exceeds a predetermined threshold. As a result, the filter parameters of said frequency are set to 1 or 0 accordingly. In some other embodiments, estimating a filter acting on the signal-to-noise ratio in each frequency bin of the first audio signal comprises applying a residual signal from a noise reduction process as a noise estimate. The noise reduction process can be based on machine learning. As in previous approaches, it is evaluated whether the signal-to-noise ratio for a given frequency exceeds a predefined threshold. As a result, the filter parameters for that frequency are set to 1 or 0 accordingly.

[0020] Further, some embodiments relate to a step of correlating the first audio signal and at least one of the plurality of second audio signals. A short-time Fourier transform (STFT) is applied to the first audio signal and at least one of the plurality of second audio signals to obtain a respective spectrogram. A cross spectrum of the respective spectrograms is then obtained and the previously obtained filter is applied to the cross spectrum. The correlated filtered spectrogram is transformed into the time domain by performing an inverse short-time Fourier transform (ISTFT). This results in a filtered correlated first audio signal.

[0021] Another example concerns the delay of sounds recorded at two different positions, both of which are associated with a first audio signal recorded at the respective position. In the proposed method, a simple tool is provided to calculate the distance between the positions associated with two or more first audio signals. This is useful because it not only estimates the possibility of crosstalk between two or more microphones (recording the first audio signals), but also provides information about the relative distance between the microphones that can be used for post-processing to estimate the position. As a result, this approach can be used to obtain information about sound sources that are far from the positions where the two (or more) first audio signals were recorded.

[0022] In one embodiment, a filter acting on the signal-to-noise ratio in each frequency bin of one of the two first audio signals is estimated in the time-frequency domain. This process is performed for both first audio signals so that two different filters are estimated. Then, the first audio signal of the two first audio signals is correlated with the second audio signal of the two first audio signals in the frequency domain and the estimated filter associated with the first audio signal of the two audio signals is applied. This process may be repeated with the second audio signal of the two first audio signals and its respective filter. A time difference between the correlated signals is evaluated. This time difference provides information about possible crosstalk in one of the first audio signals. As a result, one of the two first audio signals based on the evaluation of the time difference may be selected as the one or more first audio signals and used for further processing.

[0023] The correlating step may in some embodiments include a Short-Time Fourier Transform (STFT) performed on each of the two first audio signals to obtain a spectrogram. A cross spectrum is derived from each spectrogram, and the filter is applied to the obtained cross spectrum. An Inverse Short-Time Fourier Transform (ISTFT) is performed to obtain a filtered first audio signal to obtain a final filtered correlated first audio signal.

[0024] It is useful to place the microphones recording the second audio signals in dedicated positions. For example, the plurality of second audio signals may comprise audio signals of four voices, two of which are recorded with a maximum spatial distance of 15 cm. This distance is typically small enough to avoid accidentally recording direct and reflected sounds of the same sound source at the same time, while being large enough to provide sufficient difference when cross-correlating the second audio signals with the first audio signals without employing excessive upsampling.

[0025] Some further aspects relate to the identification and processing of moving sound sources. The identification of such sources can be difficult because moving sound sources affect the STFT by Doppler shift. Furthermore, the estimated noise may be identified as a moving sound source or as two or more sound sources at different positions. For this reason, the estimated distance between the dedicated reference point and one or more sound sources may be fed into a Kalman filter (or a particle filter if the estimated noise is non-Gaussian).

[0026] In some further embodiments, a computer system is provided comprising one or more processors and a memory. The memory is coupled to the one or more processors and comprises instructions that, when executed by the one or more processors, cause the one or more processors to perform the proposed method and its various steps. Similarly, a non-transitory computer-readable storage medium may be provided comprising computer-executable instructions for performing a method according to any of the preceding claims. [Brief description of the drawings]

[0027] Further aspects and embodiments in accordance with the proposed principles will become apparent in connection with the various embodiments and examples described in detail in conjunction with the accompanying drawings.

[0028] [Figure 1] FIG. 1 shows an embodiment of the proposed method illustrating some processing steps for determining the location of a sound source. [Diagram 2] FIG. 2 shows the steps of a frequency-weighted phase transformation with a spectral mask applied to obtain a filtered correlation signal. [Diagram 3] FIG. 3 illustrates a recording environment using multiple microphones to record more complex sound field scenarios. [Figure 4] FIG. 4 illustrates a process flow of a method according to some aspects of the proposed principles. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] The following embodiments and examples disclose different aspects and their combinations according to the proposed principles. The embodiments and examples are not necessarily drawn to scale. Similarly, different elements may be displayed enlarged or reduced in size to emphasize individual aspects. It goes without saying that the individual aspects of the embodiments and examples shown in the drawings can be combined with each other without further description without contradicting the principles according to the present invention. Some aspects show conventional structures or forms. However, it should be noted that in practice, slight differences or deviations from the ideal form may occur without contradicting the idea of ​​the present invention.

[0030] Additionally, the individual figures and aspects are not necessarily drawn to scale, nor are the proportions between the individual elements necessarily correct in nature. Some aspects have been emphasized by enlarging the view. However, terms such as "top," "upper," "lower," "lower," "larger," "smaller," and the like, are properly expressed with respect to the elements in the figures. Thus, relationships between such elements can be inferred based on the drawings.

[0031] Fig. 3 illustrates an application of the method according to the proposed principles. The scenario corresponds to a typical recording session in which multiple audio signals are recorded to obtain the sound field of a scene. In this example, a speech recording of a natural person is used, but it should be understood that the method and principles are not limited to the audio processing and localization of natural people disclosed herein. Rather, they can be used to localize any dedicated sound source relative to a reference point. The scene includes two sound sources depicted as P1, P2, which in this embodiment are two respective people having a conversation in an at least partially enclosed space. Each person holds a microphone M1, M2, respectively, in a position close to the respective body. Thus, the microphones M1, M2 can be associated with the location of the respective sound source. A number of second microphones M3, M4 are located at a position B1. The position B1 is also defined as a reference point. Thus, the persons P1, P2 are each located at a specific distance and angle relative to the reference point B1, and are also spaced apart from each other. There is a wall W on one side, and reflections occur when each sound source P1 or P2 speaks.

[0032] The microphones M1, M2, M3, and M4 are synchronized in time with each other, i.e., the sound recordings in this scenario are made using a common time base. When recording a conversation, microphone M1 records the speech of person P1 and, with some delay, also the speech of person P2. Similarly, due to the speed of sound and the distance of person P1 from reference point B1, microphones M3 and M4 record the speech of people P1 and P2 with some delay. The delay varies depending on the distance, but in any case, the direct path from the sound source to one of microphones M3 and M4 is called the direct sound.

[0033] Now, assuming there is only a single sound source P1, the distance using the direct sound, i.e. the distance to the reference point B1, can be calculated simply, i.e. by measuring the time delay between the sound signal recorded by microphone M1 and the sound signal recorded by one of the microphones M3, M4, and multiplying by the speed of sound.

[0034] Since the speed of sound depends on temperature, a temperature sensor T1 is placed near the microphones M3 and M4 to measure the air temperature and compensate for the effects of temperature changes. The above scenario is very simple and is not suitable for real-world scenarios. For example, if a wall W reflects part of the speech, it is recorded by microphone M1, but also by microphones M3 and M4 with a slight delay. Microphone M4 also records the speech. In some scenarios, the reflected sound is superimposed with the ongoing speech. Due to possible constructive interference and other effects, it may happen that the recording of the indirect reflected sound has a higher level than the direct sound. In more complex scenarios, a second sound source also provides a sound signal at the same time, resulting in a superposition of several different sound signals, some originating from sound sources P1, P2 and some being wall reflected sounds.

[0035] The present application aims to process recorded signals in such a way that it is possible to identify and locate the position of each sound source relative to a reference point.

[0036] Another application that addresses the problem of associating specific location information with a sound source is in virtual reality (VR) applications, which typically include a 360° stereoscopic video signal that includes multiple objects in a virtual environment, some of which are associated with objects that correspond to sounds.

[0037] These objects (both visual and audio) are presented to the user, for example, via binocular and stereo headphones, respectively. Binocular headphones can track the position and orientation of the user's head (e.g., using an IMU / accelerometer), so that the video and audio played to the headphones and earphones, respectively, can be adjusted accordingly to maintain the illusion of virtual reality. For example, at any given moment, only a portion of the 360° video signal is shown to the user, which corresponds to the user's current field of view in the virtual environment. As the user moves or turns his head, the portion of the 360° signal shown to the user changes to reflect how the user's view of the virtual world changes with his or her movement. Similarly, as the user moves, sounds emanating from different positions in the virtual scene can have adaptive filtering applied to the left and right headphone channels to simulate the frequency-dependent phase and amplitude changes of sounds occurring in real life due to spatial offsets between the ears and the human head, as well as scattering by the upper body.

[0038] Some VR works consist only of computer-generated images and pre-recorded or synthesized sound. However, it is becoming increasingly common to produce "live-action" VR recordings using a camera capable of recording a 360° field of view and multiple microphones capturing the sound field. The sound recorded from the microphones is processed in a manner that follows the proposed principles and aligned with the video signal to produce a VR recording that can be played over headphones or earphones as described above.

[0039] Another application that addresses the problem of associating specific location information with an audio source is in Next Generation Audio (NGA) applications, which typically include audio objects with metadata such as location.

[0040] These objects (both visual and audio) are presented to the user, for example, via head-tracked stereo headphones with binaural rendering. As a binocular headset, such headphones can track the orientation of the user's head (e.g., using an IMU / accelerometer), so that the audio played to the headphones can be adjusted accordingly to maintain the illusion of audio immersion. For example, as the user moves or rotates his head, sounds emanating from different positions in the virtual scene, or scenes recorded using this innovation, can undergo adaptive filtering of the left and right headphone channels to simulate the frequency-dependent phase and amplitude changes of sounds occurring in real life due to spatial offsets between the ears and the human head, as well as scattering of the upper body.

[0041] Reference is now made to Fig. 1, which shows the different blocks of the method according to the proposed principle. For simplicity, the method is explained using the scenario of Fig. 3 above. The method is suitable for post-processing of pre-recorded audio signals, but also for post-processing of real-time audio signals, for example during audio conferences or live events. The method starts with providing one or more first audio signals and a number of second audio signals to blocks BM1, BM2, respectively. The recorded audio signals preferably have the same digital resolution (for example 14 bits at 96 kHz) including the same sample frequency. If different resolutions and / or sampling frequencies are used, it may be desirable to resample the various audio signals to obtain signals of the same resolution and / or sampling frequency.

[0042] The upper part of the image including element 3’, R1, 30A, 31 is related to the identification of crosstalk that may occur between two or more first audio signals, i.e., audio signals recorded by microphones whose positions are to be determined. As described above, in block BM1, not only the reflected sound but also the direct sound is recorded by two microphones. In order to determine which of the two or more microphones is actually located at each sound source, the signals recorded by the two microphones are filtered and cross-correlated to obtain the time difference of cross-correlation.

[0043] For this purpose, both signals are processed using frequency-weighted phase transformation. In the first step, each signal is transformed into the frequency domain to obtain a time-frequency spectrum using STFT. The filter of the spectral mask is first derived from the spectrum by generating a smooth power spectrum S(l,k), where l is the audio signal from the microphone and k is each frame of the audio signal. For each frequency bin, a first-order filter estimates the noise n(l,k) of the current frame based on the previous frame. The overall noise n(l,k) is given by the following formula. n(l,k)=(1-α) log(S(l,k))+(n(l,k-1)) α Here, α varies depending on S(l,k)<log(n(l,k-1)). Therefore, the filter mask becomes 1 when the SNR is above a certain threshold and 0 otherwise. The result is different filter masks associated with each of the two first signals. In the next step, two pairs of the first signals are cross-correlated, and a cross-spectrum is generated by normalizing the result of the cross-correlation. Next, each estimated filter is applied to the normalized cross-spectrum, and an inverse STFT is performed to obtain the filtered correlation signal (see reference numeral 31). At this time, for the cross-spectrum of R xy the filter Fx of (signal x) is used, and for R yxIt should be noted that the cross spectrum of uses filter Fy (of signal y). The filtered correlation signal is used to estimate the signed time difference or delay of the direct sound at both microphones recording the first audio signal. The sign indicated in block 31, i.e. dt>0 or dt<0, provides information which microphone is closer to the actual sound source. This microphone (and audio signal) is then associated with the respective sound source and the corresponding filter mask.

[0044] If the association of the audio signals to each sound source is defined, i.e., only one first signal is recorded, the above steps can be omitted. Returning to FIG. 3, blocks 3, R2 to 35 illustrated at the bottom explain the various steps of estimating the distance and angle to a reference point. Block BM3 includes a plurality of second audio signals recorded by one or more second microphones whose positions are fixed relative to the reference point. The positions of each of the second microphones are slightly different so that the angle can be obtained later, but close enough so that effects such as reflections from walls can be determined and filtered. In this embodiment, there are four different second audio signals, each recorded by a different second microphone. The processing here is similar to that described for the processing of two or more first audio signals. However, in block 3, the first audio signal (the audio signal for which the distance and angle are determined) is now cross-correlated with at least one of the four second audio signals. Block 3 can be performed for each of the second audio signals to provide four filtered cross-correlated signals in total. See, for example, reference R2.

[0045] Figure 2 shows the frequency-weighted phase transformation in an exemplary embodiment. The two input signals are transformed into the frequency domain using the SFTF, from which the cross spectrum is derived. After normalizing the spectrum, the previously estimated filter (in this case the filter of the spectral mask associated with the first audio signal) is applied. The result is transformed into the time domain using the inverse SFTF.

[0046] The time delays of blocks 30B and 30A are estimated by first identifying the maximum that the peaks would have if the frequency-weighted PHAT signals were uncorrelated. For this purpose, the variance of the noise is given by sigma=mean(mask) / framesize and the maximum of the noise is derived by sqrt(sigma*2*ln(framesize)). Then, in the frequency-weighted PHAT, we search for the first value (which can include a scale for some headroom) that exceeds this maximum and re-search for a local maximum close to the first value. The position of the maximum corresponds to the time-of-flight of the direct sound (n_max / sampling frequency). The distance is then given by the time-of-flight multiplied by the speed of sound, which takes into account the temperature dependence of the speed of sound. The process of block 30B is repeated for each of the cross-spectra. In block 31, the various results are further processed using the average of the set of time-of-flight estimates. Then, in block 33, the distance is subtracted from this estimate.

[0047] To obtain the angle between the sound source and the reference point, blocks R3, 30C, and 34 to 36 are used. To avoid the effects of room reflections, in block R2 a window function is used to truncate the FW-PHAT results of the first filtered correlation signals. As shown in FIG. R2, the window function has a width that depends on the distance between the second microphones. Since the second microphones recording the second sound signals are slightly farther away, the estimated distance between the sound source and the respective second microphone may also vary. The width of the window function for truncating the first filtered correlation signals is substantially proportional to the maximum time-of-flight between the second microphones. The set of now truncated filtered correlation signals is upsampled to improve the time resolution and estimate the angle more accurately. Subsequently, the cross-correlation between pairs of upsampled truncated first signals is calculated. As a result, a total of six results are received (four truncated signals resulting in six different pairs).

[0048] The location of the maximum of the cross correlation of the upsampled truncated first signal pair corresponds to the arrival time difference of the first signal at the respective second microphone. The arrival time difference is mapped to the incidence angle using knowledge of the position of the second microphone relative to the reference point. That is, the cross correlation can be projected into the spherical spatial domain instead of the time domain. The approach of block 34 is similar to the steered response step in the SRP-PHAT approach, where six pairs of cross correlations correspond to the PHAT. The projected estimates are simply summed and a search for the maximum is performed in block 35. The location of the maximum corresponds to the arrival angle.

[0049] The process flow of the method for determining distances and angles according to the proposed principle is shown in Figure 4. The method is suitable for real-time processing as well as offline processing of multiple previously recorded audio signals forming a sound field.

[0050] The method comprises in step S1 obtaining a first audio signal recorded at a sound source whose distance and angle to a reference point needs to be determined. A plurality of second audio signals are recorded in the vicinity of the reference point or at least at known positions or locations relative to the reference point. The first audio signal and the plurality of second audio signals are synchronized in time. Such time synchronization can be achieved by referencing all audio signals during the recording session to a common time base.

[0051] Afterwards, in step S3, the various signals are optionally pre-processed. For example, the recorded audio signals can be de-noised or equalised to improve the results of the subsequent processing steps, although care must be taken not to disturb the timing of the signals. It is also useful in some embodiments to apply a method to preserve phase information of the recorded audio signals during the pre-processing step S3. Furthermore, an STFT is performed on the first audio signal and the respective second audio signal.

[0052] In this embodiment, there is only a single first audio signal associated with a single sound source. The first audio signal is processed in step S4 by estimating a filter, in particular a spectral mask filter. This filter acts on the signal-to-noise ratio of each frequency of the first audio signal in the time domain. The resulting spectral mask contains a set of "1"s and "0"s for each frequency bin.

[0053] In step S5, the first audio signal is correlated in the frequency domain with at least one of the plurality of second audio signals to obtain at least one correlation signal. This step S5 may be repeated to cross-correlate the first audio signal with each of the second audio signals. The cross-correlation is normalized before applying the filter estimated in step S4 to obtain one or more filtered correlation signals.

[0054] So far, the steps for determining distance and angle are similar.

[0055] Now, the determination of the distance between the reference point and the sound source continues with steps S6 to S8. Step S6 includes obtaining a first timing value in at least one filtered correlation signal that exceeds a dedicated threshold in the time domain. Then, in step S7, based on the first timing value, a second timing value is obtained corresponding to a threshold in at least one filtered correlation signal. Both steps S6 and S7 may use the previously described search for a maximum in the PHAT signal (i.e. the filtered correlation signal). The distance between the dedicated reference point and the sound source is based on the first and second timing values ​​obtained in step S8, respectively. Furthermore, the temperature of the air can also be taken into account. In case of a pre-recorded signal, this information is stored and used in S9 to correct for the effects of temperature affecting the speed of sound.

[0056] Steps S10 to S13 are performed to derive and estimate the angle of the sound source from the reference point. For this purpose, step S5 is performed multiple times to obtain cross-correlations of the first sound signal with the respective second sound signal. In step S10, the cross-correlated PHAT signal is truncated by applying a window function to the PHAT signal. The window function has a width proportional to the maximum time-of-flight between the second sound sources and is centered on a peak corresponding to the time-of-flight. By applying the truncation window in step S10, the part of the signal due to sound reflections on objects and other indirect sound paths is removed.

[0057] A pair of truncated correlated signals is then selected and cross correlation is applied to the selected pair in step S11. In the simplest form with two second sound sources, there is a single pair selected for the cross correlation in step S11. In more complex scenarios where multiple second sound sources are recorded, several pairs can be selected. In general, it can be considered that a larger number of selected pairs provides a more accurate determination, since possible fluctuations and statistical errors can cancel each other out. The result of step 11 is therefore one or more cross-correlated signals. These signals are upsampled in step S12 to provide a finer timing resolution that affects the accuracy of the angle determination.

[0058] The cross-correlation signals in steps S11 and S12 each comprise one or more local maxima, which due to previous truncation usually have only one correlation. The position of this maximum in the cross-correlation (or PHAT in SRP-PHAT) of the pair of truncated correlation signals corresponds to the arrival time difference between the microphone elements recording the multiple second audio signals.

[0059] The time difference of arrival can be mapped to the angle of incidence using knowledge of the position of the microphone recording the second audio signal. That is, the cross correlations can be projected onto a spherical space instead of into the time domain. This projection is performed for all cross correlation estimates for all second pairs. The projected estimates are simply summed and a search for the maximum is performed. The location of the maximum corresponds to the angle of arrival.

[0060] Further, one aspect relates to processing audio signals that are moving over time. For example, if there are multiple first microphones, an active speaker detection algorithm can be used to identify the currently active speaker and its associated first microphone. For moving audio signals, a dynamic model and Kalman filtering can be used to estimate the location of the sound source at different times. The Kalman filter tracks the estimated state of the system and the variance or uncertainty of the estimate. The estimate is updated using a state transition model and measurements.

Claims

1. A method for obtaining the position of a sound source relative to a dedicated reference point, comprising the steps of: - obtaining a first audio signal recorded at an audio source; - acquiring a plurality of second audio signals, each recorded at a known positional relationship to a dedicated reference point, said first audio signal and said plurality of second audio signals being synchronized in time; for said first audio signal, - estimating, in the time-frequency domain, a filter acting on the signal-to-noise ratio of each frequency bin of said first audio signal; - correlating said first audio signal and at least one of said plurality of second audio signals in the frequency domain to obtain at least one correlation signal; applying said filter to said at least one correlation signal to obtain at least one filtered correlation signal; obtaining a first timing value in said at least one filtered correlation signal that exceeds a dedicated threshold in said time domain; obtaining a second timing value based on said first timing value, said second timing value corresponding to a threshold value in said at least one filtered correlation signal; - estimating a distance between said dedicated reference point and said sound source based on each obtained first timing value and said second timing value; A method for providing the above.

2. for at least two filtered correlation signals recorded at a known positional relationship to said dedicated reference point, truncating said at least two filtered correlated audio signals around a specific time period; obtaining a cross-correlation between a pair of truncated filtered correlated audio signals; obtaining an angle of arrival of said filtered first audio signal by projecting said obtained cross correlation into a spherical space; The method of claim 1 further comprising:

3. A method for obtaining an angle of a sound source relative to a dedicated reference point, comprising the steps of: - obtaining a first audio signal recorded from one or more audio sources; - acquiring a plurality of second audio signals, each recorded at a known positional relationship to a dedicated reference point, said first audio signal and said plurality of second audio signals being synchronized in time; for said first audio signal, - estimating, in the time-frequency domain, a filter acting on the signal-to-noise ratio in each frequency bin of one of said first audio signals; - correlating said first audio signal and at least two of said plurality of second audio signals in the frequency domain to obtain at least two correlated signals; applying said filter to said correlation signal to obtain at least two filtered correlation signals; - truncating said at least two filtered correlated audio signals around a specific time period selected to eliminate reflections from the environment due to at least said first audio signal; obtaining a cross-correlation between pairs of truncated filtered correlated audio signals of said plurality of second audio signals; - obtaining an angle of arrival of the filtered first audio signal by projecting the obtained cross-correlation onto a spherical space; A method for providing the above.

4. The method according to claim 2 or 3, wherein the particular time period is proportional to a maximum time-of-flight between positions of the plurality of second audio signals.

5. truncating the filtered correlated audio signal upsampling a plurality of said filtered correlated audio signals, or - upsampling the truncated filtered correlated audio signal; The method according to claim 2 , comprising:

6. Obtaining the cross correlation comprises: applying a steering response to each of said truncated filtered correlated audio signals; The method according to claim 2 , comprising:

7. Correlating the first audio signal comprises: upsampling the first audio signal and the plurality of second audio signals, or upsampling said at least two correlated signals; The method of claim 1 , comprising:

8. The method of claim 1 , wherein estimating a filter of the first audio signal comprises estimating a spectral mask.

9. estimating the spectral mask filter comprises: determining a signal-to-noise ratio (SNR) based on noise for each time slice (k) depending on the previous time slice; setting a parameter of the spectral mask for a given frequency to 0 if the signal-to-noise ratio does not exceed a predetermined threshold; setting a parameter of the spectral mask for a given frequency to 1 if the signal-to-noise ratio exceeds the predetermined threshold; The method of claim 8 comprising:

10. Correlating the first audio signal with at least one of the plurality of second audio signals and applying the filter to the correlated signal includes: performing a Short-Time Fourier Transform (STFT) on the first audio signal and on the at least one of the plurality of second audio signals to obtain respective spectrograms; obtaining a cross spectrum on each of said spectrograms; applying said filter to said obtained cross spectrum; performing an Inverse Short-Time Fourier Transform (ISTFT) to obtain said filtered first audio signal; The method of any one of claims 1 to 9, comprising:

11. Estimating a filter acting on the signal to noise ratio in each frequency bin of the first audio signal comprises: - applying, for each time slice (k) of a power spectrum derived from the one or more first recorded audio signals, a quantile filter, in particular a median filter, for smoothing the power spectrum; for each time slice (k), estimating the noise depending on the previous time slice; - evaluating, for a given frequency, whether said signal-to-noise ratio exceeds a predetermined threshold and setting the parameters of said filter for said frequency to 1 or 0 accordingly; The method of any one of claims 1 to 10, comprising:

12. estimating a filter acting on the signal-to-noise ratio in each frequency bin of the first audio signal comprises applying a residual signal from a denoising process as an estimate of noise, the denoising process optionally being based on machine learning.

12. The method according to any one of claims 1 to 11.

13. For the at least two first audio signals, - estimating, in the time-frequency domain, a filter acting on the signal-to-noise ratio in each frequency bin of one of the two first audio signals; - correlating a first one of the two first audio signals with a second one of the two first audio signals in the frequency domain and applying the estimated filter to the correlated signal; - estimating the time difference between said correlation signals; selecting one of the two first audio signals as one or more of the first audio signals based on the evaluation of the time difference; The method of claim 1 , further comprising:

14. The correlating comprises: performing a Short-Time Fourier Transform (STFT) of the two first audio signals to obtain respective spectrograms; obtaining a cross spectrum on each of said spectrograms; applying said filter to the obtained cross spectrum; performing an Inverse Short-Time Fourier Transform (ISTFT) to obtain a filtered correlated first audio signal; The method of claim 13 comprising:

15. 15. The method of claim 1, wherein the plurality of second audio signals comprises four audio signals, two of the four audio signals being recorded with a maximum spatial distance of 15 cm.

16. obtaining temperature information, in particular temperature information in the vicinity of a plurality of second sound sources; - estimating said distance depending on said acquired temperature information; 16. The method of claim 1, further comprising:

17. - feeding the estimated distances between said dedicated reference points and one or more of said sound sources to a Kalman filter or a particle filter; 17. The method of claim 1, further comprising:

18. one or more processors; a memory coupled to said one or more processors and comprising instructions which, when executed by said one or more processors, cause said one or more processors to perform the method of any one of claims 1 to 17; A computer system comprising:

19. A non-transitory computer-readable storage medium comprising computer-executable instructions for performing the method of any one of claims 1 to 17.

Images