Improved localization of sound sources
By applying the expression for the general velocity vector V'(f) in the frequency domain, and combining the delay and direction of the direct and reflected acoustic paths, the problem of accuracy in three-dimensional localization of sound sources in compact microphone systems is solved, and more accurate sound source direction and distance estimation is achieved.
Patent Information
- Application Number
- CN202180078003.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-11-19
- Filing Date
- 2021-10-15
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2041-10-15
AI Technical Summary
Existing technologies struggle to accurately estimate the three-dimensional position of sound sources, especially for compact microphone systems, particularly in environments with strong reflections. This results in inaccurate angle estimations and a lack of distance information.
By applying time-frequency transformation in the frequency domain, using the expression for the general velocity vector V'(f), and combining the delay and direction of the direct and reflected acoustic paths, the three-dimensional position of the sound source is estimated. The determination of parameters DoA, d0, and z0 is optimized through an iterative process, taking into account the ratio of the omnidirectional component to the reference component.
It improves the accuracy and robustness of sound source localization, especially in the presence of strong reflections, enabling more precise determination of the direction and distance of the sound source and reducing angular deviation.
Smart Images

Figure CN116472471B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of locating sound sources, and in particular to estimating acoustic direction or "DoA" (direction of arrival) using a compact microphone system (e.g., a microphone capable of picking up sounds referred to as "surround sound" or "sound all around" as used below). Background Technology
[0002] One possible application is, for example, beamforming, which subsequently involves the spatial separation of sound sources, particularly for improving speech recognition (e.g., virtual assistants via voice interaction). This processing can also be used for 3D audio coding (pre-analysis of sound scenes to encode the main signal separately), or to allow for possible spatial editing of immersive sound content in audiovisual settings (for art, electronic music, film, or other purposes). It also allows for tracking people speaking in conference calls or detecting sound events (with or without associated video).
[0003] In existing technologies interested in surround sound (or equivalent) coding, most schemes are based on spatial components generated by frequency analysis (typically time-frequency representations generated by short-term Fourier transform or “STFT” processing, or narrowband time signal representations generated by filter banks).
[0004] The first-order surround sound signal is collected in vector form according to Equation 1 given in the appendix below. For convenience, the encoding convention of Equation 1 is given here, but it is not restrictive, as conversion to other conventions can also be implemented. Therefore, when the field is equivalent to a single plane wave from the direction described by the unit vector U1 (and thus the direction DoA of the source) and carrying the emitted signal s1(t), it can be written according to Equation 2 (Appendix).
[0005] In practice, the signal is analyzed frame by frame in the frequency domain, which gives the expression Equation 3 (Appendix), and is used for the case of a single wave in the form of Equation 4, as well as for the extension of N waves in the form of Equation 5.
[0006] One class of methods is based on the analysis of the velocity vector V(f) or the intensity vector I(f) (the first being an alternative to the second, through power normalization of the omnidirectional reference components), as shown in Equations 6 and 7.
[0007] The method using complex frequency samples is essentially based on the information contained in the real part of these vectors for position estimation (related to the active intensity and characteristics of wave propagation, as it is directly related to the gradient of the phase field).
[0008] As for the imaginary part (the part of the response associated with the energy gradient), it is considered a characteristic of steady acoustic phenomena.
[0009] Indeed, it can be seen that in the case of a single plane wave, the velocity vector consists of V = U1.
[0010] Known methods (known as "DirAC") either use time samples filtered in subbands, in which case they are real numbers and the intensity vector is also real, or they use complex frequency samples, in which case it is simply the real part of the intensity vector used to specify the direction of arrival (or more precisely, its opposite direction). Furthermore, a so-called "diffusion" coefficient is calculated, which relates to the ratio between the vector's norm and the sound field energy, making it possible to determine whether the information available at the frequency under consideration is characteristic of the directional component (in which case the vector's direction determines location) or characteristic of the "environment" (the field produced by the mixing of diffuse reverberation and / or undifferentiated secondary sound sources).
[0011] Another approach, hereinafter referred to as "VVM," is based on statistics of the velocity vector and the angular direction of its real part, weighted by certain factors related to the ratio between the real and imaginary parts and their norm. A spherical map (2D histogram, e.g., an isorectangular map) is constructed by collecting values across all frequency samples and a certain number of time frames. Therefore, this estimate is essentially based on maximum probability and is subject to a certain delay.
[0012] Another class of methods, known as "covariance" methods, sometimes as an extension of the first class, involves calculating the covariance matrix (sometimes also called the power spectral density matrix or "PSD") of the spatial components over frequency subbands. Here, the imaginary part is sometimes completely ignored. It's important to note that when the spatial components are surround sound type, the first row (or first column) of this matrix is equivalent to the intensity vector. Many of these schemes involve "subspace" methods and sometimes expensive algorithms, especially when dealing with large numbers of frequency subbands and when using higher spatial resolutions.
[0013] These "vector-based" or "matrix-based" methods attempt to identify both the "directional" components associated with a localizable sound source or path and the "directional" components associated with environmental components.
[0014] Among the observed limitations of these methods, they are hampered by the interference of reflections on direct sound (indicating the direction of the sound source), even in the case of a single, simultaneously active sound source. For example, in addition to the presence of some degree of room effect, they cannot provide sufficiently reasonable estimates and / or the estimates are often biased. Such reflective surfaces tend to cause angular deviations in the system when objects including acoustic location determination and capture devices (e.g., surround sound microphones) are placed, for example, near a table or wall (and / or if this is the case of the sound source).
[0015] In fact, location determination is often biased due to systemic interference with direct sound from reflections associated with the same sound source. When it is based on velocity vectors, primarily the real part of the velocity vector is considered, while the imaginary part is typically ignored (or at least underutilized). Acoustic reflections are considered undesirable and are not incorporated into the estimation problem. Therefore, they remain ignored and unmodeled components without considering the specific interference structures they cause.
[0016] Therefore, for the above types of applications, acoustic location is typically estimated only in terms of angle. Furthermore, there seems to be no efficient scheme for distance estimation based on a single capture point (which is considered unique for overlapping or more generally “compact” microphone systems, i.e., contained within a volume that is small in size relative to the distance to the sound source, typically around 10 cm for surround sound microphones).
[0017] However, some application contexts require additional information about the distance from the source, in addition to the source's orientation (thus requiring 3D position determination in XYZ). For example:
[0018] - Virtual navigation in a real-world environment captured in 3D (because proper modification of the source's angle and intensity depends on the relative XYZ translation between the object and the microphone),
[0019] - Locate the source to identify the person speaking (especially for connected speakers or other devices).
[0020] - Monitoring and alarm devices in residential or industrial environments
[0021] Or others.
[0022] A particularly advantageous scheme presented in document FR1911723 uses the velocity vector of sound to obtain the direction of arrival of the sound, its delay (and thus the distance to the sound source), and the delay associated with any reflections, thereby determining the location of the obstacle. Such an implementation allows for modeling the interference between direct waves and at least one indirect wave (generated by reflections), and utilizes the performance of this model over the entire velocity vector (its imaginary and real parts).
[0023] Nevertheless, while this technology is already in use, it could be used as an object for further enhancement. Summary of the Invention
[0024] This invention improves upon this situation.
[0025] A method for processing audio signals acquired by at least one microphone is proposed.
[0026] The objective is to locate at least one sound source in a space comprising at least one wall, wherein:
[0027] Apply time-frequency transformation to the acquired signal.
[0028] - Based on the acquired signal, a general velocity vector V'(f) is expressed in the frequency domain. This general velocity vector is estimated according to an expression for the velocity vector V(f), in which a reference component D(f), in addition to the omnidirectional component W(f), appears in the denominator of the expression, which has a real part and an imaginary part. The general velocity vector V'(f) represents the following composition:
[0029] *The first acoustic path, directly between the source and the microphone, is represented by the first vector U0, and
[0030] *At least a second acoustic path generated by reflections from the wall and represented by the second vector U1.
[0031] The second path has a first delay TAU1 at the microphone relative to the direct path.
[0032] - As a function of the delay TAU1, the first vector U0, and the second vector U1, at least one parameter is determined from among the following:
[0033] *The direction of the direct path (DoA),
[0034] * The distance d0 from the source to the microphone,
[0035] * The distance z0 from the source to the wall.
[0036] Therefore, the general velocity vector V'(f) mentioned above is composed of the velocity vector V(f), which is typically expressed as a function of the omnidirectional component in the denominator. The general velocity vector V'(f) replaces the "traditional" velocity vector V(f) in the sense of the aforementioned document FR1911723, and then has a "reference" component in the denominator instead of the omnidirectional component. This reference component can indeed be more "selective" regarding the direction of sound arrival. In the exemplary embodiments further given below, the reference component is particularly important. Figures 6A to 6D And 7, the direction of arrival of the sound that enables the reference component can be obtained as a first approximation by using the conventional velocity vector V(f) during the first iteration of the iterative process that gradually converges to the exact DoA.
[0037] It was then observed, within the context of this invention, that the determination of the aforementioned parameters DoA, d0, and z0 was particularly accurate and / or precise by using a general velocity vector V'(f) with a more relevant reference component instead of the velocity vector V(f). In particular, this method is more robust, especially when strong sound reflections are caused by barriers placed near the microphone or an active sound source.
[0038] The parameters DoA, d0, and z0 mentioned above have been referenced. It should be noted that, in typical embodiments, the general expression for the velocity vector also makes it possible to specifically determine the delay TAU1 given above.
[0039] In one embodiment, as described above, the method may include multiple iterations, wherein at least in part a general velocity vector V'(f) is used, with a reference component D(f) in its denominator determined based on an approximation of the direction of the direct path (DoA) obtained in previous iterations. In most cases, these iterations converge to a more accurate direction DoA.
[0040] This approach can then include a first iteration in which a “traditional” velocity vector V(f) is used instead of the general velocity vector V'(f). As described in FR1911723, the velocity vector V(f) is expressed in the frequency domain and has an omnidirectional component W(f) appearing in the denominator. Then, at the end of this first iteration, at least a first approximation of the direction of the direct path (DoA) can be determined.
[0041] Therefore, for at least the second iteration after the first iteration, the general velocity vector V'(f) estimated according to the expression of the velocity vector V(f) is used, wherein the omnidirectional component W(f) in the denominator is replaced by a reference component D(f), which is more selective in space than the omnidirectional component W(f).
[0042] For example, in one embodiment, the reference component D(f) is more selective in the direction corresponding to the first approximation of the direct path direction (DoA).
[0043] The iteration can be repeated until convergence is achieved according to a predetermined criterion. In particular, this can be a causality criterion to identify at least the first sound reflection on an obstacle (or the aforementioned “barrier”) in the sound propagation environment between the microphone and the sound source with reasonable certainty.
[0044] In one particular embodiment, in each iteration:
[0045] - An inverse frequency-to-time transformation is also applied to the expression for the general velocity vector V'(f) to obtain a series of peaks in the time domain, each related to reflections on at least one wall, in addition to peaks related to the arrival of sound along the direct path (DoA), and
[0046] - If a signal appears in the series of peaks whose time axis is smaller than the axis of the direct path peak and whose amplitude is greater than the selected threshold (possibly adaptive), then a new iteration is performed.
[0047] If the amplitude of the signal is lower than the threshold, then the causal relationship criterion is met.
[0048] Obtaining this series of peaks can generally be related to the form given in equation B4 = 39b in the appendix, and referenced below. Figure 2 Further description is needed, but this certainly applies to the general velocity vector V'(f).
[0049] The iteration of the above method can terminate, for example:
[0050] -In the first case where the amplitude of the signal is below the selected threshold, and
[0051] - In the second case where the repetition of the iteration does not cause a significant decrease in the amplitude of the signal.
[0052] In one exemplary embodiment, the following steps are performed after the second case, wherein the acquired signal is transmitted in the form of consecutive sample frames:
[0053] - For each frame, estimate the score for the presence of the sound at the start of the sound in the frame (e.g., according to Equation 53 in the Appendix), and
[0054] - Frames with scores higher than a threshold are selected for processing the audio signal acquired in the selected frames.
[0055] In fact, if the proximity of the barrier causes the first sound to be reflected immediately, making it difficult to converge to the DoA solution, it is better to look for the immediate response of these barriers to the sound initiation point (when the sound emission begins).
[0056] Regarding the respective expressions for the "conventional" and "general" velocity vectors, and in embodiments where the acquired signal is picked up by a surround sound microphone, the "conventional" velocity vector V(f) can be represented in the frequency domain by first-order surround sound components of the following type:
[0057] V(f) = 1 / W(f)[X(f),Y(f),Z(f)] T ,
[0058] W(f) is the omnidirectional component.
[0059] Meanwhile, the general velocity vector V'(f) in the frequency domain is expressed by the following types of first-order surround sound components:
[0060] V(f) = 1 / D(f)[X(f),Y(f),Z(f)] T ,
[0061] D(f) is the reference component other than the omnidirectional component.
[0062] The order considered here is first order, which allows the components of the velocity vector to be expressed in a three-dimensional reference frame, but other implementations are also possible, especially with higher surround sound orders.
[0063] In one embodiment, the estimate of the direct path direction equivalent to the first vector U0 can be determined by the average of a set of frequencies of the real part of the general velocity vector V'(f) expressed in the frequency domain (according to the form of Equation 24, which of course applies here to the general velocity vector V'(f)).
[0064] Therefore, the expression for the velocity vector in the frequency domain has made it possible to provide an estimate of the vector U0.
[0065] However, in a more advanced embodiment:
[0066] - Apply the inverse transform from frequency to time to the general velocity vector so that it can be expressed in the time domain V'(t).
[0067] - After the duration of the direct path, search for at least one maximum value in the expression for the general velocity vector V'(t)max as a function of time, and
[0068] - Derive the first delay TAU1 corresponding to the time given the maximum V'(t)max.
[0069] Furthermore, in this embodiment, one can:
[0070] - Estimate the second vector U1 as a function of the values of the normalized velocity vector V' recorded at time indices t = 0, TAU1, and 2xTAU1, so as to define vector V1 such that:
[0071] V1=V'(TAU1)-((V'(TAU1).V'(2.TAU1)) / ||V'(TAU1)||2)V'(0),
[0072] The vector U1 is then given by the following formula: U1=V1 / ||V1||.
[0073] Then:
[0074] - It is possible to determine the corresponding angles PHI0 and PHI1 of the first vector U0 and the second vector U1 relative to the wall, such that:
[0075] PHI0 = arcsin(U0.nR) and PHI1 = arcsin(U1.nR), where nR is a unit vector perpendicular to the wall, and
[0076] - As a function of the first delay TAU1, the distance d0 between the source and the microphone is determined by the following type of relationship:
[0077] d0=(TAU1xC) / ((cosPHI0 / cosPHI1)-1), where C is the speed of sound.
[0078] Furthermore, the distance z0 from the source to the wall can be determined by the following type of relationship:
[0079] z0=d0(sinPHI0–sinPHI1) / 2.
[0080] Therefore, we can determine all parameters related to the source location (e.g.) Figure 1 This is in the case of a single wall, but the model can be extended to cases with multiple walls.
[0081] Therefore, in embodiments where the space includes multiple walls:
[0082] - Apply the inverse transform from frequency to time to the general velocity vector so that it is expressed in the time domain V'(t) as a series of peaks (corresponding to the first scheme in Equation 39b in Appendix).
[0083] - Among the series of peaks, the peaks associated with reflections on the walls in the multi-faceted walls are identified, each identified peak having a time abscissa, which is a function of the first delay TAUn of the acoustic path caused by reflections on the corresponding wall n relative to the direct path.
[0084] - As a function representing each first delay TAUn, the first vector U0, and each second vector Un generated by reflections on wall n, at least one parameter is determined in the following terms:
[0085] *Direction of the direct path (DoA)
[0086] *The distance d0 from the source to the microphone, and
[0087] *At least one distance zn from the source to the wall n.
[0088] from Figure 5B It can be seen from this that Figure 5B This is an example applied to a "traditional" velocity vector, but it can also be applied to general velocity vectors. After the inverse transform (frequency versus time), the expression for the velocity vector (traditional and general) exhibits a series of peaks, which is also... Figure 2For pedagogical purposes, it is explained that the maximum value is reached for multiple values of the aforementioned delay (TAU1, 2TAU1, etc.; TAU2, 2TAU2, etc.) between the direct path and the path generated by at least one reflection from the wall, as well as for combinations of these delays (TAU1+TAU2, 2TAU1+TAU2, TAU1+2TAU2, etc.).
[0089] These peaks can then be used to specifically identify peaks at least related to reflections on wall n, and thus these peaks have multiple time x-coordinates (x1, x2, x3, etc.) for the delay TAUn associated with that wall n.
[0090] Since combinations of different delays complicate the identification of simple delays (TAU1, TAU2, TAU3, etc.) and the existence of associated walls, a first portion of peaks with the smallest positive time abscissa can be pre-selected to identify each of these portions as a peak associated with reflections on the wall (thus avoiding combinations of different delays TAU1+TAU2, 2TAU1+TAU2, TAU1+2TAU2, etc., that could appear after the first peak). However, this implementation assumes that the aforementioned causality criterion is met (otherwise, "secondary" peaks could also be obtained through combinations of delays and negative multipliers, thus their combination with "positive" delays would ultimately yield small positive time abscissas).
[0091] Therefore, ideally, the peak associated with the reflection on wall n may have a time abscissa that is a multiple of the delay TUn associated with wall n, and the first portion of the peak with the smallest positive time abscissa is pre-selected to identify the peak associated with each of the portions with a single reflection on the wall.
[0092] In embodiments where the signal acquired by the microphone is in the form of continuous samples, it is more common to apply a weighted window that decreases exponentially over time to these samples (as will be referred to below). Figure 5A (See further).
[0093] Furthermore, this window can be placed at the very beginning of the sound's starting point (or even just before it). This avoids the difficulties associated with multiple reflections.
[0094] The application of this weighted window makes it possible to obtain a first estimate with smaller deviations for parameters U0, d0, etc., due to the use of the velocity vector expression in the time domain, especially when dealing with "traditional" velocity vectors, such as in the context of the first iteration of the method. In fact, in some cases where the cumulative amplitude of the reflection is greater than the cumulative amplitude of the direct sound, the estimates of the above parameters may be biased. This is especially true when the negative time abscissa in the time expression of the velocity vector ( Figure 5B These situations can be detected when a peak is observed at the top of the curve. The application of the above-described type of weighted window allows these peaks to be returned to a point such as... Figure 5B The bottom curve shows the positive x-axis and provides a smaller estimate of the bias.
[0095] However, it should be noted that this implementation is optional, as using a general velocity vector instead of a "traditional" velocity vector already allows for nearly unbiased estimation of parameters U0, d0, etc., including in this case. However, this approach can be performed, for example, in the first iteration of the method using a "traditional" velocity vector, or even in the second case where the aforementioned iterative process fails to converge.
[0096] In one embodiment, the weights q(f) can be further iteratively applied to the velocity vector in the frequency domain (general or conventional) according to an expression of the following type, with each weight associated with a frequency band f:
[0097] q(f)=exp(-|Im(V(f)).m| / (||Im(V(f))||),
[0098] Where Im(V(f)) is the imaginary part of the velocity vector (traditional or general, simply denoted as "V(f)"), and m is a unit vector perpendicular to the plane defined by vector U0, and also a unit vector perpendicular to the wall (usually...). Figure 1 The Z-axis will be discussed in further detail below.
[0099] Such an implementation makes it possible to select the most available frequency band to determine the above parameters.
[0100] The present invention also relates to an apparatus for processing audio signals, including processing circuitry arranged to implement the above-described method.
[0101] As an explanation, Figure 4 The diagram schematically illustrates a processing circuit that may include:
[0102] - An input interface IN for receiving the signal SIG acquired by the microphone (which may include several piezoelectric plates to form these signals, for example, in a surround sound context).
[0103] The processor PROC works in conjunction with the working memory MEM to process these signals, particularly to develop expressions for general velocity vectors in order to derive the required parameters d0, U0, etc., whose values can be passed through the output interface OUT.
[0104] Such a device could take the form of a module for locating sound sources in a 3D environment, connected to a microphone (acoustic antenna or some other type). Alternatively, it could be an engine that renders sound in augmented reality based on the given location of the source in virtual space (containing one or more walls).
[0105] The present invention also relates to a computer program comprising instructions for implementing the above-described method, when these instructions are executed by a processor of a processing circuit.
[0106] For example, Figure 3A , 3B Figures 7 and 8 show examples of possible flowcharts for the algorithm of this procedure.
[0107] According to another aspect, a non-transitory computer-readable storage medium is provided on which such a program is stored.
[0108] In the detailed description below, the general velocity vector and the "conventional" general vector are specified by the same term "velocity vector" without distinction, having the same V notation (V(f); V(t)), particularly in the equations given in the appendix. When it specifically refers to the general velocity vector, the vector is explicitly specified by the term and expressed as V' (V'(f); V'(t)). Until... Figure 5B The first part of the description reviews the basic principles in the form used in document FR1911723, and includes the equations given in the appendix. Attached Figure Description
[0109] More generally, other features, details, and advantages will become apparent after reading the following detailed description and analyzing the accompanying drawings, in which:
[0110] Figure 1 [ Figure 1 The various parameters involved in locating a sound source according to one embodiment are illustrated in a graphical manner.
[0111] Figure 2 [ Figure 2 The diagram illustrates the various continuous peaks exhibited by the time expression of the velocity vector after the inverse frequency-to-time transformation (“IDFT”).
[0112] Figure 3A [ Figure 3A This illustrates the initial steps in the algorithmic processing for determining relevant parameters U0, d0, etc.
[0113] and Figure 3B [ Figure 3B It shows Figure 3A The processing steps continue.
[0114] Figure 4 [ Figure 4 The device within the scope of the invention is illustrated schematically according to one embodiment.
[0115] Figure 5A [ Figure 5A The diagram illustrates a weighted window of a sample of a signal acquired according to one embodiment, the weighted window decreasing exponentially over time.
[0116] Figure 5B [ Figure 5B The time expressions after comparing the IDFT of the velocity vectors are as follows:
[0117] - Case where the samples were not pre-processed using a weighted window (top curve),
[0118] -And cases where processing is done through a window (bottom curve).
[0119] Figure 6A [ Figure 6A ]、6B[ Figure 6B ] and 6C[ Figure 6C [Indicates when the following reference] Figure 7 The overall appearance of the relevant peaks present in the time expression of the general velocity vector V'(t) when the iteration of the described method is repeated.
[0120] Figure 6D [ Figure 6D The form of the reference component D(f) appearing in the denominator of the general velocity vector V'(f) expression during several successive iterations of this method is illustrated highly schematically and graphically.
[0121] Figure 7 [ Figure 7 The steps of the iterative method within the scope of the invention, according to the embodiments given herein by example, are illustrated schematically. Detailed Implementation
[0122] The velocity vector can be calculated in a manner known on its own. However, certain parameter settings can be suggested to improve the final result.
[0123] Typically, the spectrum B(f) of the ambient sound signal is first obtained through a series of time frames b(t) of short-term Fourier transform (or STFT), which are usually overlapping (e.g., overlapping / addition). Here, for the four components, the order of the surround sound components can be m=1 (however, calculations are applicable to higher orders without loss of generality).
[0124] Then, for each time frame, the velocity vector is calculated for all frequency samples as the ratio of the directional components X(f), Y(f), and Z(f):
[0125] - For the conventional velocity vector, the omnidirectional component W(f) (Equation 6 in the Appendix), or
[0126] - For a general velocity vector, the reference component D(f) replaces W(f) in equation 6 of the appendix.
[0127] It is also conceivable to introduce time smoothing or merging through weighted summation, as described below.
[0128] Using such ratios (X(f) / W(f), Y(f) / W(f), Z(f) / W(f); X(f) / D(f), Y(f) / D(f), Z(f) / D(f)), if the spectral composition of the audio signal does indeed excite a large number of useful frequencies (e.g., over a wide bandwidth), then the characteristics of the source signal are essentially removed in order to highlight the characteristics of the vocal tract.
[0129] In the above applications, we can consider the case of a sound source emitting a signal s(t) with stable characteristics (in position and radiation, at least over several consecutive frames), in a stable acoustic environment (reflecting walls and objects, which may cause diffraction, etc., and thus are responsible for what is commonly referred to as the "room effect," even if not in an actual "room"). These signals are picked up by surround sound microphones. The surround sound signal b(t) is generated by spatial encoding of different versions of the signal s(t) along their direct and indirect paths, a phenomenon known as the "channel effect." This results in the signal being convolved with the spatial impulse response h(t), where each channel (or dimension) is associated with a surround sound component, as shown in Equation 8 in the appendix.
[0130] This impulse response is called the SRIR "spatial room impulse response," and is typically represented as a series of time peaks:
[0131] - The first peak occurs at time t = TAU0 (propagation time), corresponding to direct sound.
[0132] The second peak at t = TAU1 corresponds to the first reflection.
[0133] wait.
[0134] Therefore, we must be able to read the direction of arrival of these wavefronts in these peaks, with the vector u given in equation 9-1 in the first approximation. n The expression for the spatial impulse response. In practice, the spatial impulse response is unknown data, but here we illustrate how some of its characteristics can be indirectly obtained by calculating the velocity vector based on the surround sound signal b(t).
[0135] To emphasize this point, we first describe the relationship between the impulse response h(t), the transmitted signal s(t), and the surround sound signal b(t) within a selected observation time interval (Equation 9-2). Specifically, this expression assumes that there is no measurement noise or other sound sources whose signals will be picked up directly or indirectly within the considered time interval. Therefore, the entire direct and indirect signals of the sources are captured within this time interval.
[0136] We show that the velocity vector obtained by performing a Fourier transform over the entire time interval is the only characteristic of the spatial impulse response. This so-called LT transform (because it is "longer" than the STFT) converts b(t), s(t), and h(t) into B(f), S(f), and H(f) according to Equation 10. This time support can correspond to a time window extended over several consecutive signal frames.
[0137] Then, the expression for the velocity vector calculated in Equation 11 is derived from the convolution expression computed in the frequency domain. This expression, Equation 11, becomes a characteristic of the channel (in other words, a characteristic of the room effect), rather than a characteristic of the transmitted signal, provided that it has non-zero energy (in fact, detectable) for each frequency f within the considered time period.
[0138] In practice, as mentioned earlier, a common approach is to perform time-frequency analysis per frame, where each short-term Fourier transform is applied to a time window. In principle, time windowing cannot verify that the observed signal comes entirely and solely from the convolution product of Equation 9. This means that, strictly speaking, the velocity vector cannot be written in a form that only represents the vocal tract (as shown on the right-hand side of Equation 11). However, in this paper, within the context of this description, as many approximations as possible are made (Equation 20, presented below), while taking advantage of the short-term analysis presented below.
[0139] In the subsequent steps, we look for a series of energy peaks that characterize the direct path of the signal emitted from the source and received by the microphone, followed by the first reflections on one or more walls, provided these reflections are identifiable. We can then focus on the initial characteristics of the spatial impulse response, namely, the first time peak, from which we infer the direction of the direct sound, and subsequent time peaks that may be characteristics of the first reflection.
[0140] To this end, we examined the effect of interference between the direct sound and at least one reflection on the complex velocity vector expression in order to estimate the relevant parameters used to define the location of the sound source.
[0141] As presented in Equation 12, for the start of the impulse response, we introduce a simplified model combining a direct path (n=0) with n specular reflections (n=1, ..., n), where gn 、TAU n and u n These are the attenuation, delay, and direction of arrival of the wave arriving at index n (the nth reflection) of the microphone system. In the following text, for simplicity, but without limiting generality, we consider the delay and attenuation relative to the direct sound, which is equivalent to setting the terms of Equation 13 for n = 0.
[0142] The corresponding frequency expression is given in equation 14, for the special case of γ0 = 1 for direct sound. Of course, for any n greater than 0, the variable γ... n They are all functions of frequency f.
[0143] Therefore, if we ignore the latter part, the frequency expression of the surrounding sound field is given by expression equation 16.
[0144] The short-term velocity vector is then represented by a regularized version of Equation 17 or Equation 18 with a non-zero EPSILON term to avoid (almost) infinite values when the component in the denominator is (almost) zero. In Equation 17 or Equation 18, the component W (specific to the conventional velocity vector) can be replaced by the reference component D to represent the general velocity vector. In fact, in general, D replaces W in the denominator, and the expression for the conventional velocity vector V corresponds to the specific case of D = W. However, for ease of illustration, the notation related to the specific case of D = W is presented in the first equation in the appendix; these notations are given for the conventional velocity vector, but can be easily converted to the general velocity vector by remembering that D is used to replace W.
[0145] Short-term analysis makes it possible to observe the frequency fingerprint (hereinafter referred to as "FDVV") over time and according to the dynamic evolution of the source signal; this is a characteristic of the submix of the inner wavefront of the spatial impulse response. The characteristic submix ("smx" of the submix) for a given observation is modeled in both the time and frequency domains according to Equation 19.
[0146] In the scheme described below, we attempt to characterize the frequency fingerprint FDVV as a function of the submatrix H using the approximation expressed in Equation 20. smx The implicit model is generated (and is usually not an exact equation except in special cases). In practice, the expression in equation 20 is given for the conventional velocity vector, and can be obtained through matrix D. Θ To adapt a general velocity vector, a matrix-based vector H is used instead of H_W, as described below.
[0147] Especially at the signal start time, the implicit model h smx (t) appears to be similar to the space impulse response h, at least in terms of relative wavefront direction and delay. earlyThe beginning of (t). Due to the implicit relative gain parameter g n Due to the influence of signal time windowing and dynamic characteristics, they may not necessarily appear consistent with those of impulse responses. This study primarily focuses on direct waves (providing the DoA) and one or more early reflections, mainly considering cases where observations exhibit specific characteristics.
[0148] For clarity, a processing example is presented below, considering only a single reflection, to estimate the conventional velocity vector in the frequency domain. The case for a general velocity vector will be discussed later. Here we discuss the case of simple interference (essentially interference between the direct sound and the first reflection), and we demonstrate how to determine the required parameters by highlighting the specific spatial frequency structure, looking not only at the real part but also the imaginary part of the velocity vector. In fact, the surround sound field is written according to Equation 21, and we derive the velocity vector from it according to Equation 22. From this expression, it can be seen that as the frequency propagates in the relevant audio spectrum as shown in Equation 23, the real and imaginary parts propagate in parallel segments (affine and linear, respectively) in 3D space. The affine segment (real part) lies on a line containing unit vectors U0 and U1 pointing to the direct and indirect waves, respectively, and these two segments are orthogonal to the mid-plane of these two vectors (therefore, the imaginary part of the vector remains itself, as it lies on the linear segment). Furthermore, assuming a homogeneous distribution of phase differences between waves (and thus a representative scan of frequencies), statistical calculations show that the average of the real parts of the velocity vectors is equal to the vector U0 represented in Equation 24, and the maximum probability is the average of U0 and U1 weighted by the various amplitudes of the waves, as represented in Equation 25. Therefore, DoA detection based on the maximum probability is affected by system angular bias, giving an intermediate direction between the direct sound and its direction. Equation 23 shows that this spatial scan is performed at a frequency period equal to the reciprocal of the delay TAU1 between the two waves. Therefore, when this spatial frequency structure can be observed, it is possible to extract the directions U0 and U1, as well as the delay TAU1, from the observations. Another way to estimate these parameters in the time domain is further given below (regarding...). Figure 2 (Description).
[0149] By pre-sensing the orientation of the reflective surface relative to the microphone's reference system, we can infer the absolute distance *d* of the source relative to the microphone from the estimates of U0, U1, and TAU1, and may even infer their elevations. In fact, as... Figure 1As shown, the distance from source S0 to microphone M is denoted as d0, and its mirror image S1 is denoted as d1 relative to the reflecting surface R, which is orthogonal to the plane formed by vectors U0 and U1. The three points (M, S0, S1) lie in the same plane orthogonal to surface R. Parameters still need to be defined to define the orientation (or tilt) of the reflecting plane. In the case of floor or ceiling reflection (detected because U1 points towards the floor or ceiling), one can utilize the assumption that it is horizontal and parallel to the XY plane of the surround sound microphone's reference frame. The distances d0 and d1 are then linked by relation 26, which also directly gives the distance from microphone M to the axis (S0, S1), where PHI0 and PHI1 are the respective elevation angles of vectors U0 and U1.
[0150] We also estimated the delay TAU1 of the reflected sound relative to the direct sound, which provides access to another relationship between distances, Equation 27, since their difference shows the sound path delay, with the factor c being the speed of sound.
[0151] By expressing d1 as a function of d0, this final quantity becomes the only unknown, which can be estimated according to Equation 28. We also obtain the distance from the source to the reflecting plane, i.e., its height or elevation z0 relative to the ground according to Equation 29, and the height or elevation of the microphone in Equation 30.
[0152] exist Figure 1 The example of reflection from the floor illustrates various parameters such as U0, U1, PHI0, PHI1, d1, and d0. Of course, similar parameters can be derived for reflection from the ceiling. In the same way, similar parameters can be derived for reflection from any other reflective surface R, whose orientation relative to the microphone's reference frame is known, determined by the normal n. R (Unit vectors orthogonal to surface R) are represented. It suffices to redefine the angles PHI0 and PHI1 relative to the reflecting surface R, typically PHI0 = arcsin(U0.n...). R ) and PHI1 = arcsin(U1.n R This could thus allow the location of these obstacles to be determined via a vector U1 associated with each reflection, for augmented reality applications or for robots that estimate location through acoustic detection.
[0153] At the orientation n of the reflective surface R If, initially unknown, one can estimate the wavefront parameters associated with at least two source locations where reflections from the same reflecting plane were detected through observations at different times, then the azimuth n can be fully estimated. RTherefore, we have a first set of parameters (U0, U1, TAU1) and at least a second set of parameters (U0', U1', TAU1'). Since U0 and U1 define a plane orthogonal to plane R, their vector product defines the axis of plane R, and the same applies to the vector product derived from U0' and U'1.
[0154] These individual vector products (non-collinear) together define the orientation of plane R.
[0155] However, a limitation of the model with only two interfering waves (direct sound and reflected wave) is that it is difficult to distinguish the various first reflections on the barrier. Furthermore, the space-frequency behavior of the velocity vector becomes rapidly more complex when additional reflections are introduced. In fact, the paths of the real and imaginary parts then combine in a meaningful way along several axes:
[0156] -In the parallel plane used for direct waves and double reflections,
[0157] - Or generally throughout the entire space.
[0158] When considering several reflecting surfaces, the complex spatial frequency distribution makes determining the parameters of the model overly cumbersome.
[0159] One solution to this problem is to perform time-frequency analysis, which is more selective in time (i.e., has a shorter time window) to allow for the spotting of simpler acoustic mixes during the onset of some amplitude (transient, rising signal), thus reducing the number of reflections that interfere with the direct sound in the mix present in the frame in question. However, in some cases, the delays associated with successive reflections may be too close to allow for the isolation of the first reflection's effect on its interference with the direct sound.
[0160] Therefore, a processing method is proposed below to make it possible to easily separate and characterize the effects of various disturbances. The first step involves transforming the fingerprint of the velocity vector to the time domain (or "TDVV" of "time-domain velocity vector") using the inverse Fourier transform shown in Equation 31. This has the effect of compressing the frequency-periodic effects associated with certain axes into sparser data, thus making them easier to analyze, as these effects manifest as complex shifts in the velocity vector. In fact, such a transformation results in a series of peaks appearing at regular time intervals, with the most significant peaks being easily detected and extracted (see, for example, [link to relevant documentation]). Figure 5B ).
[0161] A significant property is that, by construction (due to the inverse Fourier transform), the vector at t = 0 is equal to the average value of the velocity vector in the frequency domain (or the average value of its real part if we only consider the half-spectrum of the positive frequencies). Such observations are related to the estimation of the main DoA U0.
[0162] Starting with the frequency model of the velocity vectors of the two interfering waves (direct sound and reflected wave), we can effectively reformulate the denominator using the Taylor expansion of Equation 32. Using the conditions on x and γ1 given in Equation 32, we derive the (traditional) expression for the velocity vector, Equation 33, and, under the condition that the amplitude of the reflected wave is less than the amplitude of the direct sound (g1 < g0 = 1, which is typically the case at the start of sound), the inverse Fourier transform of this expression converges and is formulated as shown in Equation 34, where the first peak is identified at t = 0, giving U0 (the direction of the direct sound), followed by a series of peak characteristics of the interference of the reflected wave on the direct sound.
[0163] These peaks are placed at multiple times t = kTAU1 (non-zero integer k > 0) of the delay TAU1, and have exponentially decreasing magnitudes in the norm (according to the gain g1). By using this conventional velocity vector, they are all associated with directions collinear with the sum and difference U0-U1, and therefore orthogonal to the intermediate plane between these two vectors, and have alternating directions (signs). The advantage of transforming the velocity vectors to the time domain is the sparse and almost instantaneous presentation of the required parameters. Figure 2 ).
[0164] Therefore, in addition to the primary DoA U0, it is also possible to determine:
[0165] -Possibly for delays of TAU1 across several different walls,
[0166] Then there is the vector collinear with U0-U1, which is normalized to a unit vector n, which can be used, for example, by equation 41 (given for conventional velocity vectors), for...
[0167] - Infer that U1 is symmetric to U0 with respect to their intermediate plane, and
[0168] - Optionally, the attenuation parameter g1 (this can be modified by time-frequency analysis parameters, specifically by the shape of the analysis window and by its time position relative to the observed acoustic event. Therefore, estimating this parameter is not very useful in the application context considered here).
[0169] Observing the following time peaks makes it possible to check whether they are essentially consistent with the same series (multiple delays TAU1, multiple delays TAU2, etc.), and therefore whether they have the same interference characteristics; otherwise, it is necessary to determine the presence of multiple reflections, for example.
[0170] Below, we distinguish the case with “favorable conditions,” where, in the case of N reflections, we apply Taylor expansion according to Equation 35 to give the conventional velocity vector, provided that the sum of γ over N in Equation 35 remains less than 1. The Taylor series of the denominator in the initial expression can be rewritten using the polynomial law of Equation 36, which makes it possible to reorganize the expression of the conventional velocity vector V model into a sum of several series, where the “cross series” is denoted by SC in Equation 37. For the general velocity vector V', we find a slightly different equation in Equation B2 at the end of the appendix, which is also denoted as Equation 35b, as it corresponds to Equation 35, but is given here for the general velocity vector. Furthermore, at the end of the appendix, equations specific to the general velocity vector V' are given, and their correspondence with the previously written equations specific to the conventional velocity vector is indicated by the “b” after the equation number (Equation xxb).
[0171] Under the conditional equation 38 (equation B3 = 38b for the general velocity vector) for a regular velocity vector and any frequency f, we derive the following time series equation 39 (equation B4 = 39b for the general velocity vector) via inverse Fourier transform, where the sequence has a combinational delay SARC. We identify the first peak at t = 0, which gives U0 (the direction of the direct sound), and then for each reflection, a series of peak characteristics of the interference of that reflection on the direct sound. For example, in Figure 2 In this model, these peaks are placed at consecutive positive time axes such as TAU, 2TAU, 3TAU, etc., which are multiples of the delay TAU between the reflection on the wall and the direct path.
[0172] Then a series of characteristics of interference between several reflections on the wall and direct sound emerges (for a larger time x-axis), where delay is other combinations of its various delays (with positive integer factors).
[0173] In fact, Figure 2This sequence is illustrated in a simplified case where two reflections interfere with the direct sound. Each marker (circle, cross, and diamond, respectively) contributes to the temporal fingerprint TDVV as a function of the time x-axis, through its y-axis indicating vectors U0, U1, and U2 (characteristics of the direct sound, the first reflection, and the second reflection, respectively). Thus, we can see that the reception of the direct sound is characterized by a first peak of amplitude 1 at time zero, as indicated by the circle. The interference of the first reflection (delayed TAU1) on the direct path causes a first series of peaks at TAU1, 2xTAU1, and 3xTAU1, etc., which are marked here by a cross at one end and a circle at the other end (top-bottom). The interference of the second reflection (delayed TAU2) on the direct path causes a second series of peaks at TAU2, 2xTAU2, and 3xTAU2, etc., marked here by a diamond at one end and a circle at the other end. Then there are the elements of the “cross series,” which are the interferences between reflections (the first delay: TAU1+TAU2, then 2TAU1+TAU2, then TAU1+2TAU2, and so on). The expressions for these cross series are accessible, but they would be very long to write in general, and are not explicitly described here for the sake of brevity, especially since they are not needed to estimate the relevant parameters in the process presented here.
[0174] Below, we describe the analysis of temporal fingerprints by estimating the order of parameters.
[0175] The estimation of model parameters based on the computational time series is performed in a manner similar to that described earlier for the case of a single reflection. We first place ourselves in the most general case (excluding the specific case to be processed later), corresponding to the favorable situation where delays do not “overlap”: then, the above sequences do not show any temporal overlap, i.e., any identifiable peak belongs to only one of them. Therefore, by noting temporal peaks via an increasing delay starting from t=0, any newly detected peak with a delay TAUnew can be attributed to the already identified sequence, or the start of a new sequence can be defined. In fact, considering a set of delay properties of the already identified reflections, the first case is detected if there is a positive or partially zero integer k, generating TAUnew according to Equation 40; otherwise, we fall into the second case and increase the set of identified reflections by introducing a new delay TAUN+1, associated with the direction that can be estimated in a manner described for the case of a single reflection.
[0176] In practice, it may not be necessary to attempt to explain the numerous time peaks. We limit ourselves to the first observed peak, especially since they are the easiest to detect because their amplitude (or magnitude, in absolute value) is larger than the ones below. Thus, there are cases where the delay has a common multiple but a high rank Ki; through the above treatment, Kj (or not low) can be analyzed as a function of amplitude.
[0177] As long as the sum of the moduli of the implicit gain gn (n>0) is less than 1 (for the conventional velocity vector, Equation 38), the inverse Fourier transform (Equation 31) gives a unidirectional temporal fingerprint that evolves in positive time.
[0178] On the other hand, if the sum of the moduli of the implicit gain gn (n>0) is greater than 1, the inverse Fourier transform gives a "bidirectional" time fingerprint TDVV, and the sequence usually progresses in both positive and negative time directions. Figure 5B (The top curve in the diagram is for illustrative purposes). For example, when the amplitude of the direct wave is less than the sum of the amplitudes of the waves reflected from one or more barriers, a situation may arise where one or more reflection gains will be greater than 1. In this "adverse case," the dominant peak at time zero no longer corresponds precisely to the vector u0, but is a mixture of the latter, where a more or less significant proportion of the vector specifies the direction of reflection. This results in a positional bias (in "Estimating DoA"). Another symptom is that the norm of the dominant peak is often different from 1, typically less than 1.
[0179] This invention proposes a robust method, particularly in this case. It proposes adjusting the expression for the velocity vector by giving spatial selectivity for DoA in the D component that appears in its denominator instead of the usual omnidirectional component W.
[0180] By giving the directional scheme in the reference component D, the relative attenuation associated with each reflection of the exponent n is assigned to the denominator via the factor BETAN, while the total factor NU0 is calculated (Equation B1 in the appendix). This results in the expression for the general velocity vector of the model with N reflections being given by Equation B2 = 35b, as introduced above for the general velocity vector case. Note that the condition for the Taylor series expansion, as shown on the right-hand side of the equation, is now given by Equation B3 = 38b. It is understandable that this condition is more easily satisfied in many cases due to the additional attenuation factor BETAN. Recall that the index of the model satisfying this condition and thus has causal properties throughout the time series. Under the condition satisfied at all frequencies, the general velocity vector model in the time domain is identified in the form of Equation B4 = 39b, which is shown (in a manner similar to that of the conventional velocity vector case in Equation 39):
[0181] The first peak at -t = 0 indicates that this direction gives the DoA, and the vector U0 is obtained through the normalization equation B5.
[0182] - The number of time series corresponds to the number of reflections, each of which relates to the interference between the reflections and the direct sound, and the values of the vectors observed at regular intervals TAUn are reported in equation B6.
[0183] - and sequences with combined delays, denoted as SARC, which are not used in subsequent estimation processes.
[0184] Starting with equation B6 at the end of the appendix, we preserve the specific relationship between two consecutive vectors of the sequence, specifically the first two vectors V'(TAUn) and V'(2.TAUn). Thus, equation B7 shows the factor (-Gn / BETAn), denoted here as "RHO", whose equation B8 provides an estimate as a scalar product of the first two vectors of the same sequence, V'(TAUn) and V'(2.TAUn), divided by the square norm of the first. By reintegrating the RHO factor into equation B6, the resulting equation B9 can be rearranged to give equation B10. The right side of this equation shows the vector Un (specifically, vector U1, if we are concerned with the first reflection and its associated series), affected by the positive factor NU0 / Gn (except in possibly rare cases, such as reflections with phase reversal): thus, it can be obtained by normalizing the left side V'(TAUn) - RHO.V'(0).
[0185] Furthermore, the total factor NU0 may incorporate other influencing factors besides the reference directivity, such as the overall reduction in amplitude that may be caused by the frequency bandwidth limitation of the signal source and / or its partial masking by noise (although the latter effect is generally more difficult to model). It is worth noting that, ultimately, the direction of the vector U1 (or more generally Un) can be estimated in the same way, and for any reason related to this overall reduction in amplitude NU0.
[0186] It should also be noted that this estimation model is also applicable to conventional velocity vectors (in which case only BETAN = 1 needs to be considered).
[0187] The following is a description of a practical example of an implementation that uses a general velocity vector to determine parameters (e.g., DoA).
[0188] In this embodiment, reference is made herein. Figures 6A to 6D To illustrate with point 7, the first estimate of the delay is performed using the conventional velocity vector calculated in a "normal" manner. Figure 7 Step S71):
[0189] V(f)=1 / W(f)[X(f),Y(f),Z(f)]T
[0190] For example, consider the first-order surround sound component.
[0191] In step S721, the calculations described above are performed based on the frequency expression V(f) of the conventional velocity vector until the time expression V(t) of the conventional velocity vector is estimated.
[0192] In step S731, the time expression V(t) of the conventional velocity vector is analyzed as a time series of peaks. In this analysis, we specifically determine whether it is possible to effectively identify it (with minimal disturbances) by a unidirectional time series that develops only in positive time (as a true "causal response"), as described in equations 39 and 40 in the appendix.
[0193] When analyzing the expression for the conventional velocity vector V(t) in the time domain, we find a peak structure in the negative time domain (e.g., energy or amplitude exceeding a selected threshold THR), typically as follows: Figure 6A The peak appearing on the negative abscissa means that it cannot be reasonably identified in Equation 39, and therefore the estimate of DoA given by the peak V(t=0) is biased. Furthermore, this is a sign that the convergence condition of the Taylor series in Equation 39 is not satisfied, and therefore a sign that the proportion of indirect waves mixed with direct sound in the denominator of the calculation of V(f) is too large. In the improvement proposed here, this proportion of indirect waves is reduced by spatial filtering. This means that it is necessary to improve the directionality (which plays a role in the denominator) in the estimation of the velocity vector V(f).
[0194] Spatial filtering is then applied to the acquired surround sound data to form a beam in the direction (DoA) of V (t==0) as estimated in step S751 (step S71). In fact, although this initial estimate may be incorrect, it provides a generally accepted coarse approximation, but is sufficient to direct the beam to the direct source of the sound and attenuate reflections from sectors at greater angles.
[0195] Based on these filtered surround sound data, the modified velocity vector V'(f) in the time domain is calculated, and then V'(t) is calculated (step S781).
[0196] In the S732 test, it is determined whether a time abscissa peak less than 0 still exists in the time expression of the modified velocity vector V'(t). One can determine the signal structure present in the negative time abscissa (e.g., estimated as energy). Figure 7The value is denoted as "NRJ" in Chinese, and is given, for example, by the integration of the signal over negative time, whether it remains greater than the threshold THR, such as... Figure 6B As shown in the example, even if it is possible to observe relative to the previous Figure 6A Improvements.
[0197] In this case, the method can be repeated (S752) using a coarse estimate of the previously obtained DoA(n) to determine a reference component D(f) (denoted as D(n) of iteration n of the method in step S762), the directionality of which allows for a more selective representation of the direction of the direct sound than its estimate D(n-1) in the previous iteration, and replaces (S772) the latter D(n-1) in the estimate of the general velocity vector V'(f), then V'(t) in step S782. Therefore, the directionality of the reference component more selectively “captures” the estimated direction of the direct sound compared to the reference component in the previous iteration. In this embodiment, we do not necessarily have a surround sound order greater than 1, but we can adjust the orientation and shape of the directionality to better capture the direct sound, thereby capturing, for example, fewer reflections.
[0198] Therefore, this method can be repeated until the peak at the negative time is below the chosen threshold THR in either amplitude or energy, such as... Figure 6C As shown.
[0199] Therefore, as the iteration proceeds, the increased selectivity for the direct sound has a continuous effect on the components in the denominator of the velocity vector (traditionally, then generally), in its first-order expression. Thus, in Figure 6D In this context, we transition from the omnidirectional W component in the form of a spherical (light gray) shape to the more selective component D(1) in the form of a dark gray supercardioid in this case, and then to D(2) in the form of a narrower supercardioid with a very dark gray shape.
[0200] Return to Figure 7The method shown, but in more detail, begins with the first step S71, which constructs a conventional velocity vector V(f) in the frequency domain with an omnidirectional component W(f) in its denominator. In step S721, its expression V(t) in the time domain is estimated. Then, if, in test S731, a signal structure representing the time expression of the conventional velocity vector V(t) with a peak is identified such that the energy NRJ of this signal structure at the negative time abscissa (t < 0) remains below a fixed threshold THR (no arrow), then the current acoustic conditions allow for the direct derivation of the unbiased DoA from the conventional velocity vector. In this case, as described above, the parameters DoA, U0, U1, etc., can be determined in step S741. Otherwise (the arrow indicates exiting test S731), the direct estimation of DoA from the conventional velocity vector is biased, and at least one iteration (n = 1) must be performed to refine the velocity vector to determine the general velocity vector, as described below.
[0201] Based on the DoA estimation, even with a bias (obtained in step S751), the reference component D(1) in the frequency domain is estimated in step S761 to replace the omnidirectional component W(f) in the expression of the current “general” velocity vector V'(f) in step S771. In step S781, the time expression V'(t) of the general vector is estimated in order to determine in test S732 whether significant energy (greater than the threshold THR) is retained in the signal structure of the expression V'(t) at the negative time abscissa. If this is not the case (no arrow indicating exit from test S732), the process can be stopped at the first iteration by giving parameters such as DoA to step S742. Otherwise, the process is repeated by updating the iteration index n of the process in step S791 (here, step S79x is indicated by the index n increasing relative to the iteration of the process (step S793)) or by determining the termination of the process S792-S794.
[0202] As described above, based on the coarse DoA estimated in the previous iteration (step S752), a new reference component D(n) is estimated in step S762 to replace the old reference component D(n-1) in the denominator of the general velocity vector V'(f) in step S772. In step S782, its expression V'(t) in the time domain is determined according to this new expression of the general velocity vector V'(f) in the frequency domain. The comparison of its signal structure (energy relative to the threshold THR) in test S733 is repeated to determine whether the new DoA that can be estimated from this is biased. If this is not the case (no arrow exits test S733), the parameters, especially DoA, can be obtained in step S743, which is as follows: Figures 6A to 6CAfter three iterations in the illustrative example. Otherwise (as indicated by the arrow exiting test S733), the process must be iterated again from step S752 using the last estimated DoA, even if it is coarse and possibly biased.
[0203] The following describes an example of a possible calculation of the reference component D(f) based on the previously estimated DoA. In a form such as the equation shown in the appendix, component D... Θ (f) Typically determined by the matrixing of the surround sound components (or by vector D) Θ Weighted sum) Derivation:
[0204] D Θ (f)=D Θ .B(f), where
[0205] -B(f) is the signal vector describing the surrounding sound field in the frequency domain, for example... in Carrying signal S(f) and originating from a unit vector In the case of plane waves in the described direction, for a mixture of direct and indirect waves, we have in It is the spherical harmonic coefficient vectors, and
[0206] -D Θ It can be a vector of the "steering vector" type, causing beamforming to be typically oriented in a specific direction Θ (or it can be a unit vector). (specified), making
[0207]
[0208]
[0209] By defining "steering gain", we can achieve the following:
[0210]
[0211] Up to the first order, spherical functions It is axisymmetric; apart from orientation, the available degrees of freedom only affect the ratio between the positive and negative lobes (if any). Starting from the second order, according to vector D... Θ The coefficients mean that the main lobe (which may be in the target direction) is not necessarily symmetrical, and the side lobes can have more varied shapes and orientations. Therefore, although D is used... Θ The notation is different, but beamforming is not parameterized solely by the principal direction Θ of the target.
[0212] In the special case of directionality formed by axisymmetry, its form is: DΘ =Y(Θ).Diag(gbeamshape)
[0213] Gain It is then expressed as a scalar product. polynomial P beamshape (·)(Variations of Legendre polynomials):
[0214]
[0215] diagonal coefficient g beamshape Consider:
[0216] - On the one hand, the choice of environmental sound coding protocol, therefore the spherical harmonic function Calculation;
[0217] - On the other hand, by selecting these diagonal coefficients (e.g., the fineness and proportion of the main lobe and the sublobe), the selected beam shape is refined, thereby obtaining a standard for processing the beam shape.
[0218] For these aspects, refer to The paper by Daniel (2000) is useful, especially pages 182–186 and Figure 3.14, in which the proposed tools for spatial decoding are directly applicable to the directional construction from surround sound signals, as shown here for the D reference component.
[0219] Based on this form, note that a gain coefficient may be defined as g. beamshape The function gives selectivity to the component D(f) expressed in the context.
[0220] Refer again Figure 7 It should be noted that, depending on the acoustic conditions, a general velocity vector V'(t) with good "causality" may not be found in the time domain, thus resulting in a poor DoA estimate. See again... Figure 7 For example, when the shape of the signal, which represents the general velocity vector V'(t) in the time domain, is no longer improved, a termination criterion is added to exit the iteration of the process. Therefore, if at the end of the previous iteration, in test S733, the energy of the signal is still greater than the threshold THR (t < 0) in the negative time abscissa, and if the iteration of the process does not (or no longer) improve the estimate of the general velocity vector V'(t), this can be represented by the following two:
[0221] - Signal energy exceeding the threshold THR during negative time, and
[0222] - Signal energy that does not decrease from one iteration n-1 to the next iteration n (the "No" arrow indicating exit test S792).
[0223] The iteration of this process can then be stopped at step S794.
[0224] Otherwise (the arrow indicates the exit from test S792), this process can be performed in step S793 for subsequent iterations, starting with incrementing the iteration counter n.
[0225] In order to minimize the occurrence of the situation where the processing must stop in step S794 due to the lack of convergence of the unbiased DoA solution, in this “worst-case” acoustic situation, the teachings of the aforementioned document FR1911723 may be adopted to allow, for example, the isolation of the best frame to increase the chance of unbiased determination of the solution (e.g., the frame that initiates the signal).
[0226] In fact, as described in FR1911723, depending on the relative importance of the problem, one can assess the extent to which vector U0 provides a reasonable estimate (weak bias) of the DoA, thus providing a confidence factor for the estimate and making it possible to preferentially retain estimates for certain frames. When the risk of estimation bias proves excessive, frames least exposed to the problem can be selected, as referenced below. Figure 3A and 3B As described.
[0227] The embodiments described below can then be applied to the estimation of conventional velocity vectors, in particular, for example, as referenced above. Figure 7 The processing described is applied during the first iteration. This has already been described in FR1911723, therefore, we will now review this processing applied to the conventional velocity vector.
[0228] Therefore, one can observe the first peak of a given room through frequency analysis of time subframes. The frame in which the signal starts (rising energy, transient, etc.) is located is the frame in which observation is allowed to involve only the acoustic mix of the earliest wavefront: the direct sound and one or more reflections (therefore, according to Equation 38, the above "sum of γ" remains less than 1).
[0229] For frames containing the signal initiation point, the time window for frequency analysis may be (potentially dynamically) adjusted, for example, by giving it an asymmetrical and generally decreasing shape, such that the "bumps" of the window give more weight to the rising edge of the signal (the initiation point, transient), and thus give gradually decreasing weight to the direct sound (e.g., in an approximately exponential manner, but this is not necessary). Therefore, the amplitude of the later wavefront relative to the earlier wavefront is artificially reduced, and the convergence condition of the Taylor series is approximated, thereby ensuring unidirectional time expansion.
[0230] Below is an example of a decreasing exponential time window, applied to the signal being analyzed to achieve favorable conditions for the analysis of the resulting temporal fingerprint without substantial deviation in the estimation of the wave's direction of arrival. We perform an operation effective from time t0, which for convenience is designated as time 0, and preferably corresponds to the moment when the signal begins before silence, as shown in Equation 42, where α > 0. Furthermore, by reorganizing the convolutional forms involving s(t) and h(t), we find the form of Equation 43.
[0231] Then, equation 44 involves the properties of exponential functions, which is a reasonable choice, thus yielding the form given in equation 45, which is equivalent to establishing equation 46.
[0232] Therefore, if we model the impulse response by adding a set of specular reflections to the direct sound, we obtain equation 47.
[0233] Therefore, if the sum of α is greater than or equal to 1 (with the possibility of a “two-way series”), it is always possible to determine the attenuation factor α, thereby making the sum of gains “fitted” (Equation 48) less than 1.
[0234] We then observed that the temporal fingerprint is inherently unidirectional, which becomes apparent only at the peak in the positive time phase after applying a decreasing exponential window. Figure 5B (The lower part). We also observed that, in practice, the energy of the observed signal decreases rapidly with exponential growth, and after a relatively short truncation time, the effect of the signal truncation on the estimated value becomes negligible. In other words, in the short term, we gain the advantage of long-term analysis, which includes the entire excitation signal and its reverberation. Indeed, the observed “TDVV” conforms to the interference model without errors due to signal dynamics. Therefore, this windowing weighting has a dual nature, ideally allowing the acquisition of a usable temporal fingerprint.
[0235] In practice, when the reflection amplitude is unknown beforehand, the attenuation α must be determined. Ideally, a compromise should be struck between a value low enough to ensure the unidirectionality of the time fingerprint and a value that is not too low, to avoid reducing the chances of detecting and estimating indirect waves. For example, this value can be determined based on the attenuation factor α. EXP Through duration t EXP To determine the duration t EXP Physically, this represents the observed phenomenon (usually 5 ms), such that α = -(log a) EXP ) / t EXP .
[0236] Iterative processing (e.g., via bisection) can be implemented to adjust the attenuation value. Above the threshold attenuation value, if the obtained temporal fingerprint is detected as bidirectional and therefore may have a bias vector U0, the analysis is repeated with a stronger attenuation; otherwise, at least an estimate of U0 is used, and if the lower peaks are not obvious (because they decrease with attenuation), the analysis is repeated with the intermediate attenuation between the first two peaks, and so on, if necessary, until the vector U1 can be estimated.
[0237] Nevertheless, the exponentially decreasing window scheme may be sensitive to disturbances, especially at the beginning of the window, where they are significantly amplified. Disturbances other than noise might simply be the reverberation of the source itself at the start of windowing, if it was activated shortly before. Denoising processes could then be introduced to reduce this disturbance.
[0238] Typically, time windows of different shapes and / or sizes can be provided, or even overlap between windows, in order to maximize the chance of obtaining a “favorable fingerprint.”
[0239] The selected initial DFT size is typically larger than the analysis window.
[0240] Of course, this is in the context of processing digital audio signals sampled at a given sampling frequency, in the form of consecutive sample blocks (or "frames").
[0241] It may also selectively provide preprocessing for detection start points, transients, etc., followed by time-frequency denoising, for example by defining a mask (a time-frequency filter, possibly binary), to avoid introducing elements from other environmental and / or diffuse source sources into the interfering fingerprint. It is necessary to calculate the impulse response of the mask (the result of the inverse transform) to verify the mask's impact on peak analysis. Alternatively, it can be integrated into the frequency weighting of the fingerprints of the relevant frames to be stored so that a weighted average of the frequency fingerprints that might correspond to similar interfering mixes can be subsequently calculated (typically at the signal start point, inferred by estimating the delay after verifying that the relevant source has not moved).
[0242] Therefore, we continue to extract and observe peak values, for example, based on the norm |V(t)|: maximum peak value, and then, TAU1 (general), etc.
[0243] Then, we perform time fingerprint diagnostics by detecting (based on {tau_n} and V(sum(k_n.tau_n))):
[0244] - If temporal recirculation (a kind of circular "aliasing") occurs due to choosing FFT on too short a time support,
[0245] -If an asymptotic unidirectional sequence or an opposite bidirectional sequence exists,
[0246] Or if we are in a specific case of a sequence without significant decay (when the sum of gains sum(gn) remains close to 1), or even in a specific case of a retrograde sequence (when at least one implicit gain g_n>1).
[0247] Then we can:
[0248] - Assign a "good frame" or "good fingerprint" score (allowing for reliable estimation, possibly without DoA bias, as it is unidirectional), and store it.
[0249] - Perform estimation (Un), and
[0250] - Adjust the analysis upstream as needed by selecting an appropriate time window.
[0251] The analysis of time fingerprints has been described above, but frequency analysis can be performed more simply as follows.
[0252] It is easy to mathematically prove that, by construction, the peak at time zero equals the average value of the velocity vector over its entire spectrum (the real part is canceled out by Hermite symmetry), or even its real part if we only consider positive frequencies. We can estimate that calculating the inverse transform of the FDVV to estimate the DoA is meaningless if we are only interested in direct sound. However, a time-based TDVV check allows us to test whether such a DoA is reliable (the standard for both positive and increasing time evolution).
[0253] This favorable situation is more likely to be observed during the initial stage of the source signal, when the mixing is still not very complex. Generally, estimations at these moments are sufficient.
[0254] Furthermore, in practice, in the ideal model of interference wave mixing, the frequency and time fingerprints of VV are not always identifiable. This may be due to the concurrency of capturing other components of the sound field (insufficient SNR or SIR), where the source signal is not adequately or not always excited across a significant frequency range at critical moments due to a lack of transmission power. This may be related to more or less diffuse background noise (from other sound sources) and micro-noise.
[0255] Then, we can apply processing based on at least one or more of the following combinations:
[0256] -Based on the selection of time-frequency samples for signal initiation point detection using advanced algorithms.
[0257] - It is possible to smooth the velocity vector over several frames via the intensity vector (e.g., by the |W(f)| of the frame in question). 2 The average of V(f) weighted by the forgetting factor (which may be dynamic).
[0258] - The calculation of |W(f)| is performed on the selected signal initiation frame (if the extracted delays are the same). 2 The weighted average of V(f) is used to supplement the frequency fingerprint and merge the time fingerprint.
[0259] To save computation, it may be advisable to calculate TDVV only for frames that are detected as more consistent with their information, or even upstream of FDVV: for example, if a signal-initiated frame can be detected by simple processing, it may be advantageous to position the analysis window on the rising edge of the signal.
[0260] For good estimation of non-integer delays (fractional delays and their multiples in a time series), peak estimation can be considered through inter-sample interpolation and / or local frequency analysis (by isolating peaks in a time-constrained neighborhood), and the delay can be fine-tuned based on the phase response.
[0261] A preliminary selection of time peaks can be made based on the current estimate of the sequence's delay characteristics.
[0262] Therefore, the implemented steps can be summarized in one possible exemplary embodiment, such as Figure 3A and 3B As shown. In step S1, the Fourier transform (from time to frequency) of the surround sound signal is calculated, which can be in the form of a series of "frames" (continuous blocks of samples). For each transformed frame k (step S2), a dynamic mask can be applied to some frequency bands where the signal-to-noise ratio is below a threshold (some frequency bands may actually be highly noisy, for example, due to noise inherent in the microphone or other frequency bands, impairing the utilization of the signal picked up in that band). In particular, in step S3, a frequency band-wise noise search is preferentially performed on the "omnidirectional" component W, and in step S4, frequency bands altered by noise (e.g., exceeding a threshold, e.g., SNR < 0 dB) are masked (i.e. set to zero).
[0263] Then, in step S5, the velocity vector V(f) is calculated in the frequency domain, for example by Equation 6 (or in the form of Equation 11, Equation 18 or Equation 20).
[0264] Here, in an exemplary embodiment, we apply weights q(f) calculated as described below to assign more or less importance to the frequency band f. Such an implementation allows the velocity vector V(f) to be expressed in frequency bands where its evolution is significant. For this purpose, the optimal weights are iteratively calculated as functions of U0 and V(f). Therefore, returning to Figure 3AIn the algorithm processing, in step S6, various weights q(f) are set to 1. In step S7, we apply weighted q(f) to V(f) according to each frequency band, such that Vbar(f) = q(f)V(f). In step S8, U0 is determined for each frame k, such that:
[0265] U0(k) = E(Re(Vbar(f))), where E(x) is, for example, the expected value of x, and thus it is similar to the average of the real part of the estimated velocity vector Vbar(f) over all frequencies.
[0266] Of course, the first estimate of U0(k) is coarse. The weights with respect to the previously determined U0(k) are calculated using equation 49, which is based on the imaginary part of the vector V(f), and where vector m is a unit vector perpendicular to the plane defined by vector U0 and a normal vector perpendicular to the wall (e.g., ...). Figure 1 The direction z) is used to iteratively refine it. In step S9, the vector m is also iteratively estimated as a function of U0, and then the weights are calculated in step S10 using equation 49. The found weights are applied in step S7, and the estimate of U0 is refined until convergence is achieved at the output of test S11. At this stage, U0(k) has been estimated for different frames.
[0267] U1 can be derived from relations such as Equation 41 above. In the variant described here, U1 is determined by equations 50 to 52, which were previously applied in step S12 to the vector Vbar(f) found in step S7 to obtain a time-based representation V(t) of the velocity vector. (Refer to above) Figure 2 As seen, this embodiment makes it possible to identify various delays TAU1, TAU2, etc., in the case of different reflective surfaces. The first delay TAU1 is identified because it is the first peak of V(t) in the time after the instant of reception on the direct path. Therefore, in Equation 51, tmax(k) is the moment that maximizes the modulus of V(t)k calculated for frame k.
[0268] Test S13 checks for each frame, and for t>0, the modulus of V(t=0) is indeed greater than the modulus of V(t). In step S14, frames that do not meet this condition are discarded. Next, in step S15, various delays are determined, TAU1, then TAU2 (by removing those corresponding to delay TAU1 from the modulus of V(t)k to be compared in Equation 51), etc. Delay TAUm is given by dividing the component tmax found at each iteration m according to Equation 52 by the sampling frequency fs, taking into account that time t and tmax(k) are first represented in the form of sample indices (with time zero as a reference to index zero). Vectors U1, U2, etc., can also be calculated using Equation 50.
[0269] Other parameters can also be determined, especially d0 given by Equation 28 in step S16 (by verifying consistency with conventional room data in test S17, e.g., d0min = 0 and d0max = 5m; otherwise, the frame contains errors and can be discarded in step S14).
[0270] Step S18 may include selecting a "good" frame, representing the sound initiation point with the first reflection. The standard D(k) used to select such a frame can be illustrated by equation 53, where C(f) i (k) This refers to the amplitude (amplitude in absolute value) detected on the surround sound channel i in the time-frequency sample (t, f) generated by the first transform (time to frequency) of frame k. Epsilon indicates a non-zero positive value to avoid a zero denominator in the absence of a signal. F represents the total number of frequency subbands used.
[0271] Therefore, in step S22, only frames whose standard D(k) calculated according to Equation 53 is not less than 90% of the maximum Dmax found in step S21 can be retained from the standard of all frames D(k).
[0272] Therefore, in step S18, the D(k) value is calculated for all frames, and then in step S19, the process delivers U0(k), d0(k), and D(k) for each frame. In step S20, the D(k) values are collected so that the highest value is identified in step S21, and frames with D(k) values less than 0.9Dmax are eliminated in step S22.
[0273] Finally, in step S23, the vector U0 retained here is preferably the median (not the mean) of the vectors U0 of the various frames retained. The distance d0 is also the median of the distances d0 of the various frames retained.
[0274] Of course, the present invention is not limited to the above embodiments as examples; it extends to other variations.
[0275] The above has described an application of processing a first-order surround sound (FOA) signal. Higher orders can be used to enhance spatial resolution.
[0276] In fact, we described a first-order surround sound representation above, but it is possible to provide higher-order ones. In this case, the velocity vector calculation increases the ratio of the higher-order directional components to the component W(f), and the vector Un implicitly increases the dimension by the same amount. Increasing the dimension (more than 3) and thus increasing the spatial resolution allows vectors U0, U1, ... Un to be better distinguished from each other, and makes it easier to detect peaks V(k*TAUn) proportional to (U0-Un) in the temporal fingerprint, even when vectors U0 and Un are angularly close, which occurs in the case of grazing reflection (e.g., when the source is far from and / or close to the ground). Therefore, this makes it possible to estimate the desired parameters U0, U1, d0, etc. more accurately. It should also be noted that the fact that only three first-order components (X, Y, Z) are retained in the molecule is irrelevant to the fact that higher-order components can be used to construct the reference component in the denominator. In all cases (regardless of the denominator), one can consider improving the above treatment (and the treatment proposed in FR1911723) by adding higher-order components to the molecule, thereby increasing the dimension of the velocity vector, and in particular allowing for better distinction of its peaks in the time domain.
[0277] More generally, the velocity vector can be replaced by the ratio between the components of a spatial acoustic representation that “coincide” in the frequency domain, and can work within the coordinate system characteristics of the spatial representation.
[0278] To mitigate situations such as multiple sources, TDVV computation can be combined with artificial intelligence methods, including neural networks. Some envisioned training strategies (e.g., based on fingerprints from the model or windowed SRIRs, rather than necessarily from the raw signal) could allow the network to learn to utilize frame sequences to improve detection and estimation relevant to a given room condition.
[0279] In addition, we have already referred to the above Figures 6A to 6D Sections 7 and 8 describe the possibility of initially estimating the conventional velocity vector V(f) to determine a first coarse estimate of the DoA, and then refining the estimate of the general velocity vector V'(f) based on this coarse estimate to give a more accurate DoA. Of course, this is just one example of a possible implementation. For example, in a variant, it might be possible to immediately divide the space into several sectors, thereby providing directionality to the components in the denominator D(f) in each of these sectors, and to try using an iterative method (…). Figure 7The iterative methods of the type starting from step S761, where D(1) is simply computed as a function of the angular sector under consideration, are used to compute DoA. These iterative methods are performed in parallel for each of these sectors. This is one possible example implementation. Alternatively, “best” (or several best) angular sectors can be selected according to the validity criteria of the causal model described above, and the estimates can then be optimized in one or more selected directions, including variations in the denominator form. In fact, more generally, one can further envision evaluating multiple general velocity vectors, each associated with a different directionality (in the denominator), in the first step and / or in subsequent steps.
[0280] More generally, the search for beamforming that provides a reliable direction of arrival (or DoA) can be treated as a general optimization problem, and various strategies can be invoked under this capability. Therefore, we can identify:
[0281] - The minimization criterion (the function to be minimized) represents the effectiveness of the causal model. In the description above, in a slightly simplified and therefore potentially improved manner, this involves searching for the relative energy of the signal on the negative time x-axis;
[0282] - The parameters to be optimized are beamforming parameters, such as the coefficients of matrix D involved in equations A1 and A4, or, when choosing axisymmetric or constrained to first order, the beamform parameters g in the direction Theta(Θ) and equation A5. beamshape ;
[0283] - A set (or more) of initial parameters, typically for "omnidirectional" directionality (g beamshape = [1 0…]); or favorable directionality stored during previous use; or multiple directionality pointing to a set of directions representing space, and even expressing different forms of directionality;
[0284] - The principle of adjusting test parameters (during iteration) is that redirecting the acoustic beam in the final estimated DoA is not always a robust enough choice: therefore it is necessary, rather than stopping the algorithm due to a lack of improvement, to start over from one of the stored cases (which may be best from the perspective of minimizing the criteria) and adjust the parameters along another axis (e.g., in the form of directional parameters) or another combination of axes.
[0285] More generally, conventional approaches can be considered, including methods such as stochastic gradient or batch optimization; however, the resulting number of iterations may involve significant costs.
[0286] It should be added that, unlike common optimization tasks, the final objective parameters (usually the Un vector) are not parameters that are directly optimized, but rather parameters that are derived from them. It should be noted that there may be many different sets of beamforming parameters, all of which could be "optimal" because they conform to causal models. Therefore, they all allow the same set of parameters Un to be derived in a potentially accurate manner.
[0287] appendix
[0288] Equation 1 Equation 2 Equation 3 Equation 4 Equation 5 Equation 6 Equation 7 Equation 8 Equation 9-1 Equation 9-2 Equation 10
[0289]
[0290] Equation 11
[0291] Equation 12
[0292] Equation 13g0=1;τ0=0
[0293] Equation 14
[0294] Equation 15γ0=1
[0295] Equation 16
[0296] Equation 17
[0297] Equation 18
[0298] Equation 19
[0299]
[0300] Equation 20
[0301] Equation 21
[0302] Equation 22
[0303] Equation 23
[0304]
[0305]
[0306] Equation 24 Equation 25
[0307] Equation 26
[0308] Equation 27d1-d0=τ1c
[0309] Equation 28
[0310] Equation 29 Equation 30
[0311] Equation 31
[0312] Equation 32
[0313] Equation 33
[0314]
[0315] Equation 34
[0316] Equation 35
[0317] Equation 36
[0318] Equation 37
[0319] Equation 38
[0320] Equation 39
[0321] Equation 40
[0322] Equation 41
[0323] Equation 42
[0324] Equation 43
[0325] Equation 44e -αt =e -α(t-τ) .e -ατ
[0326] Equation 45
[0327] Equation 46
[0328]
[0329] Equation 47
[0330] Equation 48
[0331] Equation 49
[0332] Equation 50
[0333] Equation 51
[0334] Equation 52
[0335] Equation 53
[0336]
[0337] Equation A1D Θ (f)=D Θ .B(f)
[0338] Equation A2
[0339]
[0340] Equation A3
[0341] Equation A4
[0342] Equation A5D Θ =Y(Θ).Diag(g) beamshape )
[0343] Equation A6
[0344] Equation B1
[0345] Equation B2 = 35b
[0346] Equation B3 = 38b
[0347] Equation B4 = 39b
[0348]
[0349] Equation B5
[0350] Equation B6
[0351] Equation B7
[0352] Equation B8
[0353] Equation B9
[0354] Equation B10
Claims
1. A method for processing audio signals acquired by at least one microphone, The objective is to locate at least one sound source in a space comprising at least one wall, wherein: - Apply time-frequency transformation to the acquired signal. - Based on the acquired signal, a general velocity vector V'(f) is expressed in the frequency domain, which is estimated from the expression of the velocity vector V(f), in which the omnidirectional component W(f) appears in the denominator of the velocity vector V(f), and wherein, in the denominator of the general velocity vector V'(f), the omnidirectional component W(f) is replaced by a reference component D(f) that is different from the omnidirectional component W(f), the reference component D(f) being more spatially selective than the omnidirectional component W(f) in the direction corresponding to the first acoustic path, the expression of the general velocity vector V'(f) is a complex number with real and imaginary parts, and the general velocity vector V'(f) represents the following composition: * The first acoustic path, directly between the source and the microphone, is represented by the first vector U0, and * The at least second acoustic path generated by reflection from the wall and represented by the second vector U1. The second acoustic path has a first delay TAU1 at the microphone relative to the first acoustic path. - As a function of the first delay TAU1, the first vector U0, and the second vector U1, at least one parameter is determined from among the following: * The direction of the first acoustic path, * The distance d0 from the source to the microphone, * The distance z0 from the source to the wall.
2. The method of claim 1, comprising multiple iterations, wherein in at least a portion of the multiple iterations, the general velocity vector V'(f) has a reference component D(f) in its denominator determined based on an approximation of the direction of the first acoustic path obtained in a previous iteration.
3. The method of claim 2, further comprising a first iteration, wherein the general velocity vector V'(f) is replaced by the velocity vector V(f), which is expressed in the frequency domain and has an omnidirectional component W(f) appearing in the denominator, so that at the end of the first iteration, at least a first approximation of the direction of the first acoustic path is determined. And among them, For at least a second iteration following the first iteration, the general velocity vector V'(f) estimated according to the expression of the velocity vector V(f) is used, wherein the omnidirectional component W(f) in the denominator is replaced by a reference component D(f), which is more selective in space than the omnidirectional component W(f).
4. The method of claim 3, wherein the reference component D(f) is more selective than the omnidirectional component W(f) in the direction corresponding to the first approximation of the direction of the first acoustic path.
5. The method of claim 2, wherein the iteration is repeated until convergence is achieved according to a predetermined causality criterion.
6. The method according to claim 5, wherein, In each iteration: - An inverse frequency-to-time transformation is also applied to the expression of the general velocity vector V'(f) to obtain a series of peaks in the time domain, each related to reflections on at least one wall, in addition to peaks related to the arrival of sound along the first acoustic path, and - If a signal appears in the series of peaks whose time axis is smaller than the axis of the peak associated with the arrival of the sound along the first acoustic path and whose amplitude is greater than the selected threshold, then a new iteration is performed. If the amplitude of the signal is lower than the threshold, then the causal relationship criterion is met.
7. The method of claim 5, wherein the iteration terminates in the following cases: - In the first case where the amplitude of the signal is lower than the selected threshold, and - In the second case where the repetition of the iteration does not cause a significant decrease in the amplitude of the signal.
8. The method of claim 7, wherein after the second case, the following step is performed, wherein the acquired signal is transmitted in the form of consecutive sample frames: - For each frame, estimate the score for the presence of the sound at the start of the frame, and - Select frames with scores higher than a threshold for processing the audio signals acquired in the selected frames.
9. The method of claim 1, wherein the acquired signal is picked up by a surround sound microphone, and wherein the velocity vector V(f) is expressed in the frequency domain by first-order surround sound components X(f), Y(f), and Z(f) of the following types: V(f)= 1 / W(f) [X(f), Y(f), Z(f)] T , W(f) is the omnidirectional component. Furthermore, the general velocity vector V'(f) in the frequency domain is expressed by first-order surround sound components of the following types: V’(f)= 1 / D(f) [X(f), Y(f), Z(f)] T , D(f) is the reference component that is different from the omnidirectional component W(f).
10. The method according to claim 1, wherein, An estimate of the direction of the first acoustic path, which is equivalent to the first vector U0, is determined based on the average value of a set of frequencies of the real part of the general velocity vector V'(f) expressed in the frequency domain.
11. The method according to claim 1, wherein: - The inverse transformation from frequency to time is applied to the general velocity vector so that it can be expressed in the time domain V'(t). - After the duration of the first acoustic path, search for at least one maximum value in the expression for the general velocity vector V'(t)max as a function of time, and - Derive the first delay TAU1 corresponding to the time when the maximum V'(t)max is given.
12. The method according to claim 11, wherein: - Estimate the second vector U1 as a function of the values of the normalized velocity vector V' recorded at time indices t=0, TAU1, and 2.TAU1, so as to define vector V1 such that: V1 = V'(TAU1) - ( (V'(TAU1) . V'(2.TAU1)) / || V'(TAU1) ||² ) V'(0), The second vector U1 is then given by the following equation: U1 = V1 / ||V1||.
13. The method according to claim 12, wherein: - Determine the corresponding angles PHI0 and PHI1 of the first vector U0 and the second vector U1 relative to the wall, such that: PHI0 = arcsin(U0.nR) and PHI1 = arcsin(U1.nR), where nR is a unit vector perpendicular to the wall, and - As a function of the first delay TAU1, the distance d0 between the source and the microphone is determined by the following type of relationship: d0 = (TAU1xC) / ((cosPHI0 / cosPHI1)-1), where C is the speed of sound.
14. The method according to claim 13, wherein, The distance z0 from the source to the wall is determined by the following type of relationship: z0 = d0 (sinPHI0 – sinPHI1) / 2.
15. The method of claim 1, wherein the space comprises multiple walls: - The inverse transform from frequency to time is applied to the general velocity vector so that it is expressed as a series of peaks in the time domain V'(t). - Among the series of peaks, peaks associated with reflections on the walls of the multi-faceted walls are identified, each identified peak having a time abscissa, which is a function of the first delay TAUn of the acoustic path caused by reflections on the corresponding wall n relative to the first acoustic path. - As a function representing each first delay TAUn, the first vector U0, and each second vector Un generated by reflections on wall n, at least one parameter is determined in the following terms: * The direction of the first acoustic path, * The distance d0 from the source to the microphone, and * At least one distance zn from the source to the wall n.
16. The method of claim 15, wherein the peak values associated with reflections on wall n have a time abscissa, the time abscissa being a multiple of the delay TAUn associated with wall n, and wherein a first portion of the peak values having the smallest positive time abscissa is pre-selected to identify each of the first portion as a peak value associated with a single reflection on the wall.
17. An apparatus for processing audio signals, comprising processing circuitry for implementing the method according to any one of the preceding claims.
18. A non-transitory computer storage medium storing instructions for a computer program, which, when executed by a processor of a processing circuit, cause the implementation of the method according to any one of claims 1 to 16.
Citation Information
Patent Citations
Optimized coding and decoding of spatialization information for the parametric coding and decoding of multichannel audio signal
CN108885876A
Method and device for ascertaining transversal relative speed components of radar targets
CN110431437A
