Potential space kernel adaptive filtering method for sound source localization

Abstract

The invention discloses a potential space kernel adaptive filtering method for sound source localization, and relates to the field of data processing. In order to solve the problems that in the prior art, high-dimensional audio data processing is complex, feature extraction is difficult, and real-time processing is difficult to achieve on resource-limited equipment, the invention provides a potential space kernel adaptive filtering method for sound source localization, and the method combines a potential space model and kernel adaptive filtering. The feature extraction capability is improved, and the lightweight and online learning characteristics are maintained. According to the invention, by combining the potential space model and the kernel adaptive filtering method, the feature extraction capability and the nonlinear modeling capability are improved, and higher sound source localization precision and robustness are realized in complex reverberation and noise environments. The method has a lightweight structure, is low in calculation complexity, can realize millisecond-level real-time sound source localization on equipment with limited calculation resources, and is suitable for industrial application with limited resources.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a sound source localization method combining a latent space model (Latent Space Model) and a kernel adaptive filtering (Kernel Adaptive Filtering). Background Art

[0002] In industrial environments, sound is ubiquitous and contains information about the status and location of equipment and operators. Sound source localization (SSL) aims to determine the spatial location of sound-emitting objects. Existing SSL technologies are mainly divided into model-based methods and learning-based methods.

[0003] Model-based methods rely on pre-established physical or mathematical models to describe sound propagation and localize sound sources by analyzing received audio signals. However, the performance of these methods will degrade significantly in complex environments with reverberation and noise.

[0004] Learning-based methods use machine learning to learn the characteristics and patterns of sound propagation from large amounts of data to achieve more robust and accurate sound source localization. However, these methods usually require a lot of computing resources and are difficult to implement in real time on devices with limited computing resources.

[0005] With the rapid development of wearable devices and the Internet of Things, achieving high-precision and robust real-time sound source localization on embedded devices with limited computing resources has become an urgent problem to be solved. Summary of the invention

[0006] Technical problem to be solved by the invention: In order to solve the problems existing in the prior art of complex high-dimensional audio data processing, difficult feature extraction and difficulty in realizing real-time processing on resource-constrained devices, the present invention provides a latent space kernel adaptive filtering method for sound source localization, which combines the latent space model and kernel adaptive filtering, which not only improves the feature extraction capability but also maintains the characteristics of lightweight and online learning.

[0007] The present invention provides a latent space kernel adaptive filtering method for sound source localization, comprising the following steps:

[0008] Step 1. Pre-training phase:

[0009] An autoencoder is trained using a known sound source signal to minimize the reconstruction error and obtain an encoder capable of extracting key features for sound source localization; the autoencoder projects high-dimensional audio data into a low-dimensional latent space, reducing the data dimension; the autoencoder is an encoder-decoder architecture; the encoder is responsible for compressing the input data into a low-dimensional representation, using multiple convolutional layers and pooling layers to extract features; the latent representation u is finally output, representing the compressed features, and the decoder is responsible for restoring the low-dimensional representation to the original data x; the transposed convolutional layer and the convolutional layer are used to gradually restore the data, and the restored data is finally output;

[0010] Step 2. Training phase:

[0011] The encoder in the pre-trained autoencoder is used to convert the sound source signal into a potential representation. Then, the potential representation is mapped to the reproducing kernel Hilbert space through the kernel function. In the reproducing kernel Hilbert space, a nonlinear relationship model between the sound source signal and its position is established through linear filtering, and the nonlinear relationship model parameters are learned.

[0012] Step 3. Deployment phase:

[0013] In practical applications, the unknown sound source signal is processed; first, the encoder is used to obtain the latent representation, then it is mapped to the reproducing kernel Hilbert space through the kernel function, and finally the nonlinear relationship learned in the training phase is used to estimate the location of the sound source.

[0014] Furthermore, the specific method of step 1 is:

[0015] Step 1.1. Data collection and processing:

[0016] Collect the sound source signal at a known position, process the sound source signal using short-time Fourier transform, and obtain high-dimensional time-frequency domain audio data x;

[0017] Step 1.2. Autoencoder training:

[0018] Encoder: Contains 6 convolutional layers and 3 pooling layers, arranged in the order of convolutional layer-pooling layer, gradually extracting and compressing features;

[0019] Decoder: Contains 2 transposed convolutional layers and 2 convolutional layers, arranged in the order of transposed convolutional layer-convolutional layer, gradually recovering data;

[0020] Specific structure of encoder:

[0021] Convolutional layer 1: The input data passes through the first convolutional layer and outputs the feature map;

[0022] Pooling layer 1: A pooling layer is added after the first convolutional layer to reduce the size of the feature map;

[0023] Convolutional layer 2: The pooled feature map is input into the second convolutional layer to extract more complex features;

[0024] Pooling layer 2: The second convolutional layer is followed by a second pooling layer to further reduce the feature map size;

[0025] Convolutional layer 3: The pooled feature map is input into the third convolutional layer to continue extracting features;

[0026] Pooling layer 3: The third convolutional layer is followed by the third pooling layer to further compress the feature map;

[0027] Convolutional layer 4: The last convolutional layer, outputting low-dimensional feature representation;

[0028] Decoder specific structure:

[0029] Transposed convolution layer 1: The low-dimensional feature representation is input to the first transposed convolution layer to expand the feature map size;

[0030] Convolutional layer 1: The output of the transposed convolutional layer passes through the first convolutional layer to further process the features;

[0031] Transposed convolution layer 2: The feature map after the convolution layer is input to the second transposed convolution layer to continue expanding the feature map size;

[0032] Convolutional layer 2: The last convolutional layer, which outputs the same shape as the original input data;

[0033] Train an autoencoder using audio data x to minimize the reconstruction error

[0034]

[0035] Wherein, x is the reconstructed audio signal;

[0036] Training is based on reconstruction error Use the optimizer to traverse the training set, calculate the loss and perform backpropagation in each training cycle;

[0037] Step 1.3. Get the encoder:

[0038] After training, the encoder part is retained and used for subsequent feature extraction; the encoder maps high-dimensional audio data x to a low-dimensional potential representation u:

[0039] u=f e (W e x+b e ) (2)

[0040] Among them, f e (.) represents the activation function, W erepresents the weight matrix, b e Indicates bias.

[0041] Furthermore, the specific method of step 2 is:

[0042] Step 2.1. Feature extraction: Use the pre-trained encoder to convert the sound source signal into a latent representation u t ;

[0043] Step 2.2. Kernel function mapping: Use the kernel function to transform the potential representation u t High-dimensional eigenvectors mapped to reproducing kernel Hilbert space

[0044]

[0045] Among them, κ(·) represents the kernel function;

[0046] Step 2.3. Nonlinear relationship modeling: In the reproducing kernel Hilbert space, a nonlinear relationship between the potential representation and the sound source position is established by a linear filtering method; the sound source position is estimated:

[0047]

[0048] in, represents the estimated sound source position, w T represents the filter weight;

[0049] Step 2.4. Nonlinear relationship model parameter learning: Use the corresponding weight updating method to learn the nonlinear relationship model parameters w.

[0050] Furthermore, the specific method of step 3 is:

[0051] Step 3.1. Unknown sound source signal processing: For the unknown sound source signal to be located, use the encoder to obtain the low-dimensional potential representation u g ;

[0052] Step 3.2. Position estimation:

[0053] Map the latent representation to the reproducing kernel Hilbert space and then use the learned nonlinear relationship to estimate the sound source position:

[0054]

[0055] Furthermore, the weight update method of step 2.4 is:

[0056] Latent Space Kernel Least Mean Square Algorithm: It uses Gaussian kernel function, minimum mean square error criterion and stochastic gradient descent update strategy to achieve a balance between computational efficiency and positioning accuracy.

[0057] Kernel function selection: Use Gaussian kernel function κ h (u i ,u j ):

[0058]

[0059] Among them, u i 、u j represents the sound feature vector corresponding to the sound source, h represents the kernel bandwidth, ‖·‖ 2 represents the Euclidean distance;

[0060] Loss function: Use the minimum mean square error (MSE) criterion:

[0061]

[0062] Among them, J(w) represents the filter loss function, d i represents the true position corresponding to the i-th training point, N represents the size of the training set, Represents the high-dimensional feature vector of sound;

[0063] Weight Update: Update weights using stochastic gradient descent method:

[0064]

[0065] Among them, η represents the learning rate, w i represents the filter weight, e i Indicates positioning error.

[0066] Furthermore, the weight update method of step 2.4 is:

[0067] Latent Space Kernel Recursive Least Squares Algorithm: It uses Gaussian kernel function and recursive least squares method to improve the convergence speed and is suitable for scenarios that require fast response.

[0068] Kernel function selection: Gaussian kernel function;

[0069] Weight update: Update the weights using recursive least squares method:

[0070]

[0071] in, d i =[d 1 ,d 2 ,...,d i ] T , λ represents the regularization factor, I represents the identity matrix, Represents the high-dimensional feature vector of sound, d i Represents the true position corresponding to the i-th training point.

[0072] Furthermore, the weight update method of step 2.4 is:

[0073] Latent space t-distribution-based kernel adaptive filtering algorithm: It uses a t-distribution-based kernel function and an average p-th power error loss function to improve positioning accuracy and robustness in non-Gaussian noise environments;

[0074] Kernel function selection: Use the kernel function κ based on Student's t distribution v,σ (u i ,u j ):

[0075]

[0076] Among them, v represents the degree of freedom, which refers to the number of independent information in the sample data; σ 2 Represents the population variance, which affects the shape of the t distribution;

[0077] Loss function: Using the average p-th error criterion:

[0078]

[0079] Among them, J(w i ) represents the loss function, κ v,σ (e j ) represents the error metric based on the kernel function, e j represents the positioning error, p represents the power of the error, λ represents the regularization factor, and w i represents the filter weight;

[0080] Weight update: By minimizing the above loss function, solve the weight parameter w i .

[0081] Compared with the prior art, the present invention has the following beneficial effects:

[0082] High precision and strong robustness: By combining the latent space model and the kernel adaptive filtering method, the feature extraction capability and nonlinear modeling capability are improved, and higher sound source localization accuracy and robustness are achieved in complex reverberation and noise environments.

[0083] Lightweight and real-time: This method has a lightweight structure and low computational complexity. It can achieve real-time sound source localization in milliseconds on devices with limited computing resources, and is suitable for resource-constrained industrial applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 Flowchart of a latent space kernel adaptive filtering method for sound source localization.

[0085] Figure 2 Schematic diagram of the room scene for the sound source localization experiment. The hollow circles represent microphones and the solid circles represent possible sound sources.

[0086] Figure 3 This is a schematic diagram of the hydropower station scene in the sound source localization experiment. The hollow circles represent microphones and the solid circles represent possible sound sources.

[0087] Figure 4 This is an intuitive performance diagram of each algorithm in the sound source localization experimental room scene.

[0088] Figure 5 This is an intuitive performance diagram of each algorithm in the sound source localization experiment hydropower station scenario.

[0089] Figure 6 Learning curves of the latent space kernel adaptive filtering method applied to the sound source localization experiment hydropower station field, (a) X-axis estimation; (b) Y-axis estimation. DETAILED DESCRIPTION

[0090] The present invention will be further described below in conjunction with specific embodiments, but the protection scope of the present invention is not limited thereto.

[0091] Pre-training phase

[0092] 1. Data collection and processing:

[0093] The sound source signal at a known position is collected and processed using short-time Fourier transform (STFT) to obtain high-dimensional time-frequency domain audio data x.

[0094] 2. Autoencoder Training:

[0095] Use the above audio data to train the autoencoder to minimize the reconstruction error:

[0096]

[0097] Where x is the reconstructed audio signal.

[0098] 3. Get the encoder:

[0099] After training is completed, the encoder part is retained and used for subsequent feature extraction. The encoder maps high-dimensional audio data x to a low-dimensional potential representation u:

[0100] u=f e (W e x+b e ) (13)

[0101] Training phase

[0102] 1. Feature extraction: Use the pre-trained encoder to convert the sound source signal into a potential representation ut .

[0103] 2. Kernel function mapping: Use the kernel function to transform the potential representation u t Mapping to high-dimensional feature vectors in RKHS

[0104]

[0105] 3. Nonlinear relationship modeling: In RKHS, a nonlinear relationship between the potential representation and the sound source position is established through a linear filtering method. Estimating the sound source position:

[0106]

[0107] 4. Model parameter learning: Based on the selected algorithm, the model parameters w are learned using the corresponding weight update method.

[0108] Deployment Phase

[0109] 1. Unknown sound source signal processing: For the unknown sound source signal to be located, use the encoder to obtain the potential representation u g .

[0110] 2. Position estimation:

[0111] Map the latent representation to RKHS and then use the learned nonlinear relationship to estimate the sound source location:

[0112]

[0113] Example 1: Latent Space Kernel Least Mean Square Algorithm

[0114] 1. Kernel function selection: Use Gaussian kernel function:

[0115]

[0116] 2. Loss function: Use the minimum mean square error (MSE) criterion:

[0117]

[0118] 3. Weight update: Update weights using stochastic gradient descent (SGD) method:

[0119]

[0120] in,

[0121] Example 2: Latent Space Kernel Recursive Least Squares Algorithm

[0122] 1. Kernel function selection: Gaussian kernel function is also used.

[0123] 2. Weight update: Update weights using recursive least squares method:

[0124]

[0125] in, d i =[d 1 ,d 2 ,...,d i ] T .

[0126] Example 3: Kernel adaptive filtering algorithm based on t-distribution in latent space

[0127] 1. Kernel function selection: Use a kernel function based on Student's t distribution:

[0128]

[0129] 2. Loss function: Use the mean p-th error (MPE) criterion:

[0130]

[0131] 3. Weight update: By minimizing the above loss function, solve the weight parameter w i .

[0132] The above content describes in detail a latent space kernel adaptive filtering method for sound source localization, including pre-training, training and deployment stages, and provides three different algorithm implementations. This method combines the latent space model and kernel adaptive filtering technology to improve the accuracy and robustness of sound source localization while maintaining the characteristics of lightweight and real-time processing.

[0133] The present invention verifies the superiority of the latent space kernel adaptive filtering method through sound source localization experiments, and uses two real-scene sound source localization data sets to demonstrate and verify the performance of the present invention. The first real scene is a closed laboratory. The scene principle is as follows Figure 3 The second real scene is the turbine room of a hydropower station. Figure 4 As shown in the original example Figure 5 shown.

[0134] Experimental process and Figure 1 The process of adaptive data processing method for high-precision positioning is shown Figure 1 First, place the signal receivers in the following Figure 4 and Figure 5The audio signals corresponding to each position are recorded at the training data reference points shown in Figure 1, and then the training data sets corresponding to the two experiments are generated. Subsequently, in order to test the performance of the proposed data processing method, the signal receivers are placed as follows: Figure 4 and Figure 5 At the randomly generated test points shown in the figure, the audio signal at each position is recorded to generate the test data sets corresponding to the two experiments. The training data set and the test data set contain the data sequence {u(i), d(i)}, and the audio data u(i) and the corresponding actual position d(i) of the two data sets are known.

[0135] During the experiment, the input audio data u(i) and the corresponding actual position d(i) of the training data set are first input into LS-KAF for feature extraction and filter learning. Then only the audio data u(i) of the test data set is input into LS-KAF to obtain the corresponding estimated position y(i), so as to realize the function of online application. Finally, the mean square error (MSE) is used to measure the difference between the actual position d(i) of the test data set and the estimated position y(i) obtained by the test, reflecting the accuracy of positioning data processing.

[0136] The experiments were compared with the classic sound source localization algorithms (SRP-PHAT, MUSIC) based on deep learning algorithms (CNN) in the two representative indoor positioning experimental scenarios described above.

[0137] The qualitative comparison results of the performance of the adaptive data processing method for high-precision positioning proposed in this invention and other kernel adaptive filtering algorithms are as follows: Figure 6 The results show that the method proposed by the present invention has the highest sound source localization accuracy in representative scenes with different interferences.

[0138] The sound source localization prediction errors and standard deviations of LS-KAF and the comparison algorithms are shown in Tables 1 and 2. The results show that LS-KAF using the method proposed in the present invention has the highest positioning accuracy and higher robustness compared with the classic sound source localization method and the sound source localization algorithm based on deep learning.

[0139] In summary, compared with the classical sound source localization method and the sound source localization algorithm based on deep learning, the invented latent space kernel adaptive filtering method has significant advantages in the prediction accuracy of sound source localization.

Claims

1. A latent space kernel adaptive filtering method for sound source localization, comprising the following steps: Step 1. Pre-training phase: Using known sound source signals to train the autoencoder to minimize the reconstruction error, and obtain an encoder that can extract key features for sound source localization; The autoencoder projects high-dimensional audio data into a low-dimensional latent space, reducing the data dimension; the autoencoder is an encoder-decoder architecture; the encoder is responsible for compressing the input data into a low-dimensional representation, using multiple convolutional layers and pooling layers to extract features; finally outputting a latent representation u, representing the compressed features, and the decoder is responsible for restoring the low-dimensional representation to the original data x; using transposed convolutional layers and convolutional layers to gradually restore the data, and finally outputting the restored data; Step 2. Training phase: The encoder in the pre-trained autoencoder is used to convert the sound source signal into a potential representation. Then, the potential representation is mapped to the reproducing kernel Hilbert space through the kernel function. In the reproducing kernel Hilbert space, a nonlinear relationship model between the sound source signal and its position is established through linear filtering, and the nonlinear relationship model parameters are learned. Step 3. Deployment phase: In practical applications, the unknown sound source signal is processed; first, the encoder is used to obtain the latent representation, then it is mapped to the reproducing kernel Hilbert space through the kernel function, and finally the nonlinear relationship learned in the training phase is used to estimate the location of the sound source.

2. A latent space kernel adaptive filtering method for sound source localization according to claim 1, characterized in that: The specific method of step 1 is: Step 1.

1. Data collection and processing: Collect the sound source signal at a known position, process the sound source signal using short-time Fourier transform, and obtain high-dimensional time-frequency domain audio data x; Step 1.

2. Autoencoder training: Encoder: Contains 6 convolutional layers and 3 pooling layers, arranged in the order of convolutional layer-pooling layer, gradually extracting and compressing features; Decoder: Contains 2 transposed convolutional layers and 2 convolutional layers, arranged in the order of transposed convolutional layer-convolutional layer, gradually recovering data; Specific structure of encoder: Convolutional layer 1: The input data passes through the first convolutional layer and outputs the feature map; Pooling layer 1: A pooling layer is added after the first convolutional layer to reduce the size of the feature map; Convolutional layer 2: The pooled feature map is input into the second convolutional layer to extract more complex features; Pooling layer 2: The second convolutional layer is followed by a second pooling layer to further reduce the feature map size; Convolutional layer 3: The pooled feature map is input into the third convolutional layer to continue extracting features; Pooling layer 3: The third convolutional layer is followed by the third pooling layer to further compress the feature map; Convolutional layer 4: The last convolutional layer, outputting low-dimensional feature representation; Decoder specific structure: Transposed convolution layer 1: The low-dimensional feature representation is input to the first transposed convolution layer to expand the feature map size; Convolutional layer 1: The output of the transposed convolutional layer passes through the first convolutional layer to further process the features; Transposed convolution layer 2: The feature map after the convolution layer is input to the second transposed convolution layer to continue expanding the feature map size; Convolutional layer 2: The last convolutional layer, which outputs the same shape as the original input data; Train an autoencoder using audio data x to minimize the reconstruction error Wherein, x is the reconstructed audio signal; Training is based on reconstruction error Use the optimizer to traverse the training set, calculate the loss and perform backpropagation in each training cycle; Step 1.

3. Get the encoder: After training, the encoder part is retained and used for subsequent feature extraction; the encoder maps high-dimensional audio data x to a low-dimensional potential representation u: u=f e (W e x+b e ) (2) Among them, f e (.) represents the activation function, W e represents the weight matrix, b e Indicates bias.

3. The latent space kernel adaptive filtering method for sound source localization according to claim 1, characterized in that: The specific method of step 2 is: Step 2.

1. Feature extraction: Use the pre-trained encoder to convert the sound source signal into a latent representation u t ; Step 2.

2. Kernel function mapping: Use the kernel function to transform the potential representation u t High-dimensional eigenvectors mapped to reproducing kernel Hilbert space Among them, κ(·) represents the kernel function; Step 2.

3. Nonlinear relationship modeling: In the reproducing kernel Hilbert space, a nonlinear relationship between the potential representation and the sound source position is established by a linear filtering method; the sound source position is estimated: in, represents the estimated sound source position, w T represents the filter weight; Step 2.

4. Nonlinear relationship model parameter learning: Use the corresponding weight updating method to learn the nonlinear relationship model parameters w.

4. The latent space kernel adaptive filtering method for sound source localization according to claim 1, characterized in that: The specific method of step 3 is: Step 3.

1. Unknown sound source signal processing: For the unknown sound source signal to be located, use the encoder to obtain the low-dimensional potential representation u g ; Step 3.

2. Position estimation: Map the latent representation to the reproducing kernel Hilbert space and then use the learned nonlinear relationship to estimate the sound source position:

5. The latent space kernel adaptive filtering method for sound source localization according to claim 3, characterized in that: The weight update method in step 2.4 is: Latent Space Kernel Least Mean Square Algorithm: It uses Gaussian kernel function, minimum mean square error criterion and stochastic gradient descent update strategy to achieve a balance between computational efficiency and positioning accuracy. Kernel function selection: Use Gaussian kernel function κ h (u i ,u j ): Among them, u i 、u j represents the sound feature vector corresponding to the sound source, h represents the kernel bandwidth, ‖·‖ 2 represents the Euclidean distance; Loss function: Use the minimum mean square error (MSE) criterion: Among them, J(w) represents the filter loss function, d i represents the true position corresponding to the i-th training point, N represents the size of the training set, Represents the high-dimensional feature vector of sound; Weight Update: Update weights using stochastic gradient descent method: Among them, η represents the learning rate, w i represents the filter weight, e i Indicates positioning error.

6. A latent space kernel adaptive filtering method for sound source localization as claimed in claim 3, characterized in that: The weight update method in step 2.4 is: Latent Space Kernel Recursive Least Squares Algorithm: It uses Gaussian kernel function and recursive least squares method to improve the convergence speed and is suitable for scenarios that require fast response. Kernel function selection: Gaussian kernel function; Weight update: Update the weights using recursive least squares method: in, d i =[d1,d2,...,d i ] T , λ represents the regularization factor, I represents the identity matrix, Represents the high-dimensional feature vector of sound, d i Represents the true position corresponding to the i-th training point.

7. A latent space kernel adaptive filtering method for sound source localization as claimed in claim 3, characterized in that: The weight update method in step 2.4 is: Latent space t-distribution-based kernel adaptive filtering algorithm: It uses a t-distribution-based kernel function and an average p-th power error loss function to improve positioning accuracy and robustness in non-Gaussian noise environments; Kernel function selection: Use the kernel function κ based on Student's t distribution v,σ (u i ,u j ): Among them, v represents the degree of freedom, which refers to the number of independent information in the sample data; σ 2 Represents the population variance and affects the shape of the t distribution; Loss function: Using the average p-th error criterion: Among them, J(w i ) represents the loss function, κ v,σ (e j ) represents the error metric based on the kernel function, e j represents the positioning error, p represents the power of the error, λ represents the regularization factor, and w i represents the filter weight; Weight update: By minimizing the above loss function, solve the weight parameter w i .

Images