A potential space kernel adaptive filtering method for sound source localization

Abstract

The present invention discloses a latent space kernel adaptive filtering method for sound source localization, which relates to the field of data processing. In order to solve the problems existing in the prior art of complex high-dimensional audio data processing, difficulty in feature extraction, and difficulty in achieving real-time processing on resource-constrained devices, the present invention provides a latent space kernel adaptive filtering method for sound source localization, which combines the latent space model and kernel adaptive filtering, which not only improves the feature extraction capability but also maintains the characteristics of lightweight and online learning. By combining the latent space model and the kernel adaptive filtering method, the present invention improves the feature extraction capability and nonlinear modeling capability, and achieves higher sound source localization accuracy and robustness in complex reverberation and noise environments. The method has a lightweight structure and low computational complexity, and can achieve millisecond-level real-time sound source localization on devices with limited computing resources, and is suitable for resource-constrained industrial applications.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing. Specifically, it relates to a sound source localization method combining latent space model and kernel adaptive filtering. BACKGROUND

[0002] In industrial environments, sound is ubiquitous and contains information about the state and location of equipment and operators. Sound source localization (SSL) aims to determine the spatial location of sound-emitting objects. Existing SSL techniques can be mainly divided into model-based methods and learning-based methods.

[0003] Model-based methods rely on pre-established physical or mathematical models to describe sound propagation, and achieve sound source localization by analyzing received audio signals. However, in complex environments with reverberation and noise, the performance of these methods will be significantly reduced.

[0004] Learning-based methods use machine learning to learn the characteristics and patterns of sound propagation from a large amount of data to achieve more robust and accurate sound source localization. However, these methods usually require a large amount of computing resources, making it difficult to achieve real-time processing on devices with limited computing resources.

[0005] With the rapid development of wearable devices and the Internet of Things, it has become an urgent problem to achieve high-precision, strong-robustness real-time sound source localization on embedded devices with limited computing resources. SUMMARY

[0006] The technical problem solved by the present application: In order to solve the problems of high-dimensional audio data processing complexity, feature extraction difficulty and difficulty in realizing real-time processing on resource-limited devices in the prior art, the present application provides a latent space kernel adaptive filtering method for sound source localization, which combines latent space model and kernel adaptive filtering, improves feature extraction capability, and maintains lightweight and online learning characteristics.

[0007] The present application provides a latent space kernel adaptive filtering method for sound source localization, comprising the following steps:

[0008] Step 1. Pre-training phase:

[0009] An autoencoder is trained using known sound source signals to minimize reconstruction error, obtaining an encoder capable of extracting key features of sound source localization; the autoencoder projects high-dimensional audio data into a low-dimensional latent space, reducing data dimensionality; the autoencoder is an encoder-decoder architecture; the encoder is responsible for compressing input data into a low-dimensional representation, using multiple convolutional layers and pooling layers to extract features; the final output is a latent representation u, representing the compressed features; the decoder is responsible for restoring the low-dimensional representation to the original data x; transpose convolutional layers and convolutional layers are used to gradually recover the data, and the final output is the recovered data;

[0010] Step 2. Training phase:

[0011] Use the encoder in the pre-trained autoencoder to convert the sound source signal into a latent representation; then, map the latent representation to the reproducing kernel Hilbert space through the kernel function; in the reproducing kernel Hilbert space, establish a nonlinear relationship model between the sound source signal and its position through linear filtering, and learn the nonlinear relationship model parameters;

[0012] Step 3. Deployment phase:

[0013] In practical applications, unknown sound source signals are processed; first, use the encoder to obtain the latent representation, then map it to the reproducing kernel Hilbert space through the kernel function, and finally use the nonlinear relationship learned in the training phase to estimate the position of the sound source.

[0014] Further, the specific method of step 1 is:

[0015] Step 1.1. Data collection and processing:

[0016] Collect sound source signals with known positions, process the sound source signals using short-time Fourier transform to obtain high-dimensional time-frequency domain audio data x;

[0017] Step 1.2. Autoencoder training:

[0018] Encoder: contains 6 convolutional layers and 3 pooling layers, arranged in the order of convolutional layer-pooling layer, gradually extracting and compressing features;

[0019] Decoder: contains 2 transpose convolutional layers and 2 convolutional layers, arranged in the order of transpose convolutional layer-convolutional layer, gradually recovering data;

[0020] Encoder specific structure:

[0021] Convolutional layer 1: input data passes through the first convolutional layer, output feature map;

[0022] Pooling layer 1: a pooling layer is connected after the first convolutional layer to reduce the size of the feature map;

[0023] Convolutional layer 2: The feature map after pooling is input to the second convolutional layer to extract more complex features.

[0024] Pooling layer 2: The second convolutional layer is followed by the second pooling layer to further reduce the feature map size.

[0025] Convolutional layer 3: The feature map after pooling is input to the third convolutional layer to continue extracting features.

[0026] Pooling layer 3: The third convolutional layer is followed by the third pooling layer to further compress the feature map.

[0027] Convolutional layer 4: The last convolutional layer outputs a low-dimensional feature representation.

[0028] Decoder specific structure:

[0029] Transposed convolutional layer 1: The low-dimensional feature representation is input to the first transposed convolutional layer to expand the feature map size.

[0030] Convolutional layer 1: The output of the transposed convolutional layer is processed further by the first convolutional layer.

[0031] Transposed convolutional layer 2: The feature map after the convolutional layer is input to the second transposed convolutional layer to continue expanding the feature map size.

[0032] Convolutional layer 2: The last convolutional layer outputs a result with the same shape as the original input data.

[0033] Train the autoencoder using audio data x to minimize reconstruction error

[0034]

[0035] where x is the reconstructed audio signal.

[0036] Train based on reconstruction error Use the optimizer to iterate through the training set at each training cycle, calculate the loss and perform backpropagation.

[0037] Step 1.3. Obtain the encoder:

[0038] After training, keep the encoder part and use it for subsequent feature extraction; the encoder maps high-dimensional audio data x to a low-dimensional latent representation u:

[0039] u = f e (W e x + b e ) (2)

[0040] where f e (.) represents the activation function, W edenotes a weight matrix, b e denotes a bias.

[0041] Further, the specific method of step 2 is:

[0042] Step 2.1. Feature extraction: using a pre-trained encoder, the sound source signal is converted into a latent representation u t ;

[0043] Step 2.2. Kernel function mapping: using a kernel function, the latent representation u t is mapped to a high-dimensional feature vector in a reproducing kernel Hilbert space

[0044]

[0045] where κ(·) denotes a kernel function;

[0046] Step 2.3. Nonlinear relationship modeling: in the reproducing kernel Hilbert space, a nonlinear relationship between the latent representation and the sound source position is established by a linear filtering method; the estimated sound source position is:

[0047]

[0048] where denotes the estimated sound source position, w T denotes the filter weight;

[0049] Step 2.4. Nonlinear relationship model parameter learning, using the corresponding weight update method to learn the nonlinear relationship model parameter w.

[0050] Further, the specific method of step 3 is:

[0051] Step 3.1. Unknown sound source signal processing: for the unknown sound source signal to be positioned, use the encoder to obtain a low-dimensional latent representation u g ;

[0052] Step 3.2. Position estimation:

[0053] Map the latent representation to the reproducing kernel Hilbert space, and then use the learned nonlinear relationship to estimate the sound source position:

[0054]

[0055] Further, the weight update method of step 2.4 is:

[0056] Latent space kernel least mean square algorithm: using a Gaussian kernel function, a least mean square error criterion, and a stochastic gradient descent update strategy, a balance between computational efficiency and positioning accuracy is achieved;

[0057] Kernel function selection: Gaussian kernel function κ h (u i ,u j ):

[0058]

[0059] where u i , u j denote the sound feature vector corresponding to the sound source, h denotes the kernel bandwidth, and ‖·‖ 2 denotes the Euclidean distance.

[0060] Loss function: use the minimum mean square error (MSE) criterion:

[0061]

[0062] where J(w) denotes the filter loss function, d i denotes the true position corresponding to the i-th training point, N denotes the training set size, denotes the sound high-dimensional feature vector.

[0063] Weight update: use the stochastic gradient descent method to update the weight:

[0064]

[0065] where η denotes the learning rate, w i denotes the filter weight, and e i denotes the positioning error.

[0066] Further, the weight update method of step 2.4 is:

[0067] Latent space kernel recursive least squares algorithm: adopt Gaussian kernel function and recursive least squares method, which improves the convergence speed and is suitable for scenes that require fast response;

[0068] Kernel function selection: Gaussian kernel function;

[0069] Weight update: use the recursive least squares method to update the weight:

[0070]

[0071] where, d i =[d1,d2,...,d i ] T , λ denotes the regularization factor, I denotes the unit matrix, denotes the sound high-dimensional feature vector, and d i denotes the true position corresponding to the i-th training point.

[0072] Further, the weight updating method of step 2.4 is:

[0073] Kernel adaptive filtering algorithm based on t-distributed latent space: Using the t-distributed kernel function and the average p-th power error loss function, the positioning accuracy and robustness in the non-Gaussian noise environment are improved;

[0074] Kernel function selection: use the kernel function κ v,σ (u i ,u j ) based on Student's t distribution:

[0075]

[0076] Where v represents the degree of freedom, which refers to the amount of independent information in the sample data; σ 2 represents the overall variance, which affects the shape of the t distribution;

[0077] Loss function: use the average p-th power error criterion:

[0078]

[0079] Where J(w i ) represents the loss function, κ v,σ (e j ) represents the error measurement based on the kernel function, e j represents the positioning error, p represents the power of the error, λ represents the regularization factor, and w i represents the filter weight;

[0080] Weight update: by minimizing the above loss function, the weight parameter w i is solved.

[0081] Compared with the prior art, the beneficial effects of the present application are:

[0082] High precision and strong robustness: by combining the latent space model and the kernel adaptive filtering method, the feature extraction ability and the nonlinear modeling ability are improved, and higher sound source positioning accuracy and robustness are achieved in complex reverberation and noise environment.

[0083] Lightweight and real-time: the method has a lightweight structure and low computational complexity, and can realize millisecond-level real-time sound source positioning on devices with limited computing resources, suitable for resource-constrained industrial applications. BRIEF DESCRIPTION OF DRAWINGS

[0084] Figure 1 is a flowchart of a latent space kernel adaptive filtering method for sound source positioning.

[0085] Figure 2Fig. 1 is a schematic diagram of a room scene for a sound source positioning experiment, wherein hollow circles represent microphones and solid circles represent possible sound sources.

[0086] Figure 3 Fig. 2 is a schematic diagram of a hydropower station scene for a sound source positioning experiment, wherein hollow circles represent microphones and solid circles represent possible sound sources.

[0087] Figure 4 Fig. 3 is a visual performance diagram of each algorithm for a room scene in a sound source positioning experiment.

[0088] Figure 5 Fig. 4 is a visual performance diagram of each algorithm for a hydropower station scene in a sound source positioning experiment.

[0089] Figure 6 Fig. 5 is a learning curve diagram of the potential space kernel adaptive filtering method for a hydropower station scene in a sound source positioning experiment, wherein (a) is an X-axis estimation and (b) is a Y-axis estimation. DETAILED DESCRIPTION

[0090] The application will be further described below in conjunction with specific embodiments, but the scope of protection of the application is not limited thereto.

[0091] Pre-training stage

[0092] 1. Data collection and processing:

[0093] Collect the sound source signals with known positions, process the sound source signals using short-time Fourier transform (STFT) to obtain high-dimensional time-frequency domain audio data x.

[0094] 2. Autoencoder training:

[0095] Train the autoencoder using the above audio data to minimize the reconstruction error:

[0096]

[0097] wherein x is the reconstructed audio signal.

[0098] 3. Obtain the encoder:

[0099] After training, the encoder part is retained and used for subsequent feature extraction. The encoder maps the high-dimensional audio data x to a low-dimensional latent representation u:

[0100] u = f e (W e x + b e ) (13)

[0101] Training stage

[0102] 1. Feature extraction: use the pre-trained encoder to convert the sound source signal to the latent representation u t .

[0103] 2. Kernel function mapping: Use a kernel function to map the latent representation u t to a high-dimensional feature vector in RKHS

[0104]

[0105] 3. Nonlinear relationship modeling: In RKHS, establish the nonlinear relationship between the latent representation and the sound source position by linear filtering method. Estimate the sound source position:

[0106]

[0107] 4. Model parameter learning: According to the selected algorithm, learn the model parameters w using the corresponding weight update method.

[0108] Deployment phase

[0109] 1. Unknown sound source signal processing: For the unknown sound source signal to be positioned, use the encoder to obtain the latent representation u g .

[0110] 2. Position estimation:

[0111] Map the latent representation to RKHS, and then estimate the sound source position using the learned nonlinear relationship:

[0112]

[0113] Example 1: Latent space kernel least mean square algorithm

[0114] 1. Kernel function selection: Use Gaussian kernel function:

[0115]

[0116] 2. Loss function: Use the least mean square error (MSE) criterion:

[0117]

[0118] 3. Weight update: Update the weight using the stochastic gradient descent (SGD) method:

[0119]

[0120] where,

[0121] Example 2: Latent space kernel recursive least square algorithm

[0122] 1. Kernel function selection: Also use Gaussian kernel function.

[0123] 2. Weight update: Update weights using recursive least squares method:

[0124]

[0125] in, d i =[d1,d2,...,d i ] T .

[0126] Example 3: Kernel adaptive filtering algorithm based on t distribution in latent space

[0127] 1. Kernel function selection: Use a kernel function based on Student's t distribution:

[0128]

[0129] 2. Loss function: Use the mean p-th error (MPE) criterion:

[0130]

[0131] 3. Weight update: By minimizing the above loss function, solve the weight parameter w i .

[0132] This article describes a latent space kernel adaptive filtering method for sound source localization, including pre-training, training, and deployment phases, and provides three different algorithm implementations. This method combines latent space models with kernel adaptive filtering techniques to improve the accuracy and robustness of sound source localization while maintaining lightweight and real-time processing capabilities.

[0133] The present invention verifies the superiority of the latent space kernel adaptive filtering method through sound source localization experiments, and uses two real-world sound source localization datasets to demonstrate the performance of the present invention. The first real-world scenario is a closed laboratory. The principle of the scenario is as follows: Figure 3 The second real scene is the turbine room of a hydropower station, as shown in the photo. Figure 4 As shown in the original example Figure 5 shown.

[0134] Experimental process and Figure 1 The process of adaptive data processing method for high-precision positioning is shown Figure 1 First, place the signal receivers in the following Figure 4 and Figure 5 At the training data reference points shown in , the audio signals corresponding to each position are recorded, and then the training data sets corresponding to the two experiments are generated. Subsequently, in order to test the performance of the proposed data processing method, the signal receivers are placed as follows Figure 4 and Figure 5The random test points shown are recorded on each position audio signal, and the test data sets corresponding to the two experiments are generated. The training data set and the test data set contain data sequences {u(i), d(i)}, and the audio data u(i) and the corresponding actual position d(i) of the two data sets are known.

[0135] During the experiment, first, the input audio data u(i) and the corresponding actual position d(i) of the training data set are input into the LS-KAF for feature extraction and filter learning. Then, only the audio data u(i) of the test data set is input into the LS-KAF to obtain the corresponding estimated position y(i), so as to realize the function of online application. Finally, the mean square error (MSE) is used to measure the difference between the actual position d(i) of the test data set and the estimated position y(i) obtained by testing, reflecting the accuracy of the positioning data processing.

[0136] The experiment and the classical sound source positioning algorithm (SRP-PHAT, MUSIC) based on deep learning algorithm (CNN) are compared in the two representative indoor positioning experimental scenes described above.

[0137] The performance qualitative comparison results of the high-precision positioning adaptive data processing method proposed in the application and other kernel adaptive filtering algorithms are shown in Figure 6 The results show that the sound source positioning accuracy of the method proposed in the application is the highest in the representative scenes with different interference.

[0138] The sound source positioning prediction error and standard deviation of LS-KAF and the comparison algorithm are shown in Table 1 and Table 2. The results show that the LS-KAF applying the method proposed in the application has the highest positioning accuracy and higher robustness compared with the classical sound source positioning method and the sound source positioning algorithm based on deep learning.

[0139] In summary, compared with the classical sound source positioning method and the sound source positioning algorithm based on deep learning, the potential space kernel adaptive filtering method of the application has significant advantages in sound source positioning prediction accuracy.

Claims

1. A latent space kernel adaptive filtering method for sound source localization, comprising the following steps: Step 1. Pre-training phase: Using known sound source signals to train an autoencoder to minimize reconstruction error, obtaining an encoder that can extract key features for sound source localization. The autoencoder projects high-dimensional audio data into a low-dimensional latent space, reducing the data dimension. The autoencoder has an encoder-decoder architecture. The encoder compresses the input data into a low-dimensional representation and uses multiple convolutional layers and pooling layers to extract features. The encoder ultimately outputs a latent representation u, representing the compressed features. The decoder restores the low-dimensional representation to the original data x. The decoder uses transposed convolutional layers and convolutional layers to gradually restore the data, ultimately outputting the restored data. Step 2. Training phase: The encoder in the pre-trained autoencoder is used to convert the sound source signal into a latent representation. The latent representation is then mapped to the reproducing kernel Hilbert space using a kernel function. In the reproducing kernel Hilbert space, a nonlinear relationship model between the sound source signal and its position is established through linear filtering, and the parameters of the nonlinear relationship model are learned. The specific method of step 2 is: Step 2.

1. Feature extraction: Use the pre-trained encoder to convert the sound source signal into a latent representation u t ; Step 2.

2. Kernel function mapping: Use the kernel function to transform the potential representation u t High-dimensional eigenvectors mapped to the reproducing kernel Hilbert space Where κ(·) represents the kernel function; Step 2.

3. Nonlinear relationship modeling: In the reproducing kernel Hilbert space, a nonlinear relationship between the potential representation and the sound source position is established through a linear filtering method; the sound source position is estimated: in, Represents the estimated sound source position, w T represents the filter weight; Step 2.

4. Nonlinear relationship model parameter learning: Use the corresponding weight update method to learn the nonlinear relationship model parameters w; Step 3. Deployment phase: In practical applications, the unknown sound source signal is processed; first, the encoder is used to obtain the latent representation, then it is mapped to the reproducing kernel Hilbert space through the kernel function, and finally the nonlinear relationship learned in the training phase is used to estimate the position of the sound source.

2. The latent space kernel adaptive filtering method for sound source localization according to claim 1, wherein: The specific method of step 1 is: Step 1.

1. Data acquisition and processing: Collect the sound source signal at a known location, process it using short-time Fourier transform, and obtain high-dimensional time-frequency domain audio data x; Step 1.

2. Autoencoder training: Encoder: Contains 6 convolutional layers and 3 pooling layers, arranged in the order of convolutional layer-pooling layer, gradually extracting and compressing features; Decoder: Contains 2 transposed convolutional layers and 2 convolutional layers, arranged in the order of transposed convolutional layer-convolutional layer, gradually recovering data; Specific structure of encoder: Convolutional layer 1: The input data passes through the first convolutional layer and outputs a feature map; Pooling layer 1: A pooling layer is added after the first convolutional layer to reduce the size of the feature map; Convolutional layer 2: The pooled feature map is input into the second convolutional layer to extract more complex features; Pooling layer 2: The second convolutional layer is followed by a second pooling layer to further reduce the size of the feature map; Convolutional layer 3: The pooled feature map is input into the third convolutional layer to continue extracting features; Pooling layer 3: The third convolutional layer is followed by a third pooling layer to further compress the feature map; Convolutional layer 4: the last convolutional layer, outputting low-dimensional feature representation; Decoder specific structure: Transposed convolution layer 1: The low-dimensional feature representation is input to the first transposed convolution layer to expand the feature map size; Convolutional layer 1: The output of the transposed convolutional layer passes through the first convolutional layer for further feature processing; Transposed convolution layer 2: The feature map after the convolution layer is input to the second transposed convolution layer to further expand the size of the feature map; Convolutional layer 2: The last convolutional layer, which outputs a result with the same shape as the original input data; Train an autoencoder using audio data x to minimize the reconstruction error Where x is the reconstructed audio signal; Training is based on reconstruction error Use the optimizer to traverse the training set, calculate the loss and perform backpropagation in each training cycle; Step 1.

3. Get the encoder: After training is completed, the encoder part is retained and used for subsequent feature extraction; the encoder maps high-dimensional audio data x to a low-dimensional potential representation u: u=f e (W e x+b e ) (4) Among them, f e (.) represents the activation function, W e represents the weight matrix, b e Indicates bias.

3. The latent space kernel adaptive filtering method for sound source localization according to claim 1, wherein: The specific method of step 3 is: Step 3.

1. Unknown sound source signal processing: For the unknown sound source signal to be located, use the encoder to obtain the low-dimensional potential representation u g ; Step 3.

2. Position estimation: Map the latent representation to the reproducing kernel Hilbert space, and then use the learned nonlinear relationship to estimate the sound source position:

4. The latent space kernel adaptive filtering method for sound source localization according to claim 1, wherein: The weight update method in step 2.4 is: Latent Space Kernel Least Mean Square Algorithm: This algorithm uses a Gaussian kernel function, the minimum mean square error criterion, and a stochastic gradient descent update strategy to achieve a balance between computational efficiency and positioning accuracy. Kernel function selection: Use Gaussian kernel function κ h (u i ,u j ): Among them, u i 、u j Represents the sound feature vector corresponding to the sound source, h represents the kernel bandwidth, ‖‖ 2 represents the Euclidean distance; Loss function: Use the minimum mean square error (MSE) criterion: Among them, J(w) represents the filter loss function, d i represents the true position corresponding to the i-th training point, N represents the size of the training set, Represents the high-dimensional feature vector of sound; Weight update: Update weights using stochastic gradient descent method: Among them, η represents the learning rate, w i represents the filter weight, e i Indicates positioning error.

5. The latent space kernel adaptive filtering method for sound source localization according to claim 1, wherein: The weight update method in step 2.4 is: Latent Space Kernel Recursive Least Squares Algorithm: This algorithm uses a Gaussian kernel function and a recursive least squares method to improve convergence speed and is suitable for scenarios that require fast response. Kernel function selection: Gaussian kernel function; Weight update: Update the weights using recursive least squares method: in, d i =[d1,d2,...,d i ] T , λ represents the regularization factor, I represents the identity matrix, Represents the high-dimensional feature vector of the sound, d i Represents the true position corresponding to the i-th training point.

6. The latent space kernel adaptive filtering method for sound source localization according to claim 1, wherein: The weight update method in step 2.4 is: Latent space t-distributed kernel adaptive filtering algorithm: uses a t-distributed kernel function and a mean p-th power error loss function to improve positioning accuracy and robustness in non-Gaussian noise environments; Kernel function selection: Use kernel function κ based on Student's t distribution v,σ (u i ,u j ): Among them, v represents the degree of freedom, which refers to the number of independent information in the sample data; σ 2 Represents the population variance and affects the shape of the t distribution; Loss function: Use the average p-th square error criterion: Among them, J(w i ) represents the loss function, κ v,σ (e j ) represents the error metric based on the kernel function, e j represents the positioning error, p represents the power of the error, λ represents the regularization factor, and w i represents the filter weight; Weight update: By minimizing the above loss function, solve the weight parameter w i .

Images