Residual network sound source localization method and system based on fusion attention mechanism

Abstract

The invention discloses a residual network sound source localization method and system based on a fusion attention mechanism, and the method comprises the steps: adding a 1 * 1 convolution kernel, adding an extra SEAM module to form an SC-SEAM module, adding the SC-SEAM module in the last two stages of a residual network structure, forming a residual network architecture of a fusion channel attention mechanism module, and carrying out the sound source localization of the residual network. An indoor sound source positioning model is trained and generated, and the method comprises the following steps: after a microphone array obtains a to-be-positioned sound signal, STFT is carried out on the to-be-positioned sound signal received by each microphone; extracting a phase component between the STFT coefficients of the sound signals to be positioned received by each microphone, and obtaining a difference to obtain a phase difference diagram between each microphone; and inputting the phase difference image into an indoor sound source positioning model, and outputting an estimated angle value of the sound signal to be positioned. The method is high in sound source positioning accuracy and strong in robustness in indoor reverberation, noise and other multi-scene complex environments.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent perception, and specifically relates to a residual network sound source localization method and system based on a fusion attention mechanism. Background Art

[0002] With the rapid development of the intelligent era, sound source localization technology, as an important technical direction in the field of intelligent perception, has received more and more attention. Sound source localization is widely used in scenarios such as human-computer interaction, intelligent video and voice, smart home, and smart cockpit. In these scenarios, sound, as a natural and effective way of information transmission, is crucial to the construction of intelligent systems. Therefore, the development of technology to accurately perceive the location of sound sources in indoor scene environments has become a research hotspot for current scholars. Traditional sound source localization methods are usually based on the extraction process of sound features, which has a large amount of calculation and may not meet the requirements in certain specific scenarios. In a reverberant environment, the MUSIC (Multiple Signal Classification) positioning technology has an inaccurate number of signals due to multipath propagation in the received signal, which causes the performance of the MUSIC algorithm to be disturbed; another classic sound source localization method, SRP-PHAT (Steered Response Power with PhaseTransform, based on the maximum controllable response power beamforming method), has been proven by scholars to be not robust to non-stationary signals such as speech. Therefore, the biggest challenge facing traditional positioning methods is that their positioning capabilities drop sharply in indoor environments with noise and reverberation. Therefore, studying sound source localization methods that are robust to noise and reverberation has become an urgent problem to be solved. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the present invention provides a residual network sound source localization method and system based on a fusion attention mechanism, which can achieve accurate estimation of the sound source position in multiple complex environments such as indoor reverberation and noise.

[0004] On the one hand, an embodiment of the present invention provides a residual network sound source localization method based on a fusion attention mechanism, by adding a 1×1 convolution kernel and an additional SEAM module to form an SC-SEAM module, adding the SC-SEAM module to the last two stages of the residual network structure, forming a residual network architecture of a fusion channel attention mechanism module, training and generating an indoor sound source localization model, the method includes: After the microphone array obtains a sound signal to be located, the sound signal to be located received by each microphone is subjected to STFT; Extract the phase components between the STFT coefficients of the sound signal to be located received by each microphone, and obtain the difference to obtain the phase difference diagram between the microphones; The phase difference map is input into the indoor sound source localization model, and the estimated angle value of the sound signal to be located is output.

[0005] Optionally, the residual network is specifically a ResNet34 network.

[0006] Optionally, a dataset for training a generated indoor sound source localization model is constructed by synthesizing a noise signal.

[0007] Optionally, after the sound signal to be located is subjected to STFT, it can be expressed as: , in, represents the amplitude component, Indicates the nth time frame, kth frequency The phase component of the STFT coefficients of the m-th microphone received signal.

[0008] Optionally, the phase difference between the microphones is expressed as: .

[0009] Optionally, performance evaluation metrics for the indoor sound source localization model include: MAE, RMSE, and ACC.

[0010] Optionally, MAE is expressed as: , in, is the total number of time frames, is the true angle value of the i-th time frame, is the estimated angle value at the i-th time frame.

[0011] Optionally, RMSE is expressed as: , in, is the total number of time frames, It is The true angle value of the time frame, It is Estimated angle value for time frames.

[0012] Optionally, ACC is expressed as: , in, represents the number of such time frames where the estimated angle corresponds to the true angle, Represents the total number of time frames in the test dataset.

[0013] On the other hand, an embodiment of the present invention further provides a residual network sound source localization system based on a fusion attention mechanism, comprising: A microphone array, used to obtain a sound signal to be located; The STFT unit is used to perform STFT on the sound signals to be located received by each microphone; An extraction unit, used to extract the phase components between the STFT coefficients of the sound signal to be located received by each microphone; A calculation unit, used for obtaining a phase difference diagram between microphones based on the phase components between the STFT coefficients of the sound signal to be located; The indoor sound source localization model is used to take the phase difference map as input and output the estimated angle value of the sound signal to be located.

[0014] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: The present invention provides a residual network sound source localization method based on a fusion attention mechanism, which has high accuracy and strong robustness in localizing the sound source in complex environments such as indoor reverberation and noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a residual module in one embodiment of the present invention; Figure 2 This is a flow chart of the SEAM attention mechanism algorithm in one embodiment of the present invention; Figure 3 This is a schematic diagram of the ResNet34 network structure in one embodiment of the present invention; Figure 4 This is a schematic diagram of the SC-SEAM network structure in one embodiment of the present invention; Figure 5 This is a ResNet34 network architecture diagram of a fused channel attention mechanism module in one embodiment of the present invention; Figure 6 Schematic diagram of a sound source localization method based on a residual network architecture with a fusion channel attention mechanism module in one embodiment of the present invention; Figure 7 Schematic diagram of microphone array and sound source distribution in one embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to overcome the shortcomings of the prior art, the present invention provides a residual network sound source localization method and system based on a fusion attention mechanism, which can achieve accurate estimation of the sound source position in multiple complex environments such as indoor reverberation and noise.

[0017] At present, artificial intelligence technology has been widely used in the field of sound source localization. The development of artificial intelligence technology has brought revolutionary changes to the field of signal processing. Among them, deep learning, especially CNN (Convolutional Neural Networks), has achieved remarkable success in image classification. This successful experience has begun to be widely used in the field of sound source localization, providing new ideas and technical means for the processing and positioning of sound signals. Deep learning provides an end-to-end learning framework, that is, learning features and tasks directly from raw data by training the network without manually designing feature extractors. In the task of sound source localization, it means that the original sound signal can be used directly, allowing the network to automatically learn and extract features related to the positioning task. Traditional sound source localization methods usually rely on professional domain knowledge to manually design feature extractors. CNN in deep learning can automatically learn complex feature representations, so as to better adapt to different environments and acoustic scenes. CNN can have strong generalization ability through large-scale data training, so that the model can also show good performance on unseen data. In complex environments with multiple scenarios such as indoor reverberation and noise, the patent of this invention proposes a method for sound source localization using an improved convolutional neural network. The effectiveness of this method can be demonstrated by comparing it with similar methods in terms of positioning accuracy and robustness.

[0018] The present invention provides a residual network sound source localization method based on a fusion attention mechanism, which forms an SC-SEAM module by adding a 1×1 convolution kernel and an additional SEAM module, and adds the SC-SEAM module to the last two stages of the residual network structure to form a residual network architecture that integrates a channel attention mechanism module, and trains and generates an indoor sound source localization model. The method includes: After the microphone array obtains a sound signal to be located, the sound signal to be located received by each microphone is subjected to STFT; Extract the phase components between the STFT coefficients of the sound signal to be located received by each microphone, and obtain the difference to obtain the phase difference diagram between the microphones; The phase difference map is input into the indoor sound source localization model, and the estimated angle value of the sound signal to be located is output.

[0019] The specific plan is as follows: (1) Convolutional Neural Network The convolutional neural network consists of an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer.

[0020] Input Layer: The input layer is used to receive raw input data, usually image data. Image data is represented in matrix form, and each element represents a pixel value. It should be noted that the image data in the present invention refers to the time-frequency spectrum of the sound signal after processing.

[0021] Convolution Layer: The convolution layer usually uses convolution kernels to perform feature extraction and feature mapping operations on the input image. The convolution process can be expressed as formula (1): (1) In the formula, is the current layer number, is the convolution kernel corresponding to different input feature maps, is the additive bias corresponding to the output feature map.

[0022] Pooling Layer: The pooling layer is mainly used to downsample the input feature map to reduce the amount of data and the number of parameters while retaining important feature information. It moves on the input feature map by sliding the window and applies some pooling operation (such as maximum pooling or average pooling) on ​​each window.

[0023] Fully Connected Layer: In a fully connected layer, each neuron is connected to all neurons in the previous layer. The fully connected layer is usually located at the end of the neural network and is responsible for converting the feature maps of the previous layers into the final output.

[0024] Output Layer: The output layer is responsible for generating the final output of the network. The design of this layer depends on the type of problem that the network is to solve. In classification problems, the output layer usually uses the Softmax activation function to convert the network output into probabilities of each category.

[0025] (2) Residual Network Traditional neural networks train the network layer by layer by learning the mapping relationship between input and output data. However, as the number of network layers increases, the gradient may disappear, making network training more difficult. To solve this problem, the ResNet network structure introduces a residual block. Figure 1 shown.

[0026] In the residual block, the network no longer learns the mapping of the input data, but learns the residual of the input data (that is, the difference between the output and input of the current layer), and then adds this residual to the input to get the output. This design makes it easier for the network to learn identity mapping, thereby avoiding the problem of gradient disappearance. Classic ResNet network models include ResNet-18, ResNet-34, ResNet-50, ResNet-101 and ResNet-152, etc., which have different numbers of layers and parameter scales.

[0027] (3) Attention Mechanism SEAM (Squeeze and Excitation Attention Mechanism) focuses on the relationship between various dimensions and adds an attention mechanism to the channel dimension. The algorithm flow chart is as follows: Figure 2 As shown in Figure 2. The purpose of the SEAM module is to use a weight matrix to assign different weights to different positions of the image from the perspective of the channel domain, so as to obtain more important feature information. The main operations of the SEAM module include: Squeeze and Excitation.

[0028] The above steps can be described as: First, given the input feature map , let it pass The data transformation operation generates the feature map U, The squeezing operation performs global average pooling on the feature map to generate a 1×1×C vector, so that each channel is represented by a numerical value. The specific implementation process of the squeezing operation can be expressed by formula 2: (2) is the squeezing operation, H, W, C are the dimension parameters of the input feature matrix, is the input feature matrix, is the channel vector after the squeeze operation is completed.

[0029] The excitation operation is completed through two fully connected layers, through the weight Generate the desired weight information, where It is obtained through learning and is used to show the required feature correlation. The specific excitation operation can be expressed by formula 3: (3) is weighted To perform incentive operations, It is a fully connected layer operation with ReLU activation function. It is a fully connected layer operation with Sigmoid activation function. , is the weight coefficient, is the weight vector after the activation operation.

[0030] Through two fully connected layers and processing the vector obtained in the previous step, the desired channel weight vector is obtained . Channel weight value Different values ​​in represent the weight information between different channels, giving different weights to the channels. There is a hyperparameter between the two fully connected layers, the vector After the first fully connected layer, the dimension of (1×1×C) changes from (1×1×C) to (1×1×C / R), and then changes to (1×1×C) after the second fully connected layer. The activation function of the first fully connected layer is ReLU, and the second layer is Sigmoid. F Scale () The data preprocessing operation is to generate the channel weight value in the previous step Assign weights to the feature map X, and finally get the desired feature map, whose size is exactly the same as the feature map. The SEAM module does not change the size of the feature map. This operation can be expressed by formula 4: (4) is to perform weight assignment operation, is the desired feature matrix.

[0031] The channel weight values ​​generated by (1×1×C) and the input feature matrix (H×W×C) corresponds to channel multiplication, that is, the H×W values ​​of each channel in the feature map are multiplied by the channel weight value The weight of the corresponding channel in .

[0032] (4) Residual network with attention mechanism Consider combining the classic ResNet34 network with an additional attention mechanism to improve the accuracy of the model. In order for the network to learn more valuable features, the attention mechanism module can dynamically learn the weight of each channel so as to pay more attention to the channels that are meaningful to the task. The classic ResNet34 network structure is as follows: Figure 3 shown.

[0033] Then, the SEAM module is introduced to build an improved ResNet-34 network model based on the channel attention mechanism module. At the same time, the SEAM module is improved by adding a 1×1 convolution kernel to add additional spatial attention. The improved SEAM module is named SC-SEAM module, and its architecture is as follows: Figure 4 shown.

[0034] ResNet-34 usually includes 5 stages. Each stage of ResNet-34 contains multiple residual blocks. In each stage, the network reduces the size of the feature map while increasing the number of channels. SC-SEAM is a lightweight module that can be inserted into any stage of the ResNet-34 network structure. However, since the phase features are difficult to distinguish, considering that SC-SEAM is inserted into the stage of high-level feature extraction, its attention mechanism will pay more attention to relatively abstract features, which helps the model to better learn and utilize the effective features of the phase difference map for classification tasks. Therefore, only consider adding the attention mechanism during high-level feature extraction, so that the network pays more attention to the features in the phase difference map that are effective for classification tasks during learning, which is more conducive to improving the network's feature learning ability during the experiment. Based on this, the present invention considers adding the SC-SEAM module to the last two stages of the ResNet-34 network structure, and the specific network structure of ResNet34 with the improved SEAM module inserted is as follows: Figure 5 , and finally formed the ResNet34 network architecture with integrated channel attention mechanism module.

[0035] (5) Sound source angle estimation based on residual network architecture with fusion channel attention mechanism module The overall process of the residual network architecture sound source localization system framework based on the fusion channel attention mechanism module is as follows Figure 6As shown in Figure 1, since the amplitude spectrum is not used, synthetic noise signals can be used instead of actual speech to construct the training dataset. In this work, spectral white noise sources located at different angles are used to train the proposed residual network architecture fused with the channel attention mechanism module. Using synthetic noise signals to construct a dataset and train the network has significant advantages over actual speech signals. First, when constructing a training dataset, no speech database is required, making the design of ground truth labels easier. If speech signals are used to construct a training dataset, a voice activity detector (VAD) is usually required to detect silent frames, because the features from silent frames do not have a good effect on the training network, and even errors in detecting silent frames can lead to inconsistent labels, resulting in incorrect classification results. Such problems can be avoided when synthetic noise signals are used for training. The specific steps in training the network are to first convolve the sound source database signal with the simulated RIR to obtain a new signal with spatial effect, and then perform STFT (Short-Time Fourier Transform) on the sound signal received by the microphone array, extract the phase component (corresponding to the imaginary part after STFT) between the STFT coefficients of the sound signal received by each microphone, and calculate the difference between the phase components between each microphone. Finally, the phase difference map (MPD) between each microphone is obtained, and the phase difference map is fed into the improved residual neural network (i.e., the residual network architecture of the fusion channel attention mechanism module) to learn relevant features to estimate the angular position of the sound source. In this work, there is no explicit feature extraction step, but the phase difference component of the STFT coefficient of the received signal is directly provided as the input of the network model, so that the network can learn the relevant features of the sound source angle estimation from the phase difference component through training, where 70% of the phase difference map is a training set, 10% is a validation set, and 20% is a test set, and the training generates an indoor sound source localization model.

[0036] Specific application examples (1) Model construction The task goal of the project is to build an indoor sound source localization model based on the spectral features of the sound source signal, and learn the relevant features of the sound source angle estimation task by training the residual network architecture that integrates the channel attention mechanism module, without an explicit feature extraction step to calculate the input to be provided to the positioning system. Therefore, the original signal received by the microphone array is used to extract its phase component after STFT, and the phase component between the two microphones is subtracted to obtain the microphone array phase difference map (MPD) and use it as the input feature of the model. The input of the residual network framework that integrates the channel attention mechanism module is the feature representation corresponding to each STFT time frame. Consider using Nf point discrete Fourier transform (DFT) to transform the signal received by the microphone from the time domain to the frequency domain (STFT domain). In the STFT domain, the observed signal of each TF instance is represented by a complex number, so the observed signal is expressed by formula 4-8: (4-8) in, represents the amplitude component, Indicates the nth time frame, kth frequency The phase component of the STFT coefficients of the m-th microphone received signal.

[0037] In this work, the phase difference value of the STFT coefficient of the signal received by the microphone array is directly provided as the input of the positioning system. The phase component consists of the phase of the sound source and the influence of the propagation path. The idea is to allow the indoor sound source localization model to learn the relevant features of the sound source angle estimation from the microphone array phase difference component through training. The phase difference between the microphone arrays can be expressed by formula 4-9: (4-9) The phase difference refers to the phase difference between two microphones in the microphone array. Specifically, since the microphone array type used in this embodiment is a linear four-microphone array, it is assumed that the microphone numbers are , , , , according to the above description, the phase difference input into the positioning system model refers to the microphone and , and , and , and The phase difference between them, since the goal of this patent is to calculate the frame-level posterior probability of the sound source angle corresponding to the sound source signal after framing, so by dividing each time-frequency of and each microphone Combined into size The matrix forms the input features of the nth time frame, where is the total number of frequency bins (Nyquist frequency), In the matrix, refers to the total number of microphones in the microphone array, and this input feature representation is called a phase difference map. If we consider a microphone array with A microphone, , then the size of the input feature matrix is ​​expressed as Given the input feature representation, the next task is to estimate the posterior probability of the angle of each time frame. To this end, this patent proposes several supervised learning methods of residual networks fused with attention mechanisms. In this model, given the phase difference map as the input of the indoor sound source localization model, the residual network fused with attention mechanism generates the posterior probability for each sound source angle position. If the phase difference map of the nth time frame is represented as , the posterior probability generated by the improved residual network at the output is given by Given, where It is The angles corresponding to the categories are The signal block composed of time frames performs the task of sound source angle estimation. The frame-level posterior probability is obtained by of The probability of the final sound source angle estimation is obtained by averaging the frame-level posterior probabilities. The specific calculation process can be expressed by formula (4-10): (4-10) (2) Dataset construction The TIMIT speech dataset is used to build a sound source recognition model. The speech sampling frequency of the TIMIT speech dataset is 16kHz, and it contains a total of 6,300 sentences. Each of the 10 sentences is spoken by 630 people from eight major dialect areas in the United States. All sentences are manually segmented and labeled at the phoneme level. The TIMIT speech dataset includes time-based orthography, phonetic and word transcriptions, and 16-bit, 16kHz speech waveform files for each utterance.

[0038] The task of sound source estimation in this example is performed for multiple time frames, and the posterior probability of the sound source angle is estimated for each time frame. In the process of constructing training data, different levels of spectral white noise signals are considered as the basic data set, and the Pyroomacoustics platform is used to simulate the room impulse response RIRs. The configuration parameters for generating training data are shown in Table 1. Spatially uncorrelated Gaussian noise is added to the training data. The microphone array used in this embodiment is a linear four-microphone array. The angle distribution of the sound source is 0° to 180°, with a step length of 5°. There are 37 sound source positions in total, and their spatial position distribution is as shown in Figure 7 As shown. By adjusting RT60 and SNR to achieve the diversity of the dataset, the distance between the sound source and the center of the microphone array is 1m and 2m respectively, and the distance between the microphones is 10cm. Table 2 gives the parameter configurations for different room sizes used for testing, where different room sizes and positions within each room are considered to be robust under various acoustic conditions.

[0039] Table 1 Parameter configuration of each room in the training dataset

[0040] Table 2 Parameter configuration of each room in the test dataset

[0041] (3) Evaluation indicators This patent compares the proposed method with two traditional signal processing-based sound source angle estimation methods: For the MUSIC method, in order to maintain a similar comparison with the method used in this patent, the MUSIC pseudo-spectrum is calculated on each frequency sub-band of each STFT time frame, with an angular resolution of 5° in the entire angular space range, and then all frequency sub-bands are averaged to a broadband pseudo-spectrum, and the average is taken over the time frame considered in the signal block to finally obtain the predicted probability of the sound source angle. The SRP-PHAT method calculated for each time frame also performs corresponding angle estimation. For the performance evaluation of the sound source angle estimation method, three metrics are usually used: the mean absolute error (MAE) over all time frames, the root mean square error (RMSE) over all time frames, and the positioning accuracy (ACC). The mean absolute error calculated between the true angle and the estimated angle of each evaluated acoustic condition is expressed as formula 4-11: (4-11) in, is the total number of time frames, is the true angle value of the i-th time frame, is the estimated angle value at the i-th time frame.

[0042] The root mean square error (RMSE) works better when evaluating the accuracy of the prediction model because it focuses more on the variance of the error. The RMSE for each evaluated acoustic condition can be expressed by formula 4-12: (4-12) in, is the total number of time frames, It is The true angle value of the time frame, It is Estimated angle value for time frames.

[0043] This paper uses positioning accuracy (ACC) to measure the model's ability to accurately estimate angles, which is defined as the ratio of the number of such time frames where the estimated angle corresponds to the true angle to the total number of time frames. The positioning accuracy is given by Equation 4-13: (4-13) in, represents the number of such time frames where the estimated angle corresponds to the true angle, Represents the total number of time frames in the test dataset.

[0044] (4) Simulation experiment In the simulation experiment, different simulated data experiments were proposed to objectively evaluate the performance of the positioning system. In this work, the residual network architecture of the proposed fusion channel attention mechanism module was trained by using spectral white noise at different angles. A microphone array of M = 4 was considered with a microphone spacing of 10 cm. The input signal was converted to the STFT domain using a DFT length of Nf = 256 with an overlap of 50%. The sampling rate of the signal was Fs = 16 kHz, the Batch-Size was 256, and the initial learning rate e was 0.02. Various experiments were conducted on networks of different sizes. The code implementation of this patent was completed under the Pytorch framework. The architecture with the smallest average verification loss of data under different acoustic conditions was selected as the final architecture. In the test phase, the test signal was first converted to the STFT domain using the same parameters as during training, and then the phase difference map of each time frame was used as the input of the residual network architecture fused with the channel attention mechanism module. The final angle estimate of each time frame of the test signal is given by Equation 4-10.

[0045] In actual application, the position of the microphone array may be slightly offset due to various factors, such as equipment movement, unstable installation, etc. These slight position offsets may affect the performance of the algorithm, so it is necessary to verify whether the proposed method has a certain robustness through simulation experiments, that is, whether these position changes can be tolerated within a certain range without causing a sharp drop in performance. Through small perturbation simulation experiments, its performance in actual application scenarios can be better understood. The so-called slight perturbation of the microphone position, specifically, the two microphone positions in the middle of the microphone array of the test data set are moved 5mm in opposite directions, and then the sound source signal collected by the disturbed microphone array is input into the algorithm model as the test data, and finally the posterior probability of the sound source being located at each angle is obtained. The results of Tables 3 to 5 are the simulation experiment results before and after the small perturbation of the microphone position, and the values ​​in brackets are the simulation experiment results after the small perturbation. By comparing the experimental results before and after the disturbance, the adaptability of the method provided by the present invention to the slight change of the microphone position can be evaluated, and then its robustness and stability can be determined.

[0046] Table 3 The impact of microphone micro-perturbations on MAE results under different signal-to-noise ratio conditions

[0047] Table 4 The impact of small microphone disturbances on RMSE results under different signal-to-noise ratio conditions

[0048] Tables 3 and 4 record the impact of small microphone disturbances on MAE and RMSE results under different signal-to-noise ratio conditions. From the results, it can be seen that the mean absolute errors of various algorithms increase to a certain extent after small microphone disturbances. Compared with the traditional MUSIC and SRP-PHAT algorithms, the residual network architecture proposed in the present invention that integrates the channel attention mechanism module has a smaller increase in the mean absolute error and root mean square error after small microphone disturbances. Therefore, it can be seen that the method proposed in the present invention has better adaptability to small changes in the microphone position. Table 5 The impact of small microphone disturbances on ACC results under different signal-to-noise ratio conditions

[0049] Table 5 records the impact of small disturbances of the microphone on the positioning accuracy under different signal-to-noise ratio conditions. The results show that when the microphone position of the test data is slightly disturbed, the accuracy of various algorithms will decrease to a certain extent compared to before the disturbance. The degree of degradation of the positioning performance of the method proposed in this paper is smaller than that of the traditional sound source localization method. Therefore, it can be considered that the method proposed in this paper has a certain degree of robustness to small disturbances of the microphone array.

[0050] The present invention also provides a residual network sound source localization system based on a fusion attention mechanism, which forms an SC-SEAM module by adding a 1×1 convolution kernel and an additional SEAM module, and adds the SC-SEAM module to the last two stages of the residual network structure to form a residual network architecture that integrates the channel attention mechanism module, and trains and generates an indoor sound source localization model. The system includes: A microphone array, used to obtain a sound signal to be located; The STFT unit is used to perform STFT on the sound signals to be located received by each microphone; An extraction unit, used to extract the phase components between the STFT coefficients of the sound signal to be located received by each microphone; A calculation unit, used for obtaining a phase difference diagram between microphones based on the phase components between the STFT coefficients of the sound signal to be located; The indoor sound source localization model is used to take the phase difference map as input and output the estimated angle value of the sound signal to be located.

[0051] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A residual network sound source localization method based on fusion attention mechanism, characterized in that: The SC-SEAM module is formed by adding a 1×1 convolution kernel and an additional SEAM module. The SC-SEAM module is added to the last two stages of the residual network structure to form a residual network architecture that integrates the channel attention mechanism module. The indoor sound source localization model is trained and generated. The method includes: After the microphone array obtains a sound signal to be located, the sound signal to be located received by each microphone is subjected to STFT; Extract the phase components between the STFT coefficients of the sound signal to be located received by each microphone, and calculate the difference to obtain the phase difference diagram between the microphones; The phase difference map is input into the indoor sound source localization model, and the estimated angle value of the sound signal to be located is output.

2. The method according to claim 1, characterized in that The residual network is specifically a ResNet34 network.

3. The method according to claim 1, characterized in that The dataset for training the generated indoor sound source localization model is constructed by synthesizing noise signals.

4. The method according to claim 1, characterized in that After the sound signal to be located is subjected to STFT, it can be expressed as: ， in, represents the amplitude component, Indicates the nth time frame, kth frequency The phase component of the STFT coefficients of the m-th microphone received signal.

5. The method according to claim 1, characterized in that The phase difference between the microphones is expressed as: 。 6. The method according to claim 1, characterized in that The performance evaluation metrics of indoor sound source localization models include: MAE, RMSE, and ACC.

7. The method according to claim 6, characterized in that MAE is expressed as: ， in, is the total number of time frames, is the true angle value of the i-th time frame, is the estimated angle value at the i-th time frame.

8. The method according to claim 6, characterized in that RMSE is expressed as: ， in, is the total number of time frames, It is The true angle value of the time frame, It is Estimated angle value for time frames.

9. The method according to claim 6, characterized in that ACC is expressed as: ， in, represents the number of such time frames where the estimated angle corresponds to the true angle, Represents the total number of time frames in the test dataset.

10. A residual network sound source localization system based on fusion attention mechanism, characterized in that: By adding a 1×1 convolution kernel and an additional SEAM module to form an SC-SEAM module, the SC-SEAM module is added to the last two stages of the residual network structure to form a residual network architecture that integrates the channel attention mechanism module, and trains and generates an indoor sound source localization model. The system includes: A microphone array, used to obtain a sound signal to be located; The STFT unit is used to perform STFT on the sound signals to be located received by each microphone; An extraction unit, used to extract the phase components between the STFT coefficients of the sound signal to be located received by each microphone; A calculation unit, used for obtaining a phase difference diagram between microphones based on the phase components between the STFT coefficients of the sound signal to be located; The indoor sound source localization model is used to take the phase difference map as input and output the estimated angle value of the sound signal to be located.

Images