Voice identification method based on discriminative representation loss and attention convolutional network

Abstract

The invention discloses a sound identification method based on discriminative representation loss and an attention convolutional network. The method comprises the following steps: firstly, collecting an environment sound audio sample; preprocessing the audio sample, and extracting logarithmic Mel spectrum features; then inputting the logarithmic Mel spectrum features into an attention convolutional network in sequence, and supervising attention convolutional network training by adopting a distinguishing loss function determined by feature loss and cross entropy loss; and finally, inputting the logarithmic Mel spectrum features of a to-be-detected audio sample into the trained attention convolutional network to obtain a sound identification result of the to-be-detected sample. According to the method, an attention convolutional network of a parallel architecture is designed, and a distinguishing loss function is defined to supervise training of the attention convolutional network. According to the method, global and local time frequency information in the audio sample can be effectively extracted, and better intra-class compactness and inter-class separability are achieved. Compared with the traditional method, the method provided by the invention has better sound identification performance.

Description

Technical Field

[0001] The present invention relates to the technical field of audio signal processing and pattern recognition, and in particular to a sound recognition method based on discriminative representation loss and attention convolutional network. Background Art

[0002] The goal of sound event detection is to accurately identify various target sound events in audio recordings. It has applications in many areas related to machine monitoring, such as traffic monitoring, smart conference rooms, automatic assisted driving, and multimedia analysis.

[0003] At present, the classifiers for sound event detection can be divided into deep models and shallow models. Shallow models have weak generalization ability, poor stability in different environments, and limited feature extraction capabilities, and cannot capture complex features in audio data. Deep models mainly include convolutional recurrent neural networks, recurrent neural networks, and convolutional neural networks. Convolutional recurrent neural networks can perform audio recognition in complex environments, but they require a large amount of training data, high computing resource requirements, and are prone to overfitting when the amount of data is insufficient; recurrent neural networks cannot effectively capture long-term dependencies, resulting in reduced recognition accuracy; convolutional neural networks mainly focus on the extraction of local features, which can easily lead to information loss during sound recognition.

[0004] Therefore, when there is no large amount of sample data as support and the classifier is required to have strong stability in different environments, it is necessary to propose a convolutional neural network that can focus on core features without losing other local features. At the same time, in order to ensure that the classifier has higher accuracy, the audio should have stronger intra-class compactness and inter-class separation. Summary of the invention

[0005] The purpose of the present invention is to solve the defects of existing convolutional neural networks in data diversity and recognition accuracy when performing audio recognition, and to provide a sound recognition method based on discriminative representation loss and attention convolutional network. The present invention first obtains a discriminative representation loss function by integrating the cross entropy loss function and the feature loss function, and applies it to the training of the attention convolutional network. The present invention can better understand the input audio features and improve the accuracy of audio recognition under the premise of having a low amount of calculation.

[0006] The purpose of the present invention can be achieved by adopting the following technical solutions: A sound recognition method based on discriminative representation loss and attention convolutional network, the sound recognition method comprising the following steps: S1. Collecting audio samples: placing a microphone in the area to be tested, collecting the sound in the area to be tested to obtain audio samples; S2. Extracting logarithmic Mel spectrum features: performing pre-emphasis, framing and windowing processing on each collected audio sample, obtaining the linear spectrum of the audio sample by fast Fourier transform, and obtaining the logarithmic Mel spectrum features of the audio sample after filter bank and logarithmic operation; S3. Build an attention convolutional network: Build an attention convolutional network consisting of a parallel convolutional transformation module and an attention transformation module; S4. Train the attention convolutional network: Under the supervision of the discriminative loss function, the logarithmic Mel-spectrogram features of the training audio samples are used as input to train the attention convolutional network; S5. Identify the sound category of the sample to be tested: use the logarithmic Mel spectrum features of the sample to be tested as the input of the attention convolutional network to obtain the sound recognition result of the sample to be tested.

[0007] Furthermore, the process of collecting audio samples in step S1 is as follows: S1.1. Place the microphone in the area to be tested to collect the ambient sound in the area to be tested; S1.2. Set the sampling parameters, where the sampling frequency is 16kHz, the quantization accuracy is 16 bits, and the number of channels is mono.

[0008] Furthermore, the process of extracting the logarithmic Mel spectrum features in step S2 is as follows: S2.1, use a first-order high-pass filter to pre-emphasize the audio sample, the filter transfer function is H(z)=1-az -1 , where a is the pre-weighted coefficient of the filter, which is set to 0.95. Pre-emphasis can improve the audio quality and extract a purer original audio signal; S2.2, the pre-emphasized audio samples are divided into short-time frames, the frame length is 256 sampling points, the frame shift is 128 sampling points, and then the short-time frame is multiplied by the Hamming window function to obtain the windowed short-time audio frame. The time domain expression of the Hamming window function is: ,0 , where T is the frame length; S2.3, performing discrete Fourier transform on the windowed short-time audio frame to obtain a linear spectrum of the frame; S2.4, using a Mel filter to convert the linear spectrum into a Mel spectrum, and then taking the logarithm of the Mel spectrum to obtain a logarithmic Mel spectrum; S2.5. Arrange the logarithmic Mel spectra of all short time frames into a feature matrix M ij , the number of rows i of the feature matrix represents the frame number, and the number of columns j of the feature matrix represents the feature dimension of the short-time frame.

[0009] Furthermore, the attention convolutional network includes a convolutional transformation module, an attention transformation module and module fusion.

[0010] The convolution transformation module performs blueprint separable convolution on the input features, and then performs batch normalization and global normalization processing, wherein batch normalization normalizes the input audio features of different categories, which helps to accelerate convergence and reduce training time, and global normalization normalizes the input audio of the same category to avoid the influence of the average value and variance between the same category audio.

[0011] Furthermore, the process of building the attention convolutional network in step S3 is as follows: S3.1. Input feature matrix M ij Preprocessing is performed to obtain non-overlapping feature sub-graphs, which are used as inputs to the convolutional transformation module and the attention transformation module; S3.2, construct a convolution transformation module, which performs a convolution operation and a batch normalization operation on the input features in each feature dimension, and then uses the ReLU function and the global response normalization operation to transform; S3.3. Construct an attention transformation module. The branch that independently learns different feature dimensions and calculates attention weights is called an attention head. The output weight matrix of each attention head is calculated, and then a fully connected layer is used for linear transformation. The distance between each input sample and other samples of the same category is calculated to measure the intra-class compactness. The distance between each input sample and samples of different categories is calculated to measure the inter-class separability. Better intra-class compactness and inter-class separability can greatly improve the accuracy of sound recognition.

[0012] S3.4. Fusion the attention transformation module and the convolution transformation module, and use the discriminative loss function to supervise the training.

[0013] Furthermore, the preprocessing process in step S3.1 is as follows: S3.1.1. Perform convolution operations on each feature dimension independently to generate an output feature matrix with the same number of input feature dimensions.

[0014] S3.1.2. Perform a deep separable convolution operation on the output feature matrix to obtain a transformed feature matrix, and then use the ReLU function to transform the feature matrix. The deep separable convolution operation can reduce the number of parameters, increase the calculation speed, and improve the generalization ability of the network; S3.1.3. Calculate the mean and variance of all transformed feature matrices, and use the mean and variance to normalize the feature matrix of each sample. The normalization formula is: , in represents the normalized feature element, represents the feature elements in the input feature matrix, is the average value of the input sample feature elements, is the standard deviation of the characteristic elements of the input sample; S3.1.4. Scale and translate the normalized feature matrix, and concatenate the feature matrices on each feature dimension so that the feature matrices on each feature dimension have similar distributions. S3.1.5. Divide the concatenated feature matrix into feature sub-matrices of the same size and without overlapping, and then input the feature sub-matrices into the convolution transformation module and the attention transformation module respectively.

[0015] Furthermore, the depthwise separable convolution operation process in step S3.1.2 is as follows: First, spatial convolution is applied to the feature matrix on each feature dimension of the input to generate intermediate feature matrices of multiple feature dimensions; then point-by-point convolution is performed on the intermediate feature matrices of multiple dimensions to integrate the information of different feature dimensions and output the integrated feature matrix.

[0016] Depthwise separable convolution decomposes the standard convolution process into two steps: spatial convolution and point-by-point convolution, which can significantly reduce the number of parameters and computation, speed up model training, and reduce the risk of overfitting.

[0017] Furthermore, the blueprint separable convolution operation process in step S3.2 is as follows: The input feature matrix and convolution kernel are divided into multiple groups, and convolution operations are performed independently in each group. Then, the convolution results of each group are merged and the merged feature matrix is ​​output.

[0018] Blueprint separable convolution performs independent convolution operations within each group, which helps to extract features between different groups, improve the efficiency of the model and reduce computational complexity.

[0019] Furthermore, the process of calculating the attention and output weight matrix of each attention head in step S3.3 is as follows: , where X represents the input feature matrix, vector Q represents the feature vector that the attention module needs to find, vector K represents the degree of correlation between Q and other feature vectors, vector V represents the search content, and W Q , W K and W V Represents the weight matrix of three linear mappings; Assuming the number of attention heads is h, the calculation formula for multi-head attention is: M(Q, K, V)=Concat(head 1 , head 2 , ..., head p , ...,head h )W O In the formula, M(Q, K, V) represents the multi-head attention calculation result, headp represents the attention calculation result of the pth attention head, p=1, 2, ..., h, W O Represents the output weight matrix, which is used to linearly map the results of multi-head attention to obtain the final output; The attention of each head p The calculation formula is: Head p =Attention(QW p Q , KW p K , VW p V ) Among them, QW p Q , KW p K , VW p V Represents the weight matrix of the pth attention head, which is used to linearly map matrices Q, K, and V respectively. The Attention() function calculates the influence weight of each input weight matrix on each element of the output sequence.

[0020] Furthermore, the fusion process of the attention transformation module and the convolution transformation module in step S3.4 is as follows: S3.4.1. Weighted sum of the outputs of the attention transform module and the convolution transform module , where C represents the weighted sum of information, represents the information output by the attention transformation module, Represents the information of the convolution transform module output, is the module weight coefficient regulated by the discriminative representation loss function, 0< <1, can be adjusted in different application environments The value of improves the applicability of the model; S3.4.2. Group, transpose and re-divide the feature matrices of different feature dimensions to achieve shuffling, and use the fully connected layer to exchange feature information. The calculation formula for the fully connected layer exchange is:

[0021] in, is the input vector, is the weight matrix exchanged by the fully connected layer, is the bias vector exchanged by the fully connected layer, is the output vector.

[0022] Furthermore, the discriminative loss function in step S3.4 is defined as follows: The prototype of the r-th audio is denoted as M r , defined as the total number of audio samples of the rth audio class The average vector of r The calculation formula is , where R is the number of audio samples; using cross entropy loss and feature loss The total loss is summed up Jointly train the attention convolutional network, the total loss Defined as ,in, and The definition is as follows:

[0023]

[0024] Among them, among them, To predict the results, Indicates that the prediction result is the rth type of audio, Indicates that the audio sample is represented as A r When , the probability that the identification result is the rth class is, Indicates audio loss, represents the crossover loss between audios and is defined as follows:

[0025] in, Indicates other types of audio except the rth type of audio. , express In parallel with The direction of the component, express In vertical The direction of the component, express In parallel with The direction of the component, express In vertical directional component, R is the number of audio samples of the r-th audio category; in and The definition is as follows:

[0026] exist Under the supervision of r The parallel component and the vertical component Relative to the prototype A of the nth audior The maximum and minimum values ​​will be obtained respectively. If the audio classification effect is good, then A r Will be close to M r ,keep away , indicating that audio classification has a smaller cross loss; exist Under the supervision of r parallel component and the vertical component They are minimized and maximized respectively relative to other audio categories, indicating that the audio classification has good intra-class compactness and inter-class separation.

[0027] Furthermore, the training process in step S4 is as follows: S4.1, input the audio features into the attention convolutional network based on the discriminative representation loss function; S4.2. Minimize the repeated training loss function until the discriminative loss function is less than a pre-specified threshold.

[0028] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) The present invention uses an attention transformation network, which can selectively focus on the core part of the input data by adjusting the feature weights, reduce the interference of irrelevant information, and improve the stability of audio recognition.

[0029] (2) The present invention connects the convolution transformation module and the attention transformation module in parallel and optimizes the splicing parameters. It can be adaptively adjusted for different application environments, so as to focus on the core features in audio recognition without losing other local features.

[0030] (3) The present invention uses a parallel attention convolutional network, which has the characteristics of small computational scale and strong feature recognition capability.

[0031] (4) The present invention defines a discriminative representation loss function for supervised training. The discriminative representation loss function integrates cross entropy loss and feature loss. The cross entropy loss is used to reflect the difference between the output probability distribution of the attention convolutional network and the true label. The feature loss is used to reflect the intra-class compactness and inter-class separability of the audio, thereby improving the accuracy of audio recognition.

[0032] (5) The network structure of the present invention can be used as a general recognition method framework and applied to scenarios such as image recognition, which can reduce the scale of calculation and improve the classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 It is a schematic diagram of a convolution transformation module in the present invention; Figure 2 is a schematic diagram of the attention transformation module in the present invention; Figure 3 Schematic diagram of parallel component and perpendicular component in characteristic loss in the present invention; Figure 4 It is a flowchart of the sound recognition method based on discriminative representation loss and attention convolutional network disclosed in the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Example 1 Figure 4 A flowchart of a sound recognition method based on discriminative representation loss and attention convolutional network is disclosed, such as Figure 4 As shown, this embodiment discloses a sound recognition method based on discriminative representation loss and attention convolutional network, and the sound recognition method includes the following steps: S1. Collecting audio samples: placing a microphone in the area to be tested, collecting the sound in the area to be tested to obtain audio samples; In this embodiment, the specific process of collecting audio samples in step S1 includes the following steps: S1.1. Place the microphone in the area to be tested to collect the ambient sound in the area to be tested; S1.2. Set the sampling parameters, where the sampling frequency is 16kHz, the quantization accuracy is 16 bits, and the number of channels is mono.

[0037] S2. Extracting logarithmic Mel spectrum features: performing pre-emphasis, framing and windowing processing on each collected audio sample, obtaining the linear spectrum of the audio sample by fast Fourier transform, and obtaining the logarithmic Mel spectrum features of the audio sample after filter bank and logarithmic operation; In this embodiment, the official data set of DCASE2023 is used, and extracting the logarithmic mel spectrum in step S2 specifically includes the following steps: S2.1, use a first-order high-pass filter to pre-emphasize the audio sample, the filter transfer function is H(z)=1-az -1 , where a is the pre-weighted coefficient of the filter, and its value is 0.95; S2.2, dividing the pre-emphasized audio sample into short-time frames, with a frame length of 256 sampling points and a frame shift of 128 sampling points; then multiplying it with the window function to obtain a windowed short-time audio frame; S2.3, performing discrete Fourier transform on the windowed short-time audio frame to obtain a linear spectrum of the frame; S2.4, using a Mel filter to convert the linear spectrum into a Mel spectrum, and then taking the logarithm of the Mel spectrum to obtain a logarithmic Mel spectrum; S2.5. Arrange the logarithmic Mel spectra of all short time frames into a feature matrix M ij , the number of rows i of the feature matrix represents the frame number, and the number of columns j of the feature matrix represents the feature dimension of the short-time frame.

[0038] In this embodiment, the length of each frame is 256 sampling points, and the sampling frequency is 16kHz, so the time length of each frame is 256 / 16000 = 0.016 seconds. The sampling accuracy is 16 bits, that is, there are 2^16 discrete levels, so each sampling point requires 16 bits to represent, and a new frame is taken every 128 sampling points. The feature dimension after extracting the logarithmic Mel spectrum is 13, that is ,The data volume of each frame is 512 bytes.

[0039] S3. Build an attention convolutional network: Build an attention convolutional network consisting of a parallel convolutional transformation module and an attention transformation module; In this embodiment, building the attention convolutional network in step S3 specifically includes the following steps: S3.1. Input feature matrix M ij Preprocessing is performed to obtain non-overlapping feature sub-graphs, which are used as inputs to the convolutional transformation module and the attention transformation module; In this embodiment, the preprocessing process in step S3.1 is as follows: S3.1.1. Perform convolution operations on each feature dimension independently to generate an output feature matrix with the same number of input feature dimensions; S3.1.2, perform a depth-separable convolution operation on the output feature matrix to obtain a transformed feature matrix, and then use the ReLU function to transform the feature matrix; S3.1.3. Calculate the mean and variance of all transformed feature matrices, and use the above mean and variance to normalize the feature map of each sample. The normalization formula is: ,in represents the normalized feature element, represents the feature elements in the input feature matrix, is the average value of the input sample feature elements, is the standard deviation of the characteristic elements of the input sample; S3.1.4. Scale and translate the normalized feature matrix, and concatenate the feature matrices on each feature dimension so that the feature matrices on each feature dimension have similar distributions; S3.1.5. Divide the concatenated feature map into feature sub-maps of size 4×4, and then input the feature sub-maps into the convolution transformation module and the attention transformation module respectively.

[0040] In this embodiment, the depthwise separable convolution operation process in step S3.1.2 is as follows: First, spatial convolution is applied to the feature matrix on each feature dimension of the input to generate intermediate feature matrices of multiple feature dimensions; then point-by-point convolution is performed on the intermediate feature matrices of multiple dimensions to integrate the information of different feature dimensions and output the integrated feature matrix.

[0041] Depthwise separable convolution decomposes the standard convolution process into two steps: spatial convolution and point-by-point convolution, which can significantly reduce the number of parameters and computation, speed up model training, and reduce the risk of overfitting.

[0042] S3.2, construct a convolution transformation module, which performs a convolution operation and a batch normalization operation on the input features in each feature dimension, and then uses the ReLU function and the global response normalization operation to transform; In this embodiment, the specific process of building a convolutional network in step S3.2 is as follows: S3.2.1. Define the input layer: The frame length of the input sequence is 256, and the feature dimension of each frame is 13.

[0043] S3.2.2. Build a blueprint separable convolution module: divide the input feature matrix into multiple groups, perform convolution operations independently in each group, then merge the convolution results of each group, and output the merged feature matrix. The blueprint separable convolution performs independent convolution operations in each group, which helps to extract features between different groups, improve the efficiency of the model and reduce computational complexity.

[0044] S3.2.3. Perform batch normalization and RELU activation on the feature matrix output by the blueprint convolution, perform global response normalization, and output the obtained feature matrix.

[0045] S3.3. Construct an attention transformation module. The branch that independently learns different feature dimensions and calculates attention weights is called an attention head. The output weight matrix of each attention head is calculated, and then a fully connected layer is used for linear transformation. The distance between each input sample and other samples of the same category is calculated to measure the intra-class compactness, and the distance between each input sample and samples of different categories is calculated to measure the inter-class separation. In this embodiment, the attention and output weight matrix process of each attention head in step S3.3 is as follows: , where X represents the input feature matrix, vector Q represents the feature vector that the attention module needs to find, vector K represents the degree of correlation between Q and other feature vectors, vector V represents the search content, and W Q , W K and W V Represents the weight matrix of three linear mappings; Assuming the number of attention heads is h, the calculation formula for multi-head attention is: M(Q, K, V)=Concat(head 1 , head 2 , ..., head p , ...,head h )W O In the formula, M(Q, K, V) represents the multi-head attention calculation result, head p represents the attention calculation result of the pth attention head, p=1, 2, ..., h, W O Represents the output weight matrix, which is used to linearly map the results of multi-head attention to obtain the final output; The attention of each head p The calculation formula is: Head p =Attention(QW p Q , KW p K , VW p V ) Among them, QW p Q , KW p K , VW p V Represents the weight matrix of the pth attention head, which is used to linearly map matrices Q, K, and V respectively. The Attention() function calculates the influence weight of each input weight matrix on each element of the output sequence.

[0046] S3.4. Fusion the attention transformation module and the convolution transformation module, and use the discriminative loss function to supervise the training.

[0047] In this embodiment, the fusion process of the attention transformation module and the convolution transformation module in step S3.4 is as follows: S3.4.1. Weighted sum of the outputs of the attention transform module and the convolution transform module , Where C represents the weighted sum of information, represents the information output by the attention transformation module, Represents the information of the convolution transform module output, is the module weight coefficient regulated by the discriminative representation loss function; S3.4.2. Group, transpose and re-divide the feature matrices of different feature dimensions to achieve shuffling, and use the fully connected layer to exchange feature information. The calculation formula for the fully connected layer exchange is:

[0048] in, is the input vector, is the weight matrix exchanged by the fully connected layer, is the bias vector exchanged by the fully connected layer, is the output vector.

[0049] In this embodiment, the discriminative loss function in step S3.4 is defined as follows: The prototype of the r-th audio is denoted as M r , defined as the total number of audio samples of the rth audio class The average vector of M r The calculation formula is , where R is the number of audio samples; using cross entropy loss and feature loss The total loss is summed up Jointly train the attention convolutional network, the total loss Defined as ,in, and The definitions are as follows:

[0050]

[0051] To predict the results, Indicates that the prediction result is the rth type of audio, Indicates that the audio sample is represented as A r When , the probability that the identification result is the rth class is, Indicates audio loss, represents the crossover loss between audios and is defined as follows:

[0052] in, Indicates other types of audio except the rth type of audio. , express In parallel with The direction of the component, express In vertical The direction of the component, express In parallel with The direction of the component, express In vertical directional component, R is the number of audio samples of the r-th audio category; in and The definitions are as follows:

[0053] exist Under the supervision of r The parallel component and the vertical component Relative to the prototype A of the rth audio r The maximum and minimum values ​​will be obtained respectively. If the audio classification effect is good, then A r Will be close to M r ; exist Under the supervision of r parallel component and the vertical component Relative to other audio categories will be minimized and maximized respectively.

[0054] S4. Train the attention convolutional network: Under the supervision of the discriminative loss function, the logarithmic Mel-spectrogram features of the training audio samples are used as input to train the attention convolutional network; In this embodiment, the training process in step S4 is as follows: S4.1, input the audio features into the attention convolutional network based on the discriminative representation loss function; S4.2. Minimize the repeated training loss function until the discriminative loss function is less than a pre-specified threshold.

[0055] S5. Identify the sound category of the sample to be tested: use the logarithmic Mel spectrum features of the sample to be tested as the input of the attention convolutional network to obtain the sound recognition result of the sample to be tested.

[0056] In summary, this embodiment designs an attention convolutional network with a parallel architecture and defines a discriminative loss function to supervise the training of the attention convolutional network. The sound recognition method in this embodiment can effectively extract global and local time-frequency information in audio samples, and has better intra-class compactness and inter-class separation. Compared with traditional methods, the sound recognition method disclosed in this embodiment has better sound recognition performance.

[0057] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A sound recognition method based on discriminative representation loss and attention convolutional network, characterized in that: The sound recognition method comprises the following steps: S1. Collecting audio samples: placing a microphone in the area to be tested, collecting the sound in the area to be tested to obtain audio samples; S2. Extracting logarithmic Mel spectrum features: performing pre-emphasis, framing and windowing processing on each collected audio sample, obtaining the linear spectrum of the audio sample by fast Fourier transform, and obtaining the logarithmic Mel spectrum features of the audio sample after filter bank and logarithmic operation; S3. Build an attention convolutional network: Build an attention convolutional network consisting of a parallel convolutional transformation module and an attention transformation module; S4. Train the attention convolutional network: Under the supervision of the discriminative loss function, the logarithmic Mel-spectrogram features of the training audio samples are used as input to train the attention convolutional network; S5. Identify the sound category of the sample to be tested: use the logarithmic Mel spectrum features of the sample to be tested as the input of the attention convolutional network to obtain the sound recognition result of the sample to be tested.

2. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 1 is characterized in that: The process of collecting audio samples in step S1 is as follows: S1.

1. Place the microphone in the area to be tested to collect the ambient sound in the area to be tested; S1.

2. Set sampling parameters, where the sampling frequency is 8kHz-20kHz, the quantization accuracy is 4bits-64bits, and the number of channels is mono or dual-channel.

3. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 1, characterized in that: The process of extracting the logarithmic Mel spectrum features in step S2 is as follows: S2.1, use a first-order high-pass filter to pre-emphasize the audio sample, the filter transfer function is H(z)=1-az -1 , where a is the pre-weighted coefficient of the filter, ranging from 0 to 1; S2.2, dividing the pre-emphasized audio sample into short-time frames, and then multiplying them with the window function to obtain a windowed short-time audio frame; S2.3, performing discrete Fourier transform on the windowed short-time audio frame to obtain a linear spectrum of the frame; S2.4, using a Mel filter to convert the linear spectrum into a Mel spectrum, and then taking the logarithm of the Mel spectrum to obtain a logarithmic Mel spectrum; S2.

5. Arrange the logarithmic Mel spectra of all short time frames into a feature matrix M ij , the number of rows i of the feature matrix represents the frame number, and the number of columns j of the feature matrix represents the feature dimension of the short-time frame.

4. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 1, characterized in that: The process of building the attention convolutional network in step S3 is as follows: S3.

1. Input feature matrix M ij Preprocessing is performed to obtain non-overlapping feature sub-graphs, which are used as inputs to the convolutional transformation module and the attention transformation module; S3.2, construct a convolution transformation module, which performs a convolution operation and a batch normalization operation on the input features in each feature dimension, and then uses the ReLU function and the global response normalization operation to transform; S3.

3. Construct an attention transformation module. The branch that independently learns different feature dimensions and calculates attention weights is called an attention head. The output weight matrix of each attention head is calculated, and then a fully connected layer is used for linear transformation. The distance between each input sample and other samples of the same category is calculated to measure the intra-class compactness, and the distance between each input sample and samples of different categories is calculated to measure the inter-class separation. S3.

4. Fusion the attention transformation module and the convolution transformation module, and use the discriminative loss function to supervise the training.

5. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 4 is characterized in that: The pre-processing process in step S3.1 is as follows: S3.1.

1. Perform convolution operations on each feature dimension independently to generate an output feature matrix with the same number of input feature dimensions; S3.1.2, perform a depth-separable convolution operation on the output feature matrix to obtain a transformed feature matrix, and then use the ReLU function to transform the feature matrix; S3.1.

3. Calculate the mean and variance of all transformed feature matrices, and use the above mean and variance to normalize the feature map of each sample. The normalization formula is: , in represents the normalized feature element, represents the feature elements in the input feature matrix, is the average value of the input sample feature elements, is the standard deviation of the characteristic elements of the input sample; S3.1.

4. Scale and translate the normalized feature matrix, and concatenate the feature matrices on each feature dimension so that the feature matrices on each feature dimension have similar distributions; S3.1.

5. Divide the concatenated feature map into feature sub-maps of the same size and without overlap, and then input the feature sub-maps into the convolution transformation module and the attention transformation module respectively.

6. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 4, characterized in that: The process of calculating the attention and output weight matrix of each attention head in step S3.3 is as follows: In the formula, X represents the input feature matrix, vector Q represents the feature vector that the attention module needs to find, vector K represents the degree of correlation between Q and other feature vectors, vector V represents the search content, and W Q , W K and W V Represents the weight matrix of three linear mappings; Assuming the number of attention heads is h, the calculation formula for multi-head attention is: M(Q, K, V)=Concat(head1, head2, ..., head p , ...,head h )W O In the formula, M(Q, K, V) represents the multi-head attention calculation result, head p represents the attention calculation result of the pth attention head, p=1, 2, ..., h, W O Represents the output weight matrix, which is used to linearly map the results of multi-head attention to obtain the final output; The attention of each head p The calculation formula is: Head p =Attention(QW p Q , KW p K , VW p V ) Among them, QW p Q , KW p K , VW p V Represents the weight matrix of the pth attention head, which is used to linearly map matrices Q, K, and V respectively. The Attention() function calculates the influence weight of each input weight matrix on each element of the output sequence.

7. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 4, characterized in that: The fusion process of the attention transformation module and the convolution transformation module in step S3.4 is as follows: S3.4.

1. Weighted sum of the outputs of the attention transform module and the convolution transform module , where C represents the weighted sum of information, represents the information output by the attention transformation module, Represents the information of the convolution transform module output, is the module weight coefficient regulated by the discriminative representation loss function; S3.4.

2. Group, transpose and re-divide the feature matrices of different feature dimensions to achieve shuffling, and use the fully connected layer to exchange feature information. The calculation formula for the fully connected layer exchange is: in, is the input vector, is the weight matrix exchanged by the fully connected layer, is the bias vector exchanged by the fully connected layer, is the output vector.

8. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 4, characterized in that: The discriminative loss function in step S3.4 is defined as follows: The prototype of the r-th audio is denoted as M r , defined as the total number of audio samples of the rth audio class The average vector of M r The calculation formula is , where R is the number of audio samples; using cross entropy loss and feature loss The total loss is summed up Jointly train the attention convolutional network, the total loss Defined as ,in, and The definitions are as follows: in, To predict the results, Indicates that the prediction result is the rth type of audio, Indicates that the audio sample is represented as A r When , the probability that the identification result is the rth class is, Indicates audio loss, represents the crossover loss between audios and is defined as follows: in, Indicates other types of audio except the rth type of audio. , express In parallel with The direction of the component, express In vertical The direction of the component, express In parallel with The direction of the component, express In vertical directional component, R is the number of audio samples of the r-th audio category; in and The definitions are as follows: exist Under the supervision of r The parallel component and the vertical component Relative to the prototype A of the rth audio r The maximum and minimum values ​​will be obtained respectively. If the audio classification effect is good, then A r Will be close to M r ; exist Under the supervision of r parallel component and the vertical component Will be minimized and maximized respectively relative to other audio categories.

9. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 1, characterized in that: The training process in step S4 is as follows: S4.1, input the audio features into the attention convolutional network based on the discriminative representation loss function; S4.

2. Minimize the repeated training loss function until the discriminative loss function is less than a pre-specified threshold.

Images