Sound recognition method based on discriminative representation loss and attention convolutional network

Abstract

The invention discloses a sound recognition method based on discriminative representation loss and attention convolutional network, comprising the following steps: first, collecting ambient sound audio samples; then, preprocessing the audio samples and extracting log-mel spectrum features; then, inputting the log-mel spectrum features into the attention convolutional network in sequence, and using a discriminative loss function determined by feature loss and cross-entropy loss to supervise the training of the attention convolutional network; finally, inputting the log-mel spectrum features of the audio sample to be tested into the trained attention convolutional network to obtain the sound recognition result of the sample to be tested. The method of the invention designs an attention convolutional network with a parallel architecture, and defines a discriminative loss function to supervise the training of the attention convolutional network. The method of the invention can effectively extract global and local time-frequency information from audio samples, and has better intra-class compactness and inter-class separation. Compared with traditional methods, the method of the invention has better sound recognition performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio signal processing and pattern recognition, and particularly relates to a sound recognition method based on discriminative representation loss and attention convolutional network. BACKGROUND

[0002] The goal of sound event detection is to accurately recognize various target sound events in audio recordings. It has applications in many fields related to machine listening, such as traffic monitoring, intelligent conference rooms, automatic assisted driving, and multimedia analysis.

[0003] Currently, the classifiers for sound event detection can be divided into deep models and shallow models. Shallow models have weak generalization ability, poor stability in different environments, and limited feature extraction capability, and cannot capture complex features in audio data. Deep models mainly include convolutional recurrent neural networks, recurrent neural networks, and convolutional neural networks. Convolutional recurrent neural networks can recognize audio in complex environments, but require a large amount of training data, high computational resource requirements, and are prone to overfitting when the amount of data is insufficient; recurrent neural networks cannot effectively capture long-term dependencies, resulting in reduced recognition accuracy; convolutional neural networks mainly focus on extracting local features, which can lead to information loss during sound recognition.

[0004] Therefore, when there is no large amount of sample data to support and the classifier is required to have strong stability in different environments, a convolutional neural network that can focus on core features without losing other local features is needed. At the same time, in order to ensure higher accuracy of the classifier, the audio should have stronger intra-class compactness and inter-class separability. SUMMARY

[0005] The purpose of the present application is to solve the defects of existing convolutional neural networks in audio recognition in terms of data diversity and recognition accuracy, and to provide a sound recognition method based on discriminative representation loss and attention convolutional network. The present application first obtains a discriminative representation loss function by integrating a cross-entropy loss function and a feature loss function, and applies it in the training of the attention convolutional network. The present application can better understand the input audio features and improve the accuracy of audio recognition under the premise of lower computational complexity.

[0006] The purpose of the present application can be achieved by adopting the following technical solutions:

[0007] A sound recognition method based on discriminative representation loss and attention convolutional network, the sound recognition method comprising the following steps:

[0008] S1, collecting audio samples: placing a sound pickup device in the area to be measured to collect the sound of the area to be measured to obtain audio samples;

[0009] S2, extract the log-mel spectrum feature: pre-emphasize, frame and window the collected audio sample, obtain the linear spectrum of the audio sample through fast Fourier transform, and obtain the log-mel spectrum feature of the audio sample after filter bank and logarithmic operation;

[0010] S3, build an attention convolutional network: build an attention convolutional network composed of parallel convolutional transformation modules and attention transformation modules;

[0011] S4, train the attention convolutional network: under the supervision of the discriminative loss function, input the log-mel spectrum feature of the training audio sample to train the attention convolutional network;

[0012] S5, identify the sound category of the test sample: input the log-mel spectrum feature of the test sample into the attention convolutional network to obtain the sound recognition result of the test sample.

[0013] Further, the process of collecting audio samples in step S1 is as follows:

[0014] S1.1, place the pickup in the test area to collect the environmental sound of the test area;

[0015] S1.2, set the sampling parameters, wherein the sampling frequency is 16kHz, the quantization accuracy is 16bits, and the number of sound channels is single sound channel.

[0016] Further, the process of extracting the log-mel spectrum feature in step S2 is as follows:

[0017] S2.1, pre-emphasize the audio sample using a first-order high-pass filter, and the transfer function of the filter is H(z)=1-az -1 , wherein a is the pre-emphasis coefficient of the filter, and the value is 0.95. Pre-emphasis can improve audio quality and extract pure original audio signals;

[0018] S2.2, divide the pre-emphasized audio sample into short-time frames, the frame length is 256 sampling points, and the frame shift is 128 sampling points, then multiply the short-time frame by the Hamming window function to obtain the windowed short-time audio frame, and the time domain expression of the Hamming window function is: ,0 , wherein T is the frame length;

[0019] S2.3, perform discrete Fourier transform on the windowed short-time audio frame to obtain the linear spectrum of the frame;

[0020] S2.4, convert the linear spectrum to a mel spectrum using a mel filter, and take the logarithm of the mel spectrum to obtain the log-mel spectrum;

[0021] S2.5, arrange the log-mel-spectrogram of all short-time frames into a feature matrix M ij The number of rows i of the feature matrix represents the number of frames, and the number of columns j of the feature matrix represents the feature dimension of the short-time frame.

[0022] Further, the attention convolutional network comprises a convolutional transformation module, an attention transformation module and module fusion.

[0023] The convolutional transformation module performs blueprint separable convolution on the input features, and then performs batch normalization and global normalization processing, wherein the batch normalization normalizes different types of audio features, which helps to accelerate convergence and reduce training time, and the global normalization normalizes the same type of audio, avoiding the influence of the mean and variance between the same type of audio.

[0024] Further, the process of building the attention convolutional network in step S3 is as follows:

[0025] S3.1, pre-process the input feature matrix M ij to obtain non-overlapping feature subgraphs, which are used as inputs of the convolutional transformation module and the attention transformation module;

[0026] S3.2, construct a convolutional transformation module, which performs blueprint separable convolution operation and batch normalization operation on the input features in each feature dimension, and then uses ReLU function and global response normalization operation for transformation;

[0027] S3.3, construct an attention transformation module, the branch that independently learns different feature dimensions and calculates attention weights is called an attention head, the output weight matrix of each attention head is calculated, then a fully connected layer is used for linear transformation, and the distance between each input sample and other samples of the same class is calculated to measure the intra-class compactness, the distance between each input sample and samples of different classes is calculated to measure the inter-class separability, better intra-class compactness and inter-class separability can greatly improve the accuracy of sound recognition.

[0028] S3.4, fuse the attention transformation module and the convolutional transformation module, and use a discriminative loss function to supervise the training.

[0029] Further, the pre-processing process in step S3.1 is as follows:

[0030] S3.1.1, independently perform convolution operation on each feature dimension to generate an output feature matrix with the same number of input feature dimensions.

[0031] S3.1.2, a depth separable convolution operation is performed on the output feature matrix to obtain a transformed feature matrix, and a ReLU function is used to transform the feature matrix, the depth separable convolution operation can reduce the number of parameters, improve the calculation speed, and improve the generalization ability of the network;

[0032] S3.1.3, the mean and variance of all transformed feature matrices are calculated, and the mean and variance are used to normalize the feature matrix of each sample, and the normalization formula is ,

[0033] wherein represents the normalized feature element, represents the feature element in the input feature matrix, is the mean of the input sample feature element, is the standard deviation of the input sample feature element;

[0034] S3.1.4, the normalized feature matrix is scaled and translated, and the feature matrix in each feature dimension is spliced, so that the feature matrix in each feature dimension has similar distribution; S3.1.5, the spliced feature matrix is divided into feature sub-matrix of the same size and non-overlapping, and the feature sub-matrix is input into the convolution transformation module and the attention transformation module respectively.

[0035] Further, the depth separable convolution operation process in step S3.1.2 is as follows:

[0036] First, apply spatial convolution to the feature matrix in each feature dimension of the input to generate a plurality of intermediate feature matrices of different feature dimensions; then, point-by-point convolution is performed on the plurality of intermediate feature matrices of different feature dimensions to integrate the information of different feature dimensions, and an integrated feature matrix is output.

[0037] The depth separable convolution decomposes the standard convolution process into spatial convolution and point-by-point convolution, which can greatly reduce the number of parameters and the amount of calculation, speed up the model training speed, and reduce the risk of overfitting.

[0038] Further, the blueprint separable convolution operation process in step S3.2 is as follows:

[0039] The input feature matrix and the convolution kernel are divided into multiple groups, and convolution operation is independently performed in each group, and then the convolution results of each group are combined to output the combined feature matrix.

[0040] The blueprint separable convolution independently performs convolution operation in each group, which helps to extract features between different groups, and can improve the efficiency of the model and reduce the computational complexity.

[0041] Furthermore, the process of calculating the attention and output weight matrix of each attention head in step S3.3 is as follows: , where X represents the input feature matrix, vector Q represents the feature vector that the attention module needs to find, vector K represents the degree of correlation between Q and other feature vectors, vector V represents the search content, and W Q , W K and W V Represents the weight matrix of three linear mappings;

[0042] Assuming the number of attention heads is h, the calculation formula for multi-head attention is:

[0043] M(Q, K, V)=Concat(head1, head2, ..., head p , ...,head h )W O

[0044] Where M(Q, K, V) represents the result of multi-head attention calculation, head p represents the attention calculation result of the p-th attention head, p=1, 2, ..., h, W O Represents the output weight matrix, which is used to linearly map the results of multi-head attention to obtain the final output;

[0045] Attention of each head p The calculation formula is:

[0046] Head p =Attention(QW p Q , KW p K , VW p V )

[0047] Among them, QW p Q , KW p K , VW p V Represents the weight matrix of the pth attention head, which is used to linearly map matrices Q, K, and V respectively. The Attention() function calculates the influence weight of each input weight matrix on each element of the output sequence.

[0048] Furthermore, the fusion process of the attention transformation module and the convolution transformation module in step S3.4 is as follows:

[0049] S3.4.1. Weighted sum of the outputs of the attention transform module and the convolution transform module where C represents the information after weighted summation, represents the information output by the attention transformation module, represents the information output by the convolution transformation module, is the weighting coefficient of the module regulated by the discriminative representation loss function, 0 <1, different application environments can improve the applicability of the model by adjusting the value of the;

[0050] S3.4.2, group, transpose and re-partition the feature matrix of different feature dimensions to realize shuffling, and exchange feature information using a fully connected layer. The calculation formula of the fully connected layer exchange is:

[0051]

[0052] wherein, is the input vector, is the weight matrix of the fully connected layer exchange, is the bias vector of the fully connected layer exchange, is the output vector.

[0053] Further, the discriminative loss function in step S3.4 is defined as follows:

[0054] Let M r be the prototype of the rth audio, which is defined as the average vector of all audio samples of the rth audio, The calculation formula of M r is wherein R is the number of categories of audio samples; the total loss is composed of the cross-entropy loss and the feature loss together train the attention convolution network, and the total loss is defined as , wherein, and are defined as follows:

[0055]

[0056]

[0057] wherein, wherein, is the prediction result, represents that the prediction result is the rth audio, represents the probability that the recognition result is the rth when the audio sample representation is A r , and represents the audio loss, represents the cross-loss between audios and is defined as follows:

[0058]

[0059] in, Indicates other types of audio except the rth type of audio, , express In parallel with The direction component, express In vertical The direction component, express In parallel with The direction component, express In vertical directional component, R is the number of audio samples of the rth audio class;

[0060] in and The definition is as follows:

[0061] exist Under the supervision of r The parallel component of and vertical component Prototype A relative to the nth audio r The maximum and minimum values ​​will be obtained respectively. If the audio classification effect is good, then A r Will be close to M r ,keep away , indicating that audio classification has smaller cross loss;

[0062] exist Under the supervision of r parallel component and vertical component The ∑i=1,i=2,i,=1 ...

[0063] Furthermore, the training process in step S4 is as follows:

[0064] S4.1. Input the audio features into the attention convolutional network based on the discriminative representation loss function;

[0065] S4.2. Minimize the repeated training loss function until the discriminative loss function is less than a pre-specified threshold.

[0066] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0067] (1) The application uses an attention transformation network, can selectively focus on the core part of the input data by regulating the feature weight, reduces the interference of irrelevant information, and improves the stability of audio recognition.

[0068] (2) The application parallel splices the convolution transformation module and the attention transformation module, optimizes the splicing parameters, and can be adaptively adjusted for different application environments, so that the core features are focused on in audio recognition while other local features are not lost.

[0069] (3) The application uses a parallel attention convolution network, which has the characteristics of small operation scale and strong feature recognition ability.

[0070] (4) The application defines a discriminative representation loss function to supervise training, which integrates cross-entropy loss and feature loss. The cross-entropy loss reflects the difference between the output probability distribution of the attention convolution network and the true label, and the feature loss reflects the intra-class compactness and inter-class separability of the audio, thereby improving the accuracy of audio recognition.

[0071] (5) The network structure of the application can be used as a general recognition method framework and applied to image recognition and other scenes, which can reduce the operation scale and improve the classification accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0073] Figure 1 is a schematic diagram of the convolution transformation module in the application;

[0074] Figure 2 is a schematic diagram of the attention transformation module in the application;

[0075] Figure 3 is a schematic diagram of the parallel component and the vertical component in the feature loss in the application;

[0076] Figure 4 is a flowchart of the sound recognition method based on the discriminative representation loss and the attention convolution network disclosed in the application. DETAILED DESCRIPTION

[0077] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0078] Embodiment 1

[0079] Figure 4 A flowchart of a sound recognition method based on a discriminative representation loss and an attention convolutional network is disclosed, as shown in Figure 4 The present embodiment discloses a sound recognition method based on a discriminative representation loss and an attention convolutional network, which comprises the following steps:

[0080] S1, collecting an audio sample: placing a sound pickup device in a to-be-tested area to collect the sound of the to-be-tested area to obtain an audio sample;

[0081] In the present embodiment, the specific process of step S1 of collecting an audio sample comprises the following steps:

[0082] S1.1, placing a sound pickup device in a to-be-tested area to collect the environmental sound of the to-be-tested area;

[0083] S1.2, setting the sampling parameters, wherein the sampling frequency is 16 kHz, the quantization accuracy is 16 bits, and the number of sound channels is single sound channel.

[0084] S2, extracting a log-mel spectrum feature: performing pre-emphasis, framing and windowing processing on each collected audio sample, obtaining the linear spectrum of the audio sample through fast Fourier transform, and obtaining the log-mel spectrum feature of the audio sample after filter bank and logarithmic operation;

[0085] In the present embodiment, the official dataset of DCASE2023 is used, and the extraction of the log-mel spectrum in step S2 comprises the following steps:

[0086] S2.1, performing pre-emphasis processing on the audio sample using a first-order high-pass filter, and the transfer function of the filter is

[0087] H(z) = 1 - az -1 , wherein a is the pre-emphasis coefficient of the filter, and the value is 0.95;

[0088] S2.2, dividing the pre-emphasized audio sample into short-time frames, the frame length is 256 sampling points, and the frame shift is 128 sampling points; then multiplying by a window function to obtain a windowed short-time audio frame;

[0089] S2.3, Discrete Fourier Transform is performed on the windowed short-time audio frame to obtain the linear spectrum of the frame;

[0090] S2.4, the linear spectrum is converted into a Mel spectrum by using a Mel filter, and the Mel spectrum is logarithmized to obtain a logarithmic Mel spectrum;

[0091] S2.5, the logarithmic Mel spectrums of all short-time frames are arranged into a feature matrix M ij , the number of rows i of the feature matrix represents the frame number, and the number of columns j of the feature matrix represents the feature dimension of the short-time frame.

[0092] In this embodiment, the length of each frame is 256 sampling points, and the sampling frequency is 16 kHz, so the time length of each frame is 256 / 16000 = 0.016 seconds. The sampling precision is 16 bits, that is, there are 2^16 discrete levels, so each sampling point needs 16 bits to represent, and a new frame is taken every 128 sampling points, so the feature dimension after extracting the logarithmic Mel spectrum is 13, that is , the data amount of each frame is 512 bytes.

[0093] S3, building an attention convolutional network: building an attention convolutional network composed of parallel convolutional transformation modules and attention transformation modules;

[0094] In this embodiment, the step S3 of building the attention convolutional network specifically includes the following steps:

[0095] S3.1, preprocessing the input feature matrix M ij to obtain non-overlapping feature subgraphs, which are simultaneously used as the inputs of the convolutional transformation modules and the attention transformation modules;

[0096] In this embodiment, the preprocessing process in step S3.1 is as follows:

[0097] S3.1.1, independently performing convolution operation on each feature dimension to generate an output feature matrix with the same number of input feature dimensions;

[0098] S3.1.2, performing depth separable convolution operation on the output feature matrix to obtain a transformed feature matrix, and then transforming the feature matrix by using a ReLU function;

[0099] S3.1.3, calculating the mean and variance of all transformed feature matrices, and normalizing the feature maps of each sample using the above mean and variance, and the normalization formula is , wherein denotes the normalized feature element, denotes the feature element in the input feature matrix, is the average value of the input sample feature element, to input the sample feature elements;

[0100] S3.1.4, scaling and translating the normalized feature matrix, and splicing the feature matrix in each feature dimension, so that the feature matrix in each feature dimension has similar distribution;

[0101] S3.1.5, dividing the spliced feature map into a feature submap with a size of 4x4, and inputting the feature submap into a convolution transformation module and an attention transformation module, respectively.

[0102] In this embodiment, the process of the depth separable convolution operation in step S3.1.2 is as follows:

[0103] First, apply spatial convolution to the feature matrix in each feature dimension of the input to generate a plurality of intermediate feature matrices of different feature dimensions; then, perform point-by-point convolution on the plurality of intermediate feature matrices of different feature dimensions to integrate the information of different feature dimensions together, and output the integrated feature matrix.

[0104] The depth separable convolution decomposes the standard convolution process into two steps of spatial convolution and point-by-point convolution, which can greatly reduce the parameter quantity and calculation quantity, speed up the model training speed, and reduce the risk of overfitting.

[0105] S3.2, constructing a convolution transformation module, the convolution transformation module performing a depth separable convolution operation and a batch normalization operation on the input feature in each feature dimension, and then using a ReLU function and a global response normalization operation for transformation;

[0106] In this embodiment, the specific process of building the convolution network in step S3.2 is as follows:

[0107] S3.2.1, defining the input layer: the frame length of the input sequence is 256, and the feature dimension of each frame is 13.

[0108] S3.2.2, building a depth separable convolution module: dividing the input feature matrix into multiple groups, independently performing convolution operation in each group, then merging the convolution results of each group, and outputting the merged feature matrix, the depth separable convolution independently convolves in each group, which helps to extract features between different groups, and can improve the efficiency of the model and reduce the computational complexity.

[0109] S3.2.3, performing batch normalization and ReLU activation on the feature matrix output by the depth convolution, and performing global response normalization, and outputting the obtained feature matrix.

[0110] S3.3, construct attention transformation module, independent learning of different feature dimensions and calculation of attention weight branch is called attention head, calculate the output weight matrix of each attention head, then adopt full connection layer to carry out linear transformation, and calculate the distance between each input sample and other samples of the same class, which is used to measure the compactness within the class, calculate the distance between each input sample and samples of different classes, which is used to measure the separation between classes;

[0111] In the embodiment, the attention and output weight matrix process of each attention head in step S3.3 is as follows: , wherein X represents an input feature matrix, vector Q represents a feature vector that needs to be searched by the attention module, vector K represents the correlation degree of Q and other feature vectors, vector V represents the search content, W Q , W K and W V represent weight matrices of three linear mappings;

[0112] The number of attention heads is set to h, and the calculation formula of multi-head attention is as follows:

[0113] M(Q, K, V)=Concat(head1, head2,..., head p ,...,head h )W O

[0114] , wherein M(Q, K, V) represents the calculation result of multi-head attention, head p represents the attention calculation result of the pth attention head, p=1, 2, …, h, W O represents an output weight matrix, which is used to linearly map the result of multi-head attention to obtain the final output;

[0115] The attention head of each head head p is calculated according to the following formula: Head p =Attention(QW p Q , KW p K , VW p V )

[0116] , wherein QW p Q , KW p K , VW p VWp represents the weight matrix of the pth attention head, which is used for linear mapping of the matrices Q, K and V, respectively, and the Attention() function calculates the weight of the influence of each weight matrix of the input on each element of the output sequence.

[0117] S3.4, fuse the attention transformation module and the convolution transformation module, and supervise the training using a discriminative loss function.

[0118] In this embodiment, the fusion process of the attention transformation module and the convolution transformation module in step S3.4 is as follows:

[0119] S3.4.1, weighted sum the outputs of the attention transformation module and the convolution transformation module ,

[0120] where C represents the information after weighted sum, represents the information output by the attention transformation module, represents the information output by the convolution transformation module, is a module weighting coefficient regulated by the discriminative representation loss function;

[0121] S3.4.2, group, transpose and re-partition the feature matrices of different feature dimensions to realize shuffling, and exchange feature information using a fully connected layer, and the calculation formula of the fully connected layer exchange is:

[0122] where, is an input vector, is a weight matrix of the fully connected layer exchange, is a bias vector of the fully connected layer exchange, is an output vector.

[0123] In this embodiment, the discriminative loss function in step S3.4 is defined as follows:

[0124] Let M r represent the prototype of the rth audio, which is defined as the average vector of all audio samples of the rth audio , the calculation formula of M r is where R is the number of categories of audio samples; the total loss is composed of the cross-entropy loss and the feature loss , and the total loss is used to train the attention convolution network together, and the total loss is defined as , where and are defined as follows:

[0125]

[0126]

[0127] To predict the results, Indicates that the prediction result is the rth type of audio, Indicates that the audio sample is represented as A r When , the probability that the identification result is the rth class is, Indicates audio loss, represents the cross-loss between audios and is defined as follows:

[0128]

[0129] in, Indicates other types of audio except the rth type of audio, , express In parallel with The direction component, express In vertical The direction component, express In parallel with The direction component, express In vertical directional component, R is the number of audio samples of the rth audio category;

[0130] in and The definitions are as follows:

[0131] exist Under the supervision of r The parallel component of and vertical component Relative to the prototype A of the rth audio r The maximum and minimum values ​​will be obtained respectively. If the audio classification effect is good, then A r Will be close to M r ;

[0132] exist Under the supervision of r parallel component and vertical component Respectively minimized and maximized relative to other audio categories.

[0133] S4. Training the Attention Convolutional Network: Under the supervision of the discriminative loss function, the log-mel spectrum features of the training audio samples are used as input to train the Attention Convolutional Network;

[0134] In the present embodiment, the training process in step S4 is specifically as follows:

[0135] S4.1, input the audio features into the attention convolutional network based on the discriminative representation loss function;

[0136] S4.2, minimize the repeated training loss function until the discriminative loss function is less than a pre-specified threshold.

[0137] S5, recognize the sound category of the to-be-tested sample: input the log-mel spectrum features of the to-be-tested sample into the attention convolutional network to obtain the sound recognition result of the to-be-tested sample.

[0138] To sum up, the present embodiment designs an attention convolutional network with a parallel architecture and defines a discriminative loss function to supervise the training of the attention convolutional network. The sound recognition method in the present embodiment can effectively extract the global and local time-frequency information in the audio sample and has better intra-class compactness and inter-class separability. Compared with the traditional method, the sound recognition method disclosed in the present embodiment has better sound recognition performance.

[0139] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement modes and shall be included in the protection scope of the present application.

Claims

1. A sound recognition method based on discriminative representation loss and attention convolutional network, characterized in that: The sound recognition method comprises the following steps: S1. Collecting audio samples: placing a microphone in the area to be tested, collecting the sound in the area to be tested to obtain audio samples; S2. Extracting logarithmic Mel spectrum features: Pre-emphasize, frame, and window each collected audio sample, obtain the linear spectrum of the audio sample through fast Fourier transform, and obtain the logarithmic Mel spectrum features of the audio sample after filter bank and logarithmic operation; S3. Build an attention convolutional network: Build an attention convolutional network consisting of a parallel convolutional transformation module and an attention transformation module; S4. Training the Attention Convolutional Network: Under the supervision of a discriminative loss function, the logarithmic Mel-spectrogram features of the training audio samples are used as input to train the Attention Convolutional Network. The discriminative loss function is defined as follows: The prototype of the r-th audio is recorded as M r , defined as the feature vector of all audio samples of the rth audio class A r The average vector of M r The calculation formula is , where R is the number of audio sample types; using cross entropy loss and feature loss The total loss Jointly train attention convolutional networks, total loss Defined as ,in, and The definitions are as follows: in, To predict the results, Indicates that the prediction result is the rth type of audio, Indicates that the audio sample is represented as A r When , the probability that the identification result is the rth class is, Indicates audio loss, represents the cross-loss between audios and is defined as follows: in, Indicates other types of audio except the rth type of audio, , express In parallel with The direction component, express In vertical The direction component, express In parallel with The direction component, express In vertical The direction component, R r is the number of audio samples of the rth audio category; in and The definitions are as follows: exist Under the supervision of A r The parallel component of and vertical component The prototype of the r-th audio will get the maximum and minimum values ​​respectively. If the audio classification effect is good, then A r Will be close to M r ; exist Under the supervision of r parallel component and vertical component will be minimized and maximized relative to other audio categories, respectively; S5. Identify the sound category of the sample to be tested: Use the logarithmic Mel spectrum features of the sample to be tested as the input of the attention convolutional network to obtain the sound recognition result of the sample to be tested.

2. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 1 is characterized in that The process of collecting audio samples in step S1 is as follows: S1.

1. Place the microphone in the area to be tested to collect the ambient sound in the area to be tested; S1.

2. Set the sampling parameters, where the sampling frequency is 8kHz-20kHz, the quantization accuracy is 4bits-64bits, and the number of channels is mono or dual.

3. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 1 is characterized in that The process of extracting the logarithmic Mel spectrum features in step S2 is as follows: S2.1, use a first-order high-pass filter to pre-emphasize the audio sample. The transfer function of the filter is H(z)=1-az -1 , where a is the pre-weighted coefficient of the filter, ranging from 0 to 1; S2.2, dividing the pre-emphasized audio sample into short-time frames, and then multiplying them by the window function to obtain windowed short-time audio frames; S2.

3. Performing a discrete Fourier transform on the windowed short-time audio frame to obtain a linear spectrum of the frame; S2.

4. Use a Mel filter to convert the linear spectrum into a Mel spectrum, and then take the logarithm of the Mel spectrum to obtain a logarithmic Mel spectrum; S2.

5. Arrange the logarithmic Mel spectra of all short time frames into a feature matrix M ij , the number of rows i of the feature matrix represents the frame number, and the number of columns j of the feature matrix represents the feature dimension of the short-time frame.

4. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 1, characterized in that The process of building the attention convolutional network in step S3 is as follows: S3.

1. Input feature matrix M ij Preprocessing is performed to obtain non-overlapping feature subgraphs, which serve as inputs to both the convolutional transformation module and the attention transformation module. S3.

2. Construct a convolutional transformation module. The convolutional transformation module performs a blueprint-separable convolution operation and a batch normalization operation on the input features in each feature dimension, and then uses the ReLU function and the global response normalization operation to transform; S3.

3. Construct an attention transformation module. The branches that independently learn different feature dimensions and calculate attention weights are called attention heads. The output weight matrix of each attention head is calculated. Then, a fully connected layer is used to perform linear transformation. The distance between each input sample and other samples of the same category is calculated to measure intra-class compactness. The distance between each input sample and samples of different categories is calculated to measure inter-class separability. S3.

4. Fusion of the attention transformation module and the convolutional transformation module, and supervised training using a discriminative loss function.

5. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 4 is characterized in that: The pre-processing process in step S3.1 is as follows: S3.1.

1. Perform convolution operations on each feature dimension independently to generate an output feature matrix with the same number of input feature dimensions. S3.1.

2. Perform a depthwise separable convolution operation on the output feature matrix to obtain a transformed feature matrix, and then use the ReLU function to transform the feature matrix; S3.1.

3. Calculate the mean and variance of all transformed feature matrices, and use the above mean and variance to normalize the feature map of each sample. The normalization formula is: , in represents the normalized feature element, Represents the characteristic elements in the input feature matrix, is the average value of the characteristic elements of the input sample, is the standard deviation of the characteristic elements of the input sample; S3.1.

4. Scale and translate the normalized feature matrix, and concatenate the feature matrices on each feature dimension so that the feature matrices on each feature dimension have similar distributions. S3.1.

5. Divide the concatenated feature map into non-overlapping feature sub-maps of the same size, and then input the feature sub-maps into the convolution transformation module and the attention transformation module respectively.

6. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 4, characterized in that The process of calculating the attention and output weight matrix of each attention head in step S3.3 is as follows: Where X represents the input feature matrix, vector Q represents the feature vector that the attention module needs to find, vector K represents the degree of correlation between Q and other feature vectors, vector V represents the search content, and W Q , W K and W V Represents the weight matrix of three linear mappings; Assuming the number of attention heads is h, the calculation formula for multi-head attention is: M(Q, K, V)=Concat(head1, head2, ..., head p , ...,head h )W O In the formula, M(Q, K, V) represents the multi-head attention calculation result, head p represents the attention calculation result of the p-th attention head, p=1, 2, ..., h, W O Represents the output weight matrix, which is used to linearly map the results of multi-head attention to obtain the final output; Attention of each head p The calculation formula is: Head p =Attention(QW p Q , KW p K , VW p V ) Among them, QW p Q , KW p K , VW p V Represents the weight matrix of the pth attention head, which is used to linearly map matrices Q, K, and V respectively. The Attention() function calculates the influence weight of each input weight matrix on each element of the output sequence.

7. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 4, characterized in that: The fusion process of the attention transformation module and the convolution transformation module in step S3.4 is as follows: S3.4.

1. Weighted sum of the outputs of the attention transform module and the convolution transform module , where C represents the weighted sum of information, Represents the information output by the attention transformation module, Represents the information of the convolution transform module output, is the module weight coefficient regulated by the discriminative representation loss function; S3.4.

2. Group, transpose, and repartition the feature matrices of different feature dimensions to achieve shuffling, and use the fully connected layer to exchange feature information. The calculation formula for the fully connected layer exchange is: in, is the input vector, is the weight matrix exchanged by the fully connected layer, is the bias vector exchanged by the fully connected layer, is the output vector.

8. The sound recognition method based on discriminative representation loss and attention convolutional network according to claim 1, characterized in that: The training process in step S4 is as follows: S4.

1. Input the audio features into the attention convolutional network based on the discriminative representation loss function; S4.

2. Minimize the repeated training loss function until the discriminative loss function is less than a pre-specified threshold.

Images