Environmental sound classification method and system based on metric learning

Abstract

The invention discloses an environment sound classification method and system based on metric learning, and the method comprises the following steps: collecting an environment sound audio, and converting the environment sound audio into a Mel spectrogram; performing data enhancement on the Mel spectrogram; an SPA module and an FPL module are constructed, the SPA module or the FPL module is used to process the environment sound audio after data enhancement, a pre-trained convolutional neural network model is used to carry out feature extraction on the processed environment sound audio, and sound features are obtained; inputting the extracted sound features into a representation space; sound features are optimized in combination with comparison, metric learning and cross entropy loss in the representation space, and the optimized sound features are classified; according to the method, the intra-class variation is considered by using metric learning, so that the features of the same kind of samples are closer and the features of different kinds of samples are more separated in the process of carrying out the classification task, and the classification accuracy is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of environmental sound classification, and in particular to an environmental sound classification method and system based on metric learning. Background Art

[0002] Environmental sound classification focuses on identifying various daily audio events of different durations in a given audio signal, such as human voices, animal sounds, and urban noise. This task is of great significance and provides a wide range of practical applications in environmental awareness, urban noise monitoring, and security monitoring of smart devices.

[0003] In recent years, deep learning methods have made significant progress in environmental sound classification tasks, but they still face some challenges, such as limited annotated data, the diversity and complexity of environmental sounds, and how to reduce computational complexity while maintaining high classification accuracy. Currently, cross entropy loss has become one of the most commonly used loss functions in classification tasks. One of its disadvantages is that the boundaries between samples of different categories are poor, which reduces the generalization ability of the model trained by this loss function. Cross entropy loss focuses mainly on correct classification rather than optimizing the distance between categories, which may cause samples of different categories to be close in the feature space. There is no direct term in the loss function to increase the distance between samples of different categories.

[0004] In order to improve the discriminative ability of classifiers, contrastive learning was introduced. Early contrastive learning methods originated from self-supervised learning, using instance discrimination as a proxy task. With the development of technology, contrastive learning has been extended to supervised and semi-supervised learning paradigms, further using labeled data to enhance model feature representation. The main idea of ​​contrastive learning loss function is to design contrastive loss based on grouped positive and negative samples. The goal is to train the DNN model by narrowing the distance between the representations of positive sample pairs while making the representation distances between negative sample pairs far away from each other. However, contrastive loss functions usually use fixed weights for positive and negative sample pairs, which may not fully consider the degree of intra-class variation in the dataset.

[0005] Therefore, how to effectively learn more discriminative feature representations and how to design a model structure that can both improve classification performance and reduce computational complexity are issues that need to be urgently addressed in current ESC research. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides an environmental sound classification method and system based on metric learning, which aims to solve the problems mentioned in the background technology.

[0007] To achieve the above object, the present invention provides the following technical solution: an environmental sound classification method based on metric learning, comprising the following steps:

[0008] Step S1: Collect and pre-process the ambient sound audio, and convert the pre-processed ambient sound audio into a Mel-spectrogram;

[0009] Step S2: Perform data enhancement on the Mel-spectrogram, which includes random scaling and time-frequency masking operations;

[0010] Step S3: construct a SPA module and an FPL module, use the SPA module or the FPL module to process the data-enhanced mel-spectrogram, use the pre-trained convolutional neural network model MobileNetV2 to extract features from the processed mel-spectrogram, and obtain sound features;

[0011] Step S4: inputting the extracted sound features into the representation space;

[0012] Step S5: optimizing the sound features by combining the contrast loss function, the metric learning loss function, and the cross entropy loss function in the representation space, and projecting the optimized sound features to the projection layer;

[0013] Step S6: The sound features projected in step S5 are passed through a classifier, followed by a softmax activation function, to generate a probability distribution for each category, and the category with the highest probability is selected as the final result.

[0014] Furthermore, the SPA module is composed of a mean pooling layer, a 1×1 convolution layer, a ReLU activation function layer, a 1×1 convolution layer, a Sigmoid activation function layer, and an attention mechanism layer connected in sequence; first, the input data x, i.e., the Mel spectrum map after data enhancement, is average pooled through the mean pooling layer, and after average pooling, it is subjected to feature extraction through a 1×1 convolution layer, and after extraction, nonlinearity is introduced through a ReLU activation function layer, and after the introduction of nonlinearity, further feature extraction is performed through a 1×1 convolution layer, and the features of the further feature extraction are mapped to between 0 and 1 through a Sigmoid activation function layer, and then an attention weight is generated through an attention mechanism layer, and the attention weight is multiplied with the Mel spectrum map after data enhancement through element-by-element multiplication to obtain a weighted feature, i.e., the output of the SPA module.

[0015] Furthermore, the FPL module is composed of a spectral pooling layer SpectralPool2d, a depth separation convolution layer Depthwise Conv2d, a pointwise convolution layer Pointwise Conv2d, and an average pooling layer Mean connected in sequence. Pooling, 1×1 convolution layer, ReLU activation function layer, 1×1 convolution layer, Sigmoid activation function layer, attention mechanism layer; first, the input data x, that is, the Mel spectrum map after data enhancement, is pooled in the frequency domain through the spectral pooling layer, and then deep convolution is performed through the depth separation convolution layer, and finally point-by-point convolution is performed through the point-by-point convolution layer. The output of the point-by-point convolution layer is input to the average pooling layer for average pooling. After average pooling, it is subjected to feature extraction through a 1×1 convolution layer. After extraction, nonlinearity is introduced through the ReLU activation function layer. After the introduction of nonlinearity, it is further subjected to feature extraction through a 1×1 convolution layer. The features of the further feature extraction are mapped to 0 to 1 through the Sigmoid activation function layer, and then the attention weights are generated through the attention mechanism layer. The attention weights are multiplied by the output of the point-by-point convolution layer through element-by-element multiplication to obtain weighted features. The weighted features, the output of the point-by-point convolution layer and the input data x are residually connected to obtain the output of the FPL module.

[0016] Furthermore, the specific process of step S1 is: collecting ambient sound audio, performing silence elimination on the ambient sound audio, performing short-time Fourier transform on the ambient sound audio after silence elimination to obtain a short-time Fourier transform amplitude spectrum, inputting the short-time Fourier transform amplitude spectrum into a Mel filter group, and finally taking the logarithm of the output of the Mel filter group to obtain a Mel spectrum diagram; wherein, silence elimination of the ambient sound audio adopts an energy-based detection algorithm to remove the silent part in the ambient sound audio.

[0017] Furthermore, the specific process of step S2 is as follows:

[0018] Random scaling: randomly select a scaling factor from the range [0.8, 1.25] and perform linear interpolation scaling on the original waveform in the Mel-spectrogram;

[0019] Time-frequency masking: randomly select f frequency bands in the frequency dimension of the Mel-spectrogram and set them to zero; randomly select t time segments in the time dimension of the Mel-spectrogram and set them to zero.

[0020] Furthermore, the cross entropy loss function It is expressed as:

[0021]

[0022] In the formula, The 1-hot encoded true value label representing the sample, i.e. the sound feature; Represents the model output of a sample.

[0023] Furthermore, the contrast loss function It is expressed as:

[0024]

[0025] In the formula, Represents the sample y of category i i number; N represents the sample batch size; y i represents the sample of category i; y j represents the sample of category j; z i Represents y i The normalized projection vector of j Represents y j The normalized projection vector of k Represents the average of the normalized projection vectors of all samples of category k; τ Represents the temperature parameter.

[0026] Furthermore, the metric learning loss function combines the sparse pairwise loss function and adaptive positive mining, where the sparse pairwise loss function It is expressed as:

[0027]

[0028] In the formula, K Indicates the number of categories; Represents the similarity of sample pairs of different categories from category i; Represents the similarity of sample pairs of the same category as category i;

[0029] Adaptive forward mining dynamically calculates the harmonic mean of the most difficult and least difficult pairs of samples in the same category. The calculation steps are as follows:

[0030] Calculation of harmonic mean weight:

[0031]

[0032] In the formula, h i represents the harmonic mean weight; Indicates the similarity of the least difficult sample pairs of the same category; Represents the similarity of the most difficult sample pairs of the same category within the class;

[0033] Application of harmonic mean weights:

[0034]

[0035] In the formula, α i Indicates the weight setting, when When selecting h iCalculate the harmonic mean weight; when Select 0 to calculate the harmonic mean weight;

[0036] Calculation of similarity of weighted positive samples:

[0037]

[0038] Metric Learning Loss Function It is expressed as:

[0039]

[0040] Furthermore, the composite loss function obtained by combining the contrast loss function, the metric loss function, and the cross entropy loss function is expressed as:

[0041]

[0042] In the formula, α and β are weight coefficients.

[0043] A metric learning-based environmental sound classification system is used to implement a metric learning-based environmental sound classification method, comprising: a collection and conversion module, used to collect environmental sound audio and perform preprocessing, and convert the preprocessed environmental sound audio into a mel spectrum graph; a data enhancement module, used to perform data enhancement on the mel spectrum graph, and the data enhancement includes random scaling and time-frequency masking operations; a construction module, used to construct an SPA module and an FPL module, use the SPA module or the FPL module to process the data-enhanced mel spectrum graph, use a pre-trained convolutional neural network model MobileNetV2 to perform feature extraction on the processed mel spectrum graph, and obtain sound features; a transmission module, used to input the extracted sound features into a representation space; an optimization module, used to optimize the sound features by combining a contrast loss function, a metric learning loss function, and a cross entropy loss function in the representation space, and project the optimized sound features to a projection layer; a classification module, used to pass the sound features after the projecting in step S5 through a classifier, followed by a softmax activation function, to generate probability distributions of various categories, and select the category with the highest probability as the final result.

[0044] Compared with the existing technology, the present invention has the following beneficial effects:

[0045] (1) The present invention uses a method of taking intra-class variation into account by utilizing metric learning, so that in the process of classification tasks, the features of samples of the same class are closer and the features of samples of different classes are more separated, thereby effectively improving the classification accuracy.

[0046] (2) The present invention performs an average pooling operation on the frequency dimension, compresses the features on the time dimension into a global feature vector, aggregates the global information of each frequency band, performs pooling and convolution operations on the frequency dimension, adaptively adjusts the importance of different frequency bands, and suppresses the interference of background noise and irrelevant frequency components, thereby more effectively mining and utilizing the discriminative information in the sound signal.

[0047] (3) By learning the dependencies between different positions and channels in the feature map, the attention mechanism of the present invention can adaptively adjust the weights of each position in the feature map, so that the model can focus more on areas with rich information, thereby improving the quality of feature expression. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of the steps of the present invention.

[0049] Figure 2 It is a schematic diagram of the SPA module structure of the present invention.

[0050] Figure 3 It is a schematic diagram of the FPL module structure of the present invention. DETAILED DESCRIPTION

[0051] like Figure 1 As shown, the present invention provides a technical solution: an environmental sound classification method based on metric learning, comprising the following steps:

[0052] Step S1: Collect and preprocess the ambient sound audio, and convert the preprocessed ambient sound audio into a Mel-spectrogram.

[0053] The specific process of step S1 is: collecting ambient sound audio, performing silence elimination on the ambient sound audio, performing short-time Fourier transform (STFT) on the ambient sound audio after silence elimination to obtain a short-time Fourier transform amplitude spectrum, inputting the short-time Fourier transform amplitude spectrum into a Mel filter group, and finally taking the logarithm of the output of the Mel filter group to obtain a Mel spectrum diagram; in this embodiment, 44.1kHz ambient sound audio is used, the frame length of the ambient sound audio is 2048 samples, the frame shift is 1024 samples, and the number of Mel filters in the Mel filter group is 128; wherein, silence elimination of the ambient sound audio is performed by using an energy-based detection algorithm to remove the silent part in the ambient sound audio.

[0054] Step S2: Perform data enhancement on the Mel-spectrogram, which includes random scaling and time-frequency masking operations.

[0055] The specific process of step S2 is:

[0056] 1. Random scaling: Randomly select a scaling factor from the range [0.8, 1.25] and perform linear interpolation scaling on the original waveform in the Mel-spectrogram.

[0057] 2. Time-frequency masking: randomly select f frequency bands in the frequency dimension of the Mel-spectrogram and set them to zero; randomly select t time segments in the time dimension of the Mel-spectrogram and set them to zero.

[0058] Step S3: Construct the SPA module and the FPL module, use the SPA module or the FPL module to process the data-enhanced mel-spectrogram according to the needs, and use the pre-trained convolutional neural network model (MobileNetV2) to extract features from the processed mel-spectrogram to obtain sound features. The SPA module and the FPL module can further improve the discriminability and computational efficiency of the features.

[0059] like Figure 2 As shown in the figure, the SPA module is composed of a mean pooling layer (Mean Pooling), a 1×1 convolution layer (Conv), a ReLU activation function layer, a 1×1 convolution layer (Conv), a Sigmoid activation function layer, and an attention mechanism layer connected in sequence; first, the input data x, i.e., the Mel spectrum map after data enhancement, is average pooled through the mean pooling layer, and after average pooling, it is subjected to feature extraction through a 1×1 convolution layer, and after extraction, nonlinearity is introduced through a ReLU activation function layer, and after the introduction of nonlinearity, it is further subjected to feature extraction through a 1×1 convolution layer, and the features of the further feature extraction are mapped to between 0 and 1 through a Sigmoid activation function layer, and then an attention weight is generated through an attention mechanism layer, and the attention weight is multiplied by the Mel spectrum map after data enhancement through element-by-element multiplication to obtain a weighted feature, i.e., the output of the SPA module.

[0060] like Figure 3As shown in FIG. 1 , the FPL module is composed of a spectral pooling layer (SpectralPool2d), a depthwise convolutional layer (Depthwise Conv2d), a pointwise convolutional layer (Pointwise Conv2d), and a mean pooling layer (Mean The FPL module consists of a 1×1 convolution layer, a ReLU activation function layer, a 1×1 convolution layer (Conv), a Sigmoid activation function layer, and an attention mechanism layer. First, the input data x, i.e., the Mel-frequency spectrum after data enhancement, is subjected to a pooling operation in the frequency domain through a spectral pooling layer, and then a depth-wise convolution layer is used for depth-wise convolution. Finally, a point-by-point convolution layer is used for point-by-point convolution. The output of the point-by-point convolution layer is input to an average pooling layer for average pooling. After average pooling, a 1×1 convolution layer is used for feature extraction. After extraction, a nonlinearity is introduced through a ReLU activation function layer. After the nonlinearity is introduced, a 1×1 convolution layer is used for further feature extraction. The features of the further feature extraction are mapped to 0 to 1 through a Sigmoid activation function layer, and then an attention weight is generated through an attention mechanism layer. The attention weight is multiplied by the output of the point-by-point convolution layer through element-by-element multiplication to obtain a weighted feature. The weighted feature, the output of the point-by-point convolution layer, and the input data x are residually connected to obtain the output of the FPL module.

[0061] Step S4: input the extracted sound features into the representation space; in this embodiment, a fully connected layer is used to map the extracted sound features to the 1024-dimensional representation space of the pre-trained convolutional neural network model (MobileNetV2).

[0062] Step S5: In the representation space, the contrast loss function, the metric learning loss function, and the cross entropy loss function are combined to optimize the sound features, and the optimized sound features are projected to a 64-dimensional projection layer.

[0063] Among them, the cross entropy loss function It is expressed as:

[0064]

[0065] In the formula, Represents the 1-hot encoded true value label of the sample (sound feature); Represents the model output of a sample.

[0066] Contrastive loss function It is expressed as:

[0067]

[0068] In the formula, Represents the sample y of category i i number; N represents the sample batch size; y irepresents the sample of category i; y j represents the sample of category j; z i Represents y i The normalized projection vector of j Represents y j The normalized projection vector of k represents the average of the normalized projection vectors of all samples of category k; τ represents the temperature parameter.

[0069] The AdaSP loss function of metric learning is a combination of sparse pairwise loss function and adaptive positive mining. The core idea of ​​is to emphasize the most difficult negative sample pairs and the least difficult positive sample pairs, so as to maximize the difference between positive and negative sample pairs, which can be expressed as:

[0070]

[0071] In the formula, K represents the number of categories; Represents the similarity of sample pairs of different categories from category i; It represents the similarity of sample pairs of the same category as category i.

[0072] Adaptive forward mining dynamically calculates the harmonic mean of the most difficult and least difficult pairs of samples in the same category and adjusts the weights of these pairs accordingly, so that the loss function can adapt to different training stages and the degree of variation within the category. The calculation steps are as follows:

[0073] Calculation of harmonic mean weight:

[0074]

[0075] In the formula, h i represents the harmonic mean weight; Indicates the similarity of the least difficult sample pairs of the same category; Indicates the similarity between the most difficult pairs of samples in the same category within the class.

[0076] Application of harmonic mean weights:

[0077]

[0078] In the formula, α i Indicates the weight setting, when When selecting h i Calculate the harmonic mean weight; when Choose 0 to calculate the harmonic mean weight.

[0079] Calculation of similarity of weighted positive samples:

[0080]

[0081] Metric Learning Loss Function It is expressed as:

[0082]

[0083] The composite loss function obtained by combining the contrast loss function, metric learning loss function, and cross entropy loss function is expressed as:

[0084]

[0085] Where α and β are weight coefficients.

[0086] Step S6: The sound features after the projection in step S5 are passed through a classifier. The classifier consists of a fully connected layer with an output dimension of N (the number of environmental sound categories), followed by a softmax activation function to generate a probability distribution for each category, and the category with the highest probability is selected as the final result.

[0087] A metric learning-based environmental sound classification system is used to implement a metric learning-based environmental sound classification method, comprising: a collection and conversion module, used to collect environmental sound audio and perform preprocessing, and convert the preprocessed environmental sound audio into a mel spectrum graph; a data enhancement module, used to perform data enhancement on the mel spectrum graph, and the data enhancement includes random scaling and time-frequency masking operations; a construction module, used to construct an SPA module and an FPL module, use the SPA module or the FPL module to process the data-enhanced mel spectrum graph, use a pre-trained convolutional neural network model MobileNetV2 to perform feature extraction on the processed mel spectrum graph, and obtain sound features; a transmission module, used to input the extracted sound features into a representation space; an optimization module, used to optimize the sound features by combining a contrast loss function, a metric learning loss function, and a cross entropy loss function in the representation space, and project the optimized sound features to a projection layer; a classification module, used to pass the sound features after the projecting in step S5 through a classifier, followed by a softmax activation function, to generate probability distributions of various categories, and select the category with the highest probability as the final result.

[0088] The environmental sound classification method based on metric learning proposed in the present invention not only improves the classification performance, but also shows that the application of metric learning in the environmental sound classification method is better than contrastive learning and the mixed loss of contrastive learning and metric learning. After introducing the SPA module or the FPL module, the model classification accuracy can be improved without increasing or even reducing the amount of calculation, which is particularly suitable for scenarios with limited resources.

[0089] Prove the experiment;

[0090] As shown in Table 1, three different datasets are used: the ESC50 dataset, the ESC51 dataset, and the US8K dataset (Urban Sound 8K). The three methods of SoundCLR (contrastive learning), SoundMLR (metric learning), and the hybrid loss method (metric learning + contrastive learning) are compared. On each method, MobileNetV2 models with different configurations are tested, including: the basic MobileNetV2 model, the MobileNetV2 model + SPA module, and the MobileNetV2 model + FPL module; and three pooling factors of 0.25, 0.5, and 0.75 are tested.

[0091] It can be seen from Table 1 that the SoundMLR (metric learning) adopted by the present invention has the best accuracy of 91.75% in the ESC50 dataset with the MobileNetV2 model + SPA module configuration; on the ESC51 dataset, the MobileNetV2 model + FPL module combined with a 0.75 pooling factor configuration has the best accuracy of 92.16%; on the US8K dataset, the basic MobileNetV2 model configuration has the best accuracy of 85.22%;

[0092] Based on the above, we can know that SoundMLR (metric learning) is better than SoundCLR (contrastive learning) and the mixed loss method (metric learning + contrastive learning) in most cases. The MobileNetV2 model + SPA module configuration can usually improve its own performance. The MobileNetV2 model + FPL module configuration significantly reduces the amount of computation (FLOPs) while maintaining a high accuracy. At the same time, the FPL module uses a pooling factor of 0.5 to significantly reduce the amount of computation while still maintaining a relatively high accuracy.

[0093] Table 1 Classification accuracy test results of different methods

[0094]

[0095] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for classifying environmental sounds based on metric learning, characterized in that: The steps include: Step S1: Collect and pre-process the ambient sound audio, and convert the pre-processed ambient sound audio into a Mel-spectrogram; Step S2: Perform data enhancement on the Mel-spectrogram, which includes random scaling and time-frequency masking operations; Step S3: construct a SPA module and an FPL module, use the SPA module or the FPL module to process the data-enhanced mel-spectrogram, use the pre-trained convolutional neural network model MobileNetV2 to extract features from the processed mel-spectrogram, and obtain sound features; Step S4: inputting the extracted sound features into the representation space; Step S5: optimizing the sound features by combining the contrast loss function, the metric learning loss function, and the cross entropy loss function in the representation space, and projecting the optimized sound features to the projection layer; Step S6: The sound features projected in step S5 are passed through a classifier, followed by a softmax activation function, to generate a probability distribution for each category, and the category with the highest probability is selected as the final result.

2. The method for environmental sound classification based on metric learning according to claim 1, characterized in that: The SPA module is composed of a mean pooling layer, a 1×1 convolution layer, a ReLU activation function layer, a 1×1 convolution layer, a Sigmoid activation function layer, and an attention mechanism layer connected in sequence; first, the input data x, i.e., the Mel spectrum map after data enhancement, is average pooled through the mean pooling layer, and after average pooling, it is subjected to feature extraction through a 1×1 convolution layer, and after extraction, nonlinearity is introduced through a ReLU activation function layer, and after the introduction of nonlinearity, it is further subjected to feature extraction through a 1×1 convolution layer, and the features of the further feature extraction are mapped to between 0 and 1 through a Sigmoid activation function layer, and then an attention weight is generated through an attention mechanism layer, and the attention weight is multiplied by the Mel spectrum map after data enhancement through element-by-element multiplication to obtain a weighted feature, i.e., the output of the SPA module.

3. The method for environmental sound classification based on metric learning according to claim 2, characterized in that: The FPL module consists of a spectral pooling layer SpectralPool2d, a depth separation convolution layer Depthwise Conv2d, a pointwise convolution layer Pointwise Conv2d, and an average pooling layer Mean connected in sequence. Pooling, 1×1 convolution layer, ReLU activation function layer, 1×1 convolution layer, Sigmoid activation function layer, attention mechanism layer; first, the input data x, that is, the Mel spectrum map after data enhancement, is pooled in the frequency domain through the spectral pooling layer, and then deep convolution is performed through the depth separation convolution layer, and finally point-by-point convolution is performed through the point-by-point convolution layer. The output of the point-by-point convolution layer is input to the average pooling layer for average pooling. After average pooling, it is subjected to feature extraction through a 1×1 convolution layer. After extraction, nonlinearity is introduced through the ReLU activation function layer. After the introduction of nonlinearity, it is further subjected to feature extraction through a 1×1 convolution layer. The features of the further feature extraction are mapped to 0 to 1 through the Sigmoid activation function layer, and then the attention weights are generated through the attention mechanism layer. The attention weights are multiplied by the output of the point-by-point convolution layer through element-by-element multiplication to obtain weighted features. The weighted features, the output of the point-by-point convolution layer and the input data x are residually connected to obtain the output of the FPL module.

4. The method for environmental sound classification based on metric learning according to claim 3 is characterized in that: The specific process of step S1 is: collecting ambient sound audio, performing silence elimination on the ambient sound audio, performing short-time Fourier transform on the ambient sound audio after silence elimination to obtain a short-time Fourier transform amplitude spectrum, inputting the short-time Fourier transform amplitude spectrum into a Mel filter group, and finally taking the logarithm of the output of the Mel filter group to obtain a Mel spectrum diagram; wherein, silence elimination of the ambient sound audio adopts an energy-based detection algorithm to remove the silent part in the ambient sound audio.

5. The method for environmental sound classification based on metric learning according to claim 4, characterized in that: The specific process of step S2 is: Random scaling: randomly select a scaling factor from the range [0.8, 1.25] and perform linear interpolation scaling on the original waveform in the Mel-spectrogram; Time-frequency masking: randomly select f frequency bands in the frequency dimension of the Mel-spectrogram and set them to zero; randomly select t time segments in the time dimension of the Mel-spectrogram and set them to zero.

6. The method for environmental sound classification based on metric learning according to claim 5, characterized in that: Cross Entropy Loss Function It is expressed as: In the formula, The 1-hot encoded true value label representing the sample, i.e. the sound feature; Represents the model output of a sample.

7. The method for environmental sound classification based on metric learning according to claim 6, characterized in that: Contrastive loss function It is expressed as: In the formula, Represents the sample y of category i i number; N represents the sample batch size; y i represents the sample of category i; y j represents the sample of category j; z i Represents y i The normalized projection vector of j Represents y j The normalized projection vector of k represents the average of the normalized projection vectors of all samples of category k; τ represents the temperature parameter.

8. The method for environmental sound classification based on metric learning according to claim 7, characterized in that: The metric learning loss function combines the sparse pairwise loss function and adaptive positive mining, where the sparse pairwise loss function It is expressed as: In the formula, K represents the number of categories; Represents the similarity of sample pairs of different categories from category i; Represents the similarity of sample pairs of the same category as category i; Adaptive forward mining dynamically calculates the harmonic mean of the most difficult and least difficult pairs of samples in the same category. The calculation steps are as follows: Calculation of harmonic mean weight: In the formula, h i represents the harmonic mean weight; Indicates the similarity of the least difficult sample pairs of the same category; Represents the similarity of the most difficult sample pairs of the same category within the class; Application of harmonic mean weights: In the formula, α i Indicates the weight setting, when When selecting h i Calculate the harmonic mean weight; when Select 0 to calculate the harmonic mean weight; Calculation of similarity of weighted positive samples: Metric Learning Loss Function It is expressed as:

9. The method for environmental sound classification based on metric learning according to claim 8, characterized in that: The composite loss function obtained by combining the contrast loss function, the metric loss function, and the cross entropy loss function is expressed as: In the formula, α and β are weight coefficients.

10. A system for classifying environmental sounds based on metric learning, used to implement the method for classifying environmental sounds based on metric learning according to any one of claims 1 to 9, characterized in that: include: The collection and conversion module is used to collect and preprocess the ambient sound audio, and convert the preprocessed ambient sound audio into a Mel-spectrogram; The data enhancement module is used to perform data enhancement on the Mel-spectrogram, and the data enhancement includes random scaling and time-frequency masking operations; the construction module is used to construct the SPA module and the FPL module, and the SPA module or the FPL module is used to process the Mel-spectrogram after data enhancement, and the pre-trained convolutional neural network model MobileNetV2 is used to extract features from the processed Mel-spectrogram to obtain sound features; The transmission module is used to input the extracted sound features into the representation space; the optimization module is used to optimize the sound features in the representation space by combining the contrast loss function, the metric learning loss function, and the cross entropy loss function, and project the optimized sound features to the projection layer; the classification module is used to pass the sound features after the projected step S5 through the classifier, followed by the softmax activation function, to generate the probability distribution of each category, and select the category with the highest probability as the final result.

Images