Audio classification method and device based on feature decoupling and comparative learning

Abstract

The invention discloses an audio classification method and device based on feature decoupling and comparative learning, and the method comprises the following steps: 1, obtaining an audio file and corresponding label information, and carrying out the preprocessing of the audio file, and converting the audio file into waveform data; 2, an audio classification model is constructed, the preprocessed data are transmitted in for forward propagation, the model specifically comprises a feature extraction module, a reconstruction decoupling module and a comparison classification module, and the feature extraction module is mainly responsible for extracting input audio information into target coarseness information and non-target coarseness information; and the reconstruction decoupling module realizes information decoupling refinement according to mutual information upper bound combination reconstruction of the target information and the non-target information. According to the method, the target information in the audio can be effectively recognized, and meanwhile, the classification effect with accuracy and robustness is achieved.

Description

Technical Field

[0001] The present invention relates to the field of speech event classification, and in particular to an audio classification method and device based on feature decoupling and contrastive learning. Background Art

[0002] Speech event classification is to process the sound signals in the collected audio data, analyze their acoustic features and identify the corresponding event categories, and convert them into symbolic descriptions corresponding to the events in the acoustic environment, so as to perceive and understand the information in the surrounding environment. It is an important branch of speech processing research and is widely used in audio analysis tasks in different fields.

[0003] In the prior art, speech event classification frameworks are usually trained using an end-to-end model combined with cross-entropy loss. However, cross-entropy loss is easily interfered by other information in the speech, causing the model to misjudge some event categories, thereby affecting the overall classification accuracy of the model. In addition, cross-entropy loss focuses on the classification boundary and often ignores the global representation structure, making it difficult to obtain a robust representation, greatly reducing the generalization of the model. In addition, when dealing with the problem of class imbalance, it is easily affected by the category with a higher prediction frequency, resulting in low recognition accuracy for categories with fewer samples.

[0004] Therefore, in the related art, there is an urgent need for a method that can improve the accuracy and robustness of speech event classification. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides an audio event classification method and device based on feature decoupling and contrastive learning.

[0006] In order to achieve the above-mentioned invention object, the present invention provides the following technical solutions:

[0007] S1: Collect audio files and their categories, and use the pre-processed audio waveform data as model input data while generating corresponding classification labels to build an audio classification dataset;

[0008] S2: Build an audio classification model ACM (Audio Classification Module) and optimize the model ACM parameters through alternating training;

[0009] The audio classification model ACM mainly includes the feature extraction module AFEM (Audio Feature Extraction Module), the reconstruction and decoupling module RFDM (Reconstruction & Feature Disentanglement Module) and the contrastive classification module CLCM (Contrastive Learning & Classification Module):

[0010] The feature extraction module AFEM obtains the target coarse-grained information and non-target coarse-grained information in the audio representation according to the model input data in step 1;

[0011] The reconstruction decoupling module RFDM reconstructs the target coarse-grained information and the non-target coarse-grained information according to the upper bound of the mutual information, realizes the leak-free decoupling between the coarse-grained information, makes them completely separated in the semantic space, and completes the transformation of information decoupling granularity refinement;

[0012] The contrastive classification module CLCM combines supervised contrastive learning with a mask mechanism with cross entropy to achieve high-precision classification based on the robust representation of the target fine-grained information after learning decoupling;

[0013] S3, construct an audio variational fitting module VDFM (Variational Distribution Fitting Module), fit the variational probability distribution according to the target coarse-grained information and the non-target coarse-grained information and obtain an accurate upper bound of the mutual information;

[0014] S4. Input the required classified audio into the trained audio classification model ACM to obtain the corresponding category of the audio.

[0015] As a preferred embodiment of the present invention, an audio classification method based on feature decoupling and contrastive learning is characterized in that step 1 includes the following processes:

[0016] S11: converting the audio signal into indefinite-length time-series waveform data at a sampling rate of 16 kHz, and determining the corresponding fixed duration according to the audio set to ensure that it can cover 90% of the length of the audio; in addition, filtering the audio whose duration is less than 10% of the fixed duration to ensure that the audio has sufficient semantic information;

[0017] S12: For audios that are insufficient in duration, silence is added to make up the duration. For audios that are too long, the duration is fixed by random sampling when used as training sets, and by cutting the audios from the beginning when used as test sets.

[0018] As a preferred embodiment of the present invention, an audio classification method based on feature decoupling and contrastive learning is characterized in that the feature extraction module AFEM in step 2 includes a Wav2Vec 2.0 module, a target information extraction module TIEM (Target Information Extraction Module) and a non-target information extraction module NTIEM (Non-Target Information Extraction Module);

[0019] The audio is preprocessed to obtain audio waveform data a={a1,a2,a3,…,a T}, extract the audio representation I by loading the Wav2Vec 2.0 of the 960-hour pre-trained model of the Librispeech dataset t =Wav2vec2(a t ), and perform global average pooling on it in the time series dimension to obtain the compressed audio representation I = GlobalAveragePooling (I t , dim=1); on this basis, the compressed audio representation is preliminarily extracted by the target information extraction module TIEM and the non-target information extraction module NTIEM to obtain the coarse-grained representation of the target information t=TIEM(I) and the coarse-grained representation of the non-target information n=NTIEM(I).

[0020] As a preferred embodiment of the present invention, an audio classification method based on feature decoupling and contrastive learning is characterized in that the variational fitting module VDFM in step 3 includes a variational distribution average module AM ​​(Average Module) and a variational distribution standard deviation module SDM (Standard Deviation Module), which are mainly composed of a fully connected layer, ReLU and Tanh activation function;

[0021] This module assumes that the variational distribution q satisfies the Gaussian distribution, and iteratively updates the variational distribution through the variational distribution mean module AM ​​and the variational distribution standard deviation module SDM to ensure that after the variational distribution is used to replace the conditional distribution, the upper bound of the mutual information can still be estimated unbiasedly through the Club algorithm;

[0022] First, the target information is coarsely represented The mean q of the variational distribution q is obtained by the average module AM ​​and the standard deviation module SDM respectively m =AM(t) and standard deviation q d = SDM(t), where D is the feature dimension;

[0023] By varying the mean q m , standard deviation q d and coarse-grained representation of non-target information The corresponding variational distribution is Where θ is the network parameter;

[0024] Maximize its log-likelihood based on the variational distribution That is, minimize loss = -LH to achieve KL(p(t,n)||q θ (t,n))≤KL(p(t)p(n)||q θ (t,n)), thereby ensuring that when the variational fitting module outputs the variational distribution q instead of the conditional distribution p, the upper bound of the mutual information between the coarse-grained representation t of the target information calculated by the Club algorithm and the coarse-grained representation n of the non-target information is I vCLUB (t,n) is still valid, where p(t,n) is the joint distribution between t and n, p(t) is the true distribution of t, and p(n) is the true distribution of n.

[0025] As a preferred embodiment of the present invention, an audio classification method based on feature decoupling and contrastive learning is characterized in that the reconstruction decoupling module RFDM in step 2 is composed of a decoupling function FDF (Feature Disentanglement Function) and a reconstruction module RM (Reconstruction Module);

[0026] The mean q of the decoupling function FDF in the variational distribution q m , standard deviation q d and coarse-grained representation of non-target information Based on And use this as the loss function to optimize the target information extraction module TIEM and the non-target information extraction module NTIEM, so that it can finely decouple the target information fine-grained representation t from the compressed audio representation I fine-turn =TIEM(I) and non-target information fine-grained representation n fine-turn =NTIEM(I);

[0027] The reconstruction module RM is mainly composed of a fully connected layer, a BatchNorm layer, and a ReLU activation function. Its input is the splicing information I concat =concat([t fine-turn ,n fine-turn ], dim=-1), and the reconstructed compressed audio representation I is obtained through RM rec =RM(I concat ), aims to maximize the retention of audio information, minimize the information lost during the decoupling process, and achieve leak-free decoupling of the entire process.

[0028] As a preferred solution of the present invention, an audio classification method based on feature decoupling and contrastive learning is characterized in that the variational fitting module VDFM and the reconstruction decoupling module RFDM adopt an alternating training method to perform parameter learning, which is as follows:

[0029] First, the compressed audio representation I is preliminarily characterized by the target information extraction module TIEM and the non-target information extraction module NTIEM to obtain the target information coarse-grained representation t=TIEM(I) and the non-target information coarse-grained representation n=NTIEM(I);

[0030] Set the hyperparameter step of the cyclic training round, and freeze the parameters of the target information extraction module TIEM and the non-target information extraction module NTIEM through the loss function Successive cycle optimization variational fitting module VDFM;

[0031] After the variational fitting module VDFM is optimized for step times, the variational fitting module VDFM is frozen as the mutual information upper bound extractor, and the target information extraction module TIEM and the non-target information extraction module NTIEM are unfrozen. According to the mutual information upper bound I extracted by the variational fitting module VDFM vCLUB (t,n) combines the information I concat =concat([t,n],dim=-1) Input the reconstruction module RM to obtain the reconstruction loss The target information extraction module TIEM and the non-target information extraction module NTIEM are jointly optimized to obtain the fine-grained representation of the target information t fine-turn and non-target information fine-grained representation fine-turn .

[0032] As a preferred embodiment of the present invention, an audio classification method based on feature decoupling and contrastive learning is characterized in that the contrastive classification module CLCM in step 2 includes a contrastive mapping module CMM (Contrastive Mapping Module) and a class classification module CCM (Class Classification Module);

[0033] The contrast mapping module CMM consists of a fully connected layer and a ReLU activation function. After feature decoupling, the fine-grained representation of the target information is obtained. fine-turn Based on this, the feature mapping module CMM represents the target information in fine-grained form. fine-turn Mapped to a latent space suitable for supervised contrastive learning training, and the corresponding latent space tensor s = CMM (t fine-turn ); on this basis, s is normalized to obtain a normalized tensor In this way, the stability of loss function calculation is ensured, where n is the dimension of latent space tensor s. After completing the above operations, the normalized tensor z and its supervised contrast learning loss function with Mask mechanism formed by label are trained. The specific formula is as follows:

[0034]

[0035] Where Loss sup is the supervised contrastive learning loss function, I is the batch size, P(i) is the set of positive samples, i.e., samples with the same label, z is the latent space tensor, τ represents the temperature value, and α is the Mask mechanism. The Mask mechanism sets a mask for unpaired similar data in the same batch to ensure clustering in the latent space using supervised contrastive learning without data augmentation.

[0036] The input of the category classification module CCM is t fine-turn , and based on this output predict the probability distribution of the audio category And thus construct the cross entropy loss function Where C represents the number of classification categories, y i is a one-hot tag. The probability predicted by the model;

[0037] As a preferred embodiment of the present invention, an audio classification method based on feature decoupling and contrastive learning is characterized in that the contrastive classification module CLCM balances the weight between the cross entropy loss and the supervised contrastive learning loss function through the Uncertainty Loss algorithm;

[0038] First, a variance prediction module VPM (Variance Prediction Module) is constructed to predict the variance values ​​σ corresponding to the classification task and clustering task respectively. CE ,σ SUPCON = VPM(), and construct a multi-task loss function by fusing the cross entropy and supervised contrastive learning loss functions through the variance value While optimizing the classification model, task weight adaptation is achieved.

[0039] An audio classification device based on feature decoupling and contrastive learning, characterized in that the device includes: an input device, an output device, a power supply, at least one processor, and a memory communicatively connected to the processor; the memory may simultaneously correspond to instructions specified by at least one processor, and the instructions are executed by the at least one executor so that the at least one processor can execute the method described in any one of claims 1 to 8.

[0040] The beneficial effects of the present invention are:

[0041] (1) The present invention uses the Wav2Vec 2.0 model to extract rich information contained in the audio, which greatly reduces information loss compared to traditional acoustic features and significantly improves the model classification accuracy;

[0042] (2) The reconstruction decoupling module (RFDM) proposed in the present invention realizes the decoupling of target information and non-target information in the feature space under the premise of only a single label by reconstructing mutual information, thus reducing the interference of non-target information in the classification process and effectively improving the classification and recognition accuracy of the model;

[0043] (3) The contrastive classification module (CLCM) proposed in the present invention clusters audio features in latent space through a supervised contrastive learning loss function with a mask mechanism while performing classification. Features of the same type are close to each other and features of different types are separated from each other, which greatly enhances the generalization of the model.

[0044] (4) The present invention dynamically balances the weight distribution between the cross entropy loss function and the supervised contrastive learning loss function through the Uncertainty Loss algorithm, allowing the model to focus on the corresponding loss function at different training stages, thereby maximizing the optimization effect of the loss function. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of an audio classification method based on feature decoupling and contrastive learning described in Example 1 of the present invention.

[0046] Figure 2 This is a schematic diagram of a feature extraction module in an audio classification method based on feature decoupling and contrastive learning described in Example 1 of the present invention.

[0047] Figure 3 This is a schematic diagram of a variational fitting module in an audio classification method based on feature decoupling and contrastive learning described in Example 1 of the present invention.

[0048] Figure 4 This is a schematic diagram of a reconstruction decoupling module and a contrast classification module in an audio classification method based on feature decoupling and contrast learning described in Example 1 of the present invention.

[0049] Figure 5 This is a schematic diagram of an audio classification device based on feature decoupling and contrastive learning described in Example 2 of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0051] Example 1

[0052] like Figure 1 As shown, an audio classification method based on feature decoupling and contrastive learning includes the following steps:

[0053] S1: Collect audio files and their categories to build an audio classification dataset;

[0054] S2: Preprocess the audio in the dataset and convert it into audio waveform data as model input data while generating corresponding classification labels according to the category;

[0055] S3: Construct an audio classification model ACM, which includes a feature extraction module AFEM, a reconstruction decoupling module RFDM and a comparative classification module CLCM, and input the preprocessed audio features into the audio classification model for forward propagation to obtain target coarse-grained information and non-target coarse-grained information respectively;

[0056] S4: While freezing the audio classification model ACM, a variational fitting module VDFM is constructed, and the model input is trained to fit the variational upper bound of the mutual information through target coarse-grained information and non-target coarse-grained information;

[0057] S5: Frozen variational fitting module VDFM, which trains and optimizes the weights of the audio classification model ACM by integrating the cross entropy loss function, the supervised contrastive learning loss function, the variational mutual information upper bound loss function and the reconstruction function, and finally obtains the optimal weights;

[0058] S6: Load the optimal weight into the audio classification model and pass in the audio to be classified to obtain the corresponding label of the audio.

[0059] Furthermore, the preprocessing in step S2 includes the following specific steps:

[0060] S21: converting the audio signal into indefinite-length time-series waveform data at a sampling rate of 16 kHz, and determining the corresponding fixed duration according to the audio set to ensure that it can cover 90% of the length of the audio; in addition, filtering the audio whose duration is less than 10% of the fixed duration to ensure that the audio has sufficient semantic information;

[0061] S22: For audios that are insufficient in duration, silence is added to make up the duration. For audios that are too long, the duration is fixed by random sampling when used as training sets, and by cutting the audios from the beginning when used as test sets.

[0062] Further, such as Figure 2 As shown, the feature extraction module AFEM includes a Wav2Vec 2.0 module, a target information extraction module TIEM and a non-target information extraction module NTIEM;

[0063] The audio is preprocessed to obtain audio waveform data a={a1,a2,a3,…,a T}, extract the audio representation I by loading the Wav2Vec 2.0 of the 960-hour pre-trained model of the Librispeech dataset t =Wav2vec2(a t ), and perform global average pooling on it in the time series dimension to obtain the compressed audio representation I = GlobalAveragePooling (I t , dim=1); on this basis, the compressed audio representation is preliminarily extracted by the target information extraction module TIEM and the non-target information extraction module NTIEM to obtain the coarse-grained representation of the target information t=TIEM(I) and the coarse-grained representation of the non-target information n=NTIEM(I).

[0064] Further, such as Figure 3 As shown, the variational fitting module VDFM includes the variational distribution average module AM ​​and the variational distribution standard deviation module SDM, which are mainly composed of a fully connected layer, ReLU and Tanh activation functions;

[0065] This module assumes that the variational distribution q satisfies the Gaussian distribution, and iteratively updates the variational distribution through the variational distribution mean module AM ​​and the variational distribution standard deviation module SDM to ensure that after the variational distribution is used to replace the conditional distribution, the upper bound of the mutual information can still be estimated unbiasedly through the Club algorithm;

[0066] First, the target information is coarsely represented The mean q of the variational distribution q is obtained by the average module AM ​​and the standard deviation module SDM respectively m =AM(t) and standard deviation q d = SDM(t), where D is the feature dimension;

[0067] By varying the mean q m , standard deviation q d and coarse-grained representation of non-target information The corresponding variational distribution is Where θ is the network parameter;

[0068] Maximize its log-likelihood based on the variational distribution That is, minimize loss = -LH to achieve KL(p(t,n)||q θ (t,n))≤KL(p(t)p(n)||q θ (t,n)), thereby ensuring that when the variational fitting module outputs the variational distribution q instead of the conditional distribution p, the upper bound of the mutual information between the coarse-grained representation t of the target information calculated by the Club algorithm and the coarse-grained representation n of the non-target information is IvCLUB (t,n) is still valid, where p(t,n) is the joint distribution between t and n, p(t) is the true distribution of t, and p(n) is the true distribution of n.

[0069] Further, such as Figure 4 As shown, the reconstruction decoupling module RFDM consists of a decoupling function FDF and a reconstruction module RM;

[0070] The mean q of the decoupling function FDF in the variational distribution q m , standard deviation q d and coarse-grained representation of non-target information Based on And use this as the loss function to optimize the target information extraction module TIEM and the non-target information extraction module NTIEM, so that it can finely decouple the target information fine-grained representation t from the compressed audio representation I fine-turn =TIEM(I) and non-target information fine-grained representation n fine-turn =NTIEM(I);

[0071] The reconstruction module RM is mainly composed of a fully connected layer, a BatchNorm layer, and a ReLU activation function. Its input is the splicing information I concat =concat([t fine-turn ,n fine-turn ], dim=-1), and the reconstructed compressed audio representation I is obtained through RM rec =RM(I concat ), aims to maximize the retention of audio information, minimize the information lost during the decoupling process, and achieve leak-free decoupling of the entire process.

[0072] Further, such as Figure 4 As shown, the variational fitting module VDFM and the reconstruction decoupling module RFDM adopt an alternating training method to learn parameters, as follows:

[0073] First, the compressed audio representation I is preliminarily characterized by the target information extraction module TIEM and the non-target information extraction module NTIEM to obtain the target information coarse-grained representation t=TIEM(I) and the non-target information coarse-grained representation n=NTIEM(I);

[0074] Set the hyperparameter step of the cyclic training round, and freeze the parameters of the target information extraction module TIEM and the non-target information extraction module NTIEM through the loss function Successive cycle optimization variational fitting module VDFM;

[0075] After the variational fitting module VDFM is optimized for step times, the variational fitting module VDFM is frozen as the mutual information upper bound extractor, and the target information extraction module TIEM and the non-target information extraction module NTIEM are unfrozen. According to the mutual information upper bound I extracted by the variational fitting module VDFM vCLUB (t,n) combines the information I concat =concat([t,n],dim=-1) Input the reconstruction module RM to obtain the reconstruction loss The target information extraction module TIEM and the non-target information extraction module NTIEM are jointly optimized to obtain the fine-grained representation of the target information t fine-turn and non-target information fine-grained representation fine-turn .

[0076] Further, such as Figure 4 As shown, the contrast classification module CLCM includes a contrast mapping module CMM and a category classification module CCM;

[0077] The contrast mapping module CMM consists of a fully connected layer and a ReLU activation function. After feature decoupling, the fine-grained representation of the target information is obtained. fine-turn Based on this, the feature mapping module CMM represents the target information in fine-grained form. fine-turn Mapped to a latent space suitable for supervised contrastive learning training, and the corresponding latent space tensor s = CMM (t fine-turn ); on this basis, s is normalized to obtain a normalized tensor In this way, the stability of loss function calculation is ensured, where n is the dimension of latent space tensor s. After completing the above operations, the normalized tensor z and its supervised contrast learning loss function with Mask mechanism formed by label are trained. The specific formula is as follows:

[0078]

[0079] Where Loss sup is the supervised contrastive learning loss function, I is the batch size, P(i) is the set of positive samples, i.e., samples with the same label, z is the latent space tensor, τ represents the temperature value, and α is the Mask mechanism. The Mask mechanism sets a mask for unpaired similar data in the same batch to ensure clustering in the latent space using supervised contrastive learning without data augmentation.

[0080] The input of the category classification module CCM is t fine-turn , and based on this output predict the probability distribution of the audio category And thus construct the cross entropy loss function Where C represents the number of classification categories, y iis a one-hot tag. The probability predicted by the model;

[0081] Further, such as Figure 4 As shown, the contrastive classification module CLCM balances the weights between the cross entropy loss and the supervised contrastive learning loss function through the Uncertainty Loss algorithm;

[0082] First, a variance prediction module VPM (Variance Prediction Module) is constructed to predict the variance values ​​σ corresponding to the classification task and clustering task respectively. CE ,σ SUPCON = VPM(), and construct a multi-task loss function by fusing the cross entropy and supervised contrastive learning loss functions through the variance value

[0083] While optimizing the classification model, task weight adaptation is achieved.

[0084] Example 2

[0085] like Figure 5 As shown, an audio classification device based on feature decoupling and contrastive learning is characterized in that it includes an input device, an output device, a power supply, at least one processor, and a memory communicatively connected to the processor; the memory may simultaneously correspond to instructions specified by at least one processor, and the instructions are executed by the at least one executor, and the input and output interfaces include a display, a keyboard, a mouse, and a USB interface to complete data interaction operations; the power supply can be an external power supply or a rechargeable battery to provide power to the electronic device.

[0086] Those skilled in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), disks or optical disks, etc. Various media that can store program codes.

[0087] When the above-mentioned integrated unit of the present invention is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An audio classification method based on feature decoupling and contrastive learning, characterized in that: The following steps are involved: Step 1: Collect audio files and their categories, and use the pre-processed audio waveform data as model input while generating corresponding classification labels to build an audio classification dataset; Step 2: Build an audio classification model ACM (Audio Classification Module) and optimize the model ACM parameters through alternating training; The audio classification model ACM mainly includes the feature extraction module AFEM (Audio Feature Extraction Module), the reconstruction and decoupling module RFDM (Reconstruction & Feature Disentanglement Module) and the contrastive classification module CLCM (Contrastive Learning & Classification Module): The feature extraction module AFEM obtains the target coarse-grained information and non-target coarse-grained information in the audio representation according to the model input data in step 1; The reconstruction decoupling module RFDM reconstructs the target coarse-grained information and the non-target coarse-grained information according to the upper bound of the mutual information, realizes the leak-free decoupling between the coarse-grained information, makes them completely separated in the semantic space, and completes the transformation of information decoupling granularity refinement; The contrastive classification module CLCM combines supervised contrastive learning with a mask mechanism with cross entropy to achieve high-precision classification based on the robust representation of the target fine-grained information after learning decoupling; Step 3: construct an audio variational fitting module VDFM (Variational Distribution Fitting Module), fit the variational probability distribution according to the target coarse-grained information and the non-target coarse-grained information, and obtain an accurate upper bound of the mutual information; Step 4: Input the required classified audio into the trained audio classification model ACM to obtain the corresponding category of the audio.

2. The audio classification method based on feature decoupling and contrastive learning according to claim 1, characterized in that: The step 1 includes the following processes: S11: converting the audio signal into indefinite-length time-series waveform data at a sampling rate of 16 kHz, and determining the corresponding fixed duration according to the audio set to ensure that it can cover 90% of the length of the audio; in addition, filtering the audio whose duration is less than 10% of the fixed duration to ensure that the audio has sufficient semantic information; S12: For audios that are insufficient in duration, silence is added to make up the duration. For audios that are too long, the duration is fixed by random sampling when used as training sets, and by cutting the audios from the beginning when used as test sets.

3. The audio classification method based on feature decoupling and contrastive learning according to claim 1, characterized in that: The feature extraction module AFEM in step 2 includes a Wav2Vec 2.0 module, a target information extraction module TIEM (Target Information Extraction Module) and a non-target information extraction module NTIEM (Non-Target Information Extraction Module); The audio is preprocessed to obtain audio waveform data a={a1,a2,a3,…,a T }, extract the audio representation I by loading the Wav2Vec 2.0 of the 960-hour pre-trained model of the Librispeech dataset t =Wav2vec2(a t ), and perform global average pooling on it in the time series dimension to obtain the compressed audio representation I = GlobalAveragePooling (I t , dim=1); on this basis, the compressed audio representation is preliminarily extracted by the target information extraction module TIEM and the non-target information extraction module NTIEM to obtain the coarse-grained representation of the target information t=TIEM(I) and the coarse-grained representation of the non-target information n=NTIEM(I).

4. The audio classification method based on feature decoupling and contrastive learning according to claim 1, characterized in that: The variational fitting module VDFM in step 3 includes a variational distribution average module AM ​​(Average Module) and a variational distribution standard deviation module SDM (Standard Deviation Module), which are mainly composed of a fully connected layer, ReLU and Tanh activation functions; This module assumes that the variational distribution q satisfies the Gaussian distribution, and iteratively updates the variational distribution through the variational distribution mean module AM ​​and the variational distribution standard deviation module SDM to ensure that after the variational distribution is used to replace the conditional distribution, the upper bound of the mutual information can still be estimated unbiasedly through the Club algorithm; First, the target information is coarsely represented The mean q of the variational distribution q is obtained by the average module AM ​​and the standard deviation module SDM respectively m =AM(t) and standard deviation q d = SDM(t), where D is the feature dimension; By varying the mean q m , standard deviation q d and coarse-grained representation of non-target information The corresponding variational distribution is Where θ is the network parameter; Maximize its log-likelihood based on the variational distribution That is, minimize loss = -LH to achieve KL(p(t,n)||q θ (t,n))≤KL(p(t)p(n)||q θ (t,n)), thereby ensuring that when the variational fitting module outputs the variational distribution q instead of the conditional distribution p, the upper bound of the mutual information between the coarse-grained representation t of the target information calculated by the Club algorithm and the coarse-grained representation n of the non-target information is I vCLUB (t,n) is still valid, where p(t,n) is the joint distribution between t and n, p(t) is the true distribution of t, and p(n) is the true distribution of n.

5. The audio classification method based on feature decoupling and contrastive learning according to claim 1, characterized in that: In step 2, the reconstruction decoupling module RFDM is composed of a decoupling function FDF (Feature Disentanglement Function) and a reconstruction module RM (Reconstruction Module); The mean q of the decoupling function FDF in the variational distribution q m , standard deviation q d and coarse-grained representation of non-target information Based on This is used as the loss function to optimize the target information extraction module TIEM and the non-target information extraction module NTIEM, so that it can finely decouple the target information fine-grained representation t from the compressed audio representation I fine-turn =TIEM(I) and non-target information fine-grained representation n fine-turn =NTIEM(I); The reconstruction module RM is mainly composed of a fully connected layer, a BatchNorm layer, and a ReLU activation function. Its input is the splicing information I concat =concat([t fine-turn ,n fine-turn ], dim=-1), and the reconstructed compressed audio representation I is obtained through RM rec =RM(I concat ), aims to maximize the retention of audio information, minimize the information lost in the decoupling process, and achieve leak-free decoupling in the entire process.

6. The audio classification method based on feature decoupling and contrastive learning according to any one of claims 4 to 5, characterized in that: The variational fitting module VDFM and the reconstruction decoupling module RFDM use an alternating training method to learn parameters, as follows: First, the compressed audio representation I is preliminarily characterized by the target information extraction module TIEM and the non-target information extraction module NTIEM to obtain the target information coarse-grained representation t=TIEM(I) and the non-target information coarse-grained representation n=NTIEM(I); Set the hyperparameter step of the cyclic training round, and freeze the parameters of the target information extraction module TIEM and the non-target information extraction module NTIEM through the loss function Successive cycle optimization variational fitting module VDFM; After the variational fitting module VDFM is optimized for step times, the variational fitting module VDFM is frozen as the mutual information upper bound extractor, and the target information extraction module TIEM and the non-target information extraction module NTIEM are unfrozen. According to the mutual information upper bound I extracted by the variational fitting module VDFM vCLUB (t,n) combines the information I concat =concat([t,n],dim=-1) Input the reconstruction module RM to obtain the reconstruction loss The target information extraction module TIEM and the non-target information extraction module NTIEM are jointly optimized to obtain the fine-grained representation of the target information t fine-turn and non-target information fine-grained representation fine-turn .

7. The audio classification method based on feature decoupling and contrastive learning according to claim 1, characterized in that: The contrast classification module CLCM in step 2 includes a contrast mapping module CMM (Contrastive Mapping Module) and an audio classification module CCM (Class Classification Module); The contrast mapping module CMM consists of a fully connected layer and a ReLU activation function. After feature decoupling, the fine-grained representation of the target information is obtained. fine-turn Based on this, the feature mapping module CMM represents the target information in fine-grained form. fine-turn Mapped to a latent space suitable for supervised contrastive learning training, and obtain the corresponding latent space tensor s = CMM (t fine-turn ); on this basis, s is normalized to obtain a normalized tensor In this way, the stability of loss function calculation is ensured, where n is the dimension of latent space tensor s. After completing the above operations, the normalized tensor z and its supervised contrast learning loss function with Mask mechanism formed by label are trained. The specific formula is as follows: Where Loss sup is the supervised contrastive learning loss function, I is the batch size, P(i) is the set of positive samples, i.e., samples with the same label, z is the latent space tensor, τ represents the temperature value, and α is the Mask mechanism. The Mask mechanism sets a mask for unpaired similar data in the same batch to ensure clustering in the latent space using supervised contrastive learning without data augmentation. The input of the audio classification module CCM is t fine-turn , and based on this output predict the probability distribution of the audio category And thus construct the cross entropy loss function Where C represents the number of classification categories, y i is a one-hot tag. The probability predicted by the model.

8. The audio classification method based on feature decoupling and contrastive learning according to claim 7, characterized in that: The contrast classification module CLCM balances the weight between the cross entropy loss and the supervised contrast learning loss function through the Uncertainty Loss algorithm; First, a variance prediction module VPM (Variance Prediction Module) is constructed to predict the variance values ​​σ corresponding to the classification task and clustering task respectively. CE ,σ SUPCON = VPM(), and construct a multi-task loss function by fusing the cross entropy and supervised contrastive learning loss functions through the variance value While optimizing the classification model, task weight adaptation is achieved.

9. An audio classification device based on feature decoupling and contrastive learning, characterized in that: The apparatus comprises: an input device, an output device, a power supply, at least one processor, and a memory communicatively connected to the processor; the memory may simultaneously correspond to instructions specified by at least one processor, and the instructions are executed by the at least one executor so that the at least one processor can execute the method described in any one of claims 1 to 8.

Images