Audio classification network training method, electronic equipment, storage medium and computer program product
By training the teacher network to generate prediction categories in the audio classification network and determining the target tag based on the original label and prediction categories, the problem of introducing wrong labels in the re-labeling process in the multi-label audio classification is solved, and the classification accuracy of the student network is improved and Generalization ability.
Patent Information
- Application Number
- CN202411532554.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-02-07
AI Technical Summary
In multi-label audio classification, the existing technology is prone to introducing new wrong labels in the process of re-labeling, resulting in a decrease in the classification accuracy of the student network and limited generalization ability.
The audio data is trained by training the teacher network for predictive category generation and target labels are determined based on the original label and predictive category to train the student network. This method avoids the direct use of predicted categories of the teacher network as new labels, reducing the introduction of wrong labels.
It improves the training effect of students' network, improves classification accuracy and generalization ability, and reduces the error introduced due to error labels.
Smart Images
Figure CN119400203A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of audio processing technology, and in particular to a training method for an audio classification network, an electronic device, a computer-readable storage medium, and a computer program product. Background Art
[0002] Multi-label audio classification is the process of identifying audio signals and assigning multiple related category labels. With the development of artificial intelligence technology, multi-label audio classification methods have developed rapidly, thanks to the increasing number of public multi-label audio datasets. At present, in the public datasets, the labels corresponding to the audio data have the problem of label noise, that is, the labels of the audio data are mislabeled or missing. In the training process of deep neural networks, sample data and its labels have an important influence on the convergence degree and classification performance of the network model. Label noise will have a negative impact on the network model, causing the classification accuracy of the network model to decrease and the generalization ability to be limited.
[0003] To solve this problem, the relevant solution is to design a teacher network with a large number of parameters, deep network layers, and complex structure, use a public data set to train the teacher network, and use the trained teacher network to re-label the sample data in the data set to reduce the incorrect labels of the sample data in the data set, and then train the student network through the re-labeled data set.
[0004] When using the teacher network to relabel the sample data in the dataset, the predicted label output by the teacher network may be different from the original label of the sample data. In this case, the predicted label is usually selected as the new label. However, the predicted label output by the teacher network may be wrong. Over-trusting the judgment of the teacher network will reduce the error of the original label while introducing new wrong labels, resulting in the failure to train an ideal student network. Summary of the invention
[0005] The purpose of the embodiments of the present application is to provide a training method for an audio classification network and an electronic device, a storage medium, and a computer program product, so as to avoid the problem of introducing new erroneous labels during the label re-labeling process and improve the training effect of the student network.
[0006] On the one hand, the present application provides a training method for an audio classification network, comprising:
[0007] A teacher network is trained using audio data in a data set; wherein each audio data in the data set carries an original label of at least one binary classification task;
[0008] Generate a predicted category corresponding to each binary classification task for the audio data in the dataset through the trained teacher network;
[0009] Determine a target label for each audio data based on the original label of the audio data and the predicted category;
[0010] The student network is trained by audio data carrying multiple target labels as an audio classification network.
[0011] In one embodiment, the step of training a teacher network using audio data in a data set includes:
[0012] Performing a fast Fourier transform on the audio data in the data set to obtain a transformation result;
[0013] Performing Mel filter bank processing on the transformation result to obtain a processing result;
[0014] Taking a logarithm of the processing result, and performing discrete cosine transform on the logarithm to obtain an audio feature;
[0015] The teacher network is trained by audio features carrying original labels.
[0016] In one embodiment, if the original label of the audio data in any binary classification task is 1, it means that the audio data belongs to the category corresponding to the binary classification task; if the original label of the audio data in any binary classification task is 0, it means that the audio data does not belong to the category corresponding to the binary classification task; the value range of the predicted category is between 0 and 1, indicating the confidence that the audio data belongs to the category corresponding to the binary classification task;
[0017] The determining the target label of the audio data based on the original label of each audio data and the predicted category includes:
[0018] For any audio data, if the original label of the audio data in any binary classification task is 0, and the predicted category of the audio data in the binary classification task is greater than or equal to 1-T, the target label of the audio data in the binary classification task is determined to be 1; where T is a constant between 0 and 0.5;
[0019] For any audio data, if the original label of the audio data in any binary classification task is 1, and the predicted category of the audio data in the binary classification task is less than T, it is determined that the target label of the audio data in the binary classification task is 0.
[0020] In one embodiment, if the original label of the audio data in any binary classification task is 1, it means that the audio data belongs to the category corresponding to the binary classification task; if the original label of the audio data in any binary classification task is 0, it means that the audio data does not belong to the category corresponding to the binary classification task; the value range of the predicted category is between 0 and 1, indicating the confidence that the audio data belongs to the category corresponding to the binary classification task;
[0021] The determining the target label of the audio data based on the original label of each audio data and the predicted category includes:
[0022] For any audio data, if the original label of the audio data in any binary classification task is 0, and the predicted category of the audio data in the binary classification task is less than 1-T, the target label of the audio data in the binary classification task is determined to be a masked label; where T is a constant between 0 and 0.5;
[0023] For any audio data, if the original label of the audio data in any binary classification task is 1, and the predicted category of the audio data in the binary classification task is less than 0.5 and greater than or equal to T, it is determined that the target label of the audio data in the binary classification task is a blocked label.
[0024] In one embodiment, the step of training a student network using a plurality of audio data carrying target labels includes:
[0025] Performing data enhancement on the multiple audio data carrying the target labels to obtain multiple enhanced audio data;
[0026] Performing fast Fourier transform on each enhanced audio data to obtain a transform result;
[0027] Performing Mel filter bank processing on the transformation result to obtain a processing result;
[0028] Taking a logarithm of the processing result, and performing discrete cosine transform on the logarithm to obtain an audio feature;
[0029] The student network is trained by audio features carrying target labels.
[0030] In one embodiment, the step of performing data enhancement on the plurality of audio data carrying target tags to obtain a plurality of enhanced audio data includes:
[0031] Select two audio data from the plurality of audio data, and multiply the two audio data by a linear coefficient of random decibels respectively to obtain two adjusted audio data;
[0032] The two adjusted audio data are superimposed, and the target labels of the two adjusted audio data are merged to obtain enhanced audio data; wherein the enhanced audio data carries the merged target label;
[0033] By selecting two audio data for processing multiple times, multiple added audio data are obtained.
[0034] In one embodiment, the step of performing data enhancement on the plurality of audio data carrying target tags to obtain a plurality of enhanced audio data includes:
[0035] Select two audio data from the plurality of audio data, and multiply the two audio data by a linear coefficient of random decibels respectively to obtain two adjusted audio data;
[0036] respectively extracting data segments from the two adjusted audio data, splicing the extracted multiple data segments, and fusing the target labels of the two adjusted audio data to obtain enhanced audio data; wherein the enhanced audio data carries the fused target label;
[0037] By selecting two audio data for processing multiple times, multiple added audio data are obtained.
[0038] On the other hand, the present application provides an electronic device, the electronic device comprising:
[0039] processor;
[0040] a memory for storing processor-executable instructions;
[0041] Wherein, the processor is configured to execute the above-mentioned training method of the audio classification network.
[0042] Furthermore, the present application provides a computer-readable storage medium, which stores a computer program, and the computer program can be executed by a processor to complete the above-mentioned audio classification network training method.
[0043] In addition, the present application provides a computer program product, including a computer program / instruction, which implements the above-mentioned training method of the audio classification network when executed by a processor.
[0044] In the process of re-labeling sample data in a data set using a teacher network, the application scheme does not directly use the predicted category of the teacher network as a new label. Instead, the target label is determined by the original label and the predicted category, thereby avoiding the problem of directly introducing new erroneous labels due to incorrect prediction categories. This helps to subsequently train a student network with high classification accuracy and strong generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solution of the embodiments of the present application, the drawings required for use in the embodiments of the present application are briefly introduced below.
[0046] Figure 1 A schematic diagram of an application scenario of a training method for an audio classification network provided in an embodiment of the present application;
[0047] Figure 2 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application;
[0048] Figure 3 A flowchart of a method for training an audio classification network provided in one embodiment of the present application;
[0049] Figure 4 A schematic diagram of a training process of a teacher network provided in an embodiment of the present application;
[0050] Figure 5 A flowchart of a label correction method provided in one embodiment of the present application;
[0051] Figure 6 A flowchart of a label shielding method provided in one embodiment of the present application;
[0052] Figure 7 A schematic diagram of a training process of a student network provided in an embodiment of the present application;
[0053] Figure 8 A schematic diagram of the overall training process of an audio classification network provided in an embodiment of the present application;
[0054] Fig. 9 A block diagram of a training device for an audio classification network provided in one embodiment of the present application. DETAILED DESCRIPTION
[0055] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0056] Similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0057] Figure 1 Schematic diagram of application scenarios of the training method of the audio classification network provided in the embodiment of the present application. Figure 1 As shown, the application scenario includes a client 20 and a server 30; the client 20 can be a chip, a host, a mobile phone, a tablet computer, etc.; the server 30 can be a host, a server, a server cluster or a cloud computing center, which can train an audio classification network and deploy the client 20.
[0058] like Figure 2 As shown, this embodiment provides an electronic device 1, including: at least one processor 11 and a memory 12, Figure 2A processor 11 is taken as an example. The processor 11 and the memory 12 are connected via a bus 10. The memory 12 stores instructions that can be executed by the processor 11. The instructions are executed by the processor 11 so that the electronic device 1 can execute all or part of the process of the method in the following embodiment. In one embodiment, the electronic device 1 can be the above-mentioned server 30, which is used to execute the training method of the audio classification network.
[0059] The memory 12 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.
[0060] The present application also provides a computer-readable storage medium, which stores a computer program. The computer program can be executed by the processor 11 to complete the training method of the audio classification network provided in the present application.
[0061] The present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the training method of the audio classification network provided in the present application.
[0062] See also Figure 3 , is a flow chart of a training method for an audio classification network provided in an embodiment of the present application, such as Figure 3 As shown, the method may include the following steps 310 to 340.
[0063] Step 310: Train a teacher network using the audio data in the data set; wherein each audio data in the data set carries an original label of at least one binary classification task.
[0064] Here, the data set can be constructed from existing public data sets. For example, multiple public data sets can be merged to obtain the data set in this solution.
[0065] The number of original labels carried by different audio data in the data set can be the same or different. The binary classification tasks corresponding to the original labels in the data set can be selected according to the needs of subsequent applications. For binary classification tasks that do not have subsequent classification requirements, the original labels corresponding to the binary classification tasks can be ignored in the training process of this solution. For example: the audio data carries the original label of the binary classification task of "whether it is a dialect", and the audio classification network trained subsequently does not perform the binary classification task, then the original label of the binary classification task can be ignored in the training process of this solution.
[0066] By carrying the original label of at least one binary classification task, a teacher network for performing audio classification can be trained. Here, the teacher network can include multiple classifiers, the number of classifiers is determined by the sum of the binary classification tasks that all audio data need to pay attention to, and each classifier corresponds to a binary classification task.
[0067] To ensure that the teacher network can achieve good prediction results, a deep neural network with a large number of parameters, a deep number of network layers, and a complex structure can be selected to train the teacher network.
[0068] Step 320: Generate a predicted category corresponding to each binary classification task for the audio data in the dataset through the trained teacher network.
[0069] After the teacher network is trained, each audio data in the data set can be processed by the teacher network to obtain the predicted category corresponding to each binary classification task. For any audio data, if the audio data carries the original labels of m binary classification tasks, the predicted categories of these m binary classification tasks can be output by the teacher network. Here, m is a positive integer. When the teacher network processes the audio data, the predicted categories output by the classifiers other than the classifiers corresponding to the m binary classification tasks can be ignored.
[0070] Step 330: Determine a target label for each audio data based on the original label and the predicted category of the audio data.
[0071] For each audio data, the accurate target label under each binary classification task can be determined by the original label and predicted category of the audio data under each binary classification task. Here, the target label is an adjusted label that can be used for subsequent training.
[0072] Generating target labels for each binary classification task of multiple audio data in the dataset can correct the label noise existing in the dataset.
[0073] Step 340: Train a student network using audio data carrying multiple target labels as an audio classification network.
[0074] After obtaining multiple audio data with target labels, a student network can be trained as an audio classification network. Here, the student network can use the same network architecture as the teacher network or a different network architecture. For example, if the student network is to be deployed on a device with fewer computing resources, the student network will be smaller than the teacher network. The student network includes multiple classifiers, and the number of classifiers is determined by the sum of the binary classification tasks that all audio data need to focus on, and each classifier corresponds to a binary classification task.
[0075] Through the above measures, in the process of using the teacher network to relabel the sample data in the dataset, the predicted category of the teacher network is not directly used as the new label. Instead, the target label is determined by the original label and the predicted category, avoiding the problem of directly introducing new erroneous labels due to incorrect prediction categories. This helps to subsequently train a student network with high classification accuracy and strong generalization ability.
[0076] In one embodiment, see Figure 4 , is a schematic diagram of the training process of the teacher network provided in an embodiment of the present application, such as Figure 4 As shown, when executing step 310 to train the teacher network, the following steps 410 to 440 may be executed.
[0077] Step 410: Perform fast Fourier transform on the audio data in the data set to obtain a transformation result.
[0078] Step 420: Perform Mel filter bank processing on the transformation result to obtain a processing result.
[0079] Step 430: Take the logarithm of the processing result and perform discrete cosine transform on the logarithm to obtain audio features.
[0080] The audio data in the data set is time domain data, and can be processed into audio features that can be processed by a deep neural network through the preprocessing process of fast Fourier transform (FFT), Mel filter bank processing, logarithm taking and discrete cosine transform by executing the process from step 410 to step 430. Exemplarily, the audio features can be in the form of any one of vectors, feature maps, etc.
[0081] By preprocessing each audio data separately, the audio features corresponding to each audio data can be obtained.
[0082] Step 440: Train a teacher network using audio features that carry original labels.
[0083] The original label carried by each audio feature is the original label carried by the audio data corresponding to the audio feature. After obtaining multiple audio features carrying original labels, the deep neural network can be supervised trained through multiple audio features to train a teacher network for audio data classification.
[0084] Through the above measures, the audio data in the dataset can be preprocessed into audio features that can be calculated by the deep neural network, and then the teacher network can be trained through multiple audio features carrying original labels.
[0085] In one embodiment, the original label of the audio data in any binary classification task is 1 or 0. If the original label of the audio data in any binary classification task is 1, it means that the audio data belongs to the category corresponding to the binary classification task. For example: the original label corresponding to "whether it is a voice command" in any binary classification task of any audio data is 1, and the audio data belongs to a voice command. If the original label of the audio data in any binary classification task is 0, it means that the audio data does not belong to the category corresponding to the binary classification task.
[0086] The predicted category output by the teacher network for audio data in any binary classification task ranges from 0 to 1, indicating the confidence that the audio data belongs to the category corresponding to the binary classification task.
[0087] When executing step 330, the target label may be determined by correcting the label.
[0088] See also Figure 5 , is a flow chart of a label correction method provided in an embodiment of the present application, such as Figure 5 As shown, the method may include the following steps 510 to 520.
[0089] Step 510: For any audio data, if the original label of the audio data in any binary classification task is 0, and the predicted category of the audio data in the binary classification task is greater than or equal to 1-T, determine that the target label of the audio data in the binary classification task is 1; where T is a constant between 0 and 0.5.
[0090] Here, the value of T can be configured as needed. For example, T can be a value among 0.35, 0.4, 0.45, etc.
[0091] If the original label of any audio data in any binary classification task is 0, and the predicted category of the audio data in the binary classification task is greater than or equal to 1-T, it means that there is a conflict between the predicted category and the original label. At this time, since the predicted category is closer to 1, it means that the confidence that the audio data belongs to the corresponding category of the binary classification task is greater, and the original label can be corrected to 1 as the target label of the audio data in the binary classification task.
[0092] The original label is denoted as Y org , the predicted category is recorded as Y pre , the target label is denoted as Y new Step 510 can be expressed as: org =0, and Y pre ≥1-T, then Y new =1.
[0093] Step 520: For any audio data, if the original label of the audio data in any binary classification task is 1, and the predicted category of the audio data in the binary classification task is less than T, determine that the target label of the audio data in the binary classification task is 0.
[0094] If the original label of any audio data in any binary classification task is 1, and the predicted category of the audio data from the binary classification task is less than T, it means that there is a conflict between the predicted category and the original label. At this time, since the predicted category is closer to 0, it means that the confidence that the audio data does not belong to the corresponding category of the binary classification task is larger, and the original label can be corrected to 0 as the target label of the audio data in the binary classification task.
[0095] The original label is denoted as Y org , the predicted category is recorded as Y pre , the target label is denoted as Y new Step 520 can be expressed as: org =1, and Y pre <1-T, then Y new =0.
[0096] Through the above measures, the target label can be corrected when the original label needs to be corrected.
[0097] In one embodiment, the original label of the audio data in any binary classification task is 1 or 0. If the original label of the audio data in any binary classification task is 1, it means that the audio data belongs to the category corresponding to the binary classification task. If the original label of the audio data in any binary classification task is 0, it means that the audio data does not belong to the category corresponding to the binary classification task.
[0098] The predicted category output by the teacher network for audio data in any binary classification task ranges from 0 to 1, indicating the confidence that the audio data belongs to the category corresponding to the binary classification task.
[0099] When executing step 330, the target tag may be determined by shielding the tag.
[0100] See also Figure 6 , is a flow chart of a label shielding method provided in an embodiment of the present application, such as Figure 6As shown, the method may include the following steps 610 to 620.
[0101] Step 610: For any audio data, if the original label of the audio data in any binary classification task is 0, and the predicted category of the audio data in the binary classification task is less than 1-T and greater than or equal to 0.5, determine that the target label of the audio data in the binary classification task is a masked label; where T is a constant between 0 and 0.5.
[0102] Here, the value of T can be configured as needed. For example, T can be a value among 0.35, 0.4, 0.45, etc.
[0103] If the original label of any audio data in any binary classification task is 0, and the predicted category of the audio data in the binary classification task is less than 1-T and greater than or equal to 0.5, it means that the predicted category conflicts with the original label. At this time, although the predicted category is closer to 1, it is not large enough, and the predicted category may not be accurate. In this case, the target label of the audio data in the binary classification task is set to the masked label. Here, the masked label does not participate in the loss function calculation and back propagation in the subsequent training process.
[0104] The original label is denoted as Y org , the predicted category is recorded as Y pre , the target label is denoted as Y new Step 610 can be expressed as: org =0, and 0.5≤Y pre <1-T, then Y new =NaN. NaN is a special identifier for a masked label.
[0105] Step 620: For any audio data, if the original label of the audio data in any binary classification task is 1, and the predicted category of the audio data in the binary classification task is less than 0.5 and greater than or equal to T, determine that the target label of the audio data in the binary classification task is a blocked label.
[0106] If the original label of any audio data in any binary classification task is 1, and the predicted category of the audio data in the binary classification task is less than 0.5 and greater than or equal to T, it means that the predicted category conflicts with the original label. At this time, although the predicted category is closer to 0, it is not small enough, and the predicted category may not be accurate. In this case, the target label of the audio data in the binary classification task is set to the blocked label.
[0107] The original label is denoted as Y org , the predicted category is recorded as Y pre , the target label is denoted as Y new Step 620 can be expressed as: org=1, and T≤Y pre <0.5, then Y new =NaN.
[0108] Through the above measures, when there is a conflict between the original label and the predicted category and the confidence of the predicted category is insufficient, the target label can be set as a blocked label and no longer participate in the subsequent training process, thereby avoiding the introduction of new erroneous labels by correction and preventing possible erroneous original labels from participating in the training process.
[0109] In addition, for other cases other than the aforementioned correction and shielding, the original label can be considered correct. In this case, the original label can be used as the target label.
[0110] For example: the original label is recorded as Y org , the predicted category is recorded as Y pre , the target label is denoted as Y new .
[0111] Y org =0, and Y pre <0.5, then Y new = 0. Or,
[0112] Y org =1, and Y pre ≥0.5, then Y new =1.
[0113] Through the aforementioned correction and shielding adjustment strategies, it is possible to differentiate between high-confidence false labels and low-confidence false labels during the label re-labeling process, correct high-confidence false labels, and shield low-confidence false labels, thereby improving the problem of the teacher network introducing incorrect labels when re-labeling audio data and improving the accuracy of data labels.
[0114] In one embodiment, after adjusting the labels of the audio data in the dataset, the student network can be trained. Figure 7 , is a schematic diagram of the training process of the student network provided in an embodiment of the present application, such as Figure 7 As shown, when executing step 340 to train the student network, the following steps 710 to 750 may be executed.
[0115] Step 710: Perform data enhancement on multiple audio data carrying target labels to obtain multiple enhanced audio data.
[0116] In order to increase the amount of audio data in the data set and improve the subsequent training effect on the student network, data enhancement processing can be performed on multiple audio data carrying target labels to obtain multiple enhanced audio data.
[0117] Step 720: Perform fast Fourier transform on each enhanced audio data to obtain a transform result.
[0118] Step 730: Perform Mel filter bank processing on the transformation result to obtain a processing result.
[0119] Step 740: Take the logarithm of the processing result and perform discrete cosine transform on the logarithm to obtain audio features.
[0120] By executing the process from step 720 to step 740, the enhanced audio data is processed into audio features that can be processed by a deep neural network through a preprocessing process of fast Fourier transform, Mel filter bank processing, logarithm taking and discrete cosine transform.
[0121] By performing preprocessing on each enhanced audio data, the audio features corresponding to each enhanced audio data can be obtained.
[0122] Step 750: Train a student network using audio features that carry target labels.
[0123] The target label carried by each audio feature is the target label carried by the enhanced audio data corresponding to the audio feature. After obtaining multiple audio features carrying target labels, the deep neural network can be supervised trained through multiple audio features to train a student network.
[0124] Through the above measures, the enhanced audio data can be preprocessed into audio features that can be calculated by the deep neural network, and then the student network can be trained through multiple audio features carrying target labels.
[0125] In one embodiment, when executing step 710 to perform data enhancement, two audio data may be selected from a plurality of audio data, and the two audio data may be multiplied by a linear coefficient of random decibels respectively to obtain two adjusted audio data.
[0126] Here, decibel is a relative value based on the energy or amplitude of audio data, which can be expressed by the following formula (1):
[0127] dB = 10 × log 10 (E signal / E reference ) (1)
[0128] Where, dB is decibel; E signal / E reference is the relative value of energy.
[0129] If you need to increase the audio data by x dB, you can add 10×log to the decibel level. 10 (Linear coefficient), here, 10×log10 (Linear coefficient) = x dB. x is the required value, representing random decibels.
[0130] In this case, the linear coefficient may be calculated and the audio data may be multiplied by the linear coefficient to obtain the adjusted audio data.
[0131] The two adjusted audio data are superimposed, and the two adjusted audio data overlap each other. Exemplarily, the two adjusted audio data are 10 seconds each, and the superimposed audio data is still 10 seconds. The target labels of the two adjusted audio data are fused to obtain a fused target label. In this case, the target labels of the two adjusted audio data can be taken as a union to obtain a fused target label. When the target labels of the two adjusted audio data are different in any binary classification task, the target label is selected as 1, that is, it belongs to the corresponding category of the binary classification task.
[0132] In this case, the two adjusted audio data may be processed into one enhanced audio data, which carries the fused target label.
[0133] By selecting two audio data from a plurality of audio data multiple times to perform the aforementioned superposition processing flow, a plurality of enhanced audio data can be obtained.
[0134] In one embodiment, when executing step 710 to perform data enhancement, two audio data may be selected from a plurality of audio data, and the two audio data may be multiplied by a linear coefficient of random decibels respectively to obtain two adjusted audio data.
[0135] Furthermore, data segments can be respectively intercepted from the two adjusted audio data, and the intercepted multiple data segments can be spliced. Exemplarily, the two adjusted audio data are both 10 seconds, and 5 seconds of data segments are respectively intercepted from each adjusted audio data, and the spliced audio data is still 10 seconds. The target labels of the two adjusted audio data are fused to obtain a fused target label. In this case, the target labels of the two adjusted audio data can be taken as the union to obtain the fused target label. When the target labels of the two adjusted audio data are different in any binary classification task, the target label is selected as 1, that is, it belongs to the corresponding category of the binary classification task.
[0136] In this case, the two adjusted audio data may be processed into one enhanced audio data, which carries the fused target label.
[0137] By selecting two audio data from multiple audio data multiple times to perform the aforementioned splicing processing flow, multiple enhanced audio data can be obtained.
[0138] Since human auditory perception is logarithmic in nature, using a linear coefficient of random decibels to adjust audio data can make the audio data obtained by subsequent data enhancement more consistent with human auditory experience, and can flexibly adjust the intensity of audio data. At the same time, this solution takes the union of multiple target labels during data enhancement, which is more suitable for data enhancement requirements under multiple binary classification tasks. The student network trained by the audio data after data enhancement in this solution has better representation ability and higher classification accuracy.
[0139] See also Figure 8 , is a schematic diagram of the overall training process of the audio classification network provided in an embodiment of the present application, such as Figure 8 As shown, the audio data in the multi-label audio dataset is first preprocessed to obtain audio features that can be processed by the deep neural network, and then the teacher network is trained with the audio features carrying the original labels. The teacher network outputs the predicted category for the audio data in the dataset, and the label correction and shielding strategy is used to determine the target label for the audio data based on the predicted category and the original label. Furthermore, the new dataset is data enhanced, and the data is preprocessed with the data enhanced dataset to train the student network.
[0140] Fig. 9 is a block diagram of a training device for an audio classification network according to an embodiment of the present invention. Fig. 9 As shown, the device may include:
[0141] A first training module 910 is used to train a teacher network using audio data in a data set; wherein each audio data in the data set carries an original label of at least one binary classification task;
[0142] A prediction module 920, configured to generate a prediction category corresponding to each binary classification task for the audio data in the data set through a trained teacher network;
[0143] A determination module 930, configured to determine a target label of each audio data based on the original label of the audio data and the predicted category;
[0144] The second training module 940 is used to train a student network as an audio classification network using audio data carrying multiple target labels.
[0145] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned audio classification network training method, which will not be repeated here.
[0146] In several embodiments provided in the present application, the disclosed devices and methods may also be implemented in other ways. The device embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment or a part of a code, and a module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0147] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0148] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.
Claims
1. A training method for an audio classification network, characterized in that: include: A teacher network is trained using audio data in a data set; wherein each audio data in the data set carries an original label of at least one binary classification task; Generate a predicted category corresponding to each binary classification task for the audio data in the dataset through the trained teacher network; Determine a target label for each audio data based on the original label of the audio data and the predicted category; The student network is trained by audio data carrying multiple target labels as an audio classification network.
2. The method according to claim 1, characterized in that: The method of training a teacher network using the audio data in the data set includes: Performing a fast Fourier transform on the audio data in the data set to obtain a transformation result; Performing Mel filter bank processing on the transformation result to obtain a processing result; Taking a logarithm of the processing result, and performing discrete cosine transform on the logarithm to obtain an audio feature; The teacher network is trained by audio features carrying original labels.
3. The method according to claim 1, characterized in that If the original label of the audio data in any binary classification task is 1, it means that the audio data belongs to the category corresponding to the binary classification task; if the original label of the audio data in any binary classification task is 0, it means that the audio data does not belong to the category corresponding to the binary classification task; the value range of the predicted category is between 0 and 1, indicating the confidence that the audio data belongs to the category corresponding to the binary classification task; The determining the target label of the audio data based on the original label of each audio data and the predicted category includes: For any audio data, if the original label of the audio data in any binary classification task is 0, and the predicted category of the audio data in the binary classification task is greater than or equal to 1-T, the target label of the audio data in the binary classification task is determined to be 1; where T is a constant between 0 and 0.5; For any audio data, if the original label of the audio data in any binary classification task is 1, and the predicted category of the audio data in the binary classification task is less than T, it is determined that the target label of the audio data in the binary classification task is 0.
4. The method according to claim 1, characterized in that: If the original label of the audio data in any binary classification task is 1, it means that the audio data belongs to the category corresponding to the binary classification task; if the original label of the audio data in any binary classification task is 0, it means that the audio data does not belong to the category corresponding to the binary classification task; the value range of the predicted category is between 0 and 1, indicating the confidence that the audio data belongs to the category corresponding to the binary classification task; The determining the target label of the audio data based on the original label of each audio data and the predicted category includes: For any audio data, if the original label of the audio data in any binary classification task is 0, and the predicted category of the audio data in the binary classification task is less than 1-T and greater than or equal to 0.5, it is determined that the target label of the audio data in the binary classification task is a masked label; where T is a constant between 0 and 0.5; For any audio data, if the original label of the audio data in any binary classification task is 1, and the predicted category of the audio data in the binary classification task is less than 0.5 and greater than or equal to T, it is determined that the target label of the audio data in the binary classification task is a blocked label.
5. The method according to claim 1, characterized in that The student network is trained by using multiple audio data carrying target labels, including: Performing data enhancement on the multiple audio data carrying the target labels to obtain multiple enhanced audio data; Performing fast Fourier transform on each enhanced audio data to obtain a transform result; Performing Mel filter bank processing on the transformation result to obtain a processing result; Taking a logarithm of the processing result, and performing discrete cosine transform on the logarithm to obtain an audio feature; The student network is trained by audio features carrying target labels.
6. The method according to claim 5, characterized in that The step of performing data enhancement on the plurality of audio data carrying target tags to obtain a plurality of enhanced audio data includes: Select two audio data from the plurality of audio data, and multiply the two audio data by a linear coefficient of random decibels respectively to obtain two adjusted audio data; The two adjusted audio data are superimposed, and the target labels of the two adjusted audio data are merged to obtain enhanced audio data; wherein the enhanced audio data carries the merged target label; By selecting two audio data for processing multiple times, multiple added audio data are obtained.
7. The method according to claim 5, characterized in that The step of performing data enhancement on the plurality of audio data carrying target tags to obtain a plurality of enhanced audio data includes: Select two audio data from the plurality of audio data, and multiply the two audio data by a linear coefficient of random decibels respectively to obtain two adjusted audio data; respectively extracting data segments from the two adjusted audio data, splicing the extracted multiple data segments, and fusing the target labels of the two adjusted audio data to obtain enhanced audio data; wherein the enhanced audio data carries the fused target label; By selecting two audio data for processing multiple times, multiple added audio data are obtained.
8. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing processor-executable instructions; The processor is configured to execute the training method of the audio classification network according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program can be executed by a processor to complete the training method of the audio classification network described in any one of claims 1-7.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the training method of the audio classification network described in any one of claims 1 to 7 is implemented.