A training method and training device of a labeling model

Abstract

The application provides a training method and a training device of a labeling model. The labeling model comprises an encoder, a decoder and a classifier. The method comprises: inputting at least one audio data training sample into the encoder and the decoder of an initial labeling model to obtain at least one first audio processing vector; inputting the audio data training sample into an audio processing model to obtain at least one second audio processing vector; selecting a first number of first audio processing vectors as positive examples and a second number of first audio processing vectors as negative examples based on the correlation between the first audio processing vectors and the second audio processing vectors; and iteratively training at least one parameter in the encoder and the decoder based on at least the positive examples, the negative examples and the at least one audio data training sample until a preset training completion condition is met, thereby completing the training. Thus, the dependence on artificial labeling data in the labeling model training process can be reduced.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a training method and training device for a labeled model. Background Technology

[0002] In audio-related technologies within the field of artificial intelligence, it is often necessary to annotate audio data to obtain a large amount of labeled audio data for training artificial intelligence models, thereby achieving supervised learning where the model depends on the labels.

[0003] In existing technologies, labeled audio data is typically obtained through manual annotation or by training annotation models. Training annotation models also requires a large amount of manually labeled data as raw training data. However, manual annotation is inefficient, has low accuracy, and is difficult to obtain; when manually labeled data is limited, it is difficult to effectively train annotation models. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a training method and training device for annotated models, so as to reduce the dependence on manually labeled data during the training process of annotated models.

[0005] This application provides a method for training a labeled model, which includes an encoder, a decoder, and a classifier; the training method includes:

[0006] Input at least one audio data training sample into the encoder and decoder of the initial labeled model to obtain at least one first audio processing vector;

[0007] The audio data training samples are input into the audio processing model to obtain at least one second audio processing vector.

[0008] Based on the correlation between the first audio processing vector and the second audio processing vector, a first number of first audio processing vectors are selected as positive examples and a second number of first audio processing vectors are selected as negative examples.

[0009] The training is completed by iteratively training at least one parameter of the encoder and the decoder based on at least the positive example, the negative example, and at least one audio data training sample until the preset training completion condition is met.

[0010] Furthermore, based on the correlation between the first audio processing vector and the second audio processing vector, a first number of first audio processing vectors are selected as positive examples and a second number of first audio processing vectors are selected as negative examples, including:

[0011] The first audio processing vectors are sorted according to the degree of relevance corresponding to each first audio processing vector;

[0012] The first audio processing vector with the highest correlation is selected as the first positive example, and the second audio processing vector with the lowest correlation is selected as the second negative example.

[0013] Furthermore, the first quantity and / or the second quantity are preset quantities; or, the first quantity and / or the second quantity are calculated according to a preset ratio; or, the first quantity and / or the second quantity are used as training parameters, with the optimization objective of maximizing the relevance, and the first quantity and / or the second quantity are dynamically updated during the iterative training of the labeled model.

[0014] Furthermore, at least one parameter of the encoder and the decoder is iteratively trained based on at least the positive examples, the negative examples, and at least one audio data training sample, including:

[0015] The positive examples obtained in each round of training are stored in the positive example set, and the negative examples are stored in the negative example set;

[0016] For the nth round of training, positive examples and negative examples for the nth round are selected from the positive example set and the negative example set, respectively; n is a positive integer greater than 1.

[0017] The encoder and the decoder are trained for the nth round based on the nth round of positive examples, the nth round of negative examples, and at least one audio data training sample.

[0018] Furthermore, for the nth round of training, positive examples and negative examples for the nth round are selected from the positive example set and the negative example set, respectively, including:

[0019] For the positive and negative examples obtained in the first n-2 rounds of training in the positive and negative example sets, the top p positive examples and top q negative examples with the highest evaluation weights are selected according to the evaluation weights corresponding to each positive and negative example; p and q are positive integers.

[0020] The top p positive examples and top q negative examples with the highest evaluation weights are mixed with the positive and negative examples obtained in the (n-1)th round of training to obtain the positive examples and negative examples in the nth round.

[0021] Furthermore, the training method also includes:

[0022] During the nth round of training, the positive and negative examples selected from the positive and negative examples obtained in the first n-2 rounds of training correspond to the first training weights, and the positive and negative examples obtained in the (n-1)th round of training correspond to the second training weights; the first training weights and the second training weights are used as training parameters and are dynamically updated during the iterative training of the labeled model.

[0023] Furthermore, the training method also includes:

[0024] Based on the impact parameters generated by each positive example in the positive example set and / or each negative example in the negative example set when used to train the labeled model, the evaluation weight corresponding to each positive example in the positive example set and / or each negative example in the negative example set is determined; wherein, the impact parameters include the change in the labeled model loss value and / or the change in the degree of relevance caused by the introduction of positive and negative examples in this round.

[0025] Furthermore, based on the spatial distance between the first audio processing vector and the corresponding second audio processing vector, the correlation between the first audio processing vector and the corresponding second audio processing vector is determined.

[0026] Furthermore, the classifier includes a feature input layer and an embedding layer, and the method for obtaining the audio labels of the sample audio includes:

[0027] The sample audio is input into the trained encoder to obtain the audio intermediate feature representation output by the encoder;

[0028] The audio intermediate feature representation is input into the feature input layer of the classifier for preprocessing.

[0029] The preprocessed audio intermediate feature representation is input into the embedding layer of the classifier to map the feature vector corresponding to the sample audio in the embedding space.

[0030] Based on the spatial distance between the feature vector and at least one label center in the embedding space, the feature vector is clustered to determine the audio label to which the feature vector belongs, which serves as the annotation information for the sample audio.

[0031] This application embodiment also provides a training device for annotated models, the training device comprising:

[0032] The first processing module is used to input at least one audio data training sample into the encoder and decoder of the initial annotation model to obtain at least one first audio processing vector.

[0033] The second processing module is used to input the audio data training samples into the audio processing model to obtain at least one second audio processing vector.

[0034] The filtering module is used to select a first number of first audio processing vectors as positive examples and a second number of first audio processing vectors as negative examples based on the correlation between the first audio processing vector and the second audio processing vector.

[0035] The training module is used to iteratively train at least one parameter of the encoder and the decoder based on at least the positive example, the negative example and at least one audio data training sample until the preset training completion condition is met, and then the training is completed.

[0036] This application provides a training method and apparatus for annotated models. It generates a first audio processing vector based on unlabeled audio data training samples and an initial annotation model, and generates a second audio processing vector based on the unlabeled audio data training samples and the trained audio processing model. Positive and negative examples are selected based on the correlation between the first and second audio processing vectors to further iteratively train the annotation model, enabling it to learn the feature representation capabilities of the audio processing model. Furthermore, by generating positive and negative examples from unlabeled data using a self-supervised learning mechanism, the reliance on manually labeled data during the training process can be significantly reduced, thereby lowering the costs of manual annotation and review.

[0037] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A flowchart illustrating a training method for an labeled model provided in an embodiment of this application is shown;

[0040] Figure 2 This illustration shows a schematic diagram of the structure of a training device for an labeled model provided in an embodiment of this application;

[0041] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0043] Research has found that in audio-related technologies in the field of artificial intelligence, it is often necessary to annotate audio data to obtain a large amount of labeled audio data to train neural network models in order to achieve supervised learning that relies on labels.

[0044] In existing technologies, labeled audio data is typically obtained through manual annotation or by training annotation models. Training annotation models also requires a large amount of manually labeled data as raw training data. However, manual annotation is inefficient, has low accuracy, and is difficult to obtain; when manually labeled data is limited, it is difficult to effectively train annotation models.

[0045] Based on this, embodiments of this application provide a training method for annotated models to reduce the reliance on manually labeled data during the training process of annotated models.

[0046] Please see Figure 1 , Figure 1 This is a flowchart illustrating a training method for a labeled model provided in an embodiment of this application.

[0047] The annotation model in this embodiment is used to annotate the features of audio in various dimensions. The audio may include speech, instrumental music, or songs containing vocals. The model structure of the annotation model includes an encoder, a decoder, and a classifier. The encoder is used to encode the input audio data to obtain an intermediate audio feature representation. The decoder is used to decode and restore the intermediate audio feature representation. The classifier is used to map the intermediate audio feature representation to understandable audio labels, thereby achieving the annotation of the audio data.

[0048] During model training, the encoder, decoder, and classifier of the annotation model need to be trained. In model application, the trained annotation model can use the encoder and classifier to annotate unknown audio, obtaining audio labels as annotation information. For example, audio labels can include rhythm, speaker's gender, age, tone, speech rate, timbre, etc.

[0049] like Figure 1 As shown in the embodiments of this application, the training method includes:

[0050] S101. Input at least one audio data training sample into the encoder and decoder of the initial annotation model to obtain at least one first audio processing vector.

[0051] In this step, at least one audio data training sample can be input into the encoder of the initial annotation model for encoding processing to obtain the audio intermediate feature representation corresponding to each audio data training sample; the audio intermediate feature representation can be an implicit sound representation or an explicit text-based sound tag representation; then, the audio intermediate feature representation is input into the decoder for decoding and restoration to obtain the first audio processing vector corresponding to each audio data training sample.

[0052] It should be noted that the initial annotation model is a pre-trained annotation model; for the pre-training method of the annotation model, a small amount of manually annotated data can be used to train the initial annotation model, and this application does not restrict the pre-training method.

[0053] S102. Input the audio data training samples into the audio processing model to obtain at least one second audio processing vector.

[0054] The audio processing model can be a well-trained existing model that can extract the feature information of the original audio signal and transform it into a low-dimensional dense vector; for example, if the audio processing model is an embedding model, the generated second audio processing vector is an embedding vector.

[0055] S103. Based on the correlation between the first audio processing vector and the second audio processing vector, select a first number of first audio processing vectors as positive examples and select a second number of first audio processing vectors as negative examples.

[0056] Here, the correlation between the first audio processing vector and the corresponding second audio processing vector can be determined based on the spatial distance between them. Spatial distance refers to the distance between two vectors in a vector space; generally, the larger the spatial distance, the smaller the correlation. More specifically, the spatial distance between the first audio processing vector and the corresponding second audio processing vector can be determined based on calculation methods such as cosine similarity and Euclidean distance. The second audio processing vector corresponding to the first audio processing vector refers to the first and second audio processing vectors generated respectively based on the same audio data training samples.

[0057] In this step, based on the correlation between the first audio processing vector and the second audio processing vector, a first number of first audio processing vectors with higher correlation can be selected as positive examples (positive samples), and a second number of first audio processing vectors with lower correlation can be selected as negative examples (negative samples). That is, the correlation of the first audio processing vector as a positive example should be higher than the correlation of the first audio processing vector as a negative example.

[0058] This is because, in this embodiment, the second audio processing vector is regarded as the real result, the correlation between the model training result and the real result is determined, similar model training results are used as positive examples for positive incentive, and dissimilar model training results with poor performance are used as negative examples. Through training with positive and negative examples, the labeled model is prompted to make the first audio processing vector obtained after the encoding and restoration process similar to the real result in subsequent training rounds, so that the labeled model can learn the feature representation ability of the audio processing model.

[0059] In specific implementation, step S103 may include:

[0060] The first audio processing vectors are sorted according to their relevance; the first number of first audio processing vectors with the highest relevance are selected as positive examples, and the second number of first audio processing vectors with the lowest relevance are selected as negative examples.

[0061] Here, the first audio processing vectors can be sorted in descending order of their correlation, and the first few first audio processing vectors with the highest correlation can be selected as positive examples, while the last few first audio processing vectors with the lowest correlation can be selected as negative examples.

[0062] Wherein, the first quantity and / or the second quantity can be preset quantities; that is, quantities preset according to the scale of the audio data training samples. Alternatively, the first quantity and / or the second quantity can be calculated according to a preset ratio, for example, the first 10% of the first audio processing vectors are preset as positive examples, and the last 8% of the first audio processing vectors are preset as negative examples.

[0063] Alternatively, and more preferably, the first quantity and / or the second quantity are used as training parameters, with the optimization objective of maximizing the relevance, and the first quantity and / or the second quantity are dynamically updated during the iterative training of the labeled model.

[0064] Unlike existing technologies where the number of positive and negative examples is typically fixed or proportional, this embodiment sets the first and / or second quantities as training parameters and dynamically updates them during the iterative training of the labeled model to continuously optimize the parameters. The optimization goal is to maximize relevance, ensuring that the first audio processing vector obtained after encoding and reconstruction by the labeled model in subsequent training processes is more similar to the true result. For example, in reinforcement learning, the training objective is not only to optimize the encoder and decoder in the labeled model but also to optimize the selection strategy of the first and / or second quantities to maximize the reward signal, such as the final annotation accuracy. Indirectly, by setting the first and / or second quantities as training parameters and dynamically updating them during the iterative training of the labeled model, the reinforcement learning effect of the model can be further optimized in this embodiment.

[0065] Alternatively, embodiments of this application can also adjust the number of positive and negative examples based on model performance using rules or manually. During training, the model's performance is monitored (e.g., labeling accuracy, F1 score, loss function value, etc.). If the model performs poorly at a certain training stage, the number of positive and / or negative examples can be increased or decreased to provide more challenging negative examples or more explicit positive examples. For example, if the model is overfitting, the number of negative examples can be slightly increased, allowing the model to learn to distinguish some negative examples that are more difficult to differentiate.

[0066] In this way, the values ​​of positive and negative examples are also dynamic data. Whether it is manually adjusted or the model self-adjusts, compared with setting fixed values, it can further optimize the model performance and improve the model training and inference effects.

[0067] S104. At least one parameter of the encoder and the decoder is iteratively trained based on the positive example, the negative example and at least one audio data training sample until the preset training completion condition is met, and the training is completed.

[0068] In this step, the encoder and decoder are iteratively trained based on at least the positive examples, negative examples, and at least one audio data training sample to optimize each parameter to be trained. The specific training method can refer to existing technologies, and this application does not impose any limitations on it.

[0069] Training is completed when the preset training completion conditions are met, resulting in a trained encoder and decoder. Training is considered complete when any one of the following conditions is met: 1. Performance on the validation set reaches saturation; 2. Rewards become stable; 3. The maximum number of training epochs / resource limits are reached; 4. The loss function is minimized or converges.

[0070] In one possible implementation, in each round of iterative training, at least one audio data training sample corresponding to that round, as well as the positive and negative examples generated in the previous round, can be used as training data.

[0071] In another possible implementation, to stabilize training results and reduce forgetting during training, this embodiment also introduces an experience replay mechanism. Step S104 may include:

[0072] S1041. Store the positive examples obtained in each round of training into the positive example set, and store the negative examples into the negative example set.

[0073] S1042. For the nth round of training, positive examples and negative examples for the nth round are selected from the positive example set and the negative example set, respectively; n is a positive integer greater than 1.

[0074] S1043. Based on the positive examples of the nth round, the negative examples of the nth round, and at least one audio data training sample, perform the nth round of training on at least one parameter of the encoder and the decoder.

[0075] Here, the positive and negative example sets serve as experience replay buffers. As training continues, a large number of positive and negative examples generated during historical training gradually accumulate, covering the learning results of the model at different training stages. Therefore, for the nth training round, the positive and negative examples selected from the positive and negative example sets for the nth round include not only those generated in the previous round (n-1 rounds) but also significant positive and negative examples generated in previous rounds. For example, some key positive and negative examples can be selected from the positive and negative examples generated in rounds 1 to n-2 and input into the labeled model along with the positive and negative examples generated in round n-1 for training; alternatively, some positive and negative examples can be randomly selected from the positive and negative examples generated in rounds 1 to n-2 and input into the labeled model along with the positive and negative examples generated in round n-1 for training.

[0076] Generally, the number of positive examples x2 selected from the training results of the 1st to n-2nd training sessions should be less than the number of positive examples x (i.e., the first number) generated in the previous round (n-1 round), and the number of negative examples y2 should be less than the number of positive examples y (i.e., the second number) generated in the previous round (n-1 round), and x2 > y2. For example, x2 can be 30%x to 50%x, and y2 can be 10%y to 30%y.

[0077] This increases sample diversity, helps labeled models consolidate old knowledge, reduces long-distance forgetting, and stabilizes training results.

[0078] Furthermore, step S1042 also includes:

[0079] For the positive and negative examples obtained in the first n-2 rounds of training in the positive and negative example sets, select the top p positive examples and top q negative examples with the highest evaluation weights according to the evaluation weights corresponding to each positive and negative example; p and q are positive integers; mix the top p positive examples and top q negative examples with the highest evaluation weights with the positive and negative examples obtained in the (n-1)th round of training to obtain the positive examples and negative examples in the nth round.

[0080] Here, according to the evaluation weights corresponding to each positive and negative example, the top p positive examples and top q negative examples with the highest evaluation weights can be selected and mixed with the positive and negative examples obtained in the (n-1)th round of training to obtain the positive examples and negative examples in the nth round.

[0081] Specifically, the evaluation weights corresponding to each positive example in the positive example set and / or each negative example in the negative example set can be determined based on the impact parameters generated by each positive example in the positive example set and / or each negative example in the negative example set when used to train the labeled model; wherein, the impact parameters include the change in the labeled model loss value and / or the change in the degree of relevance caused by the introduction of positive and negative examples in this round.

[0082] The higher the evaluation weight, the more important it is for training the labeled model. Therefore, the top p positive examples and top q negative examples with the highest evaluation weights are selected. After each training round generates new positive and negative examples, they are stored in the positive example set and negative example set respectively, and the ranking of the evaluation weights is updated.

[0083] Furthermore, the training method in this application embodiment also includes:

[0084] During the nth round of training, the positive and negative examples selected from the positive and negative examples obtained in the first n-2 rounds of training correspond to the first training weights, and the positive and negative examples obtained in the (n-1)th round of training correspond to the second training weights; the first training weights and the second training weights are used as training parameters and are dynamically updated during the iterative training of the labeled model.

[0085] Here, a lower first training weight can be assigned to the n² positive examples and m² negative examples from the replay, while a higher second training weight can be assigned to the positive and negative examples obtained in the (n-1)th training round. Since the positive and negative examples from the new round are the most valuable for reference, assigning smaller training weights to the positive and negative examples selected in the first n-2 rounds of training compared to those obtained in the previous round not only reduces forgetting during model training but also prevents the model from getting stuck in over-memory or local optima; indirectly, this improves the labeling performance of the labeled model. For example, the first training weight can be between 0.2 and 0.5, and the second training weight can be 1. Similarly, the first and second training weights can also be used as training parameters and dynamically updated during the iterative training of the labeled model.

[0086] It should be noted that the annotation model cannot obtain the annotation information of an audio segment by directly using the pre-trained encoder and decoder. To obtain the annotation information, it is necessary to input the audio intermediate feature representation output by the pre-trained encoder into a pre-trained classifier, and then the classifier maps the audio intermediate feature representation to specific audio labels, that is, to determine the specific annotation information based on the audio intermediate feature representation.

[0087] Therefore, for the labeled model architecture in the application stage, an embedding network can be built on top of the encoder as a classifier, which includes a feature input layer and an embedding layer in sequence.

[0088] In one possible implementation, the training process employs the contrastive loss method, where the parameters of the encoder, which has either been pre-trained or fully trained, need to be fixed during the classifier training process.

[0089] Furthermore, after training, the methods for obtaining audio labels for the sample audio include:

[0090] Step 1: Input the sample audio into the trained encoder to obtain the audio intermediate feature representation output by the encoder.

[0091] Step 2: Input the audio intermediate feature representation into the feature input layer of the classifier to preprocess the audio intermediate feature representation.

[0092] The feature input layer is used to receive the encoder output and perform preprocessing such as pooling on the intermediate audio feature representation of the output.

[0093] Step 3: Input the preprocessed audio intermediate feature representation into the embedding layer of the classifier to map the feature vector corresponding to the sample audio in the embedding space.

[0094] The embedding layer consists of one or more fully connected layers, which are used to map the features output by the encoder into a fixed low-dimensional embedding space, forming a fixed-dimensional embedding space vector.

[0095] Step 4: Based on the spatial distance between the feature vector and at least one label center in the embedding space, cluster the feature vector to determine the audio label to which the feature vector belongs, and use it as the annotation information of the sample audio.

[0096] In a low-dimensional embedding space, features with the same label are close to each other, while features with different labels are far apart. Therefore, in actual inference, the encoder and classifier first obtain the feature vector of the sample audio to be inferred in the embedding space. Then, based on the spatial distance between this feature vector and the nearest label center, if the spatial distance meets the clustering distance standard, the nearest label center can be used as the audio label to which the feature vector belongs. Thus, regardless of whether similar training data exists in the training data, the classifier can determine the audio label. Therefore, the classifier training method in this embodiment can achieve accurate classification of a large number of audio labels, and the classifier also has generalization ability when audio labels change dynamically, enabling classification even for audio labels not present during the training phase, thereby improving the annotation effect of the annotation model.

[0097] This application provides a method for training an annotation model. It generates a first audio processing vector based on unlabeled audio data training samples and an initial annotation model, and generates a second audio processing vector based on the unlabeled audio data training samples and the trained audio processing model. Positive and negative examples are selected based on the correlation between the first and second audio processing vectors to train the annotation model, enabling the annotation model to learn the feature representation capabilities of the audio processing model. Furthermore, by generating positive and negative examples from unlabeled data using a self-supervised learning mechanism, the reliance on manually labeled data during the annotation model training process can be significantly reduced, thereby lowering the costs of manual annotation and review.

[0098] Based on the same inventive concept, embodiments of this application also provide a training device for labeled models. Please refer to... Figure 2 , Figure 2 This is a schematic diagram of the structure of a training device for an labeled model provided in an embodiment of this application. Figure 2 As shown, the training device 200 includes:

[0099] The first processing module 210 is used to input at least one audio data training sample into the encoder and decoder of the initial annotation model to obtain at least one first audio processing vector.

[0100] The second processing module 220 is used to input the audio data training samples into the audio processing model to obtain at least one second audio processing vector.

[0101] The filtering module 230 is used to select a first number of first audio processing vectors as positive examples and a second number of first audio processing vectors as negative examples based on the correlation between the first audio processing vector and the second audio processing vector.

[0102] The training module 240 is used to iteratively train at least one parameter of the encoder and the decoder based on at least the positive example, the negative example and at least one audio data training sample until the preset training completion condition is met, and then the training is completed.

[0103] Furthermore, when the filtering module 230 selects a first number of first audio processing vectors as positive examples and a second number of first audio processing vectors as negative examples based on the correlation between the first audio processing vector and the second audio processing vector, the filtering module 230 is used to:

[0104] The first audio processing vectors are sorted according to the degree of relevance corresponding to each first audio processing vector;

[0105] The first audio processing vector with the highest correlation is selected as the first positive example, and the second audio processing vector with the lowest correlation is selected as the second negative example.

[0106] Furthermore, the first quantity and / or the second quantity are preset quantities; or, the first quantity and / or the second quantity are calculated according to a preset ratio; or, the first quantity and / or the second quantity are used as training parameters, with the optimization objective of maximizing the relevance, and the first quantity and / or the second quantity are dynamically updated during the iterative training of the labeled model.

[0107] Furthermore, when the training module 240 iteratively trains at least one parameter of the encoder and the decoder based at least on the positive examples, the negative examples, and at least one audio data training sample, the training module 240 is used to:

[0108] The positive examples obtained in each round of training are stored in the positive example set, and the negative examples are stored in the negative example set;

[0109] For the nth round of training, positive examples and negative examples for the nth round are selected from the positive example set and the negative example set, respectively; n is a positive integer greater than 1.

[0110] The encoder and the decoder are trained for the nth round based on the nth round of positive examples, the nth round of negative examples, and at least one audio data training sample.

[0111] Furthermore, when the training module 240 selects positive examples and negative examples for the nth round from the positive example set and the negative example set respectively for the nth round of training, the training module 240 is used to:

[0112] For the positive and negative examples obtained in the first n-2 rounds of training in the positive and negative example sets, the top p positive examples and top q negative examples with the highest evaluation weights are selected according to the evaluation weights corresponding to each positive and negative example; p and q are positive integers.

[0113] The top p positive examples and top q negative examples with the highest evaluation weights are mixed with the positive and negative examples obtained in the (n-1)th round of training to obtain the positive examples and negative examples in the nth round.

[0114] Furthermore, the training module 240 is also used for:

[0115] During the nth round of training, the positive and negative examples selected from the positive and negative examples obtained in the first n-2 rounds of training correspond to the first training weights, and the positive and negative examples obtained in the (n-1)th round of training correspond to the second training weights; the first training weights and the second training weights are used as training parameters and are dynamically updated during the iterative training of the labeled model.

[0116] Furthermore, the training module 240 is also used for:

[0117] Based on the impact parameters generated by each positive example in the positive example set and / or each negative example in the negative example set when used to train the labeled model, the evaluation weight corresponding to each positive example in the positive example set and / or each negative example in the negative example set is determined; wherein, the impact parameters include the change in the labeled model loss value and / or the change in the degree of relevance caused by the introduction of positive and negative examples in this round.

[0118] Furthermore, the filtering module 230 is also used for:

[0119] Based on the spatial distance between the first audio processing vector and the corresponding second audio processing vector, the degree of correlation between the first audio processing vector and the corresponding second audio processing vector is determined.

[0120] Furthermore, the classifier includes a feature input layer and an embedding layer, and the training device further includes an inference module; when the inference module is used to infer the audio labels of the sample audio, the inference module is used for:

[0121] The sample audio is input into the trained encoder to obtain the audio intermediate feature representation output by the encoder;

[0122] The audio intermediate feature representation is input into the feature input layer of the classifier for preprocessing.

[0123] The preprocessed audio intermediate feature representation is input into the embedding layer of the classifier to map the feature vector corresponding to the sample audio in the embedding space.

[0124] Based on the spatial distance between the feature vector and at least one label center in the embedding space, the feature vector is clustered to determine the audio label to which the feature vector belongs, which serves as the annotation information for the sample audio.

[0125] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device 300 includes a processor 310, a memory 320, and a bus 330.

[0126] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 and the memory 320 communicate via the bus 330. When the machine-readable instructions are executed by the processor 310, they can perform the operations described above. Figure 1 The steps of the training method for the labeled model in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0127] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the training method for the labeled model in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0128] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0132] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training a labeled model, characterized in that, The labeled model includes an encoder, a decoder, and a classifier; the training method includes: Input at least one audio data training sample into the encoder and decoder of the initial labeled model to obtain at least one first audio processing vector; The audio data training samples are input into the audio processing model to obtain at least one second audio processing vector; wherein the audio processing model includes an embedding model, and the second audio processing vector includes an embedding vector; Based on the correlation between the first audio processing vector and the second audio processing vector, a first number of first audio processing vectors are selected as positive examples and a second number of first audio processing vectors are selected as negative examples. The training is completed by iteratively training at least one parameter of the encoder and the decoder based on at least the positive example, the negative example, and at least one audio data training sample until the preset training completion condition is met.

2. The training method according to claim 1, characterized in that, Based on the correlation between the first audio processing vector and the second audio processing vector, a first number of first audio processing vectors are selected as positive examples and a second number of first audio processing vectors are selected as negative examples, including: The first audio processing vectors are sorted according to the degree of relevance corresponding to each first audio processing vector; The first audio processing vector with the highest correlation is selected as the first positive example, and the second audio processing vector with the lowest correlation is selected as the second negative example.

3. The training method according to claim 1 or 2, characterized in that, The first quantity and / or the second quantity are preset quantities; or, the first quantity and / or the second quantity are calculated according to a preset ratio; or, the first quantity and / or the second quantity are used as training parameters, with the optimization objective of maximizing the relevance, and the first quantity and / or the second quantity are dynamically updated during the iterative training of the labeled model.

4. The training method according to claim 1, characterized in that, Iterative training of at least one parameter in the encoder and the decoder is performed based on at least the positive examples, the negative examples, and at least one audio data training sample, including: The positive examples obtained in each round of training are stored in the positive example set, and the negative examples are stored in the negative example set; For the nth round of training, positive examples and negative examples for the nth round are selected from the positive example set and the negative example set, respectively; n is a positive integer greater than 1. The encoder and the decoder are trained for the nth round based on the nth round of positive examples, the nth round of negative examples, and at least one audio data training sample.

5. The training method according to claim 4, characterized in that, For the nth round of training, positive examples and negative examples for the nth round are selected from the positive example set and the negative example set, respectively, including: For the positive and negative examples obtained in the first n-2 rounds of training in the positive and negative example sets, the top p positive examples and top q negative examples with the highest evaluation weights are selected according to the evaluation weights corresponding to each positive and negative example; p and q are positive integers. The top p positive examples and top q negative examples with the highest evaluation weights are mixed with the positive and negative examples obtained in the (n-1)th round of training to obtain the positive examples and negative examples in the nth round.

6. The training method according to claim 4, characterized in that, The training method also includes: During the nth round of training, the positive and negative examples selected from the positive and negative examples obtained in the first n-2 rounds of training correspond to the first training weights, and the positive and negative examples obtained in the (n-1)th round of training correspond to the second training weights; the first training weights and the second training weights are used as training parameters and are dynamically updated during the iterative training of the labeled model.

7. The training method according to claim 6, characterized in that, The training method also includes: Based on the impact parameters generated by each positive example in the positive example set and / or each negative example in the negative example set when used to train the labeled model, the evaluation weight corresponding to each positive example in the positive example set and / or each negative example in the negative example set is determined; wherein, the impact parameters include the change in the labeled model loss value and / or the change in the degree of relevance caused by the introduction of positive and negative examples in this round.

8. The training method according to claim 2, characterized in that, Based on the spatial distance between the first audio processing vector and the corresponding second audio processing vector, the degree of correlation between the first audio processing vector and the corresponding second audio processing vector is determined.

9. The training method according to claim 1, characterized in that, The classifier includes a feature input layer and an embedding layer, and the methods for obtaining audio labels for sample audio include: The sample audio is input into the trained encoder to obtain the audio intermediate feature representation output by the encoder; The audio intermediate feature representation is input into the feature input layer of the classifier for preprocessing. The preprocessed audio intermediate feature representation is input into the embedding layer of the classifier to map the feature vector corresponding to the sample audio in the embedding space. Based on the spatial distance between the feature vector and at least one label center in the embedding space, the feature vector is clustered to determine the audio label to which the feature vector belongs, which serves as the annotation information for the sample audio.

10. A training device for a labeled model, characterized in that, The labeled model includes an encoder, a decoder, and a classifier; the training device includes: The first processing module is used to input at least one audio data training sample into the encoder and decoder of the initial annotation model to obtain at least one first audio processing vector. The second processing module is used to input the audio data training samples into an audio processing model to obtain at least one second audio processing vector; wherein, the audio processing model includes an embedding model, and the second audio processing vector includes an embedding vector; The filtering module is used to select a first number of first audio processing vectors as positive examples and a second number of first audio processing vectors as negative examples based on the correlation between the first audio processing vector and the second audio processing vector. The training module is used to iteratively train at least one parameter of the encoder and the decoder based on at least the positive example, the negative example and at least one audio data training sample until the preset training completion condition is met, and then the training is completed.

Images