A sound model training method and a sound model training device

Abstract

This application provides a sound model training method and a sound model training apparatus. The method includes: acquiring dynamic distribution parameters; determining a target training method for each training data based on the dynamic distribution parameters; training an initial model based on each training data and a set of target training methods corresponding to each training data to obtain a target model; wherein the target training methods include at least a one-shot learning mode and a zero-shot learning mode. Through this method and apparatus, the model can simultaneously adapt to both zero-shot and one-shot learning modes, allowing the model to perform well in both modes without requiring separate training data and training strategies for zero-shot and one-shot learning, thus simplifying the model training process.

Description

Technical Field

[0001] This application relates to the field of model training technology, and in particular to a sound model training method and a sound model training device. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep learning models have made significant progress in fields such as image recognition and natural language processing. Zero-shot learning and one-shot learning, as emerging machine learning paradigms, can solve the problems of data scarcity or dynamic category expansion in traditional supervised learning. In zero-shot learning, the model can directly understand and infer new inputs based on its generalization ability, even without having encountered a specific category or task during training. In one-shot learning, the model quickly adapts to new tasks using a single reference sample, which provides key feature encodings or hints for the target task.

[0003] To achieve a model's dual capability of zero-shot and one-shot learning, two mainstream training methods are typically employed: one is to acquire independent training data for each learning mode, construct different model architectures, and perform phased training; the other is to decompose the input features into multiple attributes (such as duration, prosody, and timbre in speech signals) and then control the performance of each attribute during data generation. However, both methods have significant limitations, being complex and inefficient in training. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a sound model training method and a sound model training device, which only requires classifying the training mode based on dynamic distribution parameters for each training data, so that the model can perform well in both zero-shot learning and one-shot learning modes. The training of the model in each mode is more efficient and easier to implement, without the need to construct different training data and training strategies for zero-shot learning and one-shot learning, thus simplifying the model training process.

[0005] In a first aspect, embodiments of this application provide a sound model training method, the sound model training method comprising:

[0006] Obtain dynamic distribution parameters, and determine the target training method for each training data based on the dynamic distribution parameters;

[0007] Based on each training data and the set of target training methods corresponding to each training data, the initial model is trained to obtain the target model; wherein, the target training methods include at least a single-shot learning mode and a zero-shot learning mode.

[0008] Furthermore, determining the target training method for each piece of training data includes:

[0009] For each piece of training data, a random number is obtained; wherein, both the dynamic distribution parameter and the random number are between 0 and 1;

[0010] When the random number corresponding to the training data is greater than or equal to the dynamic distribution parameter, the target training method is determined to be a single-sample learning mode.

[0011] When the random number corresponding to the training data is less than the dynamic distribution parameter, the target training method is determined to be zero-shot learning mode.

[0012] Furthermore, when the target training method is a single-sample learning mode, training the initial model based on each training data point and its corresponding set of target training methods includes:

[0013] For each piece of training data, at least one target prompt word is obtained, and a sound feature vector of the training data is constructed based on the at least one target prompt word;

[0014] The training data is reconstructed based on the sound feature vectors to obtain the target training data;

[0015] The target training data is input into the initial model to perform single-sample learning on the initial model.

[0016] Furthermore, the process of reconstructing the training data based on the sound feature vector to obtain the target training data includes:

[0017] The sound feature vector is concatenated to the beginning of the sentence in the training data to obtain the target training data; wherein the sound feature vector of each training data has the same dimension.

[0018] Furthermore, when the target training method is zero-shot learning mode, training the initial model based on each training data point and its corresponding set of target training methods includes:

[0019] For each piece of training data, the sound representation information of that training data is extracted as the corresponding sound feature vector;

[0020] The training data and the corresponding sound feature vector are simultaneously input into the initial model to perform zero-shot learning on the initial model.

[0021] Furthermore, the sound representation information of the training data is extracted from the training data, or the sound representation information of the training data is extracted from the Mel spectrum of the training data.

[0022] Furthermore, before determining the target training method for each piece of training data, the sound model training method also includes:

[0023] Extract at least one prompt word from each piece of training data; where each prompt word describes the feature performance of the training data from different dimensions.

[0024] Furthermore, after obtaining the target model, the sound model training method further includes:

[0025] The performance of the target model in both single-shot and zero-shot learning modes is evaluated, and the dynamic distribution parameters are adjusted based on the performance evaluation results.

[0026] Furthermore, adjusting the dynamic distribution parameters based on the performance evaluation results includes:

[0027] If the performance evaluation result of the target model in the single-sample learning mode does not reach the first preset evaluation threshold, then the value of the dynamic distribution parameter is reduced.

[0028] If the performance evaluation result of the target model in zero-shot learning mode does not reach the second preset evaluation threshold, the value of the dynamic distribution parameter is increased.

[0029] Secondly, embodiments of this application also provide a sound model training device, the sound model training device comprising:

[0030] The training method determination module is used to obtain dynamic distribution parameters and determine the target training method for each training data based on the dynamic distribution parameters.

[0031] The model training module is used to train the initial model based on each training data and the set of target training methods corresponding to each training data to obtain the target model; wherein, the target training methods include at least a single-shot learning mode and a zero-shot learning mode.

[0032] This application provides a sound model training method and sound model training device. First, dynamic distribution parameters are obtained, and a target training method for each training data is determined based on the dynamic distribution parameters. Then, an initial model is trained based on each training data and the set of target training methods corresponding to each training data to obtain a target model. The target training methods include at least a single-sample learning mode and a zero-sample learning mode.

[0033] This application only requires classifying the training method for each training data point based on dynamic distribution parameters. It enables training in both zero-shot and one-shot learning modes on a traditional generative model structure using the same training dataset and strategy. This allows the model to adapt to both modes simultaneously, achieving good performance in both without complex feature decoupling or additional data preparation. Compared to traditional model training methods, the sound model training method provided in this application is more efficient and easier to implement for each mode, eliminating the need to construct different training data and strategies for zero-shot and one-shot learning, thus simplifying the model training process.

[0034] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 A flowchart illustrating a sound model training method provided in an embodiment of this application;

[0037] Figure 2 This is one of the structural schematic diagrams of a sound model training device provided in an embodiment of this application;

[0038] Figure 3 This is a second schematic diagram of the structure of a sound model training device provided in an embodiment of this application;

[0039] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0041] First, the applicable scenarios for this application will be introduced. This application can be applied to the field of model training technology.

[0042] With the rapid development of artificial intelligence technology, deep learning models have made significant progress in fields such as image recognition and natural language processing. Zero-shot learning and one-shot learning, as emerging machine learning paradigms, can solve the problems of data scarcity or dynamic category expansion in traditional supervised learning. In zero-shot learning, the model can directly understand and infer new inputs based on its generalization ability, even without having encountered a specific category or task during training. In one-shot learning, the model quickly adapts to new tasks using a single reference sample, which provides key feature encodings or hints for the target task.

[0043] Research has found that to achieve a model's dual capability of zero-shot and one-shot learning, two mainstream training methods are typically employed: one is to acquire independent training data for each learning mode, construct different model architectures, and perform phased training; the other is to decompose the input features into multiple attributes (such as duration, prosody, and timbre in speech signals) and then control the performance of each attribute during data generation. However, both methods have significant limitations, being complex to train and inefficient.

[0044] Based on this, the embodiments of this application provide a sound model training method that enables the model to perform well in both zero-shot learning and one-shot learning modes. This method is more efficient and easier to implement, eliminating the need to design different training strategies for zero-shot learning and one-shot learning, thus simplifying the model training process.

[0045] Please see Figure 1 , Figure 1 This is a flowchart illustrating a sound model training method provided in an embodiment of this application. Figure 1As shown in the embodiments of this application, the sound model training method includes:

[0046] S101, Obtain dynamic distribution parameters, and determine the target training method for each training data based on the dynamic distribution parameters.

[0047] Regarding step S101 above, in specific implementation, the dynamic distribution parameter P is obtained, and based on this dynamic distribution parameter P, the target training method for each training data point is determined. Here, the training data is audio training samples, used to train the initial sound model.

[0048] Specifically, regarding step S101 above, determining the target training method for each piece of training data includes:

[0049] Step 1011: For each piece of training data, obtain a random number.

[0050] Regarding step 1011 above, in specific implementation, for each piece of training data, obtain the random number R corresponding to that training data. Here, both the dynamic distribution parameter P and the random number R are between 0 and 1, that is, R, P∈[0,1].

[0051] Step 1012: When the random number corresponding to the training data is greater than or equal to the dynamic distribution parameter, the target training method is determined to be a single-sample learning mode.

[0052] Step 1013: When the random number corresponding to the training data is less than the dynamic distribution parameter, the target training method is determined to be zero-shot learning mode.

[0053] Here, one-shot learning refers to a model that may or may not have seen the target timbre during training, but can quickly adapt to new tasks or categories during inference using a reference sample. For example, after giving the model a timbre sample, it can adjust its output style based on that sample. Zero-shot learning refers to a model that has never encountered certain specific tasks or categories during training, but can still handle these new tasks or generate new results during inference. For example, it can generate sounds with a similar timbre even if it has never received speech data for a particular timbre.

[0054] Regarding steps 1012-1013 above, in specific implementation, when it is determined that the random number R corresponding to the training data is greater than or equal to the dynamic distribution parameter P, the target training method for the training data is determined to be the single-sample learning mode. When it is determined that the random number R corresponding to the training data is less than the dynamic distribution parameter P, the target training method for the training data is determined to be the zero-sample learning mode.

[0055] According to the sound model training method provided in this application, before determining the target training method for each piece of training data, the sound model training method further includes:

[0056] Extract at least one prompt word from each training data point.

[0057] In implementing the above steps, each training data set includes a constraint portion in addition to the data portion. This constraint portion consists of one or more prompt words, and at least one prompt word is extracted from each training data set. Here, the prompt words are used to define the feature representation of the training data, with each prompt word describing the feature representation of the training data from different dimensions. For example, when the model to be trained is a TTS (Text-to-Speech) model, the dimensions of the prompt words in the training data could be: timbre, pitch, speech rate, language, etc.

[0058] S102, based on each training data and the set of target training methods corresponding to each training data, train the initial model to obtain the target model.

[0059] Here, the target model can be a sound model used for speech generation and / or music generation.

[0060] Regarding step S102 above, in specific implementation, after the target training method for each training data is determined, the initial model is trained based on each training data and the set of target training methods corresponding to each training data to obtain the target model.

[0061] Here, according to the embodiments provided in this application, the trained model sequentially includes at least the following structures: an encoder, a decoder, and a waveform reconstruction module, wherein the encoder is selected as a learnable speaker encoder. The waveform reconstruction module can be a neural vocoder, used to convert several vectors output by the decoder into audible acoustic waveforms. The learnable speaker encoder differs from other audio encoders. Common audio encoders are usually pre-trained, but the learnable speaker encoder can be trained together with the model. In both training and inference, its input is an audio segment, and its output is a fixed-dimensional speaker embedding vector, used to represent the speaker's identity.

[0062] Thus, according to the above steps S101-S102, it is only necessary to classify the training method for each training data based on the dynamic distribution parameters. This allows the training of zero-shot learning mode and one-shot learning mode on the traditional generative model structure, enabling the model to adapt to both zero-shot learning mode and one-shot learning mode simultaneously. This allows the model to perform well in both zero-shot learning and one-shot learning modes without the need for complex feature decoupling or additional data preparation.

[0063] According to the embodiments provided in this application, regarding step S102 above, when the target training method is a single-sample learning mode, training the initial model based on each training data and its corresponding set of target training methods includes:

[0064] I: For each piece of training data, obtain at least one target prompt word, and construct the sound feature vector of the training data based on the at least one target prompt word.

[0065] Regarding step I above, in specific implementation, when the single-sample learning mode is triggered, for each piece of training data, at least one target prompt word is extracted from at least one prompt word in the training data, and a sound feature vector of the training data is constructed based on the at least one target prompt word. In this way, when the training data is input into the initial model for training, the initial model can be trained simultaneously based on the sound feature vector and the audio data.

[0066] II: Reconstruct the training data based on the sound feature vector to obtain the target training data.

[0067] Regarding step II above, in order to maintain the training effect, it is generally advisable to keep all different parts of the training data intact during implementation. Specifically, the sound feature vector can be concatenated to the beginning or end of the training data, and the concatenated data can replace the original training data to obtain the target training data.

[0068] Furthermore, regarding step II above, the reconstructing of the training data based on the sound feature vector to obtain the target training data includes:

[0069] The sound feature vector is concatenated to the beginning of the sentence in the training data to obtain the target training data.

[0070] Preferably, the sound feature vector is concatenated to the beginning of the sentence in the training data to obtain the target training data. Here, the sound feature vector of each training data has the same dimension. Specifically, in response to triggering the single-sample learning mode, a fixed-length cue word is extracted from the limited part of the training data, and the training data is reconstructed. During the concatenation, the sound feature vector is concatenated to the beginning of the sentence in the training data, rather than the end. The uniform cue word length and the uniform training data length can reduce the training cost of the model. Specifically, if each part in each training data input to the model remains in the same position and the length of each part is the same, the model does not need much computing power to obtain the similarities of the training data, thus reducing the learning difficulty of the model. However, the length of the data part of each training data is usually different. If the sound feature vector is concatenated after the data part, it is easy to cause the position of the sound feature vector in each training data to be inconsistent. In contrast, it is only necessary to extract the same length of sound feature vectors each time the training data is reconstructed to achieve uniformity of the training data. Therefore, it is relatively easy to control the length of the sound feature vectors. Setting the sound feature vectors at the beginning of the sentences in the training data can ensure that the length and position of the sound feature vectors in each training data are relatively fixed.

[0071] III: Input the target training data into the initial model to perform single-sample learning on the initial model.

[0072] Regarding step III above, in practice, the target training data is input into the initial model to perform single-sample learning. In this way, the sound feature vector originates from directly extracted cue words. The initial model can then understand the sound representation of the training data based on the cue word portion of the sound feature vector. Furthermore, because it directly uses real sound features from the training data, the timbre information of the training data will be more accurate.

[0073] According to the embodiments provided in this application, regarding step S102 above, when the target training method is zero-shot learning mode, training the initial model based on each training data and its corresponding set of target training methods includes:

[0074] A: For each piece of training data, extract the sound representation information of that training data as the corresponding sound feature vector.

[0075] Regarding step A above, in specific implementation, for each piece of training data, the sound representation information of that training data is extracted as the sound feature vector corresponding to that training data.

[0076] As an optional embodiment, the sound representation information of the training data is extracted from the training data itself, or it is extracted from the Mel spectrum of the training data, which is converted from the training data. Alternatively, the sound representation information of the training data can also be obtained from at least one cue word in the training data.

[0077] B: Input the training data and the corresponding sound feature vector into the initial model simultaneously to perform zero-shot learning on the initial model.

[0078] Regarding step B above, in specific implementation, when using this training data for training, both the training data and its corresponding sound feature vector are simultaneously input into the initial model to perform zero-shot learning. In this way, the model can also understand the relationship between sound features and audio performance based on the sound feature vector extracted from this training data. A representation vector representing the global sound representation is extracted from the training data. Although the model has not seen the timbre of the audio data during actual inference, it has already learned how to represent various sound features from the training data and the global sound representation, and can map the actual timbre obtained during inference to a portion of the sound feature vector to represent the timbre of this training data.

[0079] Preferably, the dimension of the sound feature vector is the same in both single-sample learning mode and zero-sample learning mode.

[0080] As an optional embodiment, after obtaining the target model, the sound model training method provided in this application embodiment further includes:

[0081] The performance of the target model in both single-shot and zero-shot learning modes is evaluated, and the dynamic distribution parameters are adjusted based on the performance evaluation results.

[0082] In the specific implementation of the above steps, after training the initial model with training data to obtain the target model, the performance of the target model in the single-shot learning mode and the zero-shot learning mode is evaluated to obtain the performance evaluation results, and the dynamic distribution parameters are dynamically adjusted based on the performance evaluation results.

[0083] Specifically, regarding the above steps, adjusting the dynamic distribution parameters based on the performance evaluation results includes:

[0084] (1) If the performance evaluation result of the target model in the single-sample learning mode does not reach the first preset evaluation threshold, the value of the dynamic distribution parameter is reduced.

[0085] (2) If the performance evaluation result of the target model in the zero-shot learning mode does not reach the second preset evaluation threshold, the value of the dynamic distribution parameter is increased.

[0086] Regarding steps (1)-(2) above, in specific implementation, if the performance evaluation result of the target model in the single-shot learning mode does not reach the first preset evaluation threshold, the value of the dynamic distribution parameter P is reduced, so that the model is more inclined to the zero-shot learning mode when using training data for the next round of training, and the process returns to step S102 to retrain the target model. If the performance evaluation result of the target model in the zero-shot learning mode does not reach the second preset evaluation threshold, the value of the dynamic distribution parameter P is increased, so that the model is more inclined to the single-shot learning mode when using training data for the next round of training, and the process returns to step S102 to retrain the target model. Specifically, the dynamic adjustment of the dynamic distribution parameter is based on the generation performance of the target model under the zero-shot learning mode and the single-shot learning mode. For example, in the field of TTS models, performance evaluation can be performed based on the performance parameters of the TTS model under single-shot learning and zero-shot learning models, such as WER (word error rate) and SIM (speech similarity). For training modes with poor performance, the value of the dynamic distribution parameter can be adjusted to tilt more training samples for training, flexibly controlling the performance of the model in the two modes.

[0087] Here, as an optional implementation, if the performance evaluation results of the target model fail to meet the standards in both modes, the value of the dynamic distribution parameter P can be adjusted to 0 or 1, and the model can be retrained in both modes respectively. If the performance evaluation still fails to meet the standards after retraining, the training data needs to be adjusted, or the training architecture or other model parameters of the target model need to be adjusted before retraining.

[0088] The sound model training method provided in this application first obtains dynamic distribution parameters P, and determines the target training method for each training data based on the dynamic distribution parameters P. Then, based on each training data and the set of target training methods corresponding to each training data, the initial model is trained to obtain the target model. The target training methods include at least a single-sample learning mode and a zero-sample learning mode.

[0089] This application only requires classifying the training method for each training data point based on dynamic distribution parameters. It enables training in both zero-shot and one-shot learning modes on a traditional generative model structure using the same training dataset and strategy. This allows the model to adapt to both modes simultaneously, achieving good performance in both without complex feature decoupling or additional data preparation. Compared to traditional model training methods, the sound model training method provided in this application is more efficient and easier to implement for each mode, eliminating the need to construct different training data and strategies for zero-shot and one-shot learning, thus simplifying the model training process.

[0090] Please see Figure 2 , Figure 3 , Figure 2 This is one of the structural schematic diagrams of a sound model training device provided in an embodiment of this application. Figure 3 This is a second schematic diagram of the structure of a sound model training device provided in an embodiment of this application. Figure 2 As shown, the sound model training device 200 includes:

[0091] The training method determination module 201 is used to obtain dynamic distribution parameters and determine the target training method for each training data based on the dynamic distribution parameters.

[0092] The model training module 202 is used to train the initial model based on each training data and the set of target training methods corresponding to each training data to obtain the target model; wherein, the target training methods include at least a single-shot learning mode and a zero-shot learning mode.

[0093] Furthermore, when determining the target training method for each piece of training data, the model training module 202 is also used to:

[0094] For each piece of training data, a random number is obtained; wherein, both the dynamic distribution parameter and the random number are between 0 and 1;

[0095] When the random number corresponding to the training data is greater than or equal to the dynamic distribution parameter, the target training method is determined to be a single-sample learning mode.

[0096] When the random number corresponding to the training data is less than the dynamic distribution parameter, the target training method is determined to be zero-shot learning mode.

[0097] Furthermore, when the target training method is a single-sample learning mode, when the model training module 202 trains the initial model based on each training data and its corresponding set of target training methods, the model training module 202 is also used for:

[0098] For each piece of training data, at least one target prompt word is obtained, and a sound feature vector of the training data is constructed based on the at least one target prompt word;

[0099] The training data is reconstructed based on the sound feature vectors to obtain the target training data;

[0100] The target training data is input into the initial model to perform single-sample learning on the initial model.

[0101] Furthermore, when the model training module 202 is used to reconstruct the training data based on the sound feature vector to obtain the target training data, the model training module 202 is also used to:

[0102] The sound feature vector is concatenated to the beginning of the sentence in the training data to obtain the target training data; wherein the sound feature vector of each training data has the same dimension.

[0103] Furthermore, when the target training method is zero-shot learning mode, when the model training module 202 trains the initial model based on each training data and its corresponding set of target training methods, the model training module 202 is also used for:

[0104] For each piece of training data, the sound representation information of that training data is extracted as the corresponding sound feature vector;

[0105] The training data and the corresponding sound feature vector are simultaneously input into the initial model to perform zero-shot learning on the initial model.

[0106] Furthermore, the sound representation information of the training data is extracted from the training data, or the sound representation information of the training data is extracted from the Mel spectrum of the training data.

[0107] Please see Figure 3 The sound model training device 200 further includes a prompt word acquisition module 203, which, before determining the target training method for each piece of training data, is used to:

[0108] Extract at least one prompt word from each piece of training data; where each prompt word describes the feature performance of the training data from different dimensions.

[0109] Please see Figure 3 The sound model training device 200 further includes a parameter adjustment module 204, which, after obtaining the target model, is used to:

[0110] The performance of the target model in both single-shot and zero-shot learning modes is evaluated, and the dynamic distribution parameters are adjusted based on the performance evaluation results.

[0111] Furthermore, when adjusting the dynamic distribution parameters based on performance evaluation results, the parameter adjustment module 204 is also used to:

[0112] If the performance evaluation result of the target model in the single-sample learning mode does not reach the first preset evaluation threshold, then the value of the dynamic distribution parameter is reduced.

[0113] If the performance evaluation result of the target model in zero-shot learning mode does not reach the second preset evaluation threshold, the value of the dynamic distribution parameter is increased.

[0114] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0115] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 The steps of the sound model training method in the illustrated embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0116] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the sound model training method in the illustrated embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0117] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0121] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training a sound model, characterized in that, The sound model training method includes: Obtain dynamic distribution parameters, and determine the target training method for each training data based on the dynamic distribution parameters; Based on each training data and the set of target training methods corresponding to each training data, the initial model is trained to obtain the target model; wherein, the target training methods include at least a single-shot learning mode and a zero-shot learning mode. The determination of the target training method for each piece of training data includes: For each piece of training data, a random number is obtained; wherein, both the dynamic distribution parameter and the random number are between 0 and 1; When the random number corresponding to the training data is greater than or equal to the dynamic distribution parameter, the target training method is determined to be a single-sample learning mode. When the random number corresponding to the training data is less than the dynamic distribution parameter, the target training method is determined to be zero-shot learning mode.

2. The sound model training method according to claim 1, characterized in that, When the target training method is a single-sample learning mode, training the initial model based on each training data point and its corresponding set of target training methods includes: For each piece of training data, at least one target prompt word is obtained, and a sound feature vector of the training data is constructed based on the at least one target prompt word; The training data is reconstructed based on the sound feature vectors to obtain the target training data; The target training data is input into the initial model to perform single-sample learning on the initial model.

3. The sound model training method according to claim 2, characterized in that, The process of reconstructing the training data based on the sound feature vector to obtain the target training data includes: The sound feature vector is concatenated to the beginning of the sentence in the training data to obtain the target training data; wherein the sound feature vector of each training data has the same dimension.

4. The sound model training method according to claim 1, characterized in that, When the target training method is zero-shot learning mode, training the initial model based on each training data point and its corresponding set of target training methods includes: For each piece of training data, the sound representation information of that training data is extracted as the corresponding sound feature vector; The training data and the corresponding sound feature vector are simultaneously input into the initial model to perform zero-shot learning on the initial model.

5. The sound model training method according to claim 4, characterized in that, The sound representation information of the training data is extracted from the training data, or the sound representation information of the training data is extracted from the Mel spectrum of the training data.

6. The sound model training method according to claim 2, characterized in that, Before determining the target training method for each piece of training data, the sound model training method further includes: Extract at least one prompt word from each piece of training data; where each prompt word describes the feature performance of the training data from different dimensions.

7. The sound model training method according to claim 1, characterized in that, After obtaining the target model, the sound model training method further includes: The performance of the target model in both single-shot and zero-shot learning modes is evaluated, and the dynamic distribution parameters are adjusted based on the performance evaluation results.

8. The sound model training method according to claim 7, characterized in that, The adjustment of the dynamic distribution parameters based on the performance evaluation results includes: If the performance evaluation result of the target model in the single-sample learning mode does not reach the first preset evaluation threshold, then the value of the dynamic distribution parameter is reduced. If the performance evaluation result of the target model in zero-shot learning mode does not reach the second preset evaluation threshold, the value of the dynamic distribution parameter is increased.

9. A sound model training device, characterized in that, The sound model training device includes: The training method determination module is used to obtain dynamic distribution parameters and determine the target training method for each training data based on the dynamic distribution parameters. The model training module is used to train the initial model based on each piece of training data and the set of target training methods corresponding to each piece of training data to obtain the target model; wherein, the target training methods include at least a single-shot learning mode and a zero-shot learning mode. When determining the target training method for each piece of training data, the model training module is further configured to: For each piece of training data, a random number is obtained; wherein, both the dynamic distribution parameter and the random number are between 0 and 1; When the random number corresponding to the training data is greater than or equal to the dynamic distribution parameter, the target training method is determined to be a single-sample learning mode. When the random number corresponding to the training data is less than the dynamic distribution parameter, the target training method is determined to be zero-shot learning mode.

Images