Model training method, device, electronic device and storage medium
By combining the target domain and source domain data in acoustic model training, model overfitting is avoided, and the processing capability balance of the source domain and target domain is ensured, and the overfitting problem caused by directly utilizing a small amount of adaptive data is solved.
Patent Information
- Application Number
- CN202211042444.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-29
- Publication Date
- 2024-12-31
- Estimated Expiration
- 2042-08-29
AI Technical Summary
Directly using a small amount of adaptive data in the test environment to train pre-trained acoustic models can easily lead to overfitting of the model and damage the processing capability of the source domain.
By obtaining the data of the target domain and the pre-trained adaptive model, the number of data is determined, and the corresponding amount of data is extracted from the source domain data set, the model is trained in combination with the target domain and the source domain data to avoid overfitting. Specific methods include obtaining target domain data, determining the number of data, dividing the source domain data set, extracting relevant data, and using feature extraction models and linear layers for training to ensure the balance between the target domain and the source domain.
It effectively avoids model overfitting, improves the processing capability of the target domain, and maintains the processing capability of the source domain, prevents the model output distribution from biasing towards adaptive data.
Smart Images
Figure CN115497459B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of model training technology, and in particular to a model training method, device, electronic device and storage medium. Background Art
[0002] Acoustic model adaptation is an effective robust speech recognition technology. The implementation process is as follows: the acoustic model is pre-trained using sample data from the source domain. For a specific test environment, a small amount of adaptive data (i.e., sample data from the target domain) in the test environment is used to train the pre-trained acoustic model to match the test environment.
[0003] However, directly using a small amount of adaptive data in the test environment to train the pre-trained acoustic model can easily lead to overfitting of the model, causing the output distribution of the model to be biased towards the adaptive data, thereby damaging the information already learned in the acoustic model. That is, although the processing capability of the target domain is improved, the processing capability of the source domain is reduced. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide a model training method, device, electronic device and storage medium to solve the problem that directly using a small amount of adaptive data in a test environment to train a pre-trained acoustic model easily leads to overfitting of the model. The specific technical solution is as follows:
[0005] In a first aspect, a model training method is provided, the method comprising:
[0006] Acquire first data and an adaptive model corresponding to the target domain, where the adaptive model is a model pre-trained using a data set of the source domain;
[0007] Determine a first quantity corresponding to the first data, and determine a second quantity based on the first quantity;
[0008] Extracting the second amount of second data from the data set in the source domain;
[0009] The adaptive model is trained using the first data and the second data to obtain a target model corresponding to the target domain.
[0010] In a possible implementation, the using the first data and the second data to train the adaptive model includes:
[0011] Determine a target scene corresponding to the target domain, and obtain a feature extraction model corresponding to the target scene;
[0012] Inputting the first data into the feature extraction model to obtain scene features corresponding to the output of the feature extraction model;
[0013] The adaptive model is trained using the first data, the second data and the scene features.
[0014] In a possible implementation manner, extracting the second amount of second data from the data set in the source domain includes:
[0015] Dividing the data in the source domain dataset according to scenes to obtain at least one sub-dataset, wherein one sub-dataset corresponds to one scene;
[0016] Determine sub-datasets corresponding to scenes other than the target scene as candidate sub-datasets;
[0017] The second amount of second data is extracted from all the candidate sub-data sets.
[0018] In a possible implementation manner, extracting the second amount of second data from all the candidate sub-data sets includes:
[0019] Determining the relevance between the scene of each candidate sub-data set and the target scene;
[0020] Sorting all the candidate sub-datasets in order of the relevance from low to high, and determining a preset number of candidate sub-datasets ranked top as target sub-datasets;
[0021] The second quantity of second data is extracted from the target sub-dataset.
[0022] In a possible implementation manner, extracting the second amount of second data from the target sub-data set includes:
[0023] determining a third number of the target sub-datasets;
[0024] dividing the second number by the third number to obtain an average number;
[0025] The average amount of second data is extracted from each of the target sub-data sets.
[0026] In a possible implementation manner, extracting the second amount of second data from the data set in the source domain includes:
[0027] Determine the storage capacity corresponding to each data in the data set of the source domain;
[0028] Sorting all the data in order of the storage capacity from small to large, and determining the second amount of data that is ranked first as the second data;
[0029] The second data is extracted.
[0030] In a possible implementation manner, before acquiring the adaptive model corresponding to the target domain, the method further includes:
[0031] Acquire a data set of a source domain, wherein the data set includes at least one source domain data, and one source domain data corresponds to one data label;
[0032] Inputting the source domain data into a preset model including initial parameters, obtaining a prediction label output by the preset model for the source domain data, wherein the preset model has a newly added linear layer;
[0033] Determining a prediction loss value according to the data label and the prediction label corresponding to the same source domain data;
[0034] Obtaining a preset loss weight value, and determining a total loss value of the preset model according to the preset loss weight value and the predicted loss values corresponding to all the source domain data;
[0035] When the total loss value does not reach the preset convergence condition, the initial parameters of the preset model are iteratively updated until the total loss value reaches the preset convergence condition, and the converged preset model is recorded as the adaptive model.
[0036] In a second aspect, a model training device is provided, the device comprising:
[0037] A first acquisition module, used to acquire first data and an adaptive model corresponding to a target domain, wherein the adaptive model is a model pre-trained using a data set of a source domain;
[0038] A first determining module, configured to determine a first quantity corresponding to the first data, and determine a second quantity based on the first quantity;
[0039] an extraction module, configured to extract the second quantity of second data from the data set in the source domain;
[0040] A training module is used to train the adaptive model using the first data and the second data to obtain a target model corresponding to the target domain.
[0041] In a possible implementation, the training module is specifically used to:
[0042] Determine a target scene corresponding to the target domain, and obtain a feature extraction model corresponding to the target scene;
[0043] Inputting the first data into the feature extraction model to obtain scene features corresponding to the output of the feature extraction model;
[0044] The adaptive model is trained using the first data, the second data and the scene features.
[0045] In a possible implementation, the extraction module is specifically used to:
[0046] Dividing the data in the source domain dataset according to scenes to obtain at least one sub-dataset, wherein one sub-dataset corresponds to one scene;
[0047] Determine sub-datasets corresponding to scenes other than the target scene as candidate sub-datasets;
[0048] The second amount of second data is extracted from all the candidate sub-data sets.
[0049] In a possible implementation, the extraction module is further used to:
[0050] Determining the relevance between the scene of each candidate sub-data set and the target scene;
[0051] Sorting all the candidate sub-datasets in order of the relevance from low to high, and determining a preset number of candidate sub-datasets ranked top as target sub-datasets;
[0052] The second quantity of second data is extracted from the target sub-dataset.
[0053] In a possible implementation, the extraction module is further used to:
[0054] determining a third number of the target sub-datasets;
[0055] dividing the second number by the third number to obtain an average number;
[0056] The average amount of second data is extracted from each of the target sub-data sets.
[0057] In a possible implementation, the extraction module is further used to:
[0058] Determine the storage capacity corresponding to each data in the data set of the source domain;
[0059] Sorting all the data in order of the storage capacity from small to large, and determining the second amount of data that is ranked first as the second data;
[0060] The second data is extracted.
[0061] In a possible implementation, the device further includes:
[0062] A second acquisition module is used to acquire a data set of a source domain, wherein the data set includes at least one source domain data, and one source domain data corresponds to one data label;
[0063] An input module, used to input the source domain data into a preset model containing initial parameters, and obtain a prediction label output by the preset model for the source domain data, wherein the preset model has a newly added linear layer;
[0064] A second determination module, configured to determine a prediction loss value according to the data label and the prediction label corresponding to the same source domain data;
[0065] A third acquisition module is used to obtain a preset loss weight value, and determine a total loss value of the preset model according to the preset loss weight value and the predicted loss values corresponding to all the source domain data;
[0066] An iteration module is used to iteratively update the initial parameters of the preset model when the total loss value does not reach the preset convergence condition, until the total loss value reaches the preset convergence condition, and record the converged preset model as the adaptive model.
[0067] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0068] Memory, used to store computer programs;
[0069] The processor is used to implement any method step described in the first aspect when executing a program stored in the memory.
[0070] In a fourth aspect, a computer-readable storage medium is provided, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.
[0071] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-described model training methods.
[0072] Beneficial effects of the embodiments of the present application:
[0073] The embodiment of the present application provides a model training method, device, electronic device and storage medium. The present application first obtains the first data and adaptive model corresponding to the target domain, and the adaptive model is a model pre-trained using the data set of the source domain; then, determines the first quantity corresponding to the first data, and determines the second quantity based on the first quantity; and extracts the second data of the second quantity from the data set of the source domain; finally, uses the first data and the second data to train the adaptive model to obtain the target model corresponding to the target domain. That is, the adaptive model is trained using the training data of part of the source domain together with the target domain data, which can improve the processing capability of the target domain while avoiding reducing the processing capability of the source domain, thereby avoiding the overfitting phenomenon of the model.
[0074] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0075] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0076] Figure 1 A flowchart of a model training method provided in an embodiment of the present application;
[0077] Figure 2 A flowchart of another model training method provided in an embodiment of the present application;
[0078] Figure 3 A flowchart of another model training method provided in an embodiment of the present application;
[0079] Figure 4 A flowchart of another model training method provided in an embodiment of the present application;
[0080] Figure 5 A schematic diagram of the structure of a model training device provided in an embodiment of the present application;
[0081] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0082] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0083] Since a small amount of adaptive data in the test environment is directly used to train the pre-trained acoustic model, it is easy to cause the model to overfit, making the output distribution of the model biased towards the adaptive data, thereby damaging the information already learned in the acoustic model, that is, although the processing capability of the target domain is improved, the processing capability of the source domain is reduced. To this end, the embodiment of the present application provides a model training method that can be applied to a model training platform.
[0084] The following will describe in detail a model training method provided in an embodiment of the present application in combination with a specific implementation method. Figure 1 As shown, the specific steps are as follows:
[0085] S101, obtaining first data and an adaptive model corresponding to a target domain, wherein the adaptive model is a model pre-trained using a data set of a source domain.
[0086] A model training method provided in an embodiment of the present application is used to achieve model adaptation, wherein the adaptive model can be an acoustic model or other model, which is not specifically limited here.
[0087] The source domain refers to the domain used for initial training of the adaptive model.
[0088] The target domain refers to the domain where the adaptive model is further trained to obtain the final model.
[0089] The first data is adaptive data corresponding to the target domain and used to further train the adaptive model, wherein the first data and the data of the source domain data set are generally not repeated. The first data can be determined according to the application scenario corresponding to the adaptive model. Exemplarily, assuming that the adaptive model is an acoustic model, the first data can be voice data collected from a system database such as a voice interaction system or an intelligent customer service system.
[0090] As an embodiment, the model training platform can obtain the first data and the adaptive model uploaded by the user.
[0091] As another embodiment, at least one adaptive model and at least one first data of a target domain may be pre-stored in the model training platform, and the adaptive model to be trained and the first data to be used may be determined based on user selection.
[0092] In practical applications, adaptive models can be constructed based on deep neural networks (DNN).
[0093] Optionally, in order to improve the classification effect of the adaptive model, a linear layer (i.e., an affine matrix) can be added to the deep neural network to obtain a linear network, and an adaptive model can be constructed based on the linear network, wherein the linear layer can act on the input layer or on the output layer. The linear layer is used to perform an affine transformation on the features of the corresponding layer (i.e., when the linear layer acts on the input layer, an affine transformation is performed on the features of the input layer, and when the linear layer acts on the output layer, an affine transformation is performed on the features of the output layer). The transformed features are more conducive to classification, thereby improving the classification effect of the model.
[0094] Furthermore, the added linear layer has the same dimensions as the previous layer in the model. Exemplarily, when the linear layer is added after the input layer, the linear layer has the same dimensions as the input layer. When the linear layer is added before the output layer, the linear layer has the same dimensions as the previous network layer.
[0095] S102: Determine a first quantity corresponding to the first data, and determine a second quantity based on the first quantity.
[0096] As an embodiment, after determining the first quantity corresponding to the first data, the first quantity may be directly determined as the second quantity, that is, the first quantity is equal to the second quantity.
[0097] As another embodiment, different correspondences between quantity ranges and second quantities are preset, and after determining the first quantity corresponding to the first data, the quantity range in which the first quantity is located can be determined, and the second quantity corresponding to the quantity range can be determined. For example, the quantity range is 1001-2000, and the corresponding second quantity is 1500.
[0098] S103: extract the second quantity of second data from the data set in the source domain.
[0099] S104: Use the first data and the second data to train the adaptive model to obtain a target model corresponding to the target domain.
[0100] The following is a unified description of S103-S104:
[0101] In the embodiment of the present application, after determining the second quantity, the second quantity of second data can be extracted from the data set used to train the adaptive model in the source domain. The first data and the second data are used together as training samples to train the adaptive model until the model converges to obtain the target model corresponding to the target domain. This avoids the problem of model overfitting caused by only using the first data of the target domain to train the adaptive model.
[0102] Furthermore, the target model may be a speech recognition model for recognizing speech data as text data. After obtaining the target model according to the above steps, the target model may be applied in multiple different scenarios, such as intelligent customer service scenarios, intelligent robot scenarios, or intelligent vehicle scenarios. Taking the intelligent robot scenario as an example, the target model may be set on the intelligent robot. The intelligent robot completes a round of human-computer interaction mainly by implementing the following steps:
[0103] Acquire the speech to be recognized, and perform speech recognition on the speech to be recognized through a target model to obtain a speech text corresponding to the speech to be recognized; the target model is trained by the above-mentioned model training method.
[0104] Understandably, the speech data to be recognized may be issued by the user or other intelligent robots. When the intelligent robot is used in human-computer question-answering, the speech data to be recognized may be questions raised by the user. Specifically, when the intelligent robot detects that the target user or other intelligent robot has issued the speech to be recognized, the speech to be recognized can be obtained. The speech to be recognized is input into the target model so that the speech to be recognized is recognized by the target model. That is, the speech to be recognized is converted into text, that is, speech text is obtained.
[0105] Emotion detection is performed on the speech to be recognized and the speech text to determine a speech emotion tag corresponding to the speech to be recognized and a text emotion tag corresponding to the speech text.
[0106] It is understandable that an emotion recognition model can also be set in the intelligent robot, and the emotion recognition model can be composed of a neural network, a residual network, and an attention network, so that the emotion recognition model has the ability to recognize emotions in speech. A text processing model can also be set in the intelligent robot, and the text processing model can be constructed by a transformer network, so that the text processing model can recognize emotional keywords in speech text.
[0107] Specifically, after determining the speech text corresponding to the speech to be recognized, the phoneme features, gender features and speech features of the speech to be recognized can be extracted, so that the emotion recognition is performed based on the phoneme features and gender features to assist the speech features through the emotion recognition model, and the speech emotion label corresponding to the speech to be recognized is determined. The speech text is input into the text processing model, and the speech text is subjected to keyword recognition through the text processing model to determine the emotion keywords in the speech text. A speech database is obtained, which stores a plurality of emotion text groups of different categories, and one emotion text group corresponds to one emotion label. Then, the emotion keywords are matched with the text words in each emotion text group, so that the emotion label corresponding to the emotion text group matching the emotion keyword is used as the text emotion label of the speech text.
[0108] A reply text is generated based on the voice text, the voice emotion tag and the text emotion tag.
[0109] Specifically, after determining the voice emotion label and the text emotion label, the user's current emotional state can be determined according to the voice emotion label and the text emotion label. Then, a corresponding reply text can be generated based on the emotional state and the voice text. For example, when the user's current emotional state is angry, the reply text can use more mild words.
[0110] In the embodiment of the present application, first, the first data and the adaptive model corresponding to the target domain are obtained, and the adaptive model is a model pre-trained using the data set of the source domain; then, the first quantity corresponding to the first data is determined, and the second quantity is determined based on the first quantity; and the second data of the second quantity is extracted from the data set of the source domain; finally, the first data and the second data are used to train the adaptive model to obtain the target model corresponding to the target domain. That is, the adaptive model is trained using the training data of part of the source domain together with the target domain data, which can improve the processing capability of the target domain while avoiding reducing the processing capability of the source domain, thereby avoiding the overfitting phenomenon of the model.
[0111] In another embodiment of the present application, Figure 2 As shown, S104 may include the following steps:
[0112] S201, determining a target scene corresponding to the target domain, and acquiring a feature extraction model corresponding to the target scene.
[0113] S202: Input the first data into the feature extraction model to obtain scene features corresponding to the output of the feature extraction model.
[0114] S203: Train the adaptive model by using the first data, the second data and the scene features.
[0115] The following is a unified description of S201-S203:
[0116] The feature extraction model is used to extract scene-related features (i.e., scene features) from the first data. The initial network model can be trained in advance using sample data labeled with scenes until the model converges to obtain the feature extraction model. The initial network model can be constructed based on a convolutional neural network (CNN).
[0117] In the embodiment of the present application, the target scene corresponding to the target domain can be determined, and the scene features can be extracted using the feature extraction model corresponding to the target scene, and then the adaptive model can be trained using the first data, the second data and the scene features. In this way, the adaptive model can focus on processing parameters other than the scene features, thereby improving the model's processing capabilities for different scenes.
[0118] Correspondingly, when using the target model trained based on the above method to process the data of the target scene, the data can be input into the above feature extraction model in advance to obtain the corresponding scene features, and then the data and scene features of the target scene are input into the target model for processing to output the corresponding data.
[0119] In another embodiment of the present application, Figure 3 As shown, S103 may include the following steps:
[0120] S301 , dividing the data in the data set of the source domain according to scenes to obtain at least one sub-data set, wherein one sub-data set corresponds to one scene.
[0121] The above scenarios refer to data usage scenarios, such as financial scenarios, medical scenarios, and educational scenarios.
[0122] In practical applications, in order to further prevent model overfitting, the data in the source domain dataset can be divided according to the scene to obtain a sub-dataset corresponding to each scene.
[0123] In an embodiment of the present application, each data in the source domain data set is pre-labeled with a usage scenario, and the data in the source domain data set is divided according to the scenario to obtain at least one sub-data set. The specific implementation may include: for each data in the source domain data set, determining the scenario of the data according to the labeling information of the data, and dividing the data with the same scenario into a sub-data set.
[0124] S302: Determine sub-datasets corresponding to scenes other than the target scene as candidate sub-datasets.
[0125] S303: Extract the second quantity of second data from all the candidate sub-data sets.
[0126] The following is a unified description of S302 and S303:
[0127] In the embodiment of the present application, scenes other than the target scene are determined as scenes to be extracted, and sub-data sets corresponding to the scenes to be extracted are determined as candidate sub-data sets. Then, a second amount of second data is extracted from the candidate sub-data sets. In this way, it is possible to avoid extracting data of the same scene as the target domain from the data set in the source domain, thereby further preventing the model from overfitting.
[0128] In another embodiment of the present application, Figure 4 As shown, S303 may include the following steps:
[0129] S401, determining the relevance between the scene of each candidate sub-data set and the target scene;
[0130] S402, sorting all the candidate sub-datasets in order of the relevance from low to high, and determining a preset number of candidate sub-datasets with top rankings as target sub-datasets;
[0131] S403: extract the second quantity of second data from the target sub-data set.
[0132] In an embodiment of the present application, in order to further prevent the model from overfitting, the correlation between the scene of each candidate sub-dataset and the target scene can be determined, wherein the correlation can be calculated using the vector distance (such as Euclidean distance) between the scene and the target scene, and the closer the distance, the higher the correlation. Then, a preset number of candidate sub-datasets with lower correlation are determined as target sub-datasets. Finally, a second number of second data is extracted from the target sub-dataset.
[0133] As a possible implementation, the specific implementation of extracting the second amount of second data from all the candidate sub-datasets may include: determining the third amount of the target sub-datasets, dividing the second amount by the third amount to obtain an average amount, and extracting the average amount of second data from each target sub-dataset. In this way, the same amount of data is extracted from each target sub-dataset. In this way, it is possible to avoid extracting data from only a few scenes, which may weaken the model's ability to process data from other scenes.
[0134] In another embodiment of the present application, S103 may include the following steps:
[0135] Determine the storage capacity corresponding to each data in the data set of the source domain, sort all the data in order of the storage capacity from small to large, determine a second amount of data that is ranked first as the second data, and extract the second data.
[0136] The storage capacity corresponding to each of the above data is used to represent the size of the data. It should be noted that the smaller the storage capacity, the smaller the data.
[0137] In an embodiment of the present application, by sorting each data in the source domain data set in order of storage capacity from small to large, and taking the second amount of data with the highest sorting as the second data for extraction, it is achieved to extract the smaller data in the source domain data set as the second data, which can increase the data extraction speed and thus improve the processing efficiency.
[0138] In one embodiment, before step S101, that is, before obtaining the adaptive model corresponding to the target domain, the following steps are included:
[0139] A data set of a source domain is obtained; the data set of the source domain includes at least one source domain data; one source domain data corresponds to one data label.
[0140] Understandably, when the adaptive model to be trained is an acoustic model, the source domain data may be speech data. The method of collecting speech data is not limited here. The data label is the representation corresponding to the source domain data. Exemplarily, assuming that the adaptive model is specifically a dialect speech classification model in the acoustic model, the source domain data may be speech data of different dialects, and the data label is the dialect representation corresponding to the source domain data. For example, the data label may be a label such as Cantonese, Mandarin or Sichuanese.
[0141] The source domain data is input into a preset model including initial parameters to obtain a predicted label output by the preset model for the source domain data; the preset model has a newly added linear layer.
[0142] Understandably, the preset model can be obtained by modifying the DNN network. The preset model in this embodiment adds a linear layer between the input layer and the rear hidden layer (the rear hidden layer refers to the hidden layer after the input layer and adjacent to the input layer) of the DNN network, or adds a linear layer between the output layer and the front hidden layer (the front hidden layer refers to the hidden layer before the output layer and adjacent to the output layer), that is, a newly added linear layer. The newly added linear layer has a corresponding hierarchical weight.
[0143] Furthermore, the preset model is obtained after pre-training the newly added linear layer. That is, the preset model can be trained by the above-mentioned source domain data, and during the training process, the weights of other hidden layers in the preset model are kept unchanged, and only the layer weights of the newly added linear layer are adjusted. For example, the layer weights of the newly added linear layer can be iteratively updated through the BP (Back Propagation) algorithm, so that the updated newly added linear layer can better extract features in the source domain data, thereby improving the recognition accuracy of the preset model.
[0144] Furthermore, the predicted label is a label obtained by the preset model identifying the characteristics of the source domain data and predicting the category of the source domain data. Exemplarily, assuming that the adaptive model is specifically a dialect speech classification model in the acoustic model, the predicted label is a probability value representing the source domain data belonging to labels such as Cantonese, Mandarin or Sichuanese. Among them, one source domain data corresponds to one predicted label.
[0145] A prediction loss value is determined according to the data label and the prediction label corresponding to the same source domain data.
[0146] Specifically, after the source domain data is input into a preset model containing initial parameters and the predicted label output by the preset model for the source domain data is obtained, the predicted loss value can be determined through a loss function (such as a cross entropy loss function) based on the data label and predicted label corresponding to the same source domain data.
[0147] A preset loss weight value is obtained, and a total loss value of the preset model is determined according to the preset loss weight value and the predicted loss values corresponding to all the source domain data.
[0148] It is understandable that, in the above description, it is pointed out that the newly added linear layer in the preset model is pre-trained. If only the BP algorithm is used to iteratively update the hierarchical weight of the newly added linear layer, while improving the recognition accuracy of the preset model, it will cause the preset model to overfit. In order to reduce the impact of the overfitting phenomenon on the preset model, the preset loss weight value is introduced in this embodiment to adjust the loss value of the preset model.
[0149] Specifically, after determining the predicted loss value according to the data label and the predicted label corresponding to the same source domain data, the sum of all predicted loss values is determined as the total predicted value, and the product between the weight difference (the weight difference is the difference between 1 and the preset loss weight value) and the total predicted value is determined as the first loss value. The posterior probability value obtained by forward calculation of the preset model for the source domain data is obtained, and the second loss value is determined according to the posterior probability value, the predicted label and the preset loss weight value, and then the sum of the first loss value and the second loss value is determined as the total loss value of the preset model.
[0150] When the total loss value does not reach the preset convergence condition, the initial parameters of the preset model are iteratively updated until the total loss value reaches the preset convergence condition, and the converged preset model is recorded as the adaptive model.
[0151] It can be understood that the preset convergence condition can be the condition that the total loss value is less than the set threshold, that is, when the total loss value is less than the set threshold, the training is stopped; the preset convergence condition can also be the condition that the total loss value is very small after 10,000 calculations and will not decrease anymore, that is, when the total loss value is very small after 10,000 calculations and will not decrease, the training is stopped and the preset model after convergence is recorded as the adaptive model.
[0152] Furthermore, after determining the total loss value of the preset model according to the preset loss weight value and the predicted loss value corresponding to all source domain data, if the total loss value does not reach the preset convergence condition, the initial parameters in the preset model are updated, and the source domain data corresponding to the total loss value is re-input into the updated preset model, and in this process, the total loss value corresponding to the source domain data is continuously monitored to see whether it reaches the preset convergence condition. After the total loss value of the source domain data reaches the preset convergence condition, the preset model is updated and adjusted with another source domain data, so that the total loss values corresponding to all source domain data reach the preset convergence condition, the preset model can be determined to be converged, and the converged preset model is determined as the adaptive model.
[0153] Based on the same technical concept, the present application embodiment also provides a model training device, such as Figure 5 As shown, the device comprises:
[0154] A first acquisition module 501 is used to acquire first data and an adaptive model corresponding to a target domain, wherein the adaptive model is a model pre-trained using a data set of a source domain;
[0155] A first determining module 502, configured to determine a first quantity corresponding to the first data, and determine a second quantity based on the first quantity;
[0156] An extraction module 503, configured to extract the second quantity of second data from the data set in the source domain;
[0157] The training module 504 is used to train the adaptive model using the first data and the second data to obtain a target model corresponding to the target domain.
[0158] In a possible implementation, the training module is specifically used to:
[0159] Determine a target scene corresponding to the target domain, and obtain a feature extraction model corresponding to the target scene;
[0160] Inputting the first data into the feature extraction model to obtain scene features corresponding to the output of the feature extraction model;
[0161] The adaptive model is trained using the first data, the second data and the scene features.
[0162] In a possible implementation, the extraction module is specifically used to:
[0163] Dividing the data in the source domain dataset according to scenes to obtain at least one sub-dataset, wherein one sub-dataset corresponds to one scene;
[0164] Determine sub-datasets corresponding to scenes other than the target scene as candidate sub-datasets;
[0165] The second amount of second data is extracted from all the candidate sub-data sets.
[0166] In a possible implementation, the extraction module is further used to:
[0167] Determining the relevance between the scene of each candidate sub-data set and the target scene;
[0168] Sorting all the candidate sub-datasets in order of the relevance from low to high, and determining a preset number of candidate sub-datasets ranked top as target sub-datasets;
[0169] The second quantity of second data is extracted from the target sub-dataset.
[0170] In a possible implementation, the extraction module is further used to:
[0171] determining a third number of the target sub-datasets;
[0172] dividing the second number by the third number to obtain an average number;
[0173] The average amount of second data is extracted from each of the target sub-data sets.
[0174] In a possible implementation, the extraction module is further used to:
[0175] Determine the storage capacity corresponding to each data in the data set of the source domain;
[0176] Sorting all the data in order of the storage capacity from small to large, and determining the second amount of data that is ranked first as the second data;
[0177] The second data is extracted.
[0178] In the embodiment of the present application, first, the first data and the adaptive model corresponding to the target domain are obtained, and the adaptive model is a model pre-trained using the data set of the source domain; then, the first quantity corresponding to the first data is determined, and the second quantity is determined based on the first quantity; and the second data of the second quantity is extracted from the data set of the source domain; finally, the first data and the second data are used to train the adaptive model to obtain the target model corresponding to the target domain. That is, the adaptive model is trained using the training data of part of the source domain together with the target domain data, which can improve the processing capability of the target domain while avoiding reducing the processing capability of the source domain, thereby avoiding the overfitting phenomenon of the model.
[0179] Based on the same technical concept, the embodiment of the present application also provides an electronic device, such as Figure 6 As shown, it includes a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.
[0180] Memory 113, used for storing computer programs;
[0181] The processor 111 is used to execute the program stored in the memory 113 to implement the following steps:
[0182] Acquire first data and an adaptive model corresponding to the target domain, where the adaptive model is a model pre-trained using a data set of the source domain;
[0183] Determine a first quantity corresponding to the first data, and determine a second quantity based on the first quantity;
[0184] Extracting the second amount of second data from the data set in the source domain;
[0185] The adaptive model is trained using the first data and the second data to obtain a target model corresponding to the target domain.
[0186] In a possible implementation, the using the first data and the second data to train the adaptive model includes:
[0187] Determine a target scene corresponding to the target domain, and obtain a feature extraction model corresponding to the target scene;
[0188] Inputting the first data into the feature extraction model to obtain scene features corresponding to the output of the feature extraction model;
[0189] The adaptive model is trained using the first data, the second data and the scene features.
[0190] In a possible implementation manner, extracting the second amount of second data from the data set in the source domain includes:
[0191] Dividing the data in the source domain dataset according to scenes to obtain at least one sub-dataset, wherein one sub-dataset corresponds to one scene;
[0192] Determine sub-datasets corresponding to scenes other than the target scene as candidate sub-datasets;
[0193] The second amount of second data is extracted from all the candidate sub-data sets.
[0194] In a possible implementation manner, extracting the second amount of second data from all the candidate sub-data sets includes:
[0195] Determining the relevance between the scene of each candidate sub-data set and the target scene;
[0196] Sorting all the candidate sub-datasets in order of the relevance from low to high, and determining a preset number of candidate sub-datasets ranked top as target sub-datasets;
[0197] The second quantity of second data is extracted from the target sub-dataset.
[0198] In a possible implementation manner, extracting the second amount of second data from the data set in the source domain includes:
[0199] Determine the storage capacity corresponding to each data in the data set of the source domain; sort all the data in order from small to large according to the storage capacity, and determine the second amount of data that is ranked first as the second data for extraction.
[0200] In a possible implementation manner, before using the first data and the second data to train the adaptive model, the method further includes:
[0201] A linear layer is added to the network structure of the adaptive model, wherein the linear layer is added after the input layer or before the output layer of the network structure.
[0202] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0203] The communication interface is used for communication between the above electronic device and other devices.
[0204] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0205] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0206] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned model training methods are implemented.
[0207] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any model training method in the above embodiments.
[0208] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.
[0209] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0210] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A model training method, characterized in that: Applied to acoustic model training, the method comprises: Acquire first data and an adaptive model corresponding to the target domain, where the adaptive model is a model pre-trained using a data set of the source domain; Determine a first quantity corresponding to the first data, and determine a second quantity based on the first quantity; Extracting the second amount of second data from the data set in the source domain; Using the first data and the second data to train the adaptive model and obtain the target model corresponding to the target domain, including: determining the target scene corresponding to the target domain and obtaining the feature extraction model corresponding to the target scene; inputting the first data into the feature extraction model to obtain the scene features corresponding to the output of the feature extraction model; using the first data, the second data and the scene features to train the adaptive model; Extracting the second amount of second data from the data set in the source domain includes dividing the data in the data set in the source domain according to scenes to obtain at least one sub-data set, wherein one sub-data set corresponds to one scene; determining sub-data sets corresponding to scenes other than the target scene as candidate sub-data sets; and extracting the second amount of second data from all the candidate sub-data sets; Before obtaining the adaptive model corresponding to the target domain, it also includes: obtaining a data set of the source domain, wherein the data set includes at least one source domain data, and one source domain data corresponds to a data label; inputting the source domain data into a preset model containing initial parameters, and obtaining the prediction label output by the preset model for the source domain data, wherein the preset model has a newly added linear layer; determining a prediction loss value according to the data label and the prediction label corresponding to the same source domain data; obtaining a preset loss weight value, and determining a total loss value of the preset model according to the preset loss weight value and the prediction loss values corresponding to all the source domain data; when the total loss value does not reach a preset convergence condition, iteratively updating the initial parameters of the preset model until the total loss value reaches the preset convergence condition, and recording the converged preset model as the adaptive model.
2. The model training method according to claim 1, characterized in that: The step of extracting the second amount of second data from all the candidate sub-data sets includes: Determining the relevance between the scene of each candidate sub-data set and the target scene; Sorting all the candidate sub-datasets in order of the relevance from low to high, and determining a preset number of candidate sub-datasets with top rankings as target sub-datasets; The second quantity of second data is extracted from the target sub-dataset.
3. The model training method according to claim 2, characterized in that: Extracting the second amount of second data from the target sub-data set includes: determining a third number of the target sub-datasets; dividing the second number by the third number to obtain an average number; The average amount of second data is extracted from each of the target sub-data sets.
4. The model training method according to claim 1, characterized in that: The step of extracting the second amount of second data from the data set in the source domain includes: Determine the storage capacity corresponding to each data in the data set of the source domain; Sorting all the data in order of the storage capacity from small to large, and determining the second amount of data that is ranked first as the second data; The second data is extracted.
5. A model training device, characterized in that: Applied to acoustic model training, the device comprises: A first acquisition module, used to acquire first data and an adaptive model corresponding to a target domain, wherein the adaptive model is a model pre-trained using a data set of a source domain; A first determining module, configured to determine a first quantity corresponding to the first data, and determine a second quantity based on the first quantity; an extraction module, configured to extract the second quantity of second data from the data set in the source domain; A training module, used to train the adaptive model using the first data and the second data to obtain a target model corresponding to the target domain, including: determining a target scene corresponding to the target domain and obtaining a feature extraction model corresponding to the target scene; inputting the first data into the feature extraction model to obtain scene features corresponding to the output of the feature extraction model; and training the adaptive model using the first data, the second data and the scene features; Extracting the second amount of second data from the data set in the source domain includes dividing the data in the data set in the source domain according to scenes to obtain at least one sub-data set, wherein one sub-data set corresponds to one scene; determining sub-data sets corresponding to scenes other than the target scene as candidate sub-data sets; and extracting the second amount of second data from all the candidate sub-data sets; Before obtaining the adaptive model corresponding to the target domain, it also includes: obtaining a data set of the source domain, wherein the data set includes at least one source domain data, and one source domain data corresponds to a data label; inputting the source domain data into a preset model containing initial parameters, and obtaining the prediction label output by the preset model for the source domain data, wherein the preset model has a newly added linear layer; determining a prediction loss value according to the data label and the prediction label corresponding to the same source domain data; obtaining a preset loss weight value, and determining a total loss value of the preset model according to the preset loss weight value and the prediction loss values corresponding to all the source domain data; when the total loss value does not reach a preset convergence condition, iteratively updating the initial parameters of the preset model until the total loss value reaches the preset convergence condition, and recording the converged preset model as the adaptive model.
6. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the model training method described in any one of claims 1-4 when executing the program stored in the memory.
7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the model training method described in any one of claims 1 to 4 is implemented.