A data processing model training method, a data processing method, and related devices

Abstract

The application provides a data processing model training method, a data processing method and related equipment, and the method comprises the following steps: training an initial data processing model by using training data in a first training data set to obtain a first data processing model, wherein the first training data set comprises training data in a general field; performing a replication operation on a module in the first data processing model to obtain a replicated module, and connecting the replicated module and an original module in parallel to obtain a second data processing model; and training the second data processing model by using training data in a second training data set, wherein the second training data set at least comprises training data in a specific field, and only the parameters of the replicated module are updated when the second data processing model is trained. The data processing model training method provided by the application can train a data processing model which has good processing effects on data in a general field and data in a specific field.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data processing model training method, a data processing method, and related equipment. Background Technology

[0002] Current data processing methods (such as speech recognition methods) include model-based data processing methods. Most model-based data processing methods involve training a data processing model using training data from a general domain, and then using the trained data processing model to process the data to be processed.

[0003] In practical applications, it may be necessary to process data from a general domain or data from a specific domain. However, the data processing model trained using general domain training data exhibits good processing results for general domain data, but performs poorly for data from specific domains. Summary of the Invention

[0004] In view of this, the present invention provides a data processing model training method, a data processing method, and related equipment to solve the problem that existing data processing methods are not effective for data processing in specific fields. The technical solution is as follows:

[0005] Firstly, a data processing model training method is provided, including:

[0006] The initial data processing model is trained using the training data in the first training dataset to obtain the first data processing model. The first training dataset includes multiple training data from a general domain.

[0007] The modules in the first data processing model are copied to obtain a copied module, and the copied module is connected in parallel with the original module to obtain a second data processing model;

[0008] The second data processing model is trained using training data from the second training dataset to obtain the target data processing model. The second training dataset includes at least multiple training data from a specific domain. When training the second data processing model, only the parameters of the copying module are updated.

[0009] Optionally, the copying operation of the modules in the first data processing model includes:

[0010] One or more specified modules may be selected from the modules included in the first data processing model; or, one or more modules with specified functions may be selected from the modules included in the first data processing model; or, one or more modules may be randomly selected from the modules included in the first data processing model.

[0011] Perform a copy operation on the selected module.

[0012] Optionally, the first data processing model includes several linear layers, and each linear layer includes at least a linear transformation module;

[0013] The step of selecting one or more specified modules from the modules included in the first data processing model includes:

[0014] Select one or more linear transformation modules from the modules included in the first data processing model.

[0015] Optionally, the first data processing model includes N levels of modules, wherein the nth level module is the module contained in the (n-1)th level module, and n ranges from 2 to N;

[0016] The step of randomly selecting one or more modules from the modules included in the first data processing model includes:

[0017] Randomly select several modules of the same level from the N-level modules contained in the first data processing model;

[0018] Alternatively, several modules of different levels may be randomly selected from the N-level modules contained in the first data processing model.

[0019] Optionally, the process of obtaining the second training dataset includes:

[0020] Acquire multiple training data sets from a specific domain and multiple training data sets from a general domain;

[0021] The acquired training data from multiple specific domains are mixed with the acquired training data from multiple general domains, and the mixed training data forms the second training dataset.

[0022] Optionally, in the second data processing model, the input of the copy module is the same as that of the original module, and the output of the copy module in the second data processing model is fused with the output of the original module before being input into the next module.

[0023] Secondly, a data processing method is provided, including:

[0024] Acquire data to be processed in a specific or general domain;

[0025] The data to be processed is input into a pre-trained target data processing model for data processing to obtain the data processing result.

[0026] The target data processing model is trained using any of the data processing model training methods described above.

[0027] Thirdly, a data processing model training device is provided, comprising: a first training module, a model processing module, and a second training module;

[0028] The first training module is used to train an initial data processing model using training data in a first training dataset to obtain a first data processing model, wherein the first training dataset includes multiple training data from a general domain.

[0029] The model processing module is used to perform a copying operation on the module in the first data processing model to obtain a copy module, and connect the copy module in parallel with the original module to obtain a second data processing model.

[0030] The second training module is used to train the second data processing model using training data in the second training dataset to obtain the target data processing model. The second training dataset includes at least multiple training data in a specific domain. When training the second data processing model, only the parameters of the copying module are updated.

[0031] Fourthly, a data processing device is provided, including: a memory and a processor;

[0032] The memory is used to store programs;

[0033] The processor is used to execute the program to implement each step of the data processing model training method described above.

[0034] Fifthly, a computer-readable storage medium is provided, on which a computer program is stored, wherein when executed by a processor, the computer program implements the various steps of the data processing model training method described in any of the preceding claims.

[0035] The data processing model training method provided by this invention first uses a first training dataset including training data from multiple general domains to train an initial data processing model, obtaining a first data processing model. Then, modules in the first data processing model are copied to obtain copied modules. Next, the copied modules are connected in parallel with the original modules to obtain a second data processing model. Finally, the parameters of the copied modules in the second data processing model are updated using a second training dataset including at least multiple training data from specific domains, and only on the second data processing model. The data processing model training method provided by this invention first trains a first data processing model that has good processing performance on general domain data. Then, a copied module is introduced based on the first data processing model to obtain a second data processing model. Since the copied module is obtained by copying modules in the first data processing model, its structure and parameters are the same as the corresponding original modules in the first data processing model. When training the second data processing model, the parameters of the introduced copied module are not randomly initialized, but are obtained through the first stage of training (i.e., training using general domain training data). Furthermore, the parameters of the introduced copied module are the same as those of the original modules in the first data processing model. Since the initial modules have the same structure, the model converges easily when training the second data processing model. Furthermore, during the training of the second data processing model, only the parameters of the copy module are updated, while the parameters of other modules are not updated. This parameter processing strategy allows the second data processing model to learn knowledge from specific domains without forgetting knowledge learned in the general domain. Thus, through training, the processing performance of the second data processing model for data from specific domains can be improved, while ensuring that the processing performance for data from the general domain does not decline. In other words, the target data processing model trained in the end has good processing performance for both data from specific domains and data from the general domain. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the hardware architecture involved in the present invention;

[0038] Figure 2 A flowchart illustrating the data processing model training method provided in an embodiment of the present invention;

[0039] Figure 3An example of obtaining a second data processing model by copying and paralleling some modules in a first data processing model, as provided in an embodiment of the present invention;

[0040] Figure 4 Another example of obtaining a second data processing model by copying and paralleling some modules in the first data processing model provided in the embodiments of the present invention;

[0041] Figure 5 This is an example of a second speech recognition model obtained by copying and paralleling the linear transformation module in a speech recognition model with a Conformer structure, as provided in an embodiment of the present invention.

[0042] Figure 6 This is an example of a second data processing model obtained by randomly selecting several peer modules from a first data processing model and replicating and connecting them in parallel, as provided in an embodiment of the present invention.

[0043] Figure 7 This is an example of a second data processing model obtained by randomly selecting several non-level modules from a first data processing model and replicating and connecting them in parallel, as provided in an embodiment of the present invention.

[0044] Figure 8 This is a schematic diagram illustrating the input and output of the copy module and the original module in the second data processing model provided in this embodiment of the invention.

[0045] Figure 9 This is a schematic diagram of the structure of the data processing model training device provided in an embodiment of the present invention;

[0046] Figure 10 This is a schematic diagram of the structure of a data processing device provided in an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Given that current data processing solutions (training a data processing model using training data from a general domain, and then using the trained data processing model to process the data to be processed) are not effective for data processing in specific domains, the inventors of this case conducted research to improve the processing effect for data in specific domains. The initial idea was to first train a data processing model using training data from a general domain, then mix the training data from the specific domain with the training data from the general domain, and use the mixed training data to fine-tune all the parameters of the data processing model. The fine-tuned data processing model is then used as the target data processing model, and the data to be processed is then processed based on the target data processing model.

[0049] While the above approach can improve the processing effect on data in a specific domain to some extent, the method of fine-tuning all the parameters of the data processing model will cause the data processing model to be overly biased to the distribution of data in a specific domain. This will lead to a deterioration in the general effect while fitting the distribution of the specific domain. In other words, the data processing model after full parameter fine-tuning will have a lower processing effect on data in a general domain.

[0050] To address the aforementioned issues, the inventors continued their research and, through continuous efforts, ultimately proposed a solution that can improve the processing performance of data in specific domains while maintaining the processing performance of data in general domains.

[0051] Before introducing the solution provided by this invention, the hardware architecture involved in this invention will be described first.

[0052] In one possible implementation, such as Figure 1 As shown, the hardware architecture involved in this invention may include: electronic device 101 and server 102.

[0053] For example, electronic device 101 can be any electronic product that can interact with a user through one or more means such as a keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as mobile phone, laptop, tablet computer, PDA, personal computer, wearable device, smart TV, PAD, etc.

[0054] It should be noted that, Figure 1 This is just one example; there can be many types of electronic devices, not limited to... Figure 1 The laptop in the middle.

[0055] For example, server 102 can be a single server, a server cluster consisting of multiple servers, or a cloud computing server center. Server 102 may include processors, memory, and network interfaces, etc.

[0056] For example, electronic device 101 can establish a connection and communicate with server 102 through a wireless communication network; for example, electronic device 101 can establish a connection and communicate with server 102 through a wired network.

[0057] Electronic device 101 acquires the data required for training and sends it to server 102. Server 102 uses the received data to train the target data processing model according to the data processing model training method provided by this invention. Electronic device 101 can also acquire data to be processed and send it to server 102. Server 102 processes the data to be processed based on the trained target data processing model and sends the processing result back to electronic device 101.

[0058] In another possible implementation, the hardware architecture involved in this invention may include an electronic device. The electronic device is one with strong data processing capabilities.

[0059] The electronic device can acquire the data required for training, and use the acquired data to train the target data processing model according to the data processing model training method provided in this invention. The electronic device can also acquire the data to be processed, and process the data to be processed based on the trained target data processing model to obtain the processing result.

[0060] Those skilled in the art should understand that the above-described electronic devices and servers are merely examples, and other existing or future electronic devices or servers that are applicable to this invention should also be included within the scope of protection of this invention, and are hereby incorporated by reference.

[0061] The solution provided by the present invention will be described below through the following embodiments.

[0062] Please see Figure 2 The diagram illustrates a flowchart of a data processing model training method provided in an embodiment of the present invention. The method may include:

[0063] Step S201: Use the training data in the first training dataset to train the initial data processing model and obtain the first data processing model.

[0064] From the perspective of model structure, the data processing model in this embodiment can be a deep learning-based data processing model.

[0065] From the perspective of the application domain of the model, the data processing model in this embodiment can be a model in the speech domain, such as a speech recognition model, a speech synthesis model, a speech sentiment analysis model, a speaker recognition model, a speech command recognition model, a speech translation model, a voiceprint recognition model, a speech denoising and enhancement model, a speech segmentation model, and an audio track separation model, etc. The data processing model in this embodiment can also be a model in the image domain, such as an image classification model, an object detection model, a semantic segmentation model, an instance segmentation model, an image generation model, an image super-resolution model, a face recognition model, an image registration model, a visual question answering model, etc. The data processing model in this embodiment can also be a model in the natural language domain, such as a part-of-speech tagging model, a named entity recognition model, a syntax analysis model, a semantic role tagging model, a text classification model, a text clustering model, a text generation model, a question answering model, an information extraction model, a semantic similarity calculation model, a text summarization model, a sentiment analysis model, a language model, a text proofreading and error correction model, a multilingual processing model, etc.

[0066] For example, if the data processing model is a speech recognition model, then the speech recognition model can be an end-to-end speech recognition model (such as a Conformer-based speech recognition model or an RNN-T-based speech recognition model), or a hybrid speech recognition model. Here, a hybrid speech recognition model refers to a speech recognition model with a hybrid structure based on deep neural networks and hidden Markov models (i.e., DNN+HMM). It should be noted that the above-mentioned end-to-end speech recognition model and hybrid speech recognition model are merely examples; any deep learning-based speech recognition model is applicable to this invention.

[0067] In this embodiment, the first training dataset includes multiple training data points from a general domain. Each training data point in the first training dataset is a labeled training data point from a general domain. When training the initial data processing model, the training objective is to make the processing result obtained by the data processing model from processing the training data in the first training dataset approximate the information labeled in the training data.

[0068] Specifically, the process of training the initial data processing model using the training data in the first training dataset may include: inputting the training data in the first training dataset into the initial data processing model for processing to obtain the processing result of the training data; determining the loss function (such as the cross-entropy loss function) based on the processing result of the training data and the information of the training data annotation; and updating the parameters of the initial data processing model according to the loss function.

[0069] For example, if the data processing model is a speech recognition model, then each piece of training data in the first training dataset is a training speech with text annotation in a general domain. When training the initial speech recognition model, the training objective is to make the recognition result obtained by the speech recognition model on the training speech in the first training dataset approximate the text annotated by the training speech.

[0070] Step S202: Perform a copy operation on the modules in the first data processing model to obtain a copied module, and connect the copied module in parallel with the original module to obtain the second data processing model.

[0071] The first data processing model typically includes multiple modules. This invention performs a copying operation on some or all of the multiple modules included in the first data processing model. The module obtained by copying is called a "copying module". After obtaining the copying module, the copying module can be connected in parallel with the original module. That is, after obtaining the copying module, the copying module is added on the basis of the first data processing model.

[0072] For example, such as Figure 3 As shown, the first data processing model sequentially includes module A, module B, module C, and module D. Module A and module C can be copied to obtain copied module A′ and copied module C′. After obtaining copied module A′ and copied module C′, copied module A′ can be connected in parallel with module A in the first data processing model, and copied module C′ can be connected in parallel with module C in the first data processing model, thereby obtaining a second data processing model including module A, module B, module C, module D, copied module A′, and copied module C′.

[0073] For example, the first data processing model sequentially includes module A, module B, module C, and module D. Modules B and C can be copied to obtain copied module B' and copied module C'. After obtaining copied module B' and copied module C', copied module B' and copied module C' can be treated as a whole, and module B and module C can be treated as a whole, and these two parts can be connected in parallel, such as... Figure 4 As shown, a second data processing model is obtained, which includes module A, module B, module C, module D, copy module B′, and copy module C′.

[0074] It should be noted that modules can be randomly selected from the multiple modules included in the first data processing model for copying, or modules can be selected from the multiple modules included in the first data processing model for copying according to a set selection strategy (such as selecting a specific module, selecting a module with a specific function, etc.).

[0075] Step S203: Use the training data in the second training dataset to train the second data processing model to obtain the target data processing model.

[0076] The second training dataset includes at least multiple training data points for a specific domain, each of which is labeled training data for that specific domain.

[0077] In one possible implementation, the second training dataset may include multiple domain-specific training data sets; that is, multiple domain-specific training data sets are acquired and combined to form the second training dataset. In another possible implementation, the second training dataset may include multiple domain-specific training data sets and multiple general-domain training data sets; that is, multiple domain-specific training data sets are acquired, and multiple general-domain training data sets are acquired, then the domain-specific and general-domain training data sets are mixed, and the mixed training data sets form the second training dataset. Optionally, when acquiring the multiple general-domain training data sets, a portion of the training data may be randomly acquired from the first training dataset according to a preset ratio.

[0078] When training the second data processing model using training data from the second training dataset, only the parameters of the replication module in the second data processing model are updated, while the parameters of other modules remain unchanged.

[0079] It should be noted that when training the second data processing model, only the parameters of the replication module are updated, and the parameters of other modules are not updated. This allows the second data processing model to learn knowledge in a specific domain without forgetting the knowledge learned in the general domain. Thus, it can improve the processing effect of the second data processing model on data in a specific domain, while ensuring that the processing effect on data in the general domain does not decline.

[0080] When training the second data processing model using training data from the second training dataset, the training objective is to make the processing result obtained by the second data processing model from processing the training data in the second training dataset approximate the information labeled in the training data.

[0081] Specifically, the process of training the second data processing model using training data from the second training dataset can include: inputting the training data from the second training dataset into the second data processing model for processing to obtain the processing results; determining a loss function (such as the cross-entropy loss function) based on the processing results and the labeled information of the training data; and updating the parameters of the second data processing model according to the loss function. It should be noted that when training the second data processing model, N training audio samples can be randomly selected from the second training dataset each time and input into the second data processing model; alternatively, only one training data sample can be selected from the second training dataset each time and input into the second data processing model.

[0082] The data processing model training method provided in this invention first trains a first data processing model that has good processing performance on general domain data. Then, a replication module is introduced based on the first data processing model to obtain a second data processing model. Since the replication module is obtained by copying modules in the first data processing model, the structure and parameters of the replication module are the same as those of the corresponding original modules in the first data processing model. When training the second data processing model, the parameters of the introduced replication module are not randomly initialized, but are obtained through the first stage of training (i.e., training using general domain training data). Furthermore, the replication module and the original modules... Because of the identical structure, the second data processing model converges easily (i.e., converges quickly) during training. Furthermore, during training, only the parameters of the copy module are updated, while the parameters of other modules are not updated. This parameter processing strategy allows the second data processing model to learn knowledge from specific domains without forgetting knowledge learned in the general domain. Thus, through training, the processing performance of the second data processing model for data from specific domains can be improved, while ensuring that the processing performance for data from the general domain does not decline. In other words, the target data processing model trained in the end has good processing performance for both data from specific domains and data from the general domain.

[0083] In another embodiment of the present invention, the specific implementation process of "step S202: performing a copy operation on the module in the first data processing model to obtain a copy module, and connecting the copy module in parallel with the original module to obtain a second data processing model" in the above embodiment will be described.

[0084] There are multiple ways to implement step S202. This embodiment provides the following three implementation methods:

[0085] The first implementation method is as follows: Select a specified module from the modules contained in the first data processing model; copy the selected specified module to obtain a copied module; connect the copied module in parallel with the original module to obtain the second data processing model.

[0086] The first data processing model typically contains several linear layers, each of which includes at least a linear transformation module, which is used to perform linear transformations on the input data.

[0087] For example, the module selection strategy is to select a linear transformation module from the modules included in the first data processing model. Then, the selected linear transformation module is copied to obtain several copied linear transformation modules. Then, the copied linear transformation modules can be connected in parallel with the corresponding linear transformation modules in the first data processing model to obtain the second data processing model.

[0088] Please see Figure 5 An example of a first data processing model is shown, which is a speech recognition model with a Conformer architecture. The speech recognition model with a Conformer architecture includes a data augmentation module.

[0089] The system consists of a SpecAug module, a Convolution Subsampling module, a Linear processing module, a Dropout module, and N Conformer blocks. Each Conformer block includes a first Feed Forward module and a multi-head self-attention module.

[0090] The system consists of a Multi-Head Self Attention module, a Convolution module, a Second Feedforward module, and a Layernorm module. The linear processing module is a linear layer, and the convolutional downsampling module contains a linear layer. Each Conformer block contains a linear layer in its first feedforward module, convolution module, and second feedforward module. Each linear layer contains at least a linear transformation module. Figure 5 The speech recognition model in the document contains several linear transformation modules, which can perform... Figure 5 The speech recognition model contains some or all of the linear transformation modules, which are copied to obtain several copied linear transformation modules. Then, each copied linear transformation module is connected in parallel with the corresponding linear transformation module in the speech recognition model. Figure 5 The diagram illustrates the replication of a linear transformation module within a linear layer of a convolutional downsampling module, followed by parallel connection of the replicated linear transformation module with the corresponding linear transformation module in the convolutional downsampling module. It should be noted that the replication and parallel connection of linear transformation modules within a linear layer of a convolutional downsampling module is merely an example and does not imply that replication and parallel connection are only possible for linear transformation modules within a linear layer of a convolutional downsampling module.

[0091] The second implementation method is as follows: Select a module with a specified function from the modules included in the first data processing model; copy the selected module with the specified function to obtain a copied module; connect the copied module in parallel with the original module to obtain the second data processing model.

[0092] For example, the first data processing model includes module A, module B, module C, and module D. The module selection strategy is to select modules with function F from the modules included in the first data processing model. Assuming that the function of module A and module C is F, module A and module C are copied to obtain copied module A′ and copied module C′. After obtaining copied module A′ and copied module C′, copied module A′ can be connected in parallel with module A in the first data processing model, and copied module C′ can be connected in parallel with module C in the first data processing model, thereby obtaining a second data processing model including module A, module B, module C, module D, copied module A′, and copied module C′.

[0093] The third implementation method is to randomly select a module from the modules contained in the first data processing model; copy the selected module to obtain a copied module; connect the copied module in parallel with the corresponding original module in the first data processing model to obtain the second data processing model.

[0094] Generally speaking, the first data processing model includes several large modules, some of which may contain several smaller modules, and some smaller modules may contain even smaller modules, for example, Figure 5 The Conformer-structured speech recognition model shown includes a convolutional downsampling module, which contains linear layers, and these linear layers include linear transformation modules. In this embodiment, the larger module is referred to as the first-level module, and the modules contained within the first-level module are referred to as second-level modules, and so on. That is, if the first data processing model includes N levels of modules, then the nth-level module is the module contained in the (n-1)th-level module (n ranges from 2 to N). First, it should be noted that if the first data processing model only includes first-level modules, then one or more modules are randomly selected from this level for replication and parallel connection. Second, it should be noted that which modules in the first data processing model are first-level modules, which are second-level modules, and so on, are determined according to the specific structure of the first data processing model.

[0095] In one possible implementation, several peer modules can be randomly selected from the first data processing model and copied to obtain several copied modules. Each copied module is then connected in parallel with the corresponding module in the first data processing model to obtain the second data processing model. For example, several first-level modules can be randomly selected from the first data processing model and copied to obtain several first-level copied modules. Each first-level copied module is then connected in parallel with the corresponding first-level module in the first data processing model to obtain the second data processing model.

[0096] For example, the first data processing model includes first-level module A, first-level module B, first-level module C, and first-level module D. First-level modules A and C can be copied to obtain first-level copied modules A′ and C′. After obtaining first-level copied modules A′ and C′, first-level copied module A′ can be connected in parallel with first-level module A in the first data processing model, and first-level copied module C′ can be connected in parallel with first-level module C in the first data processing model to obtain... Figure 6 The second data processing model is shown.

[0097] In another possible implementation, several modules at different levels can be randomly selected from the first data processing model and copied to obtain several copied modules. Each copied module is then connected in parallel with the corresponding module in the first data processing model to obtain the second data processing model.

[0098] For example, the first data processing model includes first-level module A, first-level module B, first-level module C, and first-level module D. First-level module A includes second-level modules a1 and a2, first-level module C includes second-level modules c1 and c2, and first-level module D includes second-level modules d1 and d2. Second-level module a1 can be copied to obtain a copied second-level module a1′, and first-level module C can be copied to obtain a copied first-level module C′. The copied second-level module a1′ is connected in parallel with second-level module a1, and the copied first-level module C′ is connected in parallel with first-level module C, resulting in... Figure 7 The second data processing model is shown.

[0099] Through any of the above implementation methods, a second data processing model incorporating a replication module can be obtained. After obtaining the second data processing model, the parameters of the replication module in the second data processing module can be fine-tuned using the training data in the second training dataset. The fine-tuned data processing model serves as the target data processing model, which exhibits good recognition performance for both general-domain and domain-specific data.

[0100] Assume that one of the replication modules in the second data processing model is M. copy Its corresponding original module is M ori ,like Figure 8 As shown, the copy module M copy The corresponding original module is M ori The input is the same, copy module M copy The output of the original module M ori The output is then merged and input into the next module.

[0101] Optionally, copy module M copy The output of the original module M oriThe outputs can be fused using a weighted summation method. Specifically, if the copy module M is used... copy The output is represented as y copy , the original module M ori The output is represented as y ori Copy module M copy The weight corresponding to the output is represented by α. copy Original module M ori The weight corresponding to the output is represented by α. ori Then, the copy module M can be copied in the manner shown in the following formula. copy The output y copy With the original module M ori The output y ori Fusion:

[0102] Y = α ori *y ori +α copy *y copy (1)

[0103] Where Y represents y copy With y ori The fusion result, weight α ori With weight α copy The sum of is 1.

[0104] That is, α ori +α copy =1.

[0105] Optional, weight α ori With weight α copy The specific value can be 1 / 2, that is, the value of the copy module M. copy The output y copy With the original module M ori The output y ori The fusion result is 1 / 2*(y ori +y copy Of course, this embodiment is not limited to this; the weight α ori With weight α copy The specific value can also be other, such as the weight α. ori The value is 2 / 5, and the weight α copy The value can be 3 / 5. For example, the weight α... ori The value is 1 / 3, and the weight α copy The value is 2 / 3, and the weight α ori With weight α copy The specific value can be determined based on the specific application scenario.

[0106] It should be noted that if the original module M ori Assuming the original module M is a linear transformation module, let's assume it's M.ori Based on the following linear transformation of y = x * W + b, then y ori and y copy It can be represented as:

[0107] y ori =x*W ori +b ori (2)

[0108] y copy =x*W copy +b copy (3)

[0109] Therefore, equation (1) above can be expressed as:

[0110] Y = α ori *(x*W ori +b ori )+α copy *(x*W copy +b copy (4)

[0111] Equation (4) above can be adjusted to:

[0112] Y = x * (α) ori *W ori+ α copy *W copy )+(α ori *b ori +α copy *b copy (5)

[0113] If α ori *W ori+ α copy *W copy Represented as W′, α ori *b ori +α copy *b copy Let b′ be the expression, then equation (5) becomes:

[0114] Y=x*W′+b′ (6)

[0115] It should be noted that after the training, W copy and b copy W will remain fixed. ori and b ori It remains fixed after the first phase of training; that is, W will remain fixed after the training ends. copy W ori b copy b ori Since these are all fixed values, α can be pre-calculated before reasoning.ori *W ori+ α copy *W copy and α ori *b ori +α copy *b copy That is, the only parameters for inference computation are W′ and b′. Therefore, if the copy module is obtained by copying the linear transformation module in the model, then introducing the copy module will not essentially introduce new parameters (the parameters before introducing the copy module are the weight parameters W). ori and bias parameter b ori The parameters after introducing the replication module are still the weight parameter W′ and the bias parameter b′. Since no new parameters are introduced during the inference phase, the computational cost of inference will not increase.

[0116] Additionally, it should be noted that, besides using the weighted summation method described above to apply the same method to the replication module M... copy The output y copy With the original module M ori The output y ori Besides fusion, other methods can be used, such as directly copying module M. copy The output y copy With the original module M ori The output y ori Summation. A fusion method using direct summation is adopted, which also avoids introducing new parameters during the inference phase.

[0117] This invention also provides a data processing method, which may include:

[0118] Step a1: Obtain the data to be processed in a specific or general domain.

[0119] Step a2: Input the data to be processed into the pre-trained target data processing model to process the data and obtain the data processing results.

[0120] The target data processing model is trained using the data processing model training method provided in the above embodiments.

[0121] Since the data processing model training method provided in the above embodiments can train a target data processing model that has a good recognition effect on both general and specific domain data, when the trained target data processing model is used to process the data to be processed, a relatively accurate data processing result can be obtained regardless of whether the data to be processed is general domain data or specific domain data.

[0122] This invention provides a data processing model training device. The data processing model training device provided in this invention is described below. The data processing model training device described below can be referred to in correspondence with the data processing model training method described above.

[0123] Please see Figure 9 The diagram shows a schematic of the structure of a data processing model training device provided in an embodiment of the present invention. The data processing model training device may include: a first training module 901, a model processing module 902, and a second training module 903.

[0124] The first training module 901 is used to train the initial data processing model using the training data in the first training dataset to obtain the first data processing model.

[0125] The first training dataset includes multiple training data from general domains.

[0126] The model processing module 902 is used to perform a copying operation on the modules in the first data processing model to obtain a copy module, and connect the copy module in parallel with the original module to obtain a second data processing model.

[0127] The second training module 903 is used to train the second data processing model using training data from the second training dataset to obtain the target data processing model.

[0128] The second training dataset includes at least multiple training data points from a specific domain. When training the second data processing model, only the parameters of the copying module are updated.

[0129] Optionally, when the model processing module 902 performs a copy operation on the modules in the first data processing model, it is specifically used for:

[0130] Select one or more specified modules from the modules included in the first data processing model, or select one or more modules with specified functions from the modules included in the first data processing model, or randomly select one or more modules from the modules included in the first data processing model; perform a copy operation on the selected modules.

[0131] The first data processing model includes several linear layers, and each linear layer includes at least a linear transformation module.

[0132] Optionally, when the model processing module 902 selects one or more specified modules from the modules included in the first data processing model, it is specifically used for:

[0133] Select one or more linear transformation modules from the modules included in the first data processing model.

[0134] Optionally, the first data processing model includes N levels of modules, wherein the nth level module is the module contained in the (n-1)th level module, and n ranges from 2 to N;

[0135] When the model processing module 902 randomly selects one or more modules from the modules included in the first data processing model, it is specifically used for:

[0136] Randomly select several modules of the same level from the N-level modules contained in the first data processing model;

[0137] Alternatively, several modules of different levels may be randomly selected from the N-level modules contained in the first speech recognition data processing model.

[0138] Optionally, the data processing model training apparatus provided in this embodiment of the invention may further include a training dataset acquisition module. The training dataset acquisition module is used to acquire a second training dataset.

[0139] When acquiring the second training dataset, the training dataset acquisition module is specifically used for:

[0140] Acquire multiple training data sets from a specific domain and multiple training data sets from a general domain;

[0141] The acquired training data from multiple specific domains are mixed with the acquired training data from multiple general domains, and the mixed training data forms the second training dataset.

[0142] In the second data processing model, the input of the copy module is the same as that of the original module, and the output of the copy module is merged with the output of the original module before being input into the next module.

[0143] The data processing model training apparatus provided in this embodiment of the invention first trains a first data processing model that has good processing effect on general domain data. Then, a replication module is introduced based on the first data processing model to obtain a second data processing model. Since the replication module is obtained by copying the modules in the first data processing model, the structure and parameters of the replication module are the same as the structure and parameters of the corresponding modules in the first data processing model. When training the second data processing model, since the parameters of the introduced replication module are not randomly initialized, but are obtained through the first stage of training (i.e., training using general domain training data), and the introduced replication module is identical to the original... Because the modules have the same structure, the model converges easily during training. Furthermore, when training the second data processing model, only the parameters of the copied modules are updated, while the parameters of other modules are not updated. This parameter processing strategy allows the second data processing model to learn knowledge from specific domains without forgetting knowledge learned in the general domain. Thus, through training, the processing performance of the second data processing model for data from specific domains can be improved, while ensuring that the processing performance for data from the general domain does not decline. In other words, the target data processing model trained in the end has good processing performance for both data from specific domains and data from the general domain.

[0144] This invention provides a processing device; please refer to [link / reference]. Figure 10 The diagram shows the structure of the processing device, which may include: a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004.

[0145] In this embodiment of the invention, the number of processor 1001, communication interface 1002, memory 1003 and communication bus 1004 is at least one, and processor 1001, communication interface 1002 and memory 1003 communicate with each other through communication bus 1004.

[0146] The processor 1001 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0147] The memory 1003 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0148] The memory stores a program, which the processor can call. The program is used for:

[0149] The initial data processing model is trained using the training data in the first training dataset to obtain the first data processing model. The first training dataset includes multiple training data from a general domain.

[0150] The modules in the first data processing model are copied to obtain a copied module, and the copied module is connected in parallel with the original module to obtain a second data processing model;

[0151] The second data processing model is trained using training data from the second training dataset to obtain the target data processing model. The second training dataset includes at least multiple training data from a specific domain. When training the second data processing model, only the parameters of the copying module are updated.

[0152] Optionally, the refined and extended functions of the program can be found in the description above.

[0153] This invention also provides a computer-readable storage medium that stores a program suitable for execution by a processor, the program being used for:

[0154] The initial data processing model is trained using the training data in the first training dataset to obtain the first data processing model. The first training dataset includes multiple training data from a general domain.

[0155] The modules in the first data processing model are copied to obtain a copied module, and the copied module is connected in parallel with the original module to obtain a second data processing model;

[0156] The second data processing model is trained using training data from the second training dataset to obtain the target data processing model. The second training dataset includes at least multiple training data from a specific domain. When training the second data processing model, only the parameters of the copying module are updated.

[0157] Optionally, the refined and extended functions of the program can be found in the description above.

[0158] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0159] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0160] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing model training method, characterized in that, include: The initial data processing model is trained using training data from the first training dataset to obtain the first data processing model. The first training dataset includes multiple training data from a general domain. The initial data processing model is an initial speech recognition model. Each training data from the general domain is a training speech with text annotations in the general domain. When training the initial speech recognition model, the training objective is to make the recognition result obtained by the initial speech recognition model from the training speech in the first training dataset approximate the corresponding annotated text, thus obtaining the first speech recognition model. The modules in the first data processing model are copied to obtain a copied module, and the copied module is connected in parallel with the original module to obtain a second data processing model; The second data processing model is trained using training data from the second training dataset to obtain the target data processing model. The second training dataset includes at least multiple training data from specific domains. When training the second data processing model, only the parameters of the copying module are updated. Each training data from a specific domain is a training speech with text annotations in that specific domain. The second data processing model is a second speech recognition model. When training the second speech recognition model, the training objective is to make the recognition result obtained by the second speech recognition model from the training speech in the second training dataset approximate the corresponding annotated text.

2. The data processing model training method according to claim 1, characterized in that, The copying operation of the modules in the first data processing model includes: One or more specified modules may be selected from the modules included in the first data processing model; or, one or more modules with specified functions may be selected from the modules included in the first data processing model; or, one or more modules may be randomly selected from the modules included in the first data processing model. Perform a copy operation on the selected module.

3. The data processing model training method according to claim 2, characterized in that, The first data processing model includes several linear layers, and each linear layer includes at least a linear transformation module; The step of selecting one or more specified modules from the modules included in the first data processing model includes: Select one or more linear transformation modules from the modules included in the first data processing model.

4. The data processing model training method according to claim 2, characterized in that, The first data processing model includes N levels of modules, where the nth level module is the module contained in the (n-1)th level module, and n ranges from 2 to N; The step of randomly selecting one or more modules from the modules included in the first data processing model includes: Randomly select several modules of the same level from the N-level modules contained in the first data processing model; Alternatively, several modules of different levels may be randomly selected from the N-level modules contained in the first data processing model.

5. The data processing model training method according to claim 1, characterized in that, The process of obtaining the second training dataset includes: Acquire multiple training data sets from a specific domain and multiple training data sets from a general domain; The acquired training data from multiple specific domains are mixed with the acquired training data from multiple general domains, and the mixed training data forms the second training dataset.

6. The data processing model training method according to claim 1, characterized in that, The copy module in the second data processing model has the same input as the original module; In the second data processing model, the output of the copy module is merged with the output of the original module and then input into the next module.

7. A data processing method, characterized in that, include: Acquire data to be processed in a specific or general domain; The data to be processed is input into a pre-trained target data processing model for processing to obtain the data processing result. The target data processing model is trained using the data processing model training method described in any one of claims 1 to 6. The data to be processed is speech data, and the target data processing model is a target speech recognition model.

8. A data processing model training device, characterized in that, include: The first training module, the model processing module, and the second training module; The first training module is used to train an initial data processing model using training data in a first training dataset to obtain a first data processing model. The first training dataset includes multiple training data in a general domain. The initial data processing model is an initial speech recognition model. Each training data in the general domain is a training speech in the general domain with annotated text. When training the initial speech recognition model, the training objective is to make the recognition result obtained by the initial speech recognition model from the training speech in the first training dataset approximate the corresponding annotated text, thereby obtaining the first speech recognition model. The model processing module is used to perform a copying operation on the module in the first data processing model to obtain a copy module, and connect the copy module in parallel with the original module to obtain a second data processing model. The second training module is used to train the second data processing model using training data from the second training dataset to obtain the target data processing model. The second training dataset includes at least multiple pieces of training data in a specific domain. When training the second data processing model, only the parameters of the copying module are updated. Each piece of training data in the specific domain is training speech with text annotation in that specific domain. The second data processing model is a second speech recognition model. When training the second speech recognition model, the training objective is to make the recognition result obtained by the second speech recognition model from the training speech in the second training dataset approximate the corresponding annotated text.

9. A data processing device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the data processing model training method as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the data processing model training method as described in any one of claims 1 to 6.

Images