Data processing method, device, electronic device and medium

Abstract

The embodiment of the present application discloses a data processing method, device, electronic device and medium, which are applied to the field of machine learning technology. The method includes: obtaining sample text data and corresponding translation text data, inputting the sample text data into a first translation model to obtain a first translation result corresponding to each text word, and inputting the sample text data into a second translation model to obtain a second translation result corresponding to each text word, obtaining important indicator parameters of the translation words corresponding to each text word, and determining the target text words from N text words according to the important indicator parameters, determining the model translation loss function according to the first translation result, the corresponding second translation result and the corresponding translation word corresponding to the target text word, and correcting the first translation model according to the model translation loss function to obtain the target translation model. By adopting the embodiment of the present application, the accuracy of the target translation model obtained can be improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a data processing method, device, electronic device and medium. Background Art

[0002] With the continuous development of computer technology, artificial intelligence (AI) technology has become more and more mature, among which artificial intelligence technology involves related technologies of machine learning.

[0003] In the prior art, the model can be trained by machine learning related technologies, and the trained model can be applied to predict data (such as text translation prediction). By inputting sample data into the model to be trained, the model can predict the prediction results for the input sample data, and then the model parameters of the model to be trained can be corrected by the difference between the prediction results and the actual data attributes of the sample data, and finally the trained target model is obtained. As for the sample data input into the model, since the content and type of the sample data are various, the sample data usually contains data that will have a negative impact on the model training, so directly using the sample data to correct the model parameters will cause the trained model to be inaccurate. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, device, electronic device and medium, which can improve the accuracy of the acquired target translation model.

[0005] On the one hand, an embodiment of the present application provides a data processing method, the method comprising:

[0006] Acquire sample text data and translated text data of the sample text data; the sample text data includes N text words, N is a positive integer; the translated text data includes a translated word corresponding to each text word in the N text words; the sample text data and the translated text data have different text types;

[0007] Inputting the sample text data into a first translation model, generating a first translation result corresponding to each text word in the first translation model, and inputting the sample text data into a second translation model, generating a second translation result corresponding to each text word in the second translation model;

[0008] Obtaining important index parameters of translation words corresponding to each of the text words, and determining a target text word from the N text words according to the important index parameters of the translation words corresponding to each of the text words;

[0009] A model translation loss function is determined based on a first translation result corresponding to the target text word, a second translation result corresponding to the target text word, and a translation word corresponding to the target text word in the translation text data, and model parameters of the first translation model are corrected based on the model translation loss function to obtain a target translation model.

[0010] On the one hand, an embodiment of the present application provides a data processing device, the device comprising:

[0011] an acquisition module, configured to acquire sample text data and translation text data of the sample text data; the sample text data comprises N text words, where N is a positive integer; the translation text data comprises a translation word corresponding to each text word in the N text words; the sample text data and the translation text data have different text types;

[0012] A processing module, configured to input the sample text data into a first translation model, generate a first translation result corresponding to each text word in the first translation model, and input the sample text data into a second translation model, generate a second translation result corresponding to each text word in the second translation model;

[0013] A determination module, used to obtain important index parameters of translation words corresponding to each of the text words, and determine a target text word from the N text words according to the important index parameters of the translation words corresponding to each of the text words;

[0014] The determination module is also used to determine a model translation loss function based on a first translation result corresponding to the target text word, a second translation result corresponding to the target text word, and a translation word corresponding to the target text word in the translation text data, and to correct the model parameters of the first translation model based on the model translation loss function to obtain a target translation model.

[0015] On the one hand, an embodiment of the present application provides an electronic device, which includes a processor and a memory, where the processor and the memory are interconnected, wherein the memory is used to store computer program instructions, and the processor is configured to execute the computer program instructions to implement some or all of the steps in the above method.

[0016] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are executed by a processor, they are used to execute some or all of the steps in the above method.

[0017] Accordingly, according to one aspect of the present application, a computer program product or a computer program is provided, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method provided above.

[0018] In the embodiment of the present application, sample text data and corresponding translation text data can be obtained, the sample text data can be input into the first translation model, the first translation result corresponding to each text word is generated in the first translation model, and the sample text data can be input into the second translation model, the second translation result corresponding to each text word is generated in the second translation model, and the important index parameters of the translation words corresponding to each text word are obtained. According to the important index parameters of the translation words corresponding to each text word, the target text words are determined from N text words, and the model translation loss function is determined according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word, and the translation words corresponding to the target text word in the translation text data, and the model parameters of the first translation model are corrected according to the model translation loss function to obtain the target translation model. By implementing the above scheme, the sample text data can be processed, and the target text words can be selected from the text words contained in the sample text data for knowledge distillation, which effectively prevents overfitting, improves the efficiency of knowledge distillation, and improves the accuracy of the target translation model obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0020] Figure 1 A schematic diagram of an application architecture provided for an embodiment of the present application;

[0021] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a scenario for determining a target translation model provided in an embodiment of the present application;

[0023] Figure 4 A flowchart of a data processing method provided in an embodiment of the present application;

[0024] Figure 5A schematic diagram of a scenario for determining a target queue provided in an embodiment of the present application;

[0025] Figure 6a A schematic diagram of a model training process provided in an embodiment of the present application;

[0026] Figure 6b A schematic diagram of a model training process provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0030] The embodiment of the present application provides a data processing method, which can enable the trained target translation model to translate text more accurately.

[0031] The data processing method proposed in the embodiment of the present application is implemented in an electronic device, which can be a server or a terminal. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application is not limited here.

[0032] The embodiments of the present application relate to the field of machine learning technology. Machine Learning (ML) is a multi-disciplinary cross-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all fields of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching.

[0033] See also Figure 1 , Figure 1 The application architecture proposed according to the technical solution of the present application can implement the data processing method of the embodiment of the present application. Specifically, Figure 1 The electronic device includes an electronic device, and a first translation model, a second translation model, and related data for training are deployed in the electronic device. The electronic device can input sample text data in the related data into the first translation model, and the first translation model outputs a first translation result, and input sample text data in the related data into the second translation model, and the second translation model outputs a second translation result. The electronic device can implement the training of the first translation model according to the first translation result, the second translation result, and the translated text data in the related data, etc. The technical solution of the present application is executed. Among them, the second translation model is a pre-trained model, the related data used to train the second translation model can be the same as or different from the related data used to train the first translation model, the model translation performance of the second translation model indicates that the model can achieve a higher accuracy of text translation, and the first translation model is trained by the second translation result generated by the second translation model, so that the model translation performance of the second translation model can be transferred to the first translation model, thereby achieving that the target translation model obtained by training has a model translation performance similar to that of the second translation model, and in the training process of the first translation model, the model parameters of the first translation model are modified, and the model parameters of the second translation model can remain unchanged.

[0034] Optionally, in some embodiments, the electronic device can process the training data through the technical solution of the present application, and select appropriate sample data from the training data for knowledge distillation to achieve selective knowledge distillation. Among them, the knowledge distillation (KD) involved in the present application is to transfer the knowledge generated by a model according to the input sample data to another model for learning. The knowledge distillation includes a teacher model (Teacher Model) and a student model (Student Model). The teacher model is a model that transfers knowledge in knowledge distillation, and the student model is a model that learns to accept knowledge in knowledge distillation. It can be understood that the student model in knowledge distillation is the above-mentioned Figure 1 The first translation model in the knowledge distillation is the teacher model mentioned above. Figure 1 The second translation model in .

[0035] Optionally, in some embodiments, the electronic device can execute the data processing method according to actual business needs. For example, the technical solution of the present application can be applied to language translation model training scenarios (such as Chinese-English translation, English-German translation, etc.), and the electronic device executes the data processing method when acquiring sample text data (such as Chinese training sample sentences) and translated text data of the sample text data (such as English translation sentences corresponding to Chinese training sample sentences), determines the target text words from the N text words included in the sample text data, and performs knowledge distillation based on the target text words, and determines the model translation loss function according to the additional supervised learning signals obtained after knowledge distillation, and corrects the model parameters of the first translation model according to the model translation loss function to obtain the target translation model, thereby realizing the training of the language translation model, and achieving better model training effect and knowledge distillation effect, as well as achieving better translation performance and accuracy. For example, the technical solution of the present application can also be applied to model compression scenarios. By executing the data processing method of the present application, model compression can be achieved and the model size can be reduced. Therefore, in the actual application of the target translation model, it can be better deployed, reducing the overhead of computing and storage resources, and reducing application costs, as well as improving translation efficiency, giving users a better user experience.

[0036] Optionally, the data involved in this application, such as important indicator parameters of the translation words corresponding to each text word included in the sample text data, can be stored in a database, or can be stored in a blockchain, such as through blockchain distributed storage, which is not limited in this application.

[0037] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0038] The solution provided in the embodiments of the present application involves technologies such as machine learning of artificial intelligence, which is specifically described by the following embodiments:

[0039] Based on the above description, the present application embodiment proposes a data processing method, which can be executed by the electronic device mentioned above. Figure 2 As shown, the process of the data processing method in the embodiment of the present application may include:

[0040] S201: Acquire sample text data and translated text data of the sample text data.

[0041] Among them, there can be one or more sample text data. When each sample text data is used to train the first translation model, the process and principle are the same, and the first translation model can be iteratively trained several times through the one or more sample text data to obtain a final trained model (such as the target translation model described below). Therefore, the following is an example of the process of training the first translation model with one sample text data. In addition, since a sample text data is composed of one or more training sample sentences, the N text words included in the sample text data can be obtained based on the training sample sentences that constitute a sample text data, where N is a positive integer and the specific value of N is determined according to the actual application scenario.

[0042] Among them, the translated text data may include a translated word corresponding to each of the N text words; the sample text data and the translated text data have different text types. It can be understood that the embodiments of the present application can be applied to any language translation model training process, such as Chinese-English translation, English-German translation, etc. If in the Chinese-English translation scenario, the text type of the sample text data is Chinese, and the text type of the corresponding translated text data is English, the sample text data can be one or more Chinese training sample sentences, and a Chinese training sample sentence is used as an example for explanation here. Correspondingly, the corresponding translated text data is the English translation sentence corresponding to the Chinese training sample sentence.

[0043] In a possible implementation, the sample text data (also called the source-side sentence) is segmented to obtain N text words, and the translation word corresponding to each text word is determined in the corresponding translation text data (also called the correct target-side sentence). For example, the sample text data is "I'm very happy", including three text words "I", "very", and "happy", and the corresponding translation text data is "I'm very happy", and the translation word corresponding to the text word "I" is "I'm", the translation word corresponding to the text word "very" is "very", and the translation word corresponding to the text word "happy" is "happy".

[0044] S202: Input the sample text data into a first translation model, generate a first translation result corresponding to each text word in the first translation model, and input the sample text data into a second translation model, generate a second translation result corresponding to each text word in the second translation model.

[0045] In one possible implementation, the first translation result includes a predicted probability of each text word corresponding to each word in a translation vocabulary. The first translation result corresponding to each text word generated in the first translation model may be specifically generated by predefining a translation vocabulary, the translation vocabulary including a plurality of words, and the translation vocabulary is used to translate text data of type A into text data of type B, type B being a text type to which the translation vocabulary belongs, the text type of the translation vocabulary is the same as the translation text data, and for the translation text data corresponding to the sample text data, the translation text data corresponding to the sample text data is obtained from the words in the translation vocabulary, so the translation words in the translation text data belong to the translation vocabulary, and the first translation model generates a predicted probability of each text word for each word in the translation vocabulary based on the translation vocabulary, that is, the first translation model predicts the probability that the predicted translation word corresponding to the text word is each word in the translation vocabulary, and determines the predicted probability of each text word generated by the first translation model for each word in the translation vocabulary as the first translation result. The specific method of inputting the sample text data into the second translation model and generating the second translation results corresponding to each text word in the second translation model can be the same as the specific method of generating the first translation result, which will not be described in detail in the embodiment of the present application.

[0046] Exemplarily, the sample text data contains a text word "XX", "XX" can be any word, and the words in the translation vocabulary are "A", "B", "C", "D", and "E". The text word is input into the first translation model, and the first translation model generates the probability that the predicted translation word corresponding to "XX" is each word in the translation vocabulary. For example, the predicted translation word corresponding to the text word "XX" obtained by the first translation model is the word "A" in the translation vocabulary with a predicted probability of 20%, the word "B" with a predicted probability of 10%, the word "C" with a predicted probability of 40%, the word "D" with a predicted probability of 5%, and the word "E" with a predicted probability of 25%, and the above predicted probabilities are used as the first translation result of the text word "XX".

[0047] S203: Obtain important indicator parameters of the translation words corresponding to each text word, and determine the target text word from the N text words according to the important indicator parameters of the translation words corresponding to each text word.

[0048] Among them, the important index parameter of the translation word can be obtained by calculating the important evaluation index value of the translation word. The important evaluation index value is an index to measure the difficulty of a text word. The larger the important evaluation index value of a text word is, the more difficult and important the text word is for the training of the first translation model. By training the model according to the target text words determined by the important evaluation index value, the benefit of the trained model can be effectively improved, and the trained model can also have higher translation accuracy and translation efficiency, and the trained model can learn more effective features. Optionally, the important evaluation index value can be the cross entropy value (Cross Entropy, CE) of the translation word corresponding to each text word.

[0049] In one possible implementation, N text words include the ith text word, i is a positive integer less than or equal to N, and the first translation result of the ith text word includes the predicted probability of the ith text word for each word in the translation vocabulary, and the specific method for obtaining the important indicator parameters of the translation word corresponding to the ith text word may be to determine the predicted probability of the translation word corresponding to the ith text word in the first translation result of the ith text word as the target predicted probability, and generate the important indicator parameters of the translation word corresponding to the ith text word according to the target predicted probability. Among them, the specific method for determining the target predicted probability may be to find the predicted probability of the corresponding translation word in the first translation result according to the translation word corresponding to the ith text word, and determine the predicted probability of the corresponding translation word as the target predicted probability. The specific method for generating the important indicator parameters of the translation word corresponding to the ith text word according to the target predicted probability may be to determine the cross entropy value of the translation word corresponding to the ith text word according to the cross entropy loss function, and determine the cross entropy value as the important indicator parameter, wherein the cross entropy loss function is:

[0050]

[0051] Where |V| is the size of the translation vocabulary, 1 is the indicator function, p(·|·) is the predicted probability given the first translation model, and θ S is the model parameter of the first translation model, y i is the predicted translation word corresponding to the i-th text word, is the correct translation word corresponding to the i-th text word, and x is the sample text data.

[0052] Exemplarily, the sample text data contains a text word "XX", and the words in the translation vocabulary are "A", "B", "C", "D", and "E". The text word is input into the first translation model, and the first translation result generated is that the predicted translation word corresponding to the text word "XX" is the word "A" in the translation vocabulary with a predicted probability of 20%, the word "B" with a predicted probability of 10%, the word "C" with a predicted probability of 40%, the word "D" with a predicted probability of 5%, and the word "E" with a predicted probability of 25%. If the correct translation word corresponding to the text word "XX" is "C", the predicted probability of 40% of the word "C" is determined as the target prediction probability, and substituted into the cross-entropy loss function to obtain the cross-entropy value of the translation word corresponding to the text word "XX".

[0053] In a possible implementation, according to the important index parameters of the translation words corresponding to each text word, the specific method of determining the target text words from N text words can be to sort the N text words according to the order of the important index parameters of the translation words corresponding to each text word from small to large, obtain the sorted N text words, obtain the sample selection quantity K, and determine the first K text words in the sorted N text words as the target text words. Wherein, K is a non-negative integer less than or equal to N.

[0054] Optionally, the sample selection quantity K can be determined by the number of text words whose important index parameters of corresponding translation words are greater than a preset threshold. For example, the sample text data contains text words "A1", "B1", "C1", "D1", and "E1", wherein the important index parameter of the translation sentence corresponding to the text word "A1" is 0.7, the important index parameter of the translation sentence corresponding to the text word "B1" is 0.9, the important index parameter of the translation sentence corresponding to the text word "C1" is 0.4, the important index parameter of the translation sentence corresponding to the text word "D1" is 0.1, and the important index parameter of the translation sentence corresponding to the text word "E1" is 0.55. The text words are sorted in descending order according to the important index parameters, i.e., "B1", "A1", "E1", "C1", and "D1". If the threshold is set to 0.6, the sample selection quantity K is 2, and the determined target text words are "A1" and "B1".

[0055] Optionally, the sample selection quantity K can also be determined based on the number of text words in the top r% after selection and sorting. For example, the sample text data contains text words "A2", "B2", "C2", "D2", and "E2", among which the important index parameter of the translation sentence corresponding to the text word "A2" is 0.7, the important index parameter of the translation sentence corresponding to the text word "B2" is 0.9, the important index parameter of the translation sentence corresponding to the text word "C2" is 0.4, the important index parameter of the translation sentence corresponding to the text word "D2" is 0.1, and the important index parameter of the translation sentence corresponding to the text word "E2" is 0.55. The samples are sorted in descending order according to the important index parameters, i.e., "B2", "A2", "E2", "C2", and "D2". If r% is 20%, the sample selection quantity K is 1, and the determined target text word is "B2".

[0056] Optionally, the sample selection quantity K can also be determined by the number of text words whose important index parameters of corresponding translation words are greater than a preset threshold and are in the top r% after sorting. For example, the sample text data contains text words "A3", "B3", "C3", "D3", and "E3", among which the important index parameter of the translation sentence corresponding to the text word "A3" is 0.7, the important index parameter of the translation sentence corresponding to the text word "B3" is 0.9, the important index parameter of the translation sentence corresponding to the text word "C3" is 0.4, the important index parameter of the translation sentence corresponding to the text word "D3" is 0.1, and the important index parameter of the translation sentence corresponding to the text word "E3" is 0.55. The text words are sorted in descending order according to the important index parameters, i.e., "B3", "A3", "E3", "C3", and "D3". If r% is set to 40% and the threshold is set to 0.8, the sample selection quantity K is 1, and the determined target text word is "B3".

[0057] S204, determining a model translation loss function according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word, and the translation word corresponding to the target text word in the translation text data, and correcting the model parameters of the first translation model according to the model translation loss function to obtain a target translation model.

[0058] In one possible implementation, during the knowledge distillation process, the first translation model obtains additional supervised learning information by learning the second translation result output by the second translation model. For word-level knowledge distillation, the cross entropy loss function is The one-hot label in is replaced with the second translation result output by the second translation model, and the cross entropy loss function is obtained. :

[0059]

[0060] Among them, q(y i =k|y ＜i ,x;θ T ) is the second translation result output by the second translation model, θ T and θ S are the model parameters of the second translation model and the first translation model respectively.

[0061] Therefore, according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word, and the translation word corresponding to the target text word in the translation text data, the model translation loss function is obtained as follows: Among them, α can be regarded as a weight, which is used to balance the values ​​of the two cross entropy loss functions.

[0062] In one possible implementation, the model parameters of the first translation model are modified according to the model translation loss function by optimizing the model translation loss function during the model training process. Implementation, that is, minimizing the translation loss function Then the target translation model is obtained. It can be understood that here only one model training is used as an example. During the training process, the above training method can be performed on the first translation model multiple times. In multiple model training processes, the same or different sample text data can be used for training. The number of model training times can be determined by the technicians according to the actual training situation. It can be that the first translation model is trained multiple times until the model converges, that is, the loss function When minimization is achieved, the model training is completed, thereby obtaining the final target translation model. In the process of multiple model training, whether the same sample text data or different sample text data is used, the process and principle of any model training are the same, and the above steps can be performed.

[0063] For example, see Figure 3 , Figure 3A scenario schematic diagram for determining a target translation model is provided, wherein sample text data includes N text words (word 1, word 2, ..., word n), the sample text data is input into a first translation model to obtain a first translation result corresponding to each text word, and the sample text data is input into a second translation model to obtain a second translation result corresponding to each text word, and important index parameters of the translation words corresponding to each text word are obtained, and then according to the important index parameters of the translation words corresponding to each text word, the target text word is determined from the N text words, the first translation result and the second translation result corresponding to the target text word are obtained, and based on the first translation result corresponding to the target text word, the second translation result corresponding to the target text word and the translation words corresponding to the target text word in the translation text data, the model parameters of the first translation model are modified to obtain the target translation model.

[0064] In the embodiment of the present application, the electronic device obtains sample text data and translated text data of the sample text data, inputs the sample text data into the first translation model, generates the first translation result corresponding to each text word in the first translation model, and inputs the sample text data into the second translation model, generates the second translation result corresponding to each text word in the second translation model, obtains the important index parameters of the translated words corresponding to each text word, and determines the target text words from N text words according to the important index parameters of the translated words corresponding to each text word, determines the model translation loss function according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word and the translated words corresponding to the target text word in the translated text data, and corrects the model parameters of the first translation model according to the model translation loss function to obtain the target translation model. By implementing the above scheme, the sample text data can be processed, and suitable target text words can be selected from the text words contained in the sample text data for knowledge distillation, which can avoid the negative impact of some text words on the overall distillation effect when performing knowledge distillation to a certain extent, and effectively prevent overfitting, thereby improving the efficiency and effect of knowledge distillation, and effectively improving the translation accuracy of the obtained target translation model.

[0065] See also Figure 4 , Figure 4 is a flow chart of a data processing method provided in an embodiment of the present application, which can be executed by the electronic device mentioned above. Figure 4 As shown, the process of the data processing method in the embodiment of the present application may include:

[0066] S401, obtaining sample text data and translated text data of the sample text data. The specific implementation of step S401 can refer to the relevant description of step S201 in the above embodiment, which will not be repeated here.

[0067] S402, input the sample text data into the first translation model, generate the first translation result corresponding to each text word in the first translation model, and input the sample text data into the second translation model, generate the second translation result corresponding to each text word in the second translation model. The specific implementation of step S402 can refer to the relevant description of step S202 in the above embodiment, which will not be repeated here.

[0068] S403: Obtain important index parameters of the translation words corresponding to each text word.

[0069] In one possible implementation, the important indicator parameter may be determined by one or more important evaluation indicator values, wherein the important evaluation indicator values ​​may include any one or more of the following: cross entropy value of the translated word, word frequency of the translated word, and length of the translated word.

[0070] Optionally, if the important evaluation index value includes the cross entropy value of the translated words, the specific acquisition method can refer to the relevant description of calculating the cross entropy value of the translated words in step S203, which will not be repeated here; Optionally, if the important evaluation index value includes the word frequency of the translated words, the specific acquisition method can be: obtain the translated text data corresponding to all the sample text data used for model training, and calculate the number of translated words in all the translated text data (set to W), and calculate the number of words that are designated translated words in the translated words included in all the translated text data (set to M), then the word frequency of the designated translated text = M / W, for example, the translated word corresponding to text word I I is the English word "happy", and the number of translated words in all translated text data is 1000, and the number of translated words "happy" in all translated text data is 10, then the word frequency of translated word I is 10 / 1000=0.01; optionally, if the important evaluation index value includes the length of the translated word, the specific acquisition method can be: calculate the character length of the translated word, for example, the translated word I corresponding to the text word I is an English word, then the character length of the translated word I is the number of letters constituting the English word, that is, if the translated word I is the English word "happy", then the length of the translated word is 5.

[0071] In a possible implementation, if the important evaluation index value includes the word frequency of the translated words, since the translated words with a smaller word frequency are selected, the word frequency of the translated words can be processed, that is, the word frequency of the translated words can be input into the formula: -log (word frequency) to obtain the word frequency of the processed translated words, and the word frequency of the processed translated words is determined as the important evaluation index value.

[0072] In a possible implementation, N text words include the ith text word, i is a positive integer less than or equal to N, if the important index parameter is determined by an important evaluation index value, then the important evaluation index value of the translation word corresponding to the ith text word is determined as the important index parameter of the translation word corresponding to the ith text word; if the important index parameter is determined by at least two important evaluation index values, then the specific method of determining the important index parameter of the translation word corresponding to the ith text word is, when at least two important evaluation index values ​​of the ith text word are obtained, the at least two evaluation index values ​​are aggregated, and then the important index parameter of the translation word corresponding to the ith text word is obtained. Among them, the specific method of aggregating at least two evaluation index values ​​to obtain the important index parameter of the translation word corresponding to the ith text word can be, obtaining the evaluation weight corresponding to each of the at least two important evaluation index values, weighting each important evaluation index value according to the evaluation weight corresponding to each important evaluation index value, obtaining the weighted index value corresponding to each important evaluation index value, and determining the important index parameter of the translation word corresponding to the ith text word according to the weighted index value corresponding to each important evaluation index value. Optionally, according to the weighted index values ​​corresponding to each important evaluation index value, determining the important index parameters of the translation word corresponding to the i-th text word can specifically be to sum the weighted index values ​​corresponding to each important evaluation index value to obtain the important index parameters of the translation word corresponding to the i-th text word. The evaluation weight can be determined by the technician according to the actual model training process, or according to the structural design of the second translation model and the first translation model, or according to factors such as the text type of the sample text data and the translation text data.

[0073] Exemplarily, the important indicator parameters are determined by three important evaluation indicator values, which are the cross entropy value of the translated words (represented by indicator value 1), the word frequency of the translated words (after processing) (represented by indicator value 2), and the length of the translated words (represented by indicator value 3). If the i-th text word is "An", and the index value 1 of the translated word corresponding to the i-th text word "An" is 1.6, the index value 2 is 0.3, and the index value 3 is 0.04, the evaluation weight of the index value 1 is 1.6, the evaluation weight of the index value 2 is 0.3, and the evaluation weight of the index value 3 is 0.04. Therefore, the corresponding important evaluation indicator values ​​are weighted according to the evaluation weights, and the weighted indicator value corresponding to the index 1 is 0.64, the weighted indicator value corresponding to the index value 2 is 0.3, and the weighted indicator value corresponding to the index value 3 is 0.2. The corresponding weighted indicator values ​​are summed to obtain the important indicator parameter of the translated word corresponding to the i-th text word, which is 1.14.

[0074] S404. Sort the N text words according to the descending order of the important indicator parameters of the translation words corresponding to each text word to obtain the sorted N text words, obtain the sample selection quantity K, and determine the first K text words among the sorted N text words as the target text words.

[0075] Wherein, K is a non-negative integer less than or equal to N.

[0076] In one possible implementation, the N text words to be sorted may be a training sample sentence or multiple training sample sentences from a model training process. If the sample text data including the N text words is multiple training sample sentences, that is, a batch of training sample sentences is divided for a training of the first translation model, and the batch of training sample sentences is used as sample text data, and then the important indicator parameters of each text word included in the batch of training sample sentences can be determined. It can be understood that when the first translation model is trained for the next time, a batch of new training sample sentences can be divided for the next training.

[0077] Optionally, the specific method of determining the important index parameters of the translation words corresponding to each text word in a batch of training sample sentences is the same as determining the important index parameters of the translation words corresponding to each text word in a training sample sentence, that is, taking a batch of training sample sentences as a training sample sentence, respectively obtaining the text words included in each training sample sentence, and the corresponding translation text data, and respectively determining the important index parameters of the translation words corresponding to the text words included in each training sample sentence, and sorting the N text words obtained based on the batch of training sample sentences according to the important index parameters of the translation words corresponding to the N text words from large to small, and then obtaining the sorted N text words. Based on this, when the sorted N text words come from the same batch of training sample sentences, the method of selecting the target text words is also called a local selection method. Optionally, the specific method of obtaining the sample selection quantity K can be determined by selecting the number of text words in the first r% after sorting, or by the number of text words whose important index parameters of the corresponding translation words are greater than a preset threshold, or by the number of text words whose important index parameters of the corresponding translation words are greater than a preset threshold and are in the first r% after sorting.

[0078] In one possible implementation, the N text words to be sorted may also come from multiple training sample sentences in multiple model training processes, that is, from different batches of training sample sentences. In this case, the specific method for obtaining sample text data is as follows: obtaining Z text words in the j-1th model training process of the first translation model, adding the Z text words to a first-in-first-out queue (FIFO), obtaining newly added sample text data in the jth model training process of the first translation model, and adding the text words contained in the newly added sample text data to the first-in-first-out queue containing the Z text words to obtain a target queue, and determining the sample text data based on the text words in the target queue. Wherein, j is a positive integer greater than 1, and Z is equal to N. The newly added sample text data may be a batch of training sample sentences. That is, each time the model is trained, a new training sample sentence is obtained, and multiple text words included in the new training sample sentence are obtained, and the multiple text words are added to a first-in-first-out queue. At this time, the first-in-first-out queue contains multiple text words included in multiple batch training sample sentences in one or more previous model trainings. After the multiple text words included in the new training sample sentence of this model training are added to the first-in-first-out queue, the target queue of this model training is obtained, and the sample text data is determined according to the N text words in the target queue. Therefore, the sample text data includes the multiple text words included in the new training sample sentence in this model training, and also includes the multiple text words included in the training sample sentence in the previous model training.

[0079] It is understandable that when the text words contained in the newly added sample text data are added to the first-in first-out queue, the text words that first enter the queue will be removed accordingly. For example, Figure 5 As shown, Figure 5 It means that when the newly added sample text data including text words in the j-th model training process are added to the first-in-first-out queue, the first-in-first-out queue already contains some text words in the previous model training process (such as the j-1st time and the j-2nd time). In the j-th model training process, as the newly added sample text data including text words are added to the first-in-first-out queue, the text words that first enter the first-in-first-out queue will be removed, thereby obtaining a target queue, which includes all text words contained in the newly added sample text data of the j-th model training and some text words in the previous model training process (such as the j-1st time), and the sample text data for the j-th model training is determined according to the text words in the target queue, and the target text words are obtained from the text words in the target queue. In addition, after the text words contained in the newly added sample text data are added to the first-in-first-out queue, if the number of text words in the first-in-first-out queue does not exceed the maximum number that the first-in-first-out queue can accommodate, the text words in the first-in-first-out queue will not be removed. Furthermore, during the j-th model training process, sample text data can be determined based on the N text words in the target queue, and a new training sample sentence in the sample text data is input into the first translation model to obtain a first translation result corresponding to each of the multiple text words included in the new training sample sentence, and the first translation result corresponding to each text word generated in the previous model training process for the training sample sentences other than the new training sample sentence in the sample text data and the first translation result corresponding to each of the multiple text words included in the new training sample sentence are determined as the first translation result corresponding to each of the N text words included in the sample text data of the j-th model training, and the same applies to the second translation result. And, obtain the important index parameters of the translation words corresponding to each of the multiple text words included in the new training sample sentence, and determine the important index parameters of the translation words corresponding to each of the text words obtained in the previous model training process in the training sample sentences other than the new training sample sentence in the sample text data and the important index parameters of the translation words corresponding to each of the multiple text words included in the new training sample sentence as the important index parameters of the translation words corresponding to each of the N text words, and determine the target text words based on the important index parameters of the translation words corresponding to each of the text words. Based on this, when the N text words to be sorted come from different batches of training sample sentences, the method of selecting the target text words is also called the global selection method, and the global selection method can be. Optionally, the specific method of determining the target text words can be the same as the implementation method of determining the target text words in the local selection method.

[0080] Among them, some text words may have a negative impact on the overall distillation effect when performing knowledge distillation. Therefore, by adaptively selecting appropriate target text words for knowledge distillation based on the training situation of the first translation model, it can have a positive benefit on the training of the first translation model. The target text words can also be called difficult samples. They are difficult samples in the training process of the first translation model. They need to be distilled through the second translation model to obtain additional supervised learning signals. The first translation model can achieve better training results by learning the additional supervised learning signals. The important indicator parameter is the indicator that measures the difficulty of a text word.

[0081] Exemplarily, the text words included in batch I are "A1", "A2", "A3", "A4", and "A5", the text words included in batch II are "B1", "B2", "B3", and "B4", and the text words included in batch III are "C1", "C2", "C3", "C4", and "C5". (1) If the method for selecting target text words is a local selection method, then, during the zth model training, batch I is used as sample text data to train the first translation model, and the important index parameters of the translation words corresponding to each text word in batch I are obtained, and the target text words are selected from the text words included in batch I for knowledge distillation. During the z+1th model training, batch II is used as sample text data to train the first translation model, and the important index parameters of the translation words corresponding to each text word in batch II are obtained, and the target text words are selected from the text words included in batch II for knowledge distillation. During the z+2th model training, batch III is used as sample text data to train the first translation model, and the important index parameters of the translation words corresponding to each text word in batch III are obtained, and the target text words are selected from the text words included in batch III for knowledge distillation. For example, Figure 6a As shown, Figure 6a A flowchart for selecting target text words and training models using the local selection method. Figure 6a The system includes a first translation model, a second translation model, relevant data for model training (multiple batches of training sample sentences, translation text data corresponding to the training sample sentences, translation vocabulary, etc.), and a server for executing the model training steps. The relevant data is stored in the server, specifically:

[0082] ①. The server divides a batch of training sample sentences for the z-th model training, obtains batch I, and uses batch I as sample text data for the z-th model training.

[0083] ②. The server inputs batch I into the first translation model to obtain a first translation result, and inputs batch I into the second translation model to obtain a second translation result.

[0084] ③. The server obtains the important indicator parameters of the translation words corresponding to each text word in batch I, and determines the target text word from each text word in batch I according to the important indicator parameters.

[0085] ④. The server trains the first translation model according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word, and the translation word corresponding to the target text word in the translation text data.

[0086] ⑤. The server divides a batch of training sample sentences for the z+1th model training to obtain batch II, and uses batch II as sample text data for the z+1th model training.

[0087] ⑥. The server inputs batch II into the first translation model to obtain a first translation result, and inputs batch II into the second translation model to obtain a second translation result.

[0088] ⑦. The server obtains the important indicator parameters of the translation words corresponding to each text word in batch II, and determines the target text word from each text word in batch II according to the important indicator parameters.

[0089] ⑧. The server trains the first translation model according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word, and the translation word corresponding to the target text word in the translation text data. It is understandable that when the z+2th model training is performed, the above steps are also performed.

[0090] (2) If the method for selecting the target text words is a global selection method, then, during the zth model training, batch I is used as the newly added sample text data, and the text words included in batch I are added to the first-in-first-out queue. Assume that the maximum number of text words that the first-in-first-out queue can accommodate is 8, and that no text words have been added to the first-in-first-out queue before, so the text words in the first-in-first-out queue are "A1", "A2", "A3", "A4", and "A5". The target queue is obtained, and the sample text data is determined based on the text words in the target queue, and the important indicator parameters of the translation words corresponding to each text word in batch I are obtained, and the target text words are selected from the text words included in batch I for knowledge distillation. The z+1th model training is performed. During training, batch II is used as the newly added sample text data, and the text words included in batch II are added to the first-in-first-out queue. Since there are already 5 text words of batch I in the first-in-first-out queue at this time, the text words included in batch II are moved into the first-in-first-out queue. The target queue obtained has some text words in batch I and text words in batch II, that is, the target queue includes text words "A2", "A3", "A4", "A5", "B1", "B2", "B3", "B4", and the target queue is determined as the sample text data for the z+1th model training, and the important indicator parameters of the translation text corresponding to the text words in batch II in the sample text data and the text words in batch I in the sample text data are obtained. The important indicator parameters of the translated texts obtained in the zth model training process are used, and the target text words are determined according to the important indicator parameters of the translated texts corresponding to each text word in the sample text data, that is, the important indicator parameters obtained in the zth model training process according to the translated texts corresponding to the text words "A2", "A3", "A4", and "A5", and the important indicator parameters obtained in the z+1 model training process according to the translated texts corresponding to the text words "B1", "B2", "B3", and "B4", respectively, are used. The target text words for knowledge distillation in the z+1 model training process are determined in each text word. Similarly, during the z+2th model training, batch III is used as the newly added sample text data, and batch III is used as the newly added sample text data. The text words included in batch III are added to the first-in-first-out queue to obtain a target queue, wherein the text words include "B2", "B3", "B4", "C1", "C2", "C3", "C4", and "C5". The important index parameters of the translation text corresponding to the text words of batch III in the sample text data and the important index parameters of the translation text corresponding to the text words of batch II in the sample text data obtained during the z+1 model training process are obtained. The target text words are determined according to the important index parameters of the translation text corresponding to each text word in the sample text data, that is, according to the important index parameters of the translation text corresponding to the text words "B2", "B3", and "B4" respectively obtained during the z+1 model training process.And according to the important index parameters obtained in the z+2 model training process of the translation texts corresponding to the text words "C1", "C2", "C3", "C4", and "C5", the target text words for knowledge distillation in the z+2 model training process are determined in each text word. For example, Figure 6b As shown, Figure 6b A flowchart for selecting target text words and training models using the global selection method. Figure 6b The system includes a first translation model, a second translation model, relevant data for model training (multiple batches of training sample sentences, translation text data corresponding to the training sample sentences, translation words, etc.), and a server for executing the model training steps. The relevant data is stored in the server, specifically:

[0091] S1. The server obtains a target queue I for the zth model training, and determines sample text data based on the target queue I, where the target queue I includes text words of batch I: the server divides a batch of training sample sentences for the zth model training to obtain batch I, and uses batch I as the newly added sample text data for the zth model training, and adds the text words included in batch I to a first-in-first-out queue. Assume that no text words have been added to the first-in-first-out queue before, so the first-in-first-out queue includes text words of batch I, obtains the target queue I, and determines the sample text data for the zth model training based on the text words included in the target queue I.

[0092] S2. The server inputs batch I into the first translation model to obtain a first translation result, and inputs batch I into the second translation model to obtain a second translation result.

[0093] S3. The server obtains important indicator parameters of the translation words corresponding to each text word in batch I, and determines the target text word from each text word in batch I according to the important indicator parameters.

[0094] S4. The server trains the first translation model according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word, and the translation word corresponding to the target text word in the translation text data.

[0095] S5. The server obtains the target queue II for the z+1th model training, and determines the sample text data according to the target queue II, wherein the target queue II includes the text words of batch I and batch II: the server divides a batch of training sample sentences for the z+1th model training to obtain batch II, and uses batch II as the newly added sample text data for the z+1th model training, and adds the text words included in batch II to the first-in-first-out queue. The first-in-first-out queue at this time is the target queue I for the zth model training. If all the text words included in batch II are added to the first-in-first-out queue, the number of text words contained in the queue exceeds the maximum number that can be accommodated. Therefore, when the text words of batch II are moved in, some of the text words in the queue that are first moved out are correspondingly moved out. Assuming that the first-in-first-out queue at this time includes some text words of batch I and text words of batch II, the target queue II is obtained, and the sample text data for the z+1th model training is determined according to the text words included in the target queue II.

[0096] S6. The server inputs batch II into the first translation model to obtain a first translation result corresponding to batch II, and inputs batch II into the second translation model to obtain a first translation result corresponding to batch II.

[0097] S7. The server obtains the important index parameters of the translation words corresponding to each text word in batch II, and determines the target text word from each text word in target queue II according to the important index parameters of the translation words corresponding to each text word in batch II and the important index parameters of the translation words corresponding to the text words in batch I in target queue II. The target text word can come from batch I or batch II.

[0098] S8. The server trains the first translation model according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word, and the translation sentence corresponding to the target text word in the translation text data. It is understandable that when the z+2th model training is performed, the above steps are also performed.

[0099] S405. Determine a model translation loss function according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word, and the translation word corresponding to the target text word in the translation text data, and correct the model parameters of the first translation model according to the model translation loss function to obtain a target translation model.

[0100] The specific method of determining the model translation loss function and obtaining the target translation model can be found in the relevant description of step S204, which will not be repeated here.

[0101] In a possible implementation, after obtaining the target translation model, the target translation model may be tested, specifically by obtaining text data to be translated, inputting the text data to be translated into the target translation model, and outputting target translation text data of the text data to be translated based on the target translation model. The text data to be translated and the sample text data have the same text type, and the target translation text data and the translation text data have the same text type.

[0102] Optionally, through a large number of tests on the technical solution of the present application, it is known that a data processing method proposed in the present application improves the translation efficiency and accuracy of the model, as well as the efficiency and effect of knowledge distillation to a certain extent. For example, in the English-German translation scenario, the neural machine translation (Neural Machine Translation, NMT) model Transformer model is trained as the first translation model, and the target translation model obtained by training based on the technical solution of the present application is tested and found that the BLEU score value (Bilingual Evaluation Understudy) has been greatly improved, that is, the translation effect of the translation model can be significantly improved. For another example, in the Chinese-English translation scenario, the Transformer model is trained as the first translation model, and the target translation model obtained by training based on the technical solution of the present application is tested and found that the translation effect of the Transformer model without knowledge distillation is compared, and the translation effect of the Transformer model without knowledge distillation selection of sample data is compared. The BLEU score value has been greatly improved, and it is explained that the technical solution of the present application can be applied to translation model training scenarios of different languages, and a stable translation effect improvement can be achieved. In addition to being applied to Transformer model training, it can also be applied to RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit) and other model training. Tests have shown that the model training effects have been greatly improved.

[0103] In an embodiment of the present application, an electronic device obtains sample text data and translated text data of the sample text data, inputs the sample text data into a first translation model, generates a first translation result corresponding to each text word in the first translation model, and inputs the sample text data into a second translation model, generates a second translation result corresponding to each text word in the second translation model, obtains important index parameters of the translated words corresponding to each text word, sorts N text words according to the order of the important index parameters of the translated words corresponding to each text word from large to small, obtains the sorted N text words, obtains the sample selection quantity K, determines the first K text words of the sorted N text words as target text words, determines a model translation loss function according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word, and the translated words corresponding to the target text word in the translated text data, and corrects the model parameters of the first translation model according to the model translation loss function to obtain a target translation model. By implementing the above scheme, the sample text data can be processed, and suitable target text words can be dynamically selected from the text words contained in the sample text data for knowledge distillation in combination with the training situation of the first translation model. This can avoid, to a certain extent, the negative impact of knowledge distillation of some text words on the overall distillation effect, and effectively prevent overfitting, thereby improving the efficiency and effect of knowledge distillation, and effectively improving the translation accuracy of the acquired target translation model.

[0104] See also Figure 7 , Figure 7 The following is a schematic diagram of the structure of a data processing device provided in this application. Figure 7 The data processing device shown is used to execute the application Figure 2 and Figure 4 For the convenience of explanation, only the part related to the embodiment of the present application is shown, and the specific technical details are not disclosed. Figure 2 and Figure 4 The test data processing device 700 may include: an acquisition module 701, a processing module 702, and a determination module 703. Among them:

[0105] The acquisition module 701 is used to acquire sample text data and translated text data of the sample text data; the sample text data includes N text words, N is a positive integer; the translated text data includes a translated word corresponding to each text word in the N text words; the sample text data and the translated text data have different text types;

[0106] Processing module 702, for inputting sample text data into a first translation model, generating a first translation result corresponding to each text word in the first translation model, and inputting sample text data into a second translation model, generating a second translation result corresponding to each text word in the second translation model;

[0107] A determination module 703 is used to obtain important index parameters of the translation words corresponding to each text word, and determine the target text word from the N text words according to the important index parameters of the translation words corresponding to each text word;

[0108] The determination module 703 is also used to determine the model translation loss function according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word, and the translation word corresponding to the target text word in the translation text data, and to correct the model parameters of the first translation model according to the model translation loss function to obtain the target translation model.

[0109] In a possible implementation, the N text words include the i-th text word, i is a positive integer less than or equal to N; the first translation result of the i-th text word includes the predicted probability of the i-th text word for each word in the translation vocabulary; the translation words in the translation text data belong to the translation vocabulary; the determination module 703, when used to obtain the important indicator parameters of the translation words corresponding to each text word, is specifically used to:

[0110] Determine the predicted probability of the translated word corresponding to the i-th text word in the first translation result of the i-th text word as the target predicted probability;

[0111] Generate important indicator parameters of the translation word corresponding to the i-th text word according to the target prediction probability.

[0112] In a possible implementation, the N text words include the i-th text word, where i is a positive integer less than or equal to N; when the determination module 703 is used to obtain the important indicator parameters of the translation word corresponding to each text word, it is specifically used to:

[0113] Obtain at least two important evaluation index values ​​for the i-th text word;

[0114] Aggregate at least two important evaluation index values ​​to obtain important index parameters of the translation word corresponding to the i-th text word.

[0115] In a possible implementation, when the determination module 703 is used to aggregate at least two important evaluation index values ​​to obtain the important index parameters of the translation word corresponding to the i-th text word, it is specifically used to:

[0116] Obtaining evaluation weights corresponding to each of at least two important evaluation indicator values;

[0117] According to the evaluation weights corresponding to each important evaluation indicator value, each important evaluation indicator value is weighted to obtain the weighted indicator value corresponding to each important evaluation indicator value;

[0118] According to the weighted index values ​​corresponding to each important evaluation index value, the important index parameters of the translation word corresponding to the i-th text word are determined.

[0119] In a possible implementation manner, when the determination module 703 is used to determine the target text word from N text words according to the important indicator parameters of the translation word corresponding to each text word, it is specifically used to:

[0120] Sorting the N text words according to the descending order of the important indicator parameters of the translation words corresponding to each text word, to obtain the sorted N text words;

[0121] The sample selection quantity K is obtained, and the first K text words among the sorted N text words are determined as target text words; K is a non-negative integer.

[0122] In a possible implementation, when the acquisition module 701 is used to acquire sample text data, it is specifically used to:

[0123] Obtain Z text words of the first translation model in the j-1th model training process; j is a positive integer greater than 1, and Z is equal to N;

[0124] Add Z text words to a first-in, first-out queue;

[0125] In the j-th model training process of the first translation model, newly added sample text data is obtained, and the text words included in the newly added sample text data are added to the first-in-first-out queue including the Z text words to obtain a target queue;

[0126] Sample text data is determined based on text terms in the target queue.

[0127] In a possible implementation, the processing module 702 is further configured to:

[0128] Acquire text data to be translated; the text data to be translated and the sample text data have the same text type;

[0129] The text data to be translated is input into a target translation model, and target translation text data of the text data to be translated is output based on the target translation model; the target translation text data and the translation text data have the same text type.

[0130] In the embodiment of the present application, the acquisition module acquires sample text data and translated text data of the sample text data, the processing module inputs the sample text data into the first translation model, generates the first translation result corresponding to each text word in the first translation model, and inputs the sample text data into the second translation model, generates the second translation result corresponding to each text word in the second translation model, the determination module acquires the important index parameters of the translated words corresponding to each text word, and determines the target text words from N text words according to the important index parameters of the translated words corresponding to each text word, the determination module determines the model translation loss function according to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word and the translated words corresponding to the target text word in the translated text data, and corrects the model parameters of the first translation model according to the model translation loss function to obtain the target translation model. By implementing the above scheme, the sample text data can be processed, and suitable target text words can be selected from the text words contained in the sample text data for knowledge distillation, which can avoid the negative impact of some text words on the overall distillation effect when performing knowledge distillation to a certain extent, and effectively prevent overfitting, thereby improving the efficiency and effect of knowledge distillation, and effectively improving the translation accuracy of the obtained target translation model.

[0131] Each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules, which is not limited in the present application.

[0132] Please attend Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device 800 includes: at least one processor 801 and a memory 802. Optionally, the electronic device may further include a network interface 803. The processor 801, the memory 802 and the network interface 803 may exchange data, the network interface 803 is controlled by the processor to send and receive messages, the memory 802 is used to store a computer program, the computer program includes program instructions, and the processor 801 is used to execute the program instructions stored in the memory 802. The processor 801 is configured to call the program instructions to execute the above method.

[0133] The memory 802 may include a volatile memory, such as a random-access memory (RAM); the memory 802 may also include a non-volatile memory, such as a flash memory, a solid-state drive (SSD), etc.; the memory 802 may also include a combination of the above types of memory.

[0134] The processor 801 may be a central processing unit (CPU) 801. In one embodiment, the processor 801 may also be a graphics processing unit (GPU) 801. The processor 801 may also be a combination of a CPU and a GPU.

[0135] In one embodiment, the memory 802 is used to store program instructions. The processor 801 can call the program instructions to perform the following steps:

[0136] Obtaining sample text data and translated text data of the sample text data; the sample text data includes N text words, N is a positive integer; the translated text data includes a translated word corresponding to each text word in the N text words; the sample text data and the translated text data have different text types;

[0137] Input the sample text data into a first translation model, generate a first translation result corresponding to each text word in the first translation model, and input the sample text data into a second translation model, generate a second translation result corresponding to each text word in the second translation model;

[0138] Obtaining important index parameters of translation words corresponding to each text word, and determining a target text word from the N text words according to the important index parameters of the translation words corresponding to each text word;

[0139] According to the first translation result corresponding to the target text word, the second translation result corresponding to the target text word and the translation word corresponding to the target text word in the translation text data, a model translation loss function is determined, and the model parameters of the first translation model are corrected according to the model translation loss function to obtain the target translation model.

[0140] In a possible implementation, the N text words include the i-th text word, i is a positive integer less than or equal to N; the first translation result of the i-th text word includes the predicted probability of the i-th text word for each word in the translation vocabulary; the translation words in the translation text data belong to the translation vocabulary; when the processor 801 is used to obtain the important indicator parameters of the translation words corresponding to each text word, it is specifically used to:

[0141] Determine the predicted probability of the translated word corresponding to the i-th text word in the first translation result of the i-th text word as the target predicted probability;

[0142] Generate important indicator parameters of the translation word corresponding to the i-th text word according to the target prediction probability.

[0143] In a possible implementation, the N text words include the i-th text word, where i is a positive integer less than or equal to N; when the processor 801 is used to obtain the important indicator parameters of the translation word corresponding to each text word, it is specifically used to:

[0144] Obtain at least two important evaluation index values ​​for the i-th text word;

[0145] Aggregate at least two important evaluation index values ​​to obtain important index parameters of the translation word corresponding to the i-th text word.

[0146] In a possible implementation manner, when the processor 801 is used to aggregate at least two important evaluation index values ​​to obtain the important index parameter of the translation word corresponding to the i-th text word, it is specifically used to:

[0147] Obtaining evaluation weights corresponding to each of at least two important evaluation indicator values;

[0148] According to the evaluation weights corresponding to each important evaluation indicator value, each important evaluation indicator value is weighted to obtain the weighted indicator value corresponding to each important evaluation indicator value;

[0149] According to the weighted index values ​​corresponding to each important evaluation index value, the important index parameters of the translation word corresponding to the i-th text word are determined.

[0150] In a possible implementation manner, when the processor 801 is used to determine the target text word from N text words according to the important indicator parameters of the translation word corresponding to each text word, it is specifically used to:

[0151] Sorting the N text words according to the descending order of the important indicator parameters of the translation words corresponding to each text word, to obtain the sorted N text words;

[0152] The sample selection quantity K is obtained, and the first K text words among the sorted N text words are determined as target text words; K is a non-negative integer.

[0153] In a possible implementation, when the processor 801 is used to obtain the sample text data, it is specifically used to:

[0154] Obtain Z text words of the first translation model in the j-1th model training process; j is a positive integer greater than 1, and Z is equal to N;

[0155] Add Z text words to a first-in, first-out queue;

[0156] In the j-th model training process of the first translation model, newly added sample text data is obtained, and the text words included in the newly added sample text data are added to a first-in-first-out queue including Z text words to obtain a target queue;

[0157] Sample text data is determined based on text terms in the target queue.

[0158] In a possible implementation, the processor 801 is further configured to:

[0159] Acquire text data to be translated; the text data to be translated and the sample text data have the same text type;

[0160] The text data to be translated is input into a target translation model, and target translation text data of the text data to be translated is output based on the target translation model; the target translation text data and the translation text data have the same text type.

[0161] In a specific implementation, the device, processor 801, memory 802, etc. described in the embodiments of the present application can execute the implementation method described in the above method embodiments, and can also execute the implementation method described in the embodiments of the present application, which will not be repeated here.

[0162] In an embodiment of the present application, a computer (readable) storage medium is also provided, which stores a computer program, which includes program instructions, and when the program instructions are executed by a processor, some or all of the steps executed in the above method embodiment can be executed. Optionally, the computer storage medium can be volatile or non-volatile.

[0163] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0164] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a computer storage medium, which can be a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The computer storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0165] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized in that: include: Acquire sample text data and translated text data of the sample text data; the sample text data includes N text words, where N is a positive integer; The translated text data includes a translated word corresponding to each of the N text words; the sample text data and the translated text data have different text types; Inputting the sample text data into a first translation model, generating a first translation result corresponding to each text word in the first translation model, and inputting the sample text data into a second translation model, generating a second translation result corresponding to each text word in the second translation model; Obtain the important indicator parameters of the translation words corresponding to each of the text words, sort the N text words in descending order of the important indicator parameters of the translation words corresponding to each of the text words, obtain the sorted N text words, and determine the first K text words of the sorted N text words as target text words, where K is a positive integer less than or equal to N; wherein the important indicator parameters of the translation words corresponding to any text word are obtained by aggregating at least two important evaluation indicator values ​​of the any text word, and the important evaluation indicator values ​​of any text word include at least two of the following: the cross entropy value, word frequency, and character length of the translation words corresponding to any text word; the cross entropy value refers to the loss value of the first translation model for the any text word under the first translation result, and the word frequency refers to the frequency of occurrence of the translation words corresponding to any text word in the translation text data; A model translation loss function is determined based on a first translation result corresponding to the target text word, a second translation result corresponding to the target text word, and a translation word corresponding to the target text word in the translation text data, and model parameters of the first translation model are corrected based on the model translation loss function to obtain a target translation model.

2. The method according to claim 1, characterized in that: The N text words include the i-th text word, i is a positive integer less than or equal to N; the first translation result of the i-th text word includes the predicted probability of the i-th text word for each word in the translation vocabulary; The translation words in the translation text data belong to the translation word library; The obtaining of important index parameters of the translation words corresponding to each of the text words includes: Determining the predicted probability of the translated word corresponding to the i-th text word in the first translation result of the i-th text word as the target predicted probability; The target prediction probability is used to determine the cross entropy value of the translation word corresponding to the i-th text word; An important indicator parameter of the translation word corresponding to the i-th text word is generated according to the cross entropy value of the translation word corresponding to the i-th text word.

3. The method according to claim 1, characterized in that The N text words include the i-th text word, where i is a positive integer less than or equal to N; The obtaining of important index parameters of the translation words corresponding to each of the text words includes: Obtaining the evaluation weight corresponding to each of the at least two important evaluation index values ​​of the i-th text word; According to the evaluation weights respectively corresponding to each of the important evaluation index values, each of the important evaluation index values ​​is weighted to obtain a weighted index value respectively corresponding to each of the important evaluation index values; According to the weighted index values ​​corresponding to each of the important evaluation index values, the important index parameters of the translation word corresponding to the i-th text word are determined.

4. The method according to claim 1, characterized in that: The obtaining of sample text data includes: Obtaining Z text words in the j-1th model training process of the first translation model, where j is a positive integer greater than 1, and Z is equal to N; Add the Z text words to a first-in-first-out queue; In the j-th model training process of the first translation model, newly added sample text data is obtained, and the text words included in the newly added sample text data are added to the first-in-first-out queue including the Z text words to obtain a target queue; The sample text data is determined according to the text words in the target queue.

5. The method according to claim 1, characterized in that The method further comprises: Acquire text data to be translated; the text data to be translated and the sample text data have the same text type; The text data to be translated is input into the target translation model, and target translation text data of the text data to be translated is output based on the target translation model; the target translation text data and the translation text data have the same text type.

6. A data processing device, characterized in that: include: An acquisition module, used for acquiring sample text data and translation text data of the sample text data; the sample text data includes N text words, N is a positive integer; the translation text data includes a translation word corresponding to each text word in the N text words; the sample text data and the translation text data have different text types; A processing module, configured to input the sample text data into a first translation model, generate a first translation result corresponding to each text word in the first translation model, and input the sample text data into a second translation model, generate a second translation result corresponding to each text word in the second translation model; A determination module is used to obtain the important indicator parameters of the translation words corresponding to each of the text words, sort the N text words in descending order of the important indicator parameters of the translation words corresponding to each of the text words, and obtain the sorted N text words, and determine the first K text words of the sorted N text words as target text words, K is a positive integer less than or equal to N; wherein, the important indicator parameters of the translation words corresponding to any text word are obtained by aggregating at least two important evaluation indicator values ​​of the any text word, and the important evaluation indicator values ​​of any text word include at least two of the following: the cross entropy value, word frequency, and character length of the translation words corresponding to any text word; the cross entropy value refers to the loss value of the first translation model for the any text word under the first translation result, and the word frequency refers to the frequency of occurrence of the translation words corresponding to any text word in the translation text data; The determination module is also used to determine a model translation loss function based on a first translation result corresponding to the target text word, a second translation result corresponding to the target text word, and a translation word corresponding to the target text word in the translation text data, and to correct the model parameters of the first translation model based on the model translation loss function to obtain a target translation model.

7. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store computer program instructions, and the processor is configured to execute the program instructions to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 5.

Images