Data processing method, device, computer equipment and storage medium

Abstract

The present application discloses a data processing method, apparatus, computer equipment, and storage medium. The method includes: obtaining a sample corpus, the sample corpus including a target sample corpus pair, the target sample corpus pair including a first sample sentence and a second sample sentence, the second sample sentence being a sentence translated from the first sample sentence; performing probability prediction processing on the j-th data in the second sample sentence based on j-1 data in the first sample sentence and the second sample sentence to obtain a first predicted probability of the j-th data; performing probability prediction processing on the j-th data in the second sample sentence based on the first j-1 data to obtain a second predicted probability of the j-th data; training an initial model based on the first predicted probability and the second predicted probability of the j-th data to obtain a target model; obtaining an input sentence, inputting the input sentence into the target model, and generating a translated output sentence. This makes text translation more accurate and improves data processing accuracy.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing in artificial intelligence, and in particular to a data processing method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the rapid development of computer technology, artificial intelligence (AI) has been applied in many fields. Deep neural networks, a type of AI technology, are being increasingly applied in fields such as speech recognition, computer vision, and natural language processing. Deep neural network models have demonstrated outstanding performance in a variety of tasks, particularly machine translation. However, when training Neural Machine Translation (NMT) models used in machine translation, word imbalance in the training data often occurs, resulting in suboptimal performance of the resulting trained translation model. Summary of the Invention

[0003] The embodiments of the present application provide a data processing method, apparatus, computer device, and storage medium, which can improve the translation effect of the translation model, make text translation more accurate, and improve data processing accuracy.

[0004] A first aspect of an embodiment of the present application discloses a data processing method, the method comprising:

[0005] Acquire a sample corpus, the sample corpus including at least one target sample corpus pair, the target sample corpus pair including a first sample sentence and a second sample sentence, the second sample sentence being a sentence translated from the first sample sentence, the second sample sentence including J data, where J is a positive integer;

[0006] Perform probability prediction processing on the j-th data in the second sample sentence based on the first sample sentence and the first j-1 data in the second sample sentence to obtain a first prediction probability of the j-th data, j∈[1,J];

[0007] Performing probability prediction processing on the j-th data of the second sample sentence based on first j-1 data in the second sample sentence to obtain a second predicted probability of the j-th data;

[0008] Training the initial model according to the first predicted probability and the second predicted probability of the j-th data in the second sample sentence to obtain a target model;

[0009] An input sentence to be translated is obtained, the input sentence is input into the target model, and a translated output sentence is generated.

[0010] A second aspect of an embodiment of the present application discloses a data processing device, the device comprising:

[0011] an acquiring unit, configured to acquire a sample corpus, the sample corpus comprising at least one target sample corpus pair, the target sample corpus pair comprising a first sample sentence and a second sample sentence, the second sample sentence being a sentence translated from the first sample sentence, the second sample sentence comprising J data, where J is a positive integer;

[0012] A first determining unit is configured to perform probability prediction processing on the j-th data in the second sample sentence based on the first sample sentence and the first j-1 data in the second sample sentence to obtain a first predicted probability of the j-th data, j∈[1, J];

[0013] a second determining unit, configured to perform probability prediction processing on the j-th data of the second sample sentence based on first j-1 data in the second sample sentence, to obtain a second predicted probability of the j-th data;

[0014] a training unit, configured to train an initial model according to a first predicted probability and a second predicted probability of the j-th data in the second sample sentence to obtain a target model;

[0015] The translation unit is used to obtain an input sentence to be translated, input the input sentence into the target model, and generate a translated output sentence.

[0016] A third aspect of an embodiment of the present application discloses a computer device, comprising a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method of the first aspect above.

[0017] A fourth aspect of an embodiment of the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method of the first aspect.

[0018] A fifth aspect of the present application discloses a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method of the first aspect.

[0019] In an embodiment of the present application, a sample corpus comprising at least one target sample corpus pair can be obtained, wherein any target sample corpus pair can include a first sample sentence and a second sample sentence translated from the first sample sentence, and the second sample sentence can include J data. After obtaining the sample corpus, the j-th data in the second sample sentence can be subjected to probability prediction processing based on the first j-1 data in the first sample sentence and the second sample sentence to obtain a first predicted probability of the j-th data; or the j-th data in the second sample sentence can be subjected to probability prediction processing based on the first j-1 data in the second sample sentence to obtain a second predicted probability of the j-th data. Thus, the initial model can be trained based on the first predicted probability and the second predicted probability of the j-th data in the second sample sentence to obtain a target model, and the target model can be used to generate a translated output sentence based on the input sentence. By implementing the above method, the attention paid to each word sample during the model training process can be dynamically adjusted by integrating the word evaluation index of the target end context information, so that the model pays more attention to some important words during training, which can then be used to improve the online translation system, make text translation more accurate, and improve data processing accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a schematic diagram of the architecture of a data processing system provided in an embodiment of the present application;

[0022] Figure 2 This is a flow chart of a data processing method provided in an embodiment of the present application;

[0023] Figure 3 This is a structural diagram of an initial training model provided in an embodiment of the present application;

[0024] Figure 4 This is a flow chart of another data processing method provided in an embodiment of the present application;

[0025] Figure 5 This is a structural diagram of another training initial model provided in an embodiment of the present application;

[0026] Figure 6 is a structural diagram of a data processing device provided in an embodiment of the present application;

[0027] Figure 7It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0029] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0030] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0031] Natural language processing (NLP) is a key area of ​​research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0032] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0033] Based on the natural language processing technology and machine learning technology mentioned in the above-mentioned artificial intelligence technology, the embodiment of the present application proposes a data processing solution, which can specifically be an implementation plan for training the initial model, and the target model obtained by training the initial model can be applied in translation scenarios, such as outputting the corresponding translation for a sentence to be translated.

[0034] Specifically, the data processing scheme generally works as follows: a sample corpus can be obtained that includes at least one target sample corpus pair, wherein any target sample corpus pair can include a first sample sentence and a second sample sentence translated from the first sample sentence, and the second sample sentence can include J data. After obtaining the sample corpus, a probability prediction process can be performed on the j-th data in the second sample sentence based on the first sample sentence and the first j-1 data in the second sample sentence to obtain a first predicted probability of the j-th data, where j is less than or equal to J; or a probability prediction process can be performed on the j-th data in the second sample sentence based on the first j-1 data in the second sample sentence to obtain a second predicted probability of the j-th data. Optionally, the first predicted probability of the j-th data can be obtained by the translation network (or referred to as a translation module or translation model, etc.) in the initial model, and the second predicted probability of the j-th data can be obtained by the language network (or referred to as a language module or language model, etc.) in the initial model.

[0035] After obtaining the first predicted probability and the second predicted probability of the j-th data point in the second sample sentence, the initial model can be trained based on the first predicted probability and the second predicted probability to obtain a target model. The target model can be the translation network in the initial model, and the target model can be used to generate a translated output sentence based on the input sentence.

[0036] Optionally, the target model that the embodiment of the present application carries out data processing method and is obtained can be applied in translation device (or translation system, translation platform etc.).In specific translation scene, this translation device can obtain input sentence, and as input sentence is " my name is Pete ".Optionally, this translation device can be provided for user input the sentence that needs to translate, i.e. input sentence, and this input sentence can be in the mode of voice input or text input or other input modes, and this application does not limit.Further, translation device can generate the translation of this input sentence after obtaining this input sentence, and as the translation of above-mentioned input sentence can be " my name is Peter ".Optionally, this translation can be displayed on the device screen of translation device in the mode of text, or output in voice mode.

[0037] By implementing the above method, the word-level adaptive training method can be used to solve the problem of word imbalance in the sample corpus. Specifically, the word evaluation index that integrates the target-side context information can be used to dynamically adjust the attention paid to each word sample during the model training process, so that the model pays more attention to some important words during training to improve the translation effect of the translation model, which can then be used to improve the online translation system, make text translation more accurate, and improve data processing accuracy.

[0038] In a specific implementation, the execution subject of the above-mentioned data processing solution can be a computer device, which includes but is not limited to a terminal or a server. In other words, the computer device can be a server or a terminal, or a system composed of a server and a terminal. Among them, the terminal mentioned above can be an electronic device, including but not limited to mobile phones, tablet computers, desktop computers, laptop computers, PDAs, vehicle-mounted devices, intelligent voice interaction devices, augmented reality / virtual reality (AR / VR) devices, helmet displays, wearable devices, smart speakers, smart home appliances, aircraft, digital cameras, cameras and other mobile Internet devices (Mobile Internet Device, MID) with network access capabilities. Among them, the server mentioned above can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, vehicle-road collaboration, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0039] It should be noted that when the computer device is a server, the embodiment of the present application provides a data processing system, such as Figure 1 As shown, the data processing system includes at least one terminal and at least one server; the terminal can obtain a sample corpus and upload the obtained sample corpus to the server (i.e., a computer device), so that the computer device can obtain the sample corpus and train the initial model based on the sample corpus to obtain a target model. Optionally, after obtaining the target model, the target model can be further applied in a translation scenario. For example, an input sentence to be translated can be obtained and input into the target model to obtain an output sentence after the input sentence is translated.

[0040] Based on the data processing solution provided above, the embodiment of the present application provides a data processing method, which can be executed by the computer device mentioned above. Figure 2 The data processing method includes but is not limited to the following steps:

[0041] S201 : Acquire a sample corpus, where the sample corpus includes at least one target sample corpus pair, and the target sample corpus pair includes a first sample sentence and a second sample sentence.

[0042] The sample corpus may include one or more target sample corpus pairs, each of which may include a first sample sentence and a second sample sentence, wherein the second sample sentence is a sentence translated from the first sample sentence, that is, the first sample sentence and the second sample sentence are sentences of two different language types. For example, the first sample sentence and the second sample sentence may be sentences of any two different language types such as Chinese, English, German, Korean, Japanese, or other language types. For example, the first sample sentence and the second sample sentence may be in Chinese and English respectively; for another example, the first sample sentence and the second sample sentence may be in English and German respectively. The second sample sentence may include J data, and the J data included in the second sample sentence may refer to the words included in the second sample sentence. For example, if the first sample sentence is "My name is Pete", the corresponding second sample sentence may be "My name is Peter", wherein the second sample sentence includes the following four data (words): my, name, is, Peter.

[0043] In one implementation, the computer device may obtain the sample corpus from a terminal, or may obtain the sample corpus from a text database storing text data, or may obtain the sample corpus from other paths. The sample corpus may include text data in various fields, for example, text data in the field of sports, text data in the field of medicine, and so on. It is understandable that in a specific embodiment of the present application, if the sample corpus involves data related to user information, the sample corpus obtained by the computer device is data authorized by the user. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent must be obtained, and the collection, use, and processing of relevant data must comply with relevant laws, regulations, and standards of relevant countries and regions.

[0044] S202 : Perform probability prediction processing on the j-th data in the second sample sentence based on the first sample sentence and the first j-1 data in the second sample sentence to obtain a first prediction probability of the j-th data.

[0045] S203 , performing probability prediction processing on the j-th data of the second sample sentence according to the first j−1 data in the second sample sentence to obtain a second predicted probability of the j-th data.

[0046] Wherein, j∈[1, J], J is a positive integer, that is, j can refer to any data among the J data included in the second sample sentence.

[0047] In steps S202 and S203, the computer device may determine a first predicted probability and a second predicted probability of the j-th data. Alternatively, the computer device may obtain the first predicted probability of the j-th data using a translation network included in the initial model, and obtain the second predicted probability of the j-th data using a language network included in the initial model.

[0048] In one implementation, Figure 3 The above is a structural diagram of training an initial model provided in an embodiment of the present application, such as Figure 3 As shown, the initial model includes a translation network and a language network. Figure 3 The translation network shown in FIG1 obtains the first predicted probability of the jth data. In a specific implementation, the first j-1 data in the first sample sentence and the second sample sentence can be input into the translation network to perform probability prediction processing on the jth data in the second sample sentence, thereby obtaining the first predicted probability of the jth data. Figure 3The language network shown in the figure obtains the second predicted probability of the j-th data point. In a specific implementation, the first j-1 data points in the second sample sentence can be input into the language network to perform probability prediction processing on the j-th data point in the second sample sentence, thereby obtaining the second predicted probability of the j-th data point. The first predicted probability can be understood as the probability predicted by the translation network that the data point at the j-th position in the second sample sentence is the j-th data point; similarly, the second predicted probability can be understood as the probability predicted by the language network that the data point at the j-th position in the second sample sentence is the j-th data point.

[0049] For example, assuming the first sample sentence is "My name is Pete" and the second sample sentence is "My name is Peter," and j = 4, then the first j-1 data points in the second sample sentence include: "My name is." Then, if the translation network is fed "My name is Pete" and "My name is," the network can predict the probability that the fourth data point is "Peter," which is the first predicted probability mentioned above. If the language network is fed "My name is," the network can predict the probability that the fourth data point is "Peter," which is the second predicted probability mentioned above.

[0050] Among them, the translation network can be a neural machine translation (NMT) network. For example, the translation network can be an RNNsearch network (a network model based on the encoder-decoder framework of a recurrent neural network (RNN)) or a LightConv network (a network model based on the encoder-decoder framework of a convolutional neural network (CNN)) or a Transformer network (a network model based on the encoder-decoder framework of a self-attention neural network (SAN)), etc. The language network can have the same architecture as the decoder in the translation network, or it can be replaced by a language network of other architectures, such as a long short-term memory network (LSTM) or a gated recurrent unit (GRU) network, etc. Among them, GRU is a variant network of LSTM with better performance.

[0051] S204: Train the initial model according to the first predicted probability and the second predicted probability of the j-th data in the second sample sentence to obtain a target model.

[0052] In one implementation, the initial model may be the model constructed by the translation network and language network described above. The target model obtained by training the initial model may be the trained translation network, which can be used to generate a translated output sentence based on an input sentence.

[0053] In one implementation, the initial model can be trained based on the first predicted probability and the second predicted probability of the j-th data in the second sample sentence to obtain a target model. For example, the training weight of the j-th data in the training process can be determined based on the first predicted probability and the second predicted probability of the j-th data in the second sample sentence. The training weight can be used to characterize the attention of the data included in the second sample sentence during the training of the initial model, or the training weight can be used to evaluate the importance of each data in the training process, wherein the training weight and attention can be positively correlated, that is, when the training weight of a certain data is greater, the corresponding attention is higher, and when the training weight of a certain data is smaller, the corresponding attention is lower. After obtaining the training weight of the j-th data, the initial model can be trained with the training weight to obtain the target model.

[0054] Optionally, a target loss function corresponding to the initial model can be obtained, and the initial model can be trained using the training weight of the j-th data and the target loss function to obtain a target model. For example, a target loss value can be calculated based on the training weight of the j-th data and the target loss function, and the initial model can be trained based on the target loss value to obtain the target model.

[0055] S205: Obtain an input sentence to be translated, input the input sentence into a target model, and generate a translated output sentence.

[0056] In one implementation, after obtaining the target model, the target model can be applied in a translation scenario. For example, an input sentence to be translated can be obtained, and after obtaining the input sentence, the input sentence can be input into the target model to generate a translated input sentence, which can be called an output sentence.

[0057] In an embodiment of the present application, a computer device can obtain a sample corpus including at least one target sample corpus pair, wherein any target sample corpus pair can include a first sample sentence and a second sample sentence translated from the first sample sentence, and the second sample sentence can include J data. After obtaining the sample corpus, the j-th data in the second sample sentence can be subjected to probability prediction processing based on the first j-1 data in the first sample sentence and the second sample sentence to obtain a first predicted probability of the j-th data; or the j-th data in the second sample sentence can be subjected to probability prediction processing based on the first j-1 data in the second sample sentence to obtain a second predicted probability of the j-th data. Thus, the initial model can be trained based on the first predicted probability and the second predicted probability of the j-th data in the second sample sentence to obtain a target model, and the target model can be used to generate a translated output sentence based on the input sentence. By implementing the above method, the attention paid to each word sample during the model training process can be dynamically adjusted by integrating the word evaluation index of the target end context information, so that the model pays more attention to some important words during training, which can be used to improve the online translation system, make text translation more accurate, and improve data processing accuracy.

[0058] Based on the above description, the embodiment of the present application further proposes another data processing method; in the embodiment of the present application, the data processing method is mainly described by taking a computer device executing the data processing method as an example, and this embodiment focuses on describing the process of training the initial model. Figure 4 As shown, the data processing method includes but is not limited to the following steps:

[0059] S401: Acquire a sample corpus, where the sample corpus includes at least one target sample corpus pair, and the target sample corpus pair includes a first sample sentence and a second sample sentence.

[0060] S402 : Perform probability prediction processing on the j-th data in the second sample sentence based on the first sample sentence and the first j-1 data in the second sample sentence to obtain a first prediction probability of the j-th data.

[0061] S403 , performing probability prediction processing on the j-th data of the second sample sentence according to the first j−1 data in the second sample sentence to obtain a second predicted probability of the j-th data.

[0062] The specific implementation of steps S401-S403 can refer to the specific description of steps S201-S203 in the above embodiment, which will not be repeated here.

[0063] S404: Determine a training weight for the jth data according to the first prediction probability and the second prediction probability of the jth data in the second sample sentence.

[0064] In one implementation, after determining the first predicted probability and the second predicted probability of the j-th data in the second sample sentence, the attention (or importance) of the j-th data in the initial model training process can be determined based on the two predicted probabilities. The attention can be represented by the training weight mentioned in this application. Among them, the training weight and the attention can be positively correlated, that is, when the training weight of a certain data is greater, the corresponding attention is higher, and when the training weight of a certain data is smaller, the corresponding attention is lower.

[0065] In one implementation, considering that the method for determining the training weight of each data in the second sample sentence is the same, this application uses a data in the second sample sentence as an example to explain the determination of the training weight of the data. The following mainly uses the j-th data in the second sample sentence as an example for explanation. The j-th data can be any data included in the second sample sentence. At the same time, it should be noted that for any target sample corpus pair in the sample corpus set, the model provided by this application processes any target sample corpus pair in the same way.

[0066] In one implementation, when determining the training weight of the jth data, the training weight of the jth data at the word level and the training weight at the sentence level can be used to jointly determine it, that is, when determining the training weight of each data, the attention of the data can be determined based on the features of two granularities (word level and sentence level) to improve the model effect of the model. Optionally, the specific implementation method of determining the training weight of the jth data can be described as follows: the training weight of the jth data at the word level can be determined based on the first predicted probability and the second predicted probability of the jth data in the second sample sentence, and the training weight can be referred to as the first training weight; the training weight of the second sample sentence where the jth data is located at the sentence level can also be determined based on the first predicted probability and the second predicted probability of each data in the j data of the second sample sentence, and the training weight can be referred to as the second training weight. After obtaining the first training weight of the jth data at the word level and the second training weight of the second sample sentence where the jth data is located at the sentence level, the training weight of the jth data can be determined based on the first training weight and the second training weight. For example, the product between the first training weight and the second training weight can be used as the training weight of the jth data.

[0067] For example, the training weight of the jth data can be expressed as follows:

[0068]

[0069] Among them, w j represents the training weight of the jth data, represents the first training weight of the jth data at the word level, w s Represents the second training weight at the sentence level for the second sample sentence where the j-th data is located.

[0070] In one implementation, in order to integrate the target end context information during the training process to improve the translation effect of the trained model, the present application proposes a machine translation adaptive training method based on conditional bilingual mutual information (CBMI), that is, using conditional bilingual mutual information as a word difficulty assessment indicator to perform word-level machine translation adaptive training. Optionally, in the present application, multi-granularity conditional bilingual mutual information can be used for adaptive weight calculation training, that is, using word-level conditional bilingual mutual information to determine the first training weight at the word level, and using sentence-level conditional bilingual mutual information to determine the second training weight at the sentence level.

[0071] The analysis shows that the conditional bilingual mutual information at the word level (abbreviated as word-level CBMI) can be derived from the formula of the conditional mutual information between the words in the source sentence (which can be understood as the sentence before translation, such as the first sample sentence) and the target sentence (which can be understood as the sentence after translation, such as the second sample sentence). The specific calculation process can be shown in the following formula (2):

[0072]

[0073] Among them, x represents the first sample sentence, y j Represents the jth data (word) in the second sample sentence, y <j Represents the first j-1 data (words) in the second sample sentence; CBMI t (x;y j ) represents the first conditional bilingual mutual information of the jth data, p NMT (y j ) represents the first predicted probability of the jth data, p LM (y j ) represents the second predicted probability of the j-th data.

[0074] It can be deduced from the above formula (2) that the initial formula in formula (2) can be simplified by decomposing the conditional joint distribution, and finally the word-level CBMI can be expressed as the logarithmic quotient of the predicted probability corresponding to the translation model (i.e., the first predicted probability output by the translation network) and the predicted probability corresponding to the language model (i.e., the second predicted probability output by the language network).

[0075] Similarly, analysis shows that the conditional bilingual mutual information at the sentence level (abbreviated as sentence-level CBMI) can be derived from the mutual information formula between the source sentence and the target sentence. The specific calculation process can be shown in the following formula (3):

[0076]

[0077] Among them, y represents the second sample sentence, CBMI s (x; y) represents the second conditional bilingual mutual information of the second sample sentence at the sentence level, |y| represents the amount of data included in the second sample sentence, and |y|=J.

[0078] It can be deduced from the above formula (3) that the conditional probability in the initial formula in formula (3) can be decomposed through the conditional probability formula, and the sentence-level CBMI can be finally expressed as the arithmetic mean of all word-level CBMIs in the sentence.

[0079] From the above description, we can also know that the conditional bilingual mutual information can be calculated through the translation model and the language model, so we can construct Figure 3 The initial model shown is used to train the translation model to implement a training method that uses multi-granularity conditional bilingual mutual information for adaptive weight calculation.

[0080] Based on the above analysis, it can be seen that the specific implementation method of determining the first training weight of the j-th data at the word level and the first training weight of the second sample sentence where the j-th data is located at the sentence level can be described as follows:

[0081] (1) A specific implementation method for determining the first training weight of the jth data at the word level may include: calculating the ratio between the first predicted probability and the second predicted probability of the jth data in the second sample sentence, and using the ratio to determine the first training weight of the jth data at the word level. For example, the ratio may be logarithmized to obtain the first conditional bilingual mutual information (which may be referred to as word-level CBMI) of the jth data at the word level. For example, the first conditional bilingual mutual information of the jth data may be as shown in the following formula (4):

[0082]

[0083] After obtaining the first conditional bilingual mutual information, the first conditional bilingual mutual information can be used to determine a first training weight of the j-th data at the word level.

[0084] Optionally, the first conditional bilingual mutual information can be determined as the first training weight of the j-th data at the word level, that is,

[0085] Optionally, considering that the fluctuations between the first conditional bilingual mutual information of each data in the second sample sentence may be large, such as the first conditional bilingual mutual information of some data in the second sample sentence may be large, while the first conditional bilingual mutual information of some data may be small, in order to reduce the volatility of the first conditional bilingual mutual information of each data, the first conditional bilingual mutual information of each data can be normalized. Among them, the normalization processing of the first conditional bilingual mutual information of each data at the word level is consistent. The following is an example of the normalization processing of the j-th data in the second sample sentence: the first conditional bilingual mutual information can be normalized first, so as to determine the normalized first conditional bilingual mutual information as the first training weight of the j-th data at the word level, that is, in, The first conditional bilingual mutual information of the jth data is normalized. Optionally, the normalization process can be implemented using a linear function normalization method, a zero-mean normalization method, etc., which is not specifically limited in this application.

[0086] For example, the linear function normalization method uses a linear function to convert the original data into the range of [0 1]. In this application, in order to implement the normalization operation within the sentence at the word level CBMI, the normalization formula corresponding to the linear function normalization method can be shown as formula (5):

[0087]

[0088] in, represents the minimum first conditional bilingual mutual information among all the data corresponding to the second sample sentence, Represents the maximum first conditional bilingual mutual information among all the data corresponding to the second sample sentence. For example, the second sample sentence is "my name is Peter", and all the data in the second sample sentence include: my, name, is, Peter, where the first conditional bilingual mutual information corresponding to each data is: P1, P2, P3, P4, and P1>P2>P3>P4. Then, for the second sample sentence, the maximum first conditional bilingual mutual information is P1, and the minimum first conditional bilingual mutual information is P4.

[0089] For example, the zero-mean normalization method normalizes the original data set to a data set with a mean of 0 and a variance of 1. The zero-mean normalization method can reduce the gap between the word-level CBMIs of each word included in a sentence to maintain a stable distribution between the word-level CBMIs of each word. In this application, in order to implement the normalization operation of the word-level CBMI within the sentence, the normalization formula corresponding to the zero-mean normalization method can be shown as formula (6):

[0090]

[0091] Among them, μ t and σ t Respectively represent the mean and variance of the first conditional bilingual mutual information corresponding to all data in the second sample sentence. It can be understood that the data can be understood as words, in other words, μ t and σ t Respectively represent the mean and variance of the word-level CBMI corresponding to all words in the second sample sentence. For example, the second sample sentence is "my name is Peter", and all the words in the second sample sentence include: my, name, is, Peter, among which the first conditional bilingual mutual information corresponding to each word is: P1, P2, P3, P4, then the mean μ can be determined based on the first conditional bilingual mutual information corresponding to these four words. t and variance σ t , such as the mean μ t =(P1+P2+P3+P4) / 4.

[0092] Optionally, after normalizing the first conditional bilingual mutual information, scaling can be performed to control the impact of the first conditional bilingual mutual information on model training. Finally, the first conditional bilingual mutual information obtained through the above two processes can be determined as the first training weight for the jth data at the word level. In a specific implementation, the first conditional bilingual mutual information can be normalized to obtain a first initial training weight for the jth data at the word level. After obtaining the first initial training weight, the first initial training weight can be scaled to obtain the first training weight for the jth data at the word level.

[0093] For example, the first training weight of the j-th data at the word level can be shown in the following formula (7):

[0094]

[0095] Among them, scale tRepresents the word-level scaling hyperparameter to control the impact of word-level CBMI on the model. The value is not limited. For example, it can be 0.1, 0.15, 0.3, etc. Experiments show that scale t When set to 0.1, the model works well. After the zero-mean normalization operation and scaling operation, the word-level training weights can be constrained to a limited range with a mean of 1.

[0096] (2) The specific implementation of determining the second training weight of the second sample sentence at the sentence level may include: first, determining the first conditional bilingual mutual information of each data at the word level among the j data, wherein the method for determining the first conditional bilingual mutual information of each data at the word level can refer to the above-mentioned determination of the first conditional bilingual mutual information of the j-th data at the word level, and will not be repeated here. After determining the first conditional bilingual mutual information corresponding to each data, the second training weight of the second sample sentence at the sentence level can be determined based on the first conditional bilingual mutual information corresponding to each data.

[0097] In one implementation, first, the second conditional bilingual mutual information of the second sample sentence at the sentence level can be determined based on the first conditional bilingual mutual information corresponding to each data point. For example, the sum of the first conditional bilingual mutual information of each data point at the word level can be calculated to determine the second conditional bilingual mutual information of the second sample sentence at the sentence level based on the sum. For example, the ratio of the sum to the number of data points included in the second sample sentence can be used as the second conditional bilingual mutual information of the second sample sentence at the sentence level, where the number of data points is the aforementioned J.

[0098] For example, the second conditional bilingual mutual information of the second sample sentence at the sentence level can be expressed as follows:

[0099]

[0100] Then, after determining the second conditional bilingual mutual information, a second training weight for the second sample sentence at the sentence level can be determined based on the second conditional bilingual mutual information. The method for determining the second training weight for the second sample sentence at the sentence level based on the second conditional bilingual mutual information can be similar to the method for determining the first training weight for the j-th data at the sentence level based on the first conditional bilingual mutual information. For example, the implementation method for determining the second training weight can be described as follows:

[0101] Optionally, the second conditional bilingual mutual information can be determined as the second training weight of the second sample sentence at the sentence level, that is, w s =CBMI s (x;y).

[0102] Optionally, considering that the fluctuations between the second conditional bilingual mutual information corresponding to each second sample sentence included in the sample corpus may be large, such as the second conditional bilingual mutual information of some second sample sentences may be large, while the second conditional bilingual mutual information of some second sample sentences may be small, in order to reduce the volatility of the second conditional bilingual mutual information of each second sample sentence, the second conditional bilingual mutual information of each second sample sentence may be normalized. Among them, the normalization processing of the second conditional bilingual mutual information of each second sample sentence at the sentence level is consistent. The following is an example of the normalization processing in a second sample sentence: the second conditional bilingual mutual information can be normalized first, so as to determine the normalized second conditional bilingual mutual information as the second training weight of the second sample sentence at the sentence level, that is, in, The normalized second conditional bilingual mutual information represents the second conditional bilingual mutual information. Optionally, the normalization process may be implemented using, for example, a linear function normalization method, a zero-mean normalization method, etc., which is not specifically limited in this application.

[0103] For example, in this application, in order to implement sentence-level CBMI to perform normalization operations between sentences, the normalization formula corresponding to the linear function normalization method can be shown as formula (9):

[0104]

[0105] in, Indicates the minimum second conditional bilingual mutual information among the second conditional bilingual mutual information corresponding to all second sample sentences included in the current batch during the training process of the initial model. represents the maximum second conditional bilingual mutual information among the second conditional bilingual mutual information corresponding to all second sample sentences.

[0106] It should be noted that, during the training process, if the target sample corpus pairs used in the current batch are all in the sample corpus, then the second sample sentences included in the current batch include: the second sample sentences of each target sample corpus pair in the sample corpus; if the target sample corpus pairs used in the current batch are part of the sample corpus, then the second sample sentences included in the current batch include: the second sample sentences of the target sample corpus pairs in the sample corpus. For example, if the sample corpus includes 7 target sample corpus pairs, and assuming that the current batch includes these 7 target sample corpus pairs, then Refers to the minimum second conditional bilingual mutual information among the second conditional bilingual mutual information of the 7 second sample sentences corresponding to the 7 target sample corpus pairs. For another example, assuming that the current batch includes 4 target sample corpus pairs, then It refers to the minimum second conditional bilingual mutual information among the second conditional bilingual mutual information of the four second sample sentences corresponding to the four target sample corpora. Correspondingly, You can refer to How to determine.

[0107] For example, in this application, the zero-mean normalization method can also be used to implement sentence-level CBMI to perform normalization operations between sentences. The zero-mean normalization method can reduce the gap between the sentence-level CBMIs of each second sample sentence included in the current batch to maintain a stable distribution between the sentence-level CBMIs of each second sample sentence. The normalization formula corresponding to the zero-mean normalization method can be shown as formula (10):

[0108]

[0109] Among them, μ s and σ s They represent the mean and variance of the second conditional bilingual mutual information corresponding to all the second sample sentences included in the current batch during the training process of the initial model, that is, μ t and σ t Respectively represent the mean and variance of the sentence-level CBMI corresponding to all target-side sentences in the current batch. For example, the current batch includes 5 second sample sentences, where the second conditional bilingual mutual information corresponding to each second sample sentence is: Q1, Q2, Q3, Q4, Q5, then μ can be determined based on the second conditional bilingual mutual information corresponding to these 5 second sample sentences. s and σ s , such as μ s =(Q1+Q2+Q3+Q4+Q5) / 5.

[0110] Optionally, after normalizing the second conditional bilingual mutual information, scaling can be performed to control its impact on model training. Finally, the second conditional bilingual mutual information obtained through these two processes can be determined as a second sentence-level training weight for the second sample sentence. In a specific implementation, the second conditional bilingual mutual information can be normalized to obtain a second initial sentence-level training weight for the second sample sentence. After obtaining this second initial training weight, the second initial training weight can be scaled to obtain a second sentence-level training weight for the second sample sentence.

[0111] For example, the second training weight of the second sample sentence at the sentence level can be shown in the following formula (11):

[0112]

[0113] Among them, scale s It represents the sentence-level scaling hyperparameter to control the impact of sentence-level CBMI on the model. The value is not limited. For example, it can be 0.1, 0.15, 0.3, etc. Experiments show that scale s When set to 0.1, the model works well. After the zero-mean normalization operation and scaling operation, the sentence-level training weights can be constrained to a limited range with a mean of 1.

[0114] S405, obtaining the target loss function of the initial model, and calculating the target loss value using the training weight of the j-th data and the target loss function.

[0115] S406: Train the initial model based on the target loss value to obtain a target model.

[0116] In step S405 and step S406, the computer device can obtain the target loss function corresponding to the initial model, and use the training weight of the j-th data and the target loss function to train the initial model to obtain the target model.

[0117] For example, the objective loss function can be expressed as follows:

[0118]

[0119] Among them, θ represents the model parameters corresponding to the translation network in the initial model.

[0120] Optionally, the target loss value can be calculated using the training weight of the jth data and the target loss function, so as to train the initial model based on the target loss value and obtain the target model. As shown in formula (12), the training weight of the jth data can be multiplied by the loss corresponding to the jth data to obtain the final adaptive loss of the jth data, and then backpropagation can be performed to train the initial model.

[0121] To better understand the data processing method provided in the embodiment of the present application, Figure 5 The structural diagram of the training initial model is further explained as shown in Figure 5As shown in the figure, the process shown in this figure can be an adaptive training process based on conditional bilingual mutual information proposed in this application. Among them, the two solid matrices represent the neural machine translation model (i.e., the translation network mentioned above) and the neural language model on the target end (i.e., the language network mentioned above); the two solid rounded rectangles represent the word-level conditional bilingual mutual information and sentence-level conditional bilingual mutual information proposed in this application; the dotted rounded rectangle represents the loss of the adaptively adjusted translation model, which is used to update the model parameters of the translation model.

[0122] based on Figure 5 The data processing method proposed in this application may include the following steps:

[0123] 1. Compare the source sentence x (i.e. the first sample sentence mentioned above) with the target sentence prefix y <j (i.e. the first j-1 data in the second sample sentence mentioned above) is input into the translation model, and the prefix of the target sentence is input into the target language model, and the two output their respective responses to the current word y j The predicted probability (i.e., the first predicted probability and the second predicted probability mentioned above);

[0124] 2. Calculate the corresponding word-level CBMI based on the predicted probabilities output by the two models;

[0125] 3. Calculate the sentence-level CBMI by averaging the word-level CBMIs corresponding to all words in the entire sentence on the target end;

[0126] 4. The two levels of CBMI are normalized and scaled at their respective granularities to form training weights at their respective granularities. Finally, the two training weights are multiplied together to obtain the final word training weight, which is then multiplied by the loss corresponding to the word for backpropagation to achieve training of the translation model.

[0127] It can be seen that the data processing method provided in this application can be roughly understood as an adaptive training method that utilizes conditional bilingual mutual information. Specifically, this method can be a method for measuring word importance in machine translation based on conditional bilingual mutual information. That is, the importance of words during training is measured based on conditional bilingual mutual information to improve the model performance of the translation model. Optionally, the conditional bilingual mutual information can be determined using the translation model and language model involved in the training process provided in this application.

[0128] In an embodiment of the present application, a machine translation adaptive training method based on context-related conditional bilingual mutual information can be used to train a translation model, wherein the word evaluation index in the training process, namely the conditional bilingual mutual information, can well integrate the target-side context information. At the same time, the index can be calculated in real time by the model during the training process to reduce the computational and storage costs of introducing context information. By integrating the word evaluation index of the target-side context information, the attention paid to each word sample during the model training process can also be dynamically adjusted, so that the model pays more attention to some important words during training, which can be used to improve the online translation system, make text translation more accurate, and improve data processing accuracy.

[0129] In one implementation, in order to demonstrate the model effect (e.g., translation effect) of the target model obtained using the data processing method provided in this application, the target model can also be tested. For example, the translation effect of the target model obtained in this application can be compared with the translation effects of other translation models. The data shown in Table 1 can be the translation effects of the translation models obtained using different training methods.

[0130] Table 1:

[0131] Model WMT14 EN-DE WMT19 ZH-EN Transformer 28.02 24.94 Transformer+Word Frequency 1 28.28(+0.26) 24.76(-0.18) Transformer+Word Frequency 2 28.24(+0.22) 25.08(+0.14) Transformer+BMI 28.46(+0.44) 25.24(+0.30) Transformer + this application 28.90(+0.88) 25.65(+0.69)

[0132] In one implementation, the test data is the English-German (En-De) dataset from the 14th International Machine Translation Contest (WMT14) and the Chinese-English (ZH-EN) dataset from the 19th International Machine Translation Contest (WMT19). Translation performance can be evaluated using the Bilingual Evaluation Understudy (BLEU). The data shown in columns 2 and 3 of Table 1 are BLEU metrics. BLEU is a method for measuring the similarity between texts and is commonly used to evaluate translation quality. In other words, BLEU can represent a translation evaluation metric, which is generally a standard method for machine translation evaluation. Higher BLEU values ​​indicate better translation performance.

[0133] In one implementation, in the translation effect test, the translation models used are all Transformer models, the difference being the training methods of the translation models. As shown in Table 1, "Transformer" represents a Transformer model obtained based on the traditional training method; "Transformer+Word Frequency 1" and "Transformer+Word Frequency 2" represent Transformer models obtained by the word frequency-based machine translation adaptive training method. These two models use the word frequency information of the target words as a word difficulty measurement indicator to perform word-level machine translation adaptive training. "Transformer+Word Frequency 1" represents a Transformer model trained by adjusting the training weights according to word frequency 1; "Transformer+Word Frequency 2" represents a Transformer model trained by adjusting the training weights according to word frequency 2.

[0134] For example, the training weight of “Transformer+word frequency 1” during training can be an exponential weight, and its specific form can be shown in the following formula (13):

[0135]

[0136] For example, the training weight of “Transformer+Word Frequency 2” during the training process can be a chi-square weight, and its specific form can be shown in the following formula (14):

[0137]

[0138] In formula (13) and formula (14), A and T are hyperparameters, Count() is the counting function, and y k Represents the kth word in the target sentence.

[0139] In one implementation, "Transformer+BMI" in Table 1 represents a Transformer model obtained by a machine translation adaptive training method based on bilingual mutual information (BMI). This training method can use bilingual mutual information (BMI) as a word difficulty assessment indicator to perform word-level machine translation adaptive training. The training weights in the training process can be BMI-type weights, that is, the training weights are BMI, where the specific calculation method of BMI can be shown in the following formula (15):

[0140]

[0141] Among them, x and x iRepresents the source sentence in the bilingual sentence pair and the i-th word in the source sentence, y j represents the jth word of the target sentence, f() is the counting function, and K represents the total number of bilingual sentence pairs.

[0142] In one implementation, “Transformer+this application” in Table 1 represents a Transformer model trained based on the data processing method of this application (i.e., a machine translation adaptive training method based on conditional bilingual mutual information).

[0143] As can be seen from Table 1, the translation effect of the Transformer model obtained by using the training method provided in this application is significantly improved compared with the Transformer models obtained by other training methods in Table 1. That is, the Transformer model obtained by this application shows better translation effect than the traditional Transformer model and the commonly used word-level adaptive training method. This also proves the effectiveness of the translation model trained by using the conditional bilingual mutual information method in this application.

[0144] From the above description, we can see that "Transformer + Word Frequency 1" and "Transformer + Word Frequency 2" can effectively improve the translation effect of low-frequency words on the target side compared to "Transformer". Compared with "Transformer + Word Frequency 1" and "Transformer + Word Frequency 2", "Transformer + BMI" can further take into account bilingual information. Compared with the Transformer models obtained by the first three training methods, it is also more suitable as a word evaluation indicator for translation tasks. However, the indicators applied in the training process of the translation model by these methods lack contextual information on the target side. That is, the indicators applied are based on context-independent word statistics, which makes it difficult to accurately evaluate each word in the target sentence. If the translation model obtained by the other training methods mentioned above performs context-dependent statistical calculations, it will introduce a large amount of computational and storage costs, making it difficult to implement in actual use. The word evaluation indicator (conditional bilingual mutual information) in the word-level adaptive training method for machine translation based on conditional bilingual mutual information proposed in this application can perform efficient context-related calculations during the training process, enabling it to integrate target-side context information and achieve real-time calculations through the model during the training process, thereby reducing the computational and storage costs of introducing context information and improving the effect of machine translation. Ultimately, this application is able to outperform existing word-level adaptive training methods in two WMT machine translation evaluation tasks.

[0145] See also Figure 6 , Figure 6: is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The data processing device described in this embodiment includes:

[0146] An acquisition unit 601 is configured to acquire a sample corpus, the sample corpus including at least one target sample corpus pair, the target sample corpus pair including a first sample sentence and a second sample sentence, the second sample sentence being a sentence translated from the first sample sentence, the second sample sentence including J data, where J is a positive integer;

[0147] A first determining unit 602 is configured to perform probability prediction processing on the j-th data in the second sample sentence based on the first sample sentence and the first j-1 data in the second sample sentence to obtain a first predicted probability of the j-th data, j∈[1, J];

[0148] A second determining unit 603 is configured to perform probability prediction processing on the j-th data of the second sample sentence based on the first j-1 data in the second sample sentence to obtain a second predicted probability of the j-th data;

[0149] A training unit 604 is configured to train the initial model according to the first predicted probability and the second predicted probability of the j-th data in the second sample sentence to obtain a target model;

[0150] The translation unit 605 is configured to obtain an input sentence to be translated, input the input sentence into the target model, and generate a translated output sentence.

[0151] In one implementation, the training unit 604 is specifically configured to:

[0152] Determining a training weight for the j-th data according to the first predicted probability and the second predicted probability of the j-th data in the second sample sentence;

[0153] Obtaining the target loss function of the initial model, and calculating the target loss value using the training weight of the j-th data and the target loss function;

[0154] The initial model is trained based on the target loss value to obtain a target model.

[0155] In one implementation, the training unit 604 is specifically configured to:

[0156] Determining a first training weight of the j-th data at a word level according to the first predicted probability and the second predicted probability of the j-th data in the second sample sentence;

[0157] determining a second training weight for the second sample sentence at a sentence level according to the first predicted probability and the second predicted probability of each data item in the j data items of the second sample sentence;

[0158] The training weight of the j-th data is determined according to the first training weight and the second training weight.

[0159] In one implementation, the training unit 604 is specifically configured to:

[0160] Calculating a ratio between a first predicted probability and a second predicted probability of the j-th data in the second sample sentence;

[0161] Performing logarithmic processing on the ratio to obtain the first conditional bilingual mutual information of the j-th data at the word level;

[0162] The first conditional bilingual mutual information is used to determine a first training weight of the j-th data at a word level.

[0163] In one implementation, the training unit 604 is specifically configured to:

[0164] Determine the first conditional bilingual mutual information of each data at the word level in the j data, and calculate the sum of the first conditional bilingual mutual information of each data at the word level;

[0165] Determining second conditional bilingual mutual information of the second sample sentence at a sentence level according to the sum value;

[0166] The second conditional bilingual mutual information is used to determine a second training weight of the second sample sentence at a sentence level.

[0167] In one implementation, the training unit 604 is specifically configured to:

[0168] Normalizing the first conditional bilingual mutual information to obtain a first initial training weight of the j-th data at a word level;

[0169] The first initial training weight is scaled to obtain a first training weight of the j-th data at the word level.

[0170] In one implementation, the first predicted probability is obtained by performing probability prediction processing on the j-th data in the second sample sentence through the translation network in the initial model; the second predicted probability is obtained by performing probability prediction processing on the j-th data in the second sample sentence through the language network in the initial model; and the target model includes the translation network in the trained initial model.

[0171] It is understood that the division of units in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, other division methods may be used. The functional units in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0172] See also Figure 7 , Figure 7 7 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device includes: a processor 701 and a memory 702. Optionally, the computer device may also include a network interface 703. The processor 701, memory 702, and network interface 703 can exchange data.

[0173] The processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0174] The memory 702 may include a read-only memory and a random access memory, and provides program instructions and data to the processor 701. A portion of the memory 702 may also include a non-volatile random access memory. When the processor 701 calls the program instructions, it is used to execute:

[0175] Acquire a sample corpus, the sample corpus including at least one target sample corpus pair, the target sample corpus pair including a first sample sentence and a second sample sentence, the second sample sentence being a sentence translated from the first sample sentence, the second sample sentence including J data, where J is a positive integer;

[0176] Perform probability prediction processing on the j-th data in the second sample sentence based on the first sample sentence and the first j-1 data in the second sample sentence to obtain a first prediction probability of the j-th data, j∈[1,J];

[0177] Performing probability prediction processing on the j-th data of the second sample sentence based on first j-1 data in the second sample sentence to obtain a second predicted probability of the j-th data;

[0178] Training the initial model according to the first predicted probability and the second predicted probability of the j-th data in the second sample sentence to obtain a target model;

[0179] An input sentence to be translated is obtained, the input sentence is input into the target model, and a translated output sentence is generated.

[0180] In one implementation, the processor 701 is specifically configured to:

[0181] Determining a training weight for the j-th data according to the first predicted probability and the second predicted probability of the j-th data in the second sample sentence;

[0182] Obtaining the target loss function of the initial model, and calculating the target loss value using the training weight of the j-th data and the target loss function;

[0183] The initial model is trained based on the target loss value to obtain a target model.

[0184] In one implementation, the processor 701 is specifically configured to:

[0185] Determining a first training weight of the j-th data at a word level according to the first predicted probability and the second predicted probability of the j-th data in the second sample sentence;

[0186] determining a second training weight for the second sample sentence at a sentence level according to the first predicted probability and the second predicted probability of each data item in the j data items of the second sample sentence;

[0187] The training weight of the j-th data is determined according to the first training weight and the second training weight.

[0188] In one implementation, the processor 701 is specifically configured to:

[0189] Calculating a ratio between a first predicted probability and a second predicted probability of the j-th data in the second sample sentence;

[0190] Performing logarithmic processing on the ratio to obtain the first conditional bilingual mutual information of the j-th data at the word level;

[0191] The first conditional bilingual mutual information is used to determine a first training weight of the j-th data at a word level.

[0192] In one implementation, the processor 701 is specifically configured to:

[0193] Determine the first conditional bilingual mutual information of each data at the word level in the j data, and calculate the sum of the first conditional bilingual mutual information of each data at the word level;

[0194] Determining second conditional bilingual mutual information of the second sample sentence at a sentence level according to the sum value;

[0195] The second conditional bilingual mutual information is used to determine a second training weight of the second sample sentence at a sentence level.

[0196] In one implementation, the processor 701 is specifically configured to:

[0197] Normalizing the first conditional bilingual mutual information to obtain a first initial training weight of the j-th data at a word level;

[0198] The first initial training weight is scaled to obtain a first training weight of the j-th data at the word level.

[0199] In one implementation, the first predicted probability is obtained by performing probability prediction processing on the j-th data in the second sample sentence through the translation network in the initial model; the second predicted probability is obtained by performing probability prediction processing on the j-th data in the second sample sentence through the language network in the initial model; and the target model includes the translation network in the trained initial model.

[0200] The embodiment of the present application further provides a computer storage medium in which program instructions are stored. When the program is executed, the program may include: Figure 2 or Figure 4 Part or all of the steps of the data processing method in the corresponding embodiment.

[0201] It should be noted that for the aforementioned various method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0202] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0203] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps performed in the above-described method embodiments.

[0204] The above is a detailed introduction to a data processing method, device, computer equipment and storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data processing method, characterized in that: include: Acquire a sample corpus, the sample corpus including at least one target sample corpus pair, the target sample corpus pair including a first sample sentence and a second sample sentence, the second sample sentence being a sentence translated from the first sample sentence, the second sample sentence including J data, where J is a positive integer; Perform probability prediction processing on the j-th data in the second sample sentence based on the first sample sentence and the first j-1 data in the second sample sentence to obtain a first prediction probability of the j-th data, j∈[1,J]; Performing probability prediction processing on the j-th data of the second sample sentence based on first j-1 data in the second sample sentence to obtain a second predicted probability of the j-th data; Calculating a ratio between a first predicted probability and a second predicted probability of the j-th data in the second sample sentence; Performing logarithmic processing on the ratio to obtain first conditional bilingual mutual information of the j-th data at the word level; and determining a first training weight of the j-th data at the word level using the first conditional bilingual mutual information; determining a second training weight for the second sample sentence at a sentence level according to the first predicted probability and the second predicted probability of each data item in the j data items of the second sample sentence; Determining the training weight of the j-th data according to the first training weight and the second training weight; Obtaining a target loss function of the initial model, calculating a target loss value using the training weight of the j-th data and the target loss function, and training the initial model based on the target loss value to obtain a target model; An input sentence to be translated is obtained, the input sentence is input into the target model, and a translated output sentence is generated.

2. The method according to claim 1, characterized in that The determining, based on the first predicted probability and the second predicted probability of each data item in the j data items of the second sample sentence, a second training weight for the second sample sentence at the sentence level includes: Determine the first conditional bilingual mutual information of each data at the word level in the j data, and calculate the sum of the first conditional bilingual mutual information of each data at the word level; Determining second conditional bilingual mutual information of the second sample sentence at a sentence level according to the sum value; The second conditional bilingual mutual information is used to determine a second training weight of the second sample sentence at a sentence level.

3. The method according to claim 1, characterized in that The determining of a first training weight of the j-th data at a word level by using the first conditional bilingual mutual information includes: Normalizing the first conditional bilingual mutual information to obtain a first initial training weight of the j-th data at a word level; The first initial training weight is scaled to obtain a first training weight of the j-th data at the word level.

4. The method according to any one of claims 1 to 3, characterized in that The first predicted probability is obtained by performing probability prediction processing on the j-th data in the second sample sentence through the translation network in the initial model; the second predicted probability is obtained by performing probability prediction processing on the j-th data in the second sample sentence through the language network in the initial model; and the target model includes the translation network in the trained initial model.

5. A data processing device, characterized in that: include: an acquiring unit, configured to acquire a sample corpus, the sample corpus comprising at least one target sample corpus pair, the target sample corpus pair comprising a first sample sentence and a second sample sentence, the second sample sentence being a sentence translated from the first sample sentence, the second sample sentence comprising J data, where J is a positive integer; A first determining unit is configured to perform probability prediction processing on the j-th data in the second sample sentence based on the first sample sentence and the first j-1 data in the second sample sentence to obtain a first predicted probability of the j-th data, j∈[1, J]; a second determining unit, configured to perform probability prediction processing on the j-th data of the second sample sentence based on first j-1 data in the second sample sentence, to obtain a second predicted probability of the j-th data; a training unit, configured to calculate a ratio between a first predicted probability and a second predicted probability of the j-th data in the second sample sentence; Performing logarithmic processing on the ratio to obtain first conditional bilingual mutual information of the j-th data at the word level; and determining a first training weight of the j-th data at the word level using the first conditional bilingual mutual information; Determining a second training weight for the second sample sentence at a sentence level based on the first predicted probability and the second predicted probability of each of the j data in the second sample sentence; determining a training weight for the j-th data based on the first training weight and the second training weight; Obtaining a target loss function of the initial model, calculating a target loss value using the training weight of the j-th data and the target loss function, and training the initial model based on the target loss value to obtain a target model; The translation unit is used to obtain an input sentence to be translated, input the input sentence into the target model, and generate a translated output sentence.

6. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 4.

8. A computer program product, characterized in that The computer program product comprises computer instructions, and the computer instructions are executed by a processor to implement the method according to any one of claims 1 to 4.

Images