Sentence translation method, device, equipment and medium

Abstract

The present application provides a sentence translation method, device, equipment and medium, which relates to the field of machine translation technology and is used to improve the fidelity of translation results. The method includes: simulating the overconfident translation process of the first translation model with the help of a first translation auxiliary model, thereby quantifying the fidelity training loss of the first translation model according to the difference between the probabilities output by the first translation auxiliary model and the first translation model, and directly and effectively adjusting the model parameters of the first translation model based on the fidelity training loss, thereby improving the translation fidelity of the translation sentence output by the trained target translation model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, in particular to the field of machine translation technology, and provides a sentence translation method, device, equipment and medium. Background Art

[0002] Translation models are widely used in the field of automatic translation. Translation models can essentially be understood as a joint language model, which predicts the next element based on the sentence to be translated and the translated text until the translation result of the sentence to be translated is obtained.

[0003] At present, the fidelity of the translation model is an important indicator for evaluating the training effect of the translation model. Fidelity refers to the degree of consistency between the translation result output by the translation model and the sentence to be translated. How to improve the fidelity of the translation model is an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide a sentence translation method, apparatus, device and medium for improving the fidelity of translation results.

[0005] On the one hand, a sentence translation method is provided, comprising:

[0006] Obtain a target sentence to be translated;

[0007] Inputting the target sentence into the trained target translation model to obtain a target translation sentence;

[0008] The target translation model is obtained by adjusting the parameters of the first translation model to be trained based on a fidelity training loss value during the training of the first translation model to be trained using a first sample training data set. Each first sample training data includes a first sample sentence to be translated and a corresponding first sample translation sentence. The fidelity training loss value is obtained based on the difference between the first probability and the second probability corresponding to each output position. The first probability corresponding to each output position is the output probability of the first target element obtained based on the first translation model and the corresponding first sample training data. The second probability corresponding to each output position is the output probability of the first target element obtained based on the first translation auxiliary model and the corresponding first sample translation sentence.

[0009] On the one hand, a sentence translation device is provided, comprising:

[0010] A sentence acquisition module, used to obtain a target sentence to be translated;

[0011] A sentence translation module, used for inputting the target sentence into a trained target translation model to obtain a target translation sentence;

[0012] The target translation model is obtained by adjusting the parameters of the first translation model to be trained based on a fidelity training loss value during the training of the first translation model to be trained using a first sample training data set. Each first sample training data includes a first sample sentence to be translated and a corresponding first sample translation sentence. The fidelity training loss value is obtained based on the difference between the first probability and the second probability corresponding to each output position. The first probability corresponding to each output position is the output probability of the first target element obtained based on the first translation model and the corresponding first sample training data. The second probability corresponding to each output position is the output probability of the first target element obtained based on the first translation auxiliary model and the corresponding first sample translation sentence.

[0013] In a possible embodiment, the device further includes a model training module, wherein the model training module is specifically configured to train a target translation model obtained by:

[0014] Based on the first sample training data set, the first translation model to be trained is trained multiple times until a first model convergence condition is met, and the first translation model that meets the first preset condition in the multiple times of iterative training is used as the target translation model, wherein one iterative training includes the following operations:

[0015] Selecting first sample training data from the first sample training data set;

[0016] Inputting the first sample training data into the first translation model to obtain first probabilities corresponding to respective output positions, wherein the first probability corresponding to each output position is determined based on the first sample sentence to be translated in the first sample training data and the historical translation text associated with the corresponding output position, wherein each element in the first sample translation sentence in the first sample training data corresponds to each output position one by one, and the historical translation text is the text content in the first sample translation sentence that is located before the element corresponding to the corresponding output position;

[0017] Inputting the first sample translation sentence into the first translation auxiliary model to obtain second probabilities corresponding to the respective output positions, wherein the second probability corresponding to each output position is determined based on the historical translation text associated with the corresponding output position;

[0018] Obtaining a fidelity training loss value based on a difference between the first probability and the second probability corresponding to each of the output positions;

[0019] Based on the fidelity training loss value, parameters of the first translation model are adjusted.

[0020] In a possible embodiment, the model training module is specifically used for:

[0021] Based on the first translation model, encoding the first sample sentence to be translated in the first sample training data to obtain an encoding result;

[0022] Based on the first translation model, the following operations are performed for each output position:

[0023] For one output position, based on the encoding result, decoding the historical translation text associated with the one output position to obtain a first decoding result of the one output position;

[0024] A first decoding result of the one output position is normalized to obtain a first probability corresponding to the one output position.

[0025] In a possible embodiment, the model training module is specifically used for:

[0026] Based on the first translation auxiliary model, the following operations are performed for each output position:

[0027] Decoding the historical translation text associated with an output position to obtain a second decoding result of the output position;

[0028] The second decoding result of the one output position is normalized to obtain a second probability corresponding to the one output position.

[0029] In a possible embodiment, the model training module is specifically used for:

[0030] Determine an element-level training loss value based on a probability difference between a first probability and a second probability corresponding to each of the output positions, wherein the element-level training loss value is negatively correlated with the probability difference of each of the output positions;

[0031] Based on the probability difference between the first probability and the second probability corresponding to each of the output positions, a sentence-level training loss value is determined, wherein if the first ratio is greater than a ratio threshold, the sentence-level loss value is zero, and if the first ratio is less than or equal to the ratio threshold, the sentence-level loss value is negatively correlated with the probability difference of each output position, and the first ratio is: based on the probability difference of each output position, the ratio of the number of elements having a probability difference less than the difference threshold to the total number of each element in the first sample translation sentence;

[0032] The fidelity training loss value is obtained based on at least one of the element-level training loss value and the sentence-level training loss value.

[0033] In a possible embodiment, the model training module is specifically used for:

[0034] For each output location, perform the following operations respectively:

[0035] Substituting the probability difference of an output position into a preset function to obtain a first training loss value of the output position, wherein the preset function is a monotonically decreasing function;

[0036] Determining a weight of the one output position based on the first probability corresponding to the one output position, wherein the weight of the one output position is negatively correlated with the first probability corresponding to the one output position;

[0037] Based on the weight of the one output position, weighting the first training score loss value to obtain a second training score loss value of the one output position;

[0038] Based on the obtained second training loss values ​​corresponding to the respective output positions, the element-level training loss value is obtained.

[0039] In a possible embodiment, the model training module is specifically used for:

[0040] Based on at least one of the element-level training loss value and the sentence-level training loss value, obtaining an initial fidelity training loss value;

[0041] The fidelity training loss value is determined based on the initial fidelity training loss value and the initial translation training loss value, wherein the initial translation training loss value is determined based on the first probability of each output position and is used to represent an error between the translation sentence predicted by the first translation model and the first sample translation sentence.

[0042] In a possible embodiment, the model training module is further used for:

[0043] Before adjusting the parameters of the first translation model based on the fidelity training loss value, determining a first translation training loss value of the first translation model based on the first probabilities corresponding to the respective output positions, the first translation training loss value representing an error between the predicted translation sentence and the first sample translation sentence;

[0044] Determining a second translation training loss value of the first translation auxiliary model based on the second probabilities corresponding to the respective output positions, wherein the second translation training loss value represents an error between the predicted translation sentence and the first sample translation sentence;

[0045] Performing a weighted summation on the first translation training loss value and the second translation training loss value to obtain a first joint training loss value;

[0046] When adjusting the parameters of the first translation model based on the fidelity training loss value, the parameters of the first translation model and the first translation auxiliary model are adjusted based on the first joint training loss value.

[0047] In a possible embodiment, the model training module is further used for:

[0048] Before the first translation model to be trained is trained multiple times iteratively, based on the second sample training data set, the second translation model to be trained and the second translation auxiliary model to be trained are subjected to multiple rounds of joint pre-training until the second model convergence condition is met, the second translation model that meets the second preset condition in the multiple rounds of joint pre-training is used as the first translation model, and the second translation auxiliary model that meets the third preset condition in the multiple rounds of joint pre-training is used as the first translation auxiliary model, each second sample training data includes a second sample sentence to be translated and a corresponding second sample translation sentence, and each round of joint pre-training includes the following operations:

[0049] Selecting second sample training data from the second sample training data set;

[0050] Input the second sample training data into the second translation model to obtain third probabilities corresponding to respective output positions, wherein the third probability corresponding to each output position is the output probability of the second target element at the corresponding output position;

[0051] Inputting the second sample translation sentence in the second sample training data into the second translation auxiliary model to obtain a fourth probability, wherein the fourth probability of each output position is the output probability of the second target element at the corresponding output position;

[0052] Determining a third translation training loss value of the second translation model based on the third probabilities corresponding to the respective output positions, wherein the third translation training loss value represents an error between the predicted translation sentence and the second sample translation sentence;

[0053] Determining a fourth translation training loss value of the second translation auxiliary model based on the fourth probabilities corresponding to the respective output positions, wherein the fourth translation training loss value represents an error between the predicted translation sentence and the second sample translation sentence;

[0054] Performing a weighted summation on the third translation training loss value and the fourth translation training loss value to determine a second joint training loss value;

[0055] Based on the second joint training loss value, parameters of the second translation model and the second translation auxiliary model are adjusted respectively.

[0056] In a possible embodiment, the second translation model at least includes the same structure as the second translation auxiliary model, and the second translation model shares parameters with the same structure as the second translation auxiliary model.

[0057] An embodiment of the present application provides a computer device, including:

[0058] at least one processor, and

[0059] a memory communicatively coupled to the at least one processor;

[0060] The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements any of the sentence translation methods discussed above by executing the instructions stored in the memory.

[0061] An embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores computer instructions. When the computer instructions are executed on a computer, the computer executes any of the sentence translation methods discussed above.

[0062] Since the embodiment of the present application adopts the above technical solution, it has at least the following technical effects:

[0063] In an embodiment of the present application, when training the first translation model, a first translation auxiliary model is introduced, and the first translation auxiliary model is completely output according to the historical translation text, that is, the first translation auxiliary model is equivalent to simulating the process of translation in the first translation model that relies on the historical translation text, so that the difference between the first probability of the first translation model at each output position and the second probability of the first translation auxiliary model at the corresponding output position is equivalent to representing the translation fidelity of the first translation model, thereby determining the fidelity training loss value of the first translation model according to the difference, and adjusting the translation fidelity of the first translation model according to the fidelity training loss value, which can effectively and directly improve the translation fidelity of the first translation model, thus improving the translation fidelity of the target translation model, thereby improving the translation effect of the translation sentence obtained by the target translation model. Further, since the translation result output by the target translation model is more faithful to the original text, the translation fluency of the target translation model can also be improved to a certain extent. Moreover, an embodiment of the present application provides a method for quantifying the fidelity training loss of a translation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of the structure of a transformer model provided in an embodiment of the present application;

[0065] Figure 2 A method flow chart of a sentence translation method provided in an embodiment of the present application;

[0066] Figure 3 A flowchart of a sentence translation method provided in an embodiment of the present application;

[0067] Figure 4 A process example diagram of a sentence translation method provided in an embodiment of the present application;

[0068] Figure 5 A flow chart of a method for training a first translation model provided in an embodiment of the present application;

[0069] Figure 6 An example diagram of alignment of a first sample translation sentence and various output positions provided in an embodiment of the present application;

[0070] Figure 7 A schematic diagram of the structure of a first translation model provided in an embodiment of the present application;

[0071] Figure 8 A schematic diagram of the structure of a first translation assistance model provided in an embodiment of the present application;

[0072] Fig. 9 Provided for the embodiments of this application Figure 8 A schematic diagram of the structure of the second decoding module;

[0073] Fig.10 A flow chart of a method for determining element level loss value provided in an embodiment of the present application;

[0074] Fig.11 A flow chart of a joint pre-training method provided in an embodiment of the present application;

[0075] Fig.12 A schematic diagram of an interaction process between a server and a terminal provided in an embodiment of the present application;

[0076] Fig.13 An example diagram of a translation process of a terminal during a translation process provided in an embodiment of the present application;

[0077] Fig.14 A schematic diagram of the structure of a sentence translation device provided in an embodiment of the present application;

[0078] Fig.15 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to better understand the technical solution provided by the embodiments of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0080] In order to help those skilled in the art better understand the technical solution of the present application, the terms involved in the present application are introduced below.

[0081] 1. Sentences to be translated and translated sentences: Sentences to be translated refer to sentences that need to be translated, and translated sentences refer to the translation results of sentences to be translated. The translated sentences in this application can also be called translation results. It should be noted that sentences to be translated and translated sentences are relative concepts. For example, when translating sentence A, the translated sentence B is the translated sentence; when translating sentence B, the translated sentence A is the translated sentence. For the sake of ease of description, when training a translation model, the sentences to be translated and translated sentences used can be respectively referred to as sample sentences to be translated and sample translated sentences.

[0082] 2. Translation model: refers to the model used for text translation. The translation model is essentially a joint language model that predicts the next word based on the historical translation results in the sentences to be translated and the translated sentences. For example, the Neural Machine Translation (NMT) model is a statistical model that uses a neural network model to learn machine translation. Its main advantage is that a system can be trained directly through the sentences to be translated and the translated sentences, and no special statistical machine learning system is required.

[0083] 3. Translation auxiliary model: refers to other models used in the process of training the translation model. For example, the language model (LM) is an important technology for natural language processing. The most common data in natural language processing is text data. A natural language text can be regarded as a discrete time series. Assuming that the words in a text of length T are w1, w2, ..., wT, then in the discrete time series, wt (1≤t≤T) can be regarded as the output or label of the model at time step t.

[0084] 4. Transformer: It is a neural network framework, including an encoding part and a decoding part. The encoding part includes multiple encoders connected in sequence, and the decoding part includes multiple decoders connected in sequence. Each encoder and decoder includes a self-attention layer, and the self-attention layer can adopt a multi-head attention mechanism. The use of the attention mechanism can make the model fully interact with the input sentence, effectively capture the global information of the input sentence, and obtain a rich context representation.

[0085] Please refer to Figure 1, is a model structure of a transformer model, the transformer model includes an encoding part and a decoding part, the encoding part includes multiple encoders, and the decoding part includes multiple decoders, Figure 1 In the example, the encoding part includes 3 encoders and the decoding part includes 3 decoders, but in practice, the number of encoders included in the encoding part and the number of decoders included in the decoding part are not limited. The encoding output of the last encoder in the encoding part can be input to each decoder in the decoding part, and each decoder decodes the decoding output of the previous decoder based on the encoding output of the last encoder.

[0086] 5. Fluency: The fluency of the translation result in the target language environment.

[0087] 6. Fidelity: The degree to which the sentence to be translated is fully expressed, which can be further understood as the degree to which the translation result contains the language information in the sentence to be translated.

[0088] 7. Hallucination: It refers to the translation results that are generally fluent but do not match the original content. The degree of deviation between the translation results and the original content can be called the degree of overconfidence, or the degree of overconfidence.

[0089] 8. Fidelity training loss value: It is a measurement indicator or optimization target of the fidelity of the translation model. The fidelity training loss value of the translation model is closely related to the fidelity and fluency of the translation model. The smaller the fidelity training loss value, to a certain extent, it indicates that at least one of the fidelity and fluency of the translation model is too small; the larger the fidelity training loss value of the translation model, to a certain extent, it can indicate that at least one of the fidelity and fluency of the translation model is too large.

[0090] 9. Cloud technology: A general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technical network system require a large amount of computing and storage resources, such as video websites, image websites, and more portal websites. With the rapid development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the backend system for logical processing. Data of different levels will be processed separately, and all kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0091] 10. Cloud computing: It is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0092] 11. Artificial Intelligence (AI): It is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0093] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0094] 12. Natural Language Processing (NLP): It is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0095] It should be noted that, in the embodiments of the present application, “at least one” refers to one or more than one, and “plurality” refers to two or more than two.

[0096] At present, the translation model pays too much attention to the translated results, resulting in poor fidelity of the translated sentences. For example, the sentence to be translated is "Chinese citizens San Zhang and Si Li lost contact during their trip to A", and the output result of the translation model is "Chinese citizens San Zhang and Si Li lost their ties during a trip to A, China said on Tuesday". It can be seen that the fluency of the translated sentence is good, but the "China said on Tuesday" in the translated sentence is actually fabricated, which makes it impossible to correctly express the meaning of the sentence to be translated.

[0097] In view of this, the embodiments of the present application provide a sentence translation method, apparatus, device and medium. The design concept of the sentence translation method in the embodiments of the present application is introduced below.

[0098] In the embodiment of the present application, when training the translation model, an auxiliary translation model is directly introduced. The auxiliary translation model completely relies on sample translation sentences for output, which is equivalent to measuring the overconfidence of the language model in the translation model to be trained, thereby quantifying the fidelity training loss value of the translation model to be trained according to the difference between the output probability of the model to be translated and the output probability of the auxiliary translation model. Based on the fidelity training loss value, the parameters of the translation model are adjusted, which can increase the attention of the translation model to the sentences to be translated, thereby improving the fidelity of the translated sentences. Moreover, while being faithful to the sentences to be translated, it can also improve the fluency of the translated sentences, thereby improving the translation effect as a whole.

[0099] Based on the above design ideas, the application scenarios of the sentence translation method of the embodiment of the present application are introduced below.

[0100] Please refer to Figure 2 , which is an application scenario diagram of a sentence translation method provided in an embodiment of the present application, and the application scenario diagram includes a terminal 210, a server 220, and an application 211 running in the terminal 210.

[0101] The terminal 210 and the server 220 may be directly or indirectly connected to each other through one or more communication networks. The communication network may be, for example, a wired network or a wireless network, such as a mobile cellular network or a wireless fidelity (Wireless-Fidelity, WIFI) network, or other possible networks, which are not limited in the embodiments of the present application.

[0102] The user can use the corresponding functions of the application 211 in the terminal 210. For example, the terminal 210 can generate a translation request in response to a translation operation, and send the translation request to the server 220. The translation request is used to request the server 220 to translate the target sentence to be translated. The translation operation is, for example, an input operation of the target sentence, or clicking a translation control for the target sentence. Among them, the application 211 is, for example, an application pre-installed in the terminal 210, or an application embedded in a third-party application, or a web page running in a browser, etc., and this application does not limit this.

[0103] The server 220 may be used to support the corresponding functions of the application 211. For example, after receiving the translation request, the server 120 may call the trained target translation model to translate the target sentence to obtain the target translation sentence, wherein the training process of the target translation model and the sentence translation process involved will be introduced below.

[0104] It should be noted that, in the embodiment of the present application, the number of the terminal 210 may be one or more, and the number of the server 220 may be one or more. Figure 2 The example above takes two terminals and one server as an example. In practice, there is no limit on the number of terminals or servers.

[0105] It should be noted that the above sentence translation method can be applied to various specific application scenarios, for example, in bilingual translation scenarios. The sentence translation method can also be applied to various application scenarios involving sentence translation, such as virtual reality, augmented reality, unmanned driving, smart furniture, smart office, smart wear, smart transportation, smart city, drone, robot, etc., and this application does not limit this.

[0106] In a possible application scenario, the model parameters of the trained target translation model in this application, or the translation history records (such as the target sentence and the target translation sentence), etc. can be stored using cloud storage technology. Cloud storage is a new concept that extends and develops from the concept of cloud computing. For example, distributed cloud storage. A distributed cloud storage system built on distributed cloud storage refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (or storage nodes) in the network to work together through application software or application interfaces to provide external data storage and business access functions.

[0107] In a possible application scenario, in order to reduce communication delay, servers 220 may be deployed in various regions, or different servers 220 may serve the regions corresponding to the terminals 210 for load balancing. Multiple servers 220 can share data through blockchain, and multiple servers 220 are equivalent to a data sharing system composed of multiple servers 220. For example, the terminal 210 is located at location a and communicates with the server 220, and the terminal 210 is located at location b and communicates with other servers.

[0108] Each server 220 in the data sharing system has a node identifier corresponding to the server 220. Each server 220 in the data sharing system can store the node identifiers of other servers 220 in the data sharing system, so that the generated blocks can be broadcast to other servers 220 in the data sharing system according to the node identifiers of other servers 220. Each server 220 can maintain a node identifier list as shown in the following table, and store the server 220 name and the node identifier in the node identifier list. The node identifier can be an Internet Protocol (IP) address for interconnection between networks and any other information that can be used to identify the node.

[0109] The terminal mentioned above is an electronic device used by the user, which can be a computer device with certain computing capabilities and running applications, such as a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, game device, smart TV or smart wearable device. The server 220 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0110] Of course, the sentence translation method provided in the embodiment of the present application is not limited to Figure 2 The application scenarios shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto. Figure 2 The functions that can be implemented by each device in the application scenario shown will be described in the subsequent method embodiments and will not be described in detail here.

[0111] Please refer to Figure 3 , which is a flow chart of a sentence translation method provided in an embodiment of the present application, which can be Figure 2The method may be executed by a server or a terminal, or may be executed by a server and a terminal together. In the embodiment of the present application, execution by a server is taken as an example. The process of the method is described as follows.

[0112] Step 31, obtaining a target sentence to be translated.

[0113] The server may obtain a target sentence to be translated from the terminal. The target sentence may be in any language, such as Chinese, English or French.

[0114] As an embodiment, the server may also simultaneously obtain the target translation language corresponding to the target sentence from the terminal, where the target translation language refers to the language type into which the target sentence needs to be translated.

[0115] Step 32, inputting the target sentence into the trained target translation model to obtain the target translation sentence; wherein the target translation model is obtained by adjusting the parameters of the first translation model based on the fidelity training loss value during the training of the first translation model to be trained using the first sample training data set.

[0116] The server may pre-store a target translation model, or the server may call a target translation model on another device, and use the target translation model to translate the target sentence, thereby obtaining a target translation sentence. The target translation model may be obtained by training a first translation model to be trained based on the first sample training data set, and during the training process, the parameters of the first translation model may be adjusted based on the fidelity training loss value.

[0117] The first sample training data set includes at least one first sample training data, and each first sample training data includes a first sample sentence to be translated and a first sample translated sentence used to represent a translation result of the first sample sentence to be translated.

[0118] As an embodiment, if the target translation model is a multilingual translation model, that is, it can translate multiple languages, then each first sample sentence to be translated in the first sample training data set can be in multiple languages, and similarly, each first sample translation sentence in the first sample training data set can also be in multiple languages. If the target translation model is a monolingual translation model, then each first sample sentence to be translated in the first sample training data set can be in one language, each first sample translation sentence in the first sample training data set can be in one language, and any first sample sentence to be translated and the corresponding first sample translation sentence are in different languages.

[0119] When training the first translation model according to the first sample training data set, the server can input the first sample translation sentence and the corresponding first sample translation sentence into the first translation model. The first translation model predicts the first probability of each output position based on the first sample translation sentence and the corresponding first sample translation sentence. Each output position can be understood as each position corresponding to the translation sentence predicted by the first translation model. The first probability is the probability of the corresponding first target element at each output position output by the first translation model. The first translation model can output the probability of each candidate element at an output position. The first target element at the corresponding output position is, for example, the real element corresponding to the output position in the first sample translation sentence, or the element with the highest probability among the candidate elements. The candidate element can be understood as an element in the element library corresponding to the first translation model. The element library can be pre-configured by the server, or can be accumulated during the training process based on the first sample training data set.

[0120] Similarly, the first server inputs the corresponding first sample translation sentence into the first translation auxiliary model, and the first translation auxiliary model outputs the second probability of the first target element of each output position according to the first sample translation sentence. The various output positions can refer to the content discussed in the previous text, which will not be repeated here. The first target element can refer to the content discussed in the previous text, which will not be repeated here. The first translation auxiliary model can be a pre-trained model. Since the first translation auxiliary model outputs completely according to the first sample translation sentence and has nothing to do with the first sample sentence to be translated, this is equivalent to simulating the process of over-reliance on historical translation texts for translation during the translation process, so that the overconfidence of the first translation model can be quantified based on the second probability output by the first translation auxiliary model.

[0121] The server determines a fidelity training loss value based on a difference between the first probability and the second probability at the same output position, and adjusts parameters of the first translation model based on the fidelity training loss value.

[0122] It should be noted that the first translation model to be trained refers to the first translation model, relative to the target translation model, which is obtained without being trained with the first sample training data set. In fact, the first translation model may also be a model that has been pre-trained with other training data in advance.

[0123] For example, see Figure 4, is a process example diagram of a sentence translation method provided by an embodiment of the present application, the first sample sentence to be translated is "She is my friend", the corresponding first sample translation sentence is "She is my friend", the server inputs the first sample sentence to be translated and the first sample translation sentence into the first translation model, the first translation model has a first probability of "She" at the first output position of 0.5, a first probability of "is" at the second output position of 0.6, a first probability of "my" at the third output position of 0.4, and a first probability of "friend" at the fourth output position of 0.35. Similarly, the first server inputs the first sample translation sentence "She is my friend" into the first translation auxiliary model, the first translation auxiliary model has a second probability of "She" at the first output position of 0.6, a second probability of "is" at the second output position of 0.8, a second probability of "my" at the third output position of 0.7, and a second probability of "friend" at the fourth output position of 0.3.

[0124] exist Figure 3 In the illustrated embodiment, the reason why the translation model is overconfident is that the translation model pays too much attention to historical translation results, and the first translation auxiliary model is output based on the first sample translation sentence. That is to say, the first translation auxiliary model is equivalent to simulating the process of paying too much attention to historical translation texts in the first translation model. Therefore, the fidelity training loss value can be calculated based on the difference between the first probability output by the first translation model and the second probability output by the first translation auxiliary model. The server can adjust the first translation model according to the fidelity training loss value. That is to say, the embodiment of the present application provides a method for quantifying the fidelity training loss value of the first translation model, and directly adjusts the parameters of the first translation model based on the fidelity training loss value, thereby fundamentally improving the fidelity of the first translation model and correspondingly improving the fidelity of the translation results output by the target translation model.

[0125] The target translation model in step 32 may be obtained by the server from other devices, where other devices refer to devices other than the server, or may be obtained by the server training the first translation model. The following takes the process of the server training the first translation model as an example to introduce the process of training the first translation model and obtaining the target translation model.

[0126] Please refer to Figure 5 , is a flow chart of a method for training a first translation model provided in an embodiment of the present application, the flow chart of the method specifically includes:

[0127] Step 51, based on the first sample training data set, performing multiple iterative training on the first translation model to be trained until the first model convergence condition is met, and taking the first translation model that meets the first preset condition in the multiple iterative training as the target translation model, wherein one iterative training includes the following operations:

[0128] Step 511, selecting first sample training data from a first sample training data set;

[0129] Step 512, inputting the first sample training data into the first translation model to obtain first probabilities corresponding to each output position;

[0130] Step 513, inputting the first sample translation sentence into the first translation auxiliary model to obtain second probabilities corresponding to each output;

[0131] Step 514, obtaining a fidelity training loss value based on the difference between the first probability and the second probability corresponding to each output position;

[0132] Step 515: Adjust the parameters of the first translation model based on the fidelity training loss value.

[0133] In each iterative training, a corresponding first translation model can be obtained, which can be further understood as the parameters of the first translation model. After multiple iterative trainings, the server is equivalent to obtaining multiple first translation models. The server can screen the first translation models that meet the first preset conditions from the multiple first translation models. The first preset conditions can be, for example, the first translation model output by the last iterative training in multiple iterative trainings is used as the target translation model, or the first translation model whose translation indicators meet the indicator conditions is used as the target translation model. The translation indicators can be, for example, the fidelity of the translation results or other indicators. The translation indicators can be determined based on the first verification set. The meaning of the first verification set can refer to the first sample training data set, which will not be repeated here.

[0134] In an embodiment of the present application, the first translation model may be trained by an iterative training method, wherein each iterative training is performed by using first sample training data selected from a first sample training data set, and the selected first sample training data may be one or more.

[0135] In step 512, the first translation model can determine the first probability corresponding to an output position based on the first sample sentence to be translated and the historical translation text associated with an output position. Each element in the first sample translation sentence in the first sample training data corresponds to each output position one by one, and the historical translation text associated with the corresponding position is the text content in the first sample translation sentence that is located before the element corresponding to the corresponding output position. The element can be understood as a constituent unit of the first sample translation sentence, for example, an element can be a word, or a letter, etc.

[0136] In order to clearly explain the meaning of the historical translation text, the following examples are given:

[0137] Please refer to Figure 6 , is an example diagram of the alignment of the first sample translation sentence with each output position, such as Figure 6 As shown, the first sample translation sentence is "He comes from China", and each element (here, an element is a word) in the first sample translation sentence corresponds to each output position one by one. Specifically, "He" corresponds to the first output position, "comes" corresponds to the second output position, "from" corresponds to the third output position, and "China" corresponds to the fourth output position. Taking the third output position 61 as an example, the historical translation text 62 corresponding to the third output position 61 is "He comes".

[0138] As an embodiment, an implementation method of step 512 is as follows:

[0139] Step 1.1: Based on a first translation model, encode a first sample sentence to be translated in first sample training data to obtain a first encoding result.

[0140] Step 1.2: Based on the first translation model, a historical translation text associated with an output position is decoded to obtain a decoding result of the output position.

[0141] Step 1.3: Based on the first translation model, the decoding result of the output position is normalized to obtain a first probability corresponding to the output position.

[0142] In an embodiment of the present application, the server may encode the first sample sentence to be translated, and the encoding may be, for example, one or both of embedded learning or deep coding, so as to obtain an encoding result of the first sample sentence to be translated, and the encoding result may be understood as semantic information of the first sample sentence to be translated.

[0143] Based on the encoding result, the server decodes the historical translation text associated with the output position, thereby obtaining a first decoding result at the output position, and the first decoding result can be understood as the semantic information of the element corresponding to the output position. The server normalizes the first decoding result to obtain a first probability corresponding to the output position.

[0144] In the embodiment of the present application, the first decoding result output by the first translation model is respectively related to the encoding result and the historical translation text, thereby avoiding the situation where the first translation model pays too much attention to the first sample sentence to be translated or pays too much attention to the historical translation text.

[0145] As an example, please refer to Figure 7 , is a schematic diagram of the structure of a first translation model provided in an embodiment of the present application, the first translation model includes: an encoding module, a first decoding module and a first normalization module. The encoding module is used to encode the first sample sentence to be translated to obtain an encoding result; the first decoding module is used to decode the historical translation text of the corresponding output position based on the encoding result, so as to obtain a first decoding result; the first normalization module is used to normalize the first decoding result to obtain a first probability of the corresponding output position.

[0146] Furthermore, the encoding module may be implemented by one or more encoders, such as an encoder in a transformer model. Similarly, the first decoding module may be implemented by one or more decoders, such as a decoder in a transformer model. The first normalization module may include one or more normalization layers.

[0147] As a possible embodiment, the first translation model may also adopt a neural machine translation model based on a recurrent neural network (RNN), or directly adopt a transformer model, etc.

[0148] By analogy, the server may perform the above processing on each output position, thereby obtaining the first probability corresponding to each output position.

[0149] Similarly, in step 513, the first translation assistance model can determine the second probability corresponding to an output position based on the historical translation text associated with the output position. The meaning of the historical translation text can refer to the content discussed above, which will not be repeated here.

[0150] As an embodiment, an implementation method of step 513 is as follows:

[0151] Step 2.1, based on the first translation assistance model, decoding a historical translation text associated with an output position to obtain a second decoding result of the output position;

[0152] Step 2.2: Based on the first translation auxiliary model, normalize the second decoding result of the output position to obtain a second probability corresponding to the output position.

[0153] In an embodiment of the present application, the server can directly predict the second probability of belonging to the corresponding first target element at a certain output position based on the historical translation text at the output position. That is to say, the output of the first translation auxiliary model is only related to the historical translation text and has nothing to do with the first sentence to be translated. This can accurately simulate the process of over-focusing on the historical translation text in the first translation model, so as to facilitate the subsequent more accurate determination of the fidelity training loss value of the first translation model.

[0154] As an example, please refer to Figure 8 , is a structural schematic diagram of a first translation auxiliary model provided in an embodiment of the present application, the first translation auxiliary model includes a second decoding module and a second normalization layer, the second decoding module decodes a historical translation text associated with an output position, and the second normalization layer normalizes the second decoding result of the second decoding module to obtain a second probability of the output position.

[0155] For further information, please refer to Fig. 9 ,for Figure 8 A schematic diagram of the structure of the second decoding module in the embodiment of the present invention, wherein the second decoding module includes an embedding layer and N decoders, each decoder includes a self-attention layer, a third normalization layer, a feedforward network and a fourth normalization layer, and the value of N is an integer greater than or equal to 1.

[0156] Specifically, embedding learning is performed on the first sample translation sentence through the embedding layer to obtain the sample serialization representation of the first sample translation sentence; the sample serialization representation is processed through the self-attention layer using the self-attention mechanism to obtain the attention representation of each output position; the attention representation corresponding to the same output position and the elements in the sample serialization representation are weighted to obtain the first weighted result at each output position; the first weighted result at each output position is normalized through the third normalization layer to obtain the first normalized result at each output position; the first normalized result at each output position is feedforward processed through the feedforward network; the feedforward processing result and the first normalized result at the same output position are weighted to obtain the second weighted result at each output position; the second weighted result at each output position is normalized through the fourth normalization layer to obtain the second normalized result corresponding to each output position.

[0157] By analogy, after being processed by N decoders, it passes through the second normalization module to obtain the second probability at each output position.

[0158] There are many ways to calculate the fidelity training loss value, and the following examples are given:

[0159] Calculation method 1:

[0160] The fidelity training loss is expressed as the element-based Token-level Objective (MTO). When the element is a word, the element-based training loss can also be called the word-level training loss or the word-level optimization objective.

[0161] Specifically, the server determines an element-level training loss value based on a probability difference between the first probability and the second probability of each output position, wherein the element-level training loss value is negatively correlated with the probability difference of each output position. The element-level training loss value reflects the difference between each element predicted by the first translation model and the first auxiliary model.

[0162] Please refer to Fig.10 , is a flow chart of a method for determining an element level loss value provided in an embodiment of the present application, the flow chart of the method comprising:

[0163] Step 1001, for each output position, perform the following operations respectively: substitute the probability difference of an output position into a preset function to obtain a first training loss value at the output position, where the preset function is a monotonically decreasing function;

[0164] Step 1002, determining a weight of the output position according to the first probability of the output position, wherein the weight of the output position is negatively correlated with the first probability corresponding to the output position;

[0165] Step 1003: weighting the first training score loss value of the output position according to the weight of the output position, so as to obtain a second training score loss value of the output position;

[0166] Step 1004, based on the second training loss value corresponding to each output position, an element-level loss function is obtained.

[0167] In an embodiment of the present application, when the first probability at a certain output position is large, it means that the first translation model has learned the element well and does not need to pay too much attention to this element. Therefore, the training score loss value corresponding to the output position can be appropriately reduced; when the first probability at a certain output position is small, the first translation model needs to strengthen the training of this element. Therefore, the training score loss at this output position can be appropriately increased. That is to say, in an embodiment of the present application, according to the first probability at the output position and the learning effect of the first translation model on each element, the size of the training score loss value at the corresponding output position is flexibly adjusted, which is conducive to improving the training effect of the first translation model.

[0168] The following is an introduction to the calculation process of element-level loss value:

[0169] Step 3.1, determining the probability difference (also called Margin) between the first translation model and the first translation auxiliary model at a certain output position. An example of a calculation formula for the probability difference at a certain output position is as follows:

[0170] Δ(t)=p NMT (y t |y <t ,x)-p LM (y t |y <t ) (1)

[0171] Among them, Δ(t) is Margin, or it can be understood as the fidelity of the first translation model, p NMT (y t |y <t ,x) represents the first probability of the first translation model at an output position, which can be abbreviated as p NMT (t), p LM (y t |y <t ) represents the second probability of the first translation auxiliary model at the output position, x represents the first sample sentence to be translated, y <t Indicates the historical translation text associated with the output position, y t Represents the first target element at this output position.

[0172] Specifically, Δ(t) is negatively correlated with the overconfidence of the first translation model. Specifically, when Δ(t) is large, it means that the first translation model is obviously better than the first translation auxiliary model, y t The correlation with the first sample sentence to be translated x is high, which means that the first translation model is not overconfident. When Δ(t) is moderate, the first translation model may be slightly overconfident and there is room for further enhancement. When Δ(t) is small, the first translation model is too confident and may not be able to correctly translate the first sample translation sentence.

[0173] Step 3.2: Determine the first training loss value at the output position The first training loss The specific expression of can be any of the following:

[0174] (1) Linear function: (1-Δ(t)) / 2

[0175] (2) Cube function: (1-Δ(t) 3 ) / 2

[0176] (3) Quintic function: (1-Δ(t) 5 ) / 2

[0177] (4) Logarithmic function: Among them, α is a hyperparameter

[0178] (5) sigmoid function: 1 / (1+e -Δ(t) )

[0179] In the embodiment of the present application, the linear function form is simpler, and the cubic function, quintic function, logarithmic function and sigmoid function can correspond to different slopes for different Δ(t), thereby widening the gap between the fidelity training loss values ​​determined in different training processes, which is conducive to quickly training the first translation model.

[0180] Step 3.3, determine the second training loss value at the output position, and sum the second training loss values ​​at each output position to obtain the initial element-level training loss value.

[0181] As an embodiment, the weight at a certain output position is: fixed weight or (1-p NMT (y t |y <t ,x)) to the Mth power, the fixed weight is, for example, 1, and M is a number greater than or equal to 1. When the weight corresponding to the output position is 1, the amount of calculation is smaller than that of calculating the element-level training loss value.

[0182] In an embodiment of the present application, when the weight corresponding to the output position is related to the first probability, the training loss value corresponding to each element can be flexibly adjusted, which is conducive to determining a more reasonable element-level training loss value.

[0183] In step 3.4, based on the initial element-level training loss value, determine the element-level training loss value. An example of a formula for calculating the element-level training loss value is as follows:

[0184]

[0185] Among them, (1-p NMT (y t |y <t ,x)) represents the weight of a certain output position, T represents the total number of output positions, Element-wise training loss value.

[0186] The calculation result of formula (2) can be used as the fidelity training loss value, or the calculation result of formula (2) can be weighted to obtain the fidelity training loss value, or the fidelity training loss value can be determined based on the element-level training loss value and the initial translation training loss value. The server can also determine the initial translation training loss value of the first translation model based on the first probability of each output position. The initial translation training loss value is calculated, for example, using a cross entropy loss function. A formula for calculating the fidelity training loss value is:

[0187]

[0188] in, represents the faithfulness training loss value, represents the initial translation training loss value of the first translation model, that is, the error between the translation sentence predicted by the first translation model and the first sample translation sentence, λ M is a hyperparameter.

[0189] As an example, For example, the cross entropy loss function of the first translation model can be used. In this case, The calculation formula example is as follows:

[0190]

[0191] in, P(y|x) represents the first sentence probability of the translation sentence predicted by the first translation model. The first sentence probability can be represented by the conditional probability of the translation sentence, for example.

[0192] In calculation method 1, the server uses the probability difference of each output position to calculate the element-level training loss value, and the element-level training loss value is negatively correlated with the probability difference of each output position, providing a simple and reasonable quantization method for the fidelity training loss value.

[0193] Calculation method 2:

[0194] The fidelity training loss value is expressed as a margin-based sentence-level objective (MSO), which can also be called a sentence-level optimization objective.

[0195] Specifically, the server determines the sentence-level training loss value based on the probability difference between the first probability and the second probability corresponding to each output position. If the first ratio is greater than the ratio threshold, the sentence-level loss value is zero; if the first ratio is less than or equal to the ratio threshold, the sentence-level loss value is negatively correlated with the probability difference of each output position. The first ratio is: based on the probability difference of each output position, the ratio between the number of elements with a probability difference less than the difference threshold and the total number of elements in the first sample translation sentence.

[0196] Compared with the first translation auxiliary model, the first translation model has additional information about the first sample sentence to be translated. Therefore, in theory, the prediction result of the first translation model should be more accurate than the prediction result of the first translation auxiliary model, that is, Δ(t) should be greater than 0 at each output position. However, in fact, there are elements with Δ(t) less than 0. If there are a large number of elements with Δ(t) less than 0 corresponding to each output position, it means that the first translation model predicts the translation result of the first sentence to be translated with poor accuracy, that is, the translation result of the first translation model for the first sentence to be translated is an illusion, that is, the fidelity of the translation result is poor. Therefore, in the embodiment of the present application, the sentence-level training loss value of this type of first sample training data can be set to 0, which is equivalent to treating this type of first sample training data as "dirty" data and eliminating this type of first sample training data, thereby improving the effect of training the first translation model.

[0197] Of course, if there are a small number of elements with Δ(t) less than 0 corresponding to each output position, it means that the accuracy of the translation result predicted by the first translation model for the first sentence to be translated is within an acceptable range. Therefore, in the embodiment of the present application, the sentence-level loss value can be determined based on the probability difference corresponding to each output position, and the sentence-level training loss value is negatively correlated with the probability difference of each output position.

[0198] As an embodiment, a calculation formula of a sentence-level training loss value is as follows:

[0199]

[0200] Among them, is the indicative function, when hour, otherwise It changes dynamically during the training phase, k is the ratio threshold, is the element-level training loss value discussed above, This is the first ratio.

[0201] Among them, the first ratio The calculation formula example is as follows:

[0202]

[0203] Among them, #{y t ∈y:Δ(t)<0} refers to the number of elements in y whose Δ(t) is less than 0. t :y t ∈y} refers to the total number of elements included in the first sample translation sentence y.

[0204] As an embodiment, the ratio threshold may be set to 0.3.

[0205] In calculation method 2, sentence-level training loss values ​​are introduced to exclude first sample training data with poor prediction accuracy in the first sample training data set, thereby excluding some sample training data with large local errors, thereby improving the training effect of the first translation model. In addition, during the training process, the ability of the first translation model to distinguish "dirty" data is gradually improved, thereby further improving the training effect of the first translation model.

[0206] Calculation method three:

[0207] The weighted sum of the sentence-level training loss and the element-level training loss represents the faithfulness training loss.

[0208] When weighted summing the sentence-level training loss value and the element-level training loss value, the weights of the sentence-level training loss value and the element-level training loss value can be the same or different, and this application does not limit this. The calculation methods of the sentence-level training loss value and the element-level training loss value can refer to the content discussed above, and will not be repeated here.

[0209] In an embodiment of the present application, the sentence-level training loss value and the element-level training loss value are combined to calculate the fidelity training loss value. While quantifying the training loss value of the first translation model in terms of translation fidelity, the "dirty" data in the first sample training data set can also be excluded to improve the training effect of the first translation model.

[0210] Calculation method 4:

[0211] Determine an initial fidelity training loss value using at least one of a sentence-level training loss value and an element-level training loss value;

[0212] A fidelity training loss value is determined based on an initial training loss value and an initial translation training loss value, wherein the initial translation training loss value is determined based on the first probability of each output position and is used to represent an error between the translation sentence predicted by the first translation model and the first sample translation sentence.

[0213] For example, the element-level training loss value in calculation method 1 can be determined as the initial fidelity training loss value, the sentence-level training loss value in calculation method 2 can be determined as the initial fidelity training loss value, or the weighted sum of the sentence-level training loss value and the element-level training loss value in calculation method 2 can be determined as the initial fidelity training loss value. Furthermore, the method for determining the initial translation training loss value can refer to the content discussed above and will not be repeated here. The server performs a weighted summation of the initial translation training loss value and the initial fidelity training loss value to obtain a fidelity training loss value.

[0214] In calculation method 4, not only the fidelity of the first translation model is quantified, but also the translation loss of the first translation model is introduced, which is beneficial to improving the overall translation effect of the first translation model and correspondingly improving the translation effect of the output result of the target translation model.

[0215] After determining the fidelity training loss value, the server may adjust the parameters of the first translation model according to the fidelity training loss value. For example, the server may use a gradient descent algorithm to adjust the parameters of the first translation model.

[0216] As an embodiment, in order to improve the training effect of the first translation auxiliary model and the first translation model, a first joint training loss value of the first translation auxiliary model and the first translation model is introduced in the embodiment of the present application. The first joint training loss value is used to represent the translation loss of the first translation auxiliary model and the first translation model. Furthermore, when training the first translation auxiliary model and the first translation model, the parameters of the first translation auxiliary model and the first translation model can be adjusted based on the fidelity training loss value and the first joint training loss value.

[0217] As an embodiment, a method for calculating the first joint training loss value specifically includes:

[0218] Step 4.1, determining a first translation training loss value of a first translation model based on first probabilities corresponding to respective output positions;

[0219] Step 4.2, determining a second translation training loss value of the first translation auxiliary model based on the second probability of each output position;

[0220] Step 4.3, performing weighted summation on the first translation training loss value and the second translation training loss value to obtain a first joint training loss value.

[0221] In step 4.1, the server may use the product of the first probabilities corresponding to each output position as the first sentence probability of the translation sentence predicted by the first translation model, and determine the first translation training loss value of the first translation model based on the first sentence probability. The first translation training loss value is used to represent the error between the predicted translation sentence and the first sample translation sentence.

[0222] As an embodiment, the calculation formula of the first translation training loss value may refer to the above formula (4).

[0223] Similarly, in step 4.2, the server may multiply the product of the second probabilities corresponding to the respective output positions and determine the first translation training loss value of the first translation model based on the second sentence probability.

[0224] As an embodiment, the calculation formula of the second translation training loss value is:

[0225]

[0226] As an embodiment, a calculation formula for calculating the first joint training loss value is as follows:

[0227]

[0228] Among them, λ LM is a hyperparameter.

[0229] After calculating the first joint training loss value, the server may adjust parameters of the first translation model and the first translation auxiliary model according to the first joint training loss value.

[0230] In an embodiment of the present application, the first joint training loss value and the fidelity training loss value are used to jointly train the first translation model and the first translation auxiliary model. While ensuring the translation fidelity of the first translation model, the translation effects of the first translation model and the first translation auxiliary model can also be guaranteed. Moreover, the first translation model and the first translation auxiliary model are trained synchronously to improve the training efficiency.

[0231] As an embodiment, in order to ensure the accuracy of the translation sentence predicted by the first translation auxiliary model, before adjusting the parameters of the first translation auxiliary model according to the fidelity training loss value, the first translation auxiliary model and the first translation model can be jointly pre-trained. Fig.11 , is a flow chart of a joint pre-training method provided in an embodiment of the present application, the flow chart of the method comprising:

[0232] Step 110, based on the second sample training data set, multiple rounds of joint pre-training are performed on the second translation model to be trained and the second translation auxiliary model to be trained until the second model convergence condition is met, and the second translation model that meets the second preset condition in the multiple rounds of joint pre-training is used as the first translation model, and the second translation auxiliary model that meets the third preset condition in the multiple rounds of joint pre-training is used as the first translation auxiliary model, wherein each second sample training data includes a second sample sentence to be translated and a corresponding second sample translation sentence, and each round of joint pre-training includes the following operations:

[0233] Step 1101, selecting second sample training data from a second sample training data set;

[0234] Step 1102, inputting the second sample training data into the second translation model to obtain third probabilities corresponding to each output position;

[0235] The third probability corresponding to each output position is the output probability of the second target element at the corresponding output position.

[0236] Step 1103, inputting the second sample translation sentence in the second sample training data into the second translation auxiliary model to obtain a fourth probability;

[0237] Wherein, the fourth probability of each output position is the output probability of the second target element of the corresponding output position;

[0238] Step 1104, determining a third translation training loss value of the second translation model based on the third probabilities corresponding to the respective output positions;

[0239] Step 1105, determining a fourth translation training loss value of the second translation auxiliary model based on the fourth probabilities corresponding to the output positions;

[0240] Step 1106, performing weighted summation on the third translation training loss value and the fourth translation training loss value to determine a second joint training loss value;

[0241] As an embodiment, the method for determining the second joint training loss value may refer to the above formula (8).

[0242] Step 1107: Based on the second joint training loss value, adjust the parameters of the second translation model and the second translation auxiliary model respectively.

[0243] The second preset condition and the third preset condition may refer to the content of the first preset condition mentioned above, and will not be repeated here.

[0244] In each round of joint pre-training, a corresponding second translation model and a second translation auxiliary model can be obtained. A second translation model can be further understood as the parameters of the second translation model, and a second translation auxiliary model can be further understood as the parameters of the second translation auxiliary model. After multiple rounds of joint pre-training, the server can obtain multiple second translation models, and the server can select a second translation model that meets the second preset condition from these multiple second translation models. Similarly, the server can also select a second translation auxiliary model that meets the third preset condition from multiple second translation auxiliary models.

[0245] In an embodiment of the present application, the server may perform joint pre-training on the first translation model and the first translation auxiliary model based on the second sample training data set before adjusting the first translation model based on the fidelity training loss value. During the joint pre-training process, the translation loss values ​​of the first translation model and the first translation auxiliary model may be calculated respectively, and the translation loss values ​​of the first translation model and the first translation auxiliary model may be weightedly summed to obtain a second joint training loss value, thereby adjusting the parameters of the second translation model and the second translation auxiliary model respectively according to the second joint training loss value.

[0246] It should be noted that the first sample training data set and the second sample training data set may be completely identical data sets, partially identical data sets, or completely different data sets, and this application does not limit this.

[0247] In an embodiment of the present application, the second translation model and the second translation auxiliary model can be jointly pre-trained in advance, so that the first translation auxiliary model obtained after the joint pre-training can output more accurate translation sentences, facilitate the subsequent accurate determination of the fidelity training loss of the first translation auxiliary model, and improve the training effect of the second translation auxiliary model.

[0248] exist Fig.11 In the illustrated embodiment, since joint pre-training is required to obtain the first translation auxiliary model and the first translation model, in the embodiment of the present application, in order to reduce the amount of joint pre-training, in the embodiment of the present application, the second translation model at least includes the same structure as the second translation auxiliary model, and the second translation model and the second translation auxiliary model have the same structure and share parameters.

[0249] Taking the second translation model as the previous text Figure 7 The structure shown, the second translation auxiliary model is the previous Figure 8 Taking the structure shown as an example, the first decoding module in the second translation model can share parameters with the second decoding module in the second translation auxiliary model, and the first normalization module in the second translation model can share parameters with the second normalization module in the second translation auxiliary model.

[0250] Based on the above Figure 2 The following application scenarios are combined Fig.12 The schematic diagram of the interaction process between the server and the terminal shown in the figure introduces the sentence translation method in the embodiment of the present application:

[0251] Step 121 : In response to the translation operation, the terminal obtains a target sentence to be translated and generates a translation request.

[0252] The translation operation and translation request may refer to the contents discussed above and will not be elaborated here.

[0253] Step 122: The terminal sends a translation request to the server.

[0254] Step 123: The server inputs the target sentence into the trained target translation model to obtain the target translation sentence.

[0255] The training process of the target translation model can refer to the content discussed in the previous article and will not be repeated here.

[0256] Step 124: The server sends the target translation sentence to the terminal.

[0257] After obtaining the target translation sentence, the server may send the target translation sentence to the terminal, and the terminal may display the target translation sentence.

[0258] In an embodiment of the present application, the server can provide translation results to the terminal in a timely manner with the help of the target translation model. Since the target translation model is obtained by directly adjusting the training loss based on the fidelity, the translation sentence translated by the target translation result has higher fidelity. In addition, deploying the target translation model on the server side can relatively reduce the processing amount on the terminal side.

[0259] For example, see Fig.13 , which is an example diagram of the translation process of the terminal during the translation process provided in the embodiment of the present application, the user long presses the message 131, and the message 131 is specifically Fig.13 In (1) of the figure, “Hallo, Ich bin ein Personal vermittler von A company”, the terminal displays Fig.13 The control list shown in (1) includes a translation control 132. When the user clicks the translation control 132, it is equivalent to performing a translation operation. The terminal responds to the translation operation and generates a translation request. The translation request carries the message 131. At this time, the message 131 is used as a target sentence, and the server obtains the target translation sentence and sends it to the terminal. The terminal displays as shown in FIG. Fig.13The target translation sentence 133 shown in (2) is specifically "Hello, I am a recruiter of Company A".

[0260] In order to verify the translation effect of the target translation model in the embodiment of the present application, the embodiment of the present application uses the English (En) -> German (Deutsch, De) data set in the Workshop on Machine Translation (WMT) database to evaluate the bilingual evaluation understudy (BLEU) index of the target translation model in the embodiment of the present application and the conventional NMT model. The evaluation results are shown in Table 1 below:

[0261] Table 1

[0262]

[0263]

[0264] It should be noted that the NMT model trained in the conventional way refers to the NMT model directly trained using bilingual parallel corpus; +MTO means that when training the NMT model, the element-level training loss value is introduced to adjust the parameters of the NMT model; +MSO means that when training the NMT model, the sentence-level training loss value is introduced to adjust the parameters of the NMT model.

[0265] From Table 1 above, it can be seen that the translation model trained by introducing the fidelity training loss value in the embodiment of the present application performs better in terms of BLEU index than the conventional NMT model and the NMT model based on LM joint training.

[0266] Similarly, the English->French (Fr) dataset and the Chinese (Zh)->English dataset in the WMT database were used to evaluate the BLEU index of the target translation model, as shown in Table 2 below:

[0267] Table 2

[0268] Model name and how the model was trained BLEU indicator of En→Fr BLEU index of Zh→En Conventional NMT (NMT model trained in a conventional way) 41.07 25.75 +LM (introducing NMT trained jointly with LM) 41.14 25.75 +MTO (introducing element-level training loss) 41.56 26.94 +MSO (introducing sentence-level training loss value) 41.70 27.25

[0269] It can be seen from Table 2 above that the translation model obtained by introducing MTO or MSO training in the embodiment of the present application has a BLEU index in translating English into French, and a BLEU index in translating Chinese into English, which are better than the BLEU index of the conventional NMT model and the NMT model introduced with LM joint training.

[0270] Furthermore, in order to further verify the training effect of the target translation model, the target translation model was manually evaluated based on translating Chinese into English. The manual evaluation results are shown in Table 3 below:

[0271] Table 3

[0272]

[0273]

[0274] The average scores in Table 3 refer to the weighted average results of fidelity and fluency.

[0275] From Table 3 above, it can be seen that the target translation model trained after introducing MTO and MSO has better overall performance in terms of fidelity and fluency.

[0276] Based on the same inventive concept, the present application embodiment provides a sentence translation device, which can realize the functions of the terminal or server discussed above, please refer to Fig.14 , the device comprises:

[0277] The sentence obtaining module 1401 is used to obtain the target sentence to be translated;

[0278] The sentence translation module 1402 is used to input the target sentence into the trained target translation model to obtain the target translation sentence;

[0279] The target translation model is obtained by adjusting the parameters of the first translation model based on the fidelity training loss value during the training of the first translation model to be trained using the first sample training data set. Each first sample training data includes a first sample sentence to be translated and a corresponding first sample translation sentence. The fidelity training loss value is obtained based on the difference between the first probability and the second probability corresponding to each output position. The first probability corresponding to each output position is the output probability of the first target element obtained based on the first translation model and the corresponding first sample training data. The second probability corresponding to each output position is the output probability of the first target element obtained based on the first translation auxiliary model and the corresponding first sample translation sentence.

[0280] In a possible embodiment, the device further includes a model training module 1403, and the model training module 1403 is specifically configured to train the target translation model obtained by:

[0281] Based on the first sample training data set, the first translation model to be trained is trained multiple times until the first model convergence condition is met, and the first translation model that meets the first preset condition in the multiple iterative trainings is used as the target translation model, wherein one iterative training includes the following operations:

[0282] Selecting first sample training data from a first sample training data set;

[0283] Inputting the first sample training data into the first translation model, obtaining first probabilities corresponding to respective output positions, wherein the first probability corresponding to each output position is determined based on the first sample sentence to be translated in the first sample training data and the historical translation text associated with the corresponding output position, wherein each element in the first sample translation sentence in the first sample training data corresponds to each output position one by one, and the historical translation text is the text content in the first sample translation sentence that is located before the element corresponding to the corresponding output position;

[0284] Inputting the first sample translation sentence into the first translation assistance model to obtain second probabilities corresponding to respective output positions, wherein the second probability corresponding to each output position is determined based on historical translation texts associated with the corresponding output position;

[0285] Obtaining a fidelity training loss value based on a difference between the first probability and the second probability corresponding to each output position;

[0286] Based on the faithfulness training loss value, the parameters of the first translation model are adjusted.

[0287] In a possible embodiment, the model training module 1403 is specifically used for:

[0288] Based on the first translation model, encoding the first sample sentence to be translated in the first sample training data to obtain an encoding result;

[0289] Based on the first translation model, the following operations are performed for each output position:

[0290] For an output position, based on the encoding result, decoding a historical translation text associated with the output position to obtain a first decoding result of the output position;

[0291] A first decoding result of an output position is normalized to obtain a first probability corresponding to the output position.

[0292] In a possible embodiment, the model training module 1403 is specifically used for:

[0293] Based on the first translation auxiliary model, the following operations are performed for each output position:

[0294] Decoding a historical translation text associated with an output position to obtain a second decoding result of the output position;

[0295] A second decoding result of an output position is normalized to obtain a second probability corresponding to the output position.

[0296] In a possible embodiment, the model training module 1403 is specifically used for:

[0297] Determine an element-level training loss value based on a probability difference between a first probability and a second probability corresponding to each output position, wherein the element-level training loss value is negatively correlated with the probability difference of each output position;

[0298] Determine a sentence-level training loss value based on a probability difference between a first probability and a second probability corresponding to each output position, wherein if the first ratio is greater than a ratio threshold, the sentence-level loss value is zero, and if the first ratio is less than or equal to the ratio threshold, the sentence-level loss value is negatively correlated with the probability difference of each output position, and the first ratio is: based on the probability difference of each output position, determine the ratio of the number of elements whose probability difference is less than the difference threshold to the total number of each element in the first sample translation sentence;

[0299] A fidelity training loss value is obtained based on at least one of an element-level training loss value and a sentence-level training loss value.

[0300] In a possible embodiment, the model training module 1403 is specifically used for:

[0301] Based on at least one of the element-level training loss value and the sentence-level training loss value, obtaining an initial fidelity training loss value;

[0302] A fidelity training loss value is determined based on an initial fidelity training loss value and an initial translation training loss value, wherein the initial translation training loss value is determined based on a first probability of each output position and is used to represent an error between a translation sentence predicted by a first translation model and a first sample translation sentence.

[0303] In a possible embodiment, the model training module 1403 is specifically used for:

[0304] For each output location, do the following:

[0305] Substituting the probability difference of an output position into a preset function to obtain a first training loss value of an output position, wherein the preset function is a monotonically decreasing function;

[0306] Determining a weight of an output position based on a first probability corresponding to an output position, wherein the weight of an output position is negatively correlated with the first probability corresponding to the output position;

[0307] Based on a weight of an output position, weighting the first training point loss value to obtain a second training point loss value of an output position;

[0308] Based on the obtained second training loss values ​​corresponding to each output position, an element-level training loss value is obtained.

[0309] In a possible embodiment, the model training module 1403 is further used to:

[0310] Based on the fidelity training loss value, before adjusting the parameters of the first translation model, determining a first translation training loss value of the first translation model based on first probabilities corresponding to respective output positions, the first translation training loss value representing an error between the predicted translation sentence and the first sample translation sentence;

[0311] Determine a second translation training loss value of the first translation auxiliary model based on the second probabilities corresponding to the respective output positions, where the second translation training loss value represents an error between the predicted translation sentence and the first sample translation sentence;

[0312] Performing a weighted summation on the first translation training loss value and the second translation training loss value to obtain a first joint training loss value;

[0313] When adjusting the parameters of the first translation model based on the fidelity training loss value, the parameters of the first translation model and the first translation auxiliary model are adjusted based on the first joint training loss value.

[0314] In a possible embodiment, the model training module 1403 is further used to:

[0315] Before performing multiple iterative training on the first translation model to be trained, based on the second sample training data set, multiple rounds of joint pre-training are performed on the second translation model to be trained and the second translation auxiliary model to be trained until the second model convergence condition is met, and the second translation model that meets the second preset condition in the multiple rounds of joint pre-training is used as the first translation model, and the second translation auxiliary model that meets the third preset condition in the multiple rounds of joint pre-training is used as the first translation auxiliary model, each second sample training data includes a second sample sentence to be translated and a corresponding second sample translation sentence, and each round of joint pre-training includes the following operations:

[0316] Selecting second sample training data from the second sample training data set;

[0317] Input the second sample training data into the second translation model to obtain the third probability corresponding to each output position, wherein the third probability corresponding to each output position is the output probability of the second target element at the corresponding output position;

[0318] Inputting the second sample translation sentence in the second sample training data into the second translation auxiliary model to obtain a fourth probability, wherein the fourth probability of each output position is the output probability of the second target element at the corresponding output position;

[0319] Determine a third translation training loss value of the second translation model based on the third probabilities corresponding to the respective output positions, where the third translation training loss value represents an error between the predicted translation sentence and the second sample translation sentence;

[0320] Determining a fourth translation training loss value of the second translation auxiliary model based on fourth probabilities corresponding to respective output positions, where the fourth translation training loss value represents an error between the predicted translation sentence and the second sample translation sentence;

[0321] Performing a weighted summation on the third translation training loss value and the fourth translation training loss value to determine a second joint training loss value;

[0322] Based on the second joint training loss value, parameters of the second translation model and the second translation auxiliary model are adjusted respectively.

[0323] In a possible embodiment, the second translation model at least includes the same structure as the second translation auxiliary model, and the second translation model shares parameters with the same structure as the second translation auxiliary model.

[0324] As an embodiment, the model training module 1403 is an optional module.

[0325] It should be noted that Fig.14 The device shown can also implement any of the sentence translation methods discussed above, which will not be described in detail here.

[0326] Based on the same inventive concept, the embodiment of the present application provides a computer device, which can implement the functions of the server or terminal in the above text, please refer to Fig.15 , the computer device includes a processor 1501 and a memory 1502.

[0327] The processor 1501 may be a central processing unit (CPU) or a digital processing unit, etc. The specific connection medium between the memory 1502 and the processor 1501 is not limited in the embodiment of the present application. Fig.15 In the embodiment, the memory 1502 and the processor 1501 are connected via a bus 1503. The bus 1503 is connected to the processor 1501 via a bus 1503. Fig.15 The connections between other components are shown in bold lines, and are not intended to be limiting. Bus 1503 can be divided into address bus, data bus, control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0328] The memory 1502 may be a volatile memory, such as a random-access memory (RAM); the memory 1502 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 1502 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1502 may be a combination of the above memories.

[0329] The processor 1501 is used to execute the sentence translation method discussed above when calling the computer program stored in the memory 1502, and can also be used to implement the above Fig.14 Functions of the device shown.

[0330] Based on the same inventive concept, an embodiment of the present application provides a computer storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes any of the sentence translation methods discussed above.

[0331] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0332] Based on the same inventive concept, an embodiment of the present application provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned sentence translation method.

[0333] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, etc., various media that can store program codes.

[0334] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0335] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A sentence translation method, characterized in that: include: Obtain a target sentence to be translated; Inputting the target sentence into the trained target translation model to obtain a target translation sentence; The target translation model is obtained by adjusting the parameters of the first translation model to be trained based on a fidelity training loss value in the process of training the first translation model to be trained using a first sample training data set, each first sample training data includes a first sample sentence to be translated and a corresponding first sample translation sentence, the fidelity training loss value is obtained based on the difference between the first probability and the second probability corresponding to each output position, the first probability corresponding to each output position is the output probability of the first target element obtained based on the first translation model and the corresponding first sample training data, and the second probability corresponding to each output position is the output probability of the first target element obtained based on the first translation auxiliary model and the corresponding first sample translation sentence; The fidelity training loss value is obtained by the following steps: Determine an element-level training loss value based on a probability difference between a first probability and a second probability corresponding to each of the output positions, wherein the element-level training loss value is negatively correlated with the probability difference of each of the output positions; Based on the probability difference between the first probability and the second probability corresponding to each of the output positions, a sentence-level training loss value is determined, wherein if the first ratio is greater than a ratio threshold, the sentence-level training loss value is zero, and if the first ratio is less than or equal to the ratio threshold, the sentence-level training loss value is negatively correlated with the probability difference of each output position, and the first ratio is: based on the probability difference of each output position, the ratio of the number of elements having a probability difference less than the difference threshold to the total number of each element in the first sample translation sentence; The fidelity training loss value is obtained based on at least one of the element-level training loss value and the sentence-level training loss value.

2. The method according to claim 1, characterized in that The target translation model is trained in the following way: Based on the first sample training data set, the first translation model to be trained is trained multiple times until a first model convergence condition is met, and the first translation model that meets the first preset condition in the multiple times of iterative training is used as the target translation model, wherein one iterative training includes the following operations: Selecting first sample training data from the first sample training data set; Inputting the first sample training data into the first translation model to obtain first probabilities corresponding to respective output positions, wherein the first probability corresponding to each output position is determined based on the first sample sentence to be translated in the first sample training data and the historical translation text associated with the corresponding output position, wherein each element in the first sample translation sentence in the first sample training data corresponds to each output position one by one, and the historical translation text is the text content in the first sample translation sentence that is located before the element corresponding to the corresponding output position; Inputting the first sample translation sentence into the first translation auxiliary model to obtain second probabilities corresponding to the respective output positions, wherein the second probability corresponding to each output position is determined based on the historical translation text associated with the corresponding output position; Obtaining a fidelity training loss value based on a difference between the first probability and the second probability corresponding to each of the output positions; Based on the fidelity training loss value, parameters of the first translation model are adjusted.

3. The method according to claim 2, characterized in that The step of inputting the first sample training data into the first translation model to obtain first probabilities corresponding to respective output positions includes: Based on the first translation model, encoding the first sample sentence to be translated in the first sample training data to obtain an encoding result; Based on the first translation model, the following operations are performed for each output position: For one output position, based on the encoding result, decoding the historical translation text associated with the one output position to obtain a first decoding result of the one output position; The first decoding result of the one output position is normalized to obtain a first probability corresponding to the one output position.

4. The method according to claim 2, characterized in that Inputting the first sample translation sentence into the first translation auxiliary model to obtain second probabilities corresponding to each of the output positions includes: Based on the first translation auxiliary model, the following operations are performed for each output position: Decoding the historical translation text associated with an output position to obtain a second decoding result of the output position; The second decoding result of the one output position is normalized to obtain a second probability corresponding to the one output position.

5. The method according to claim 1, characterized in that The obtaining the fidelity training loss value based on at least one of the element-level training loss value and the sentence-level training loss value comprises: Based on at least one of the element-level training loss value and the sentence-level training loss value, obtaining an initial fidelity training loss value; The fidelity training loss value is determined based on the initial fidelity training loss value and the initial translation training loss value, wherein the initial translation training loss value is determined based on the first probability of each output position and is used to represent an error between the translation sentence predicted by the first translation model and the first sample translation sentence.

6. The method according to claim 1, characterized in that The determining of the element-level training loss value based on the probability difference between the first probability and the second probability of each of the output positions comprises: For each output location, perform the following operations respectively: Substituting the probability difference of an output position into a preset function to obtain a first training loss value of the output position, wherein the preset function is a monotonically decreasing function; Determining a weight of the one output position based on the first probability corresponding to the one output position, wherein the weight of the one output position is negatively correlated with the first probability corresponding to the one output position; Based on the weight of the one output position, weighting the first training score loss value to obtain a second training score loss value of the one output position; Based on the obtained second training loss values ​​corresponding to the respective output positions, the element-level training loss value is obtained.

7. The method according to any one of claims 2 to 4, characterized in that: Before adjusting the parameters of the first translation model based on the fidelity training loss value, the method further includes: Determining a first translation training loss value of the first translation model based on the first probabilities corresponding to the respective output positions, wherein the first translation training loss value represents an error between the predicted translation sentence and the first sample translation sentence; Determining a second translation training loss value of the first translation auxiliary model based on the second probabilities corresponding to the respective output positions, wherein the second translation training loss value represents an error between the predicted translation sentence and the first sample translation sentence; Performing a weighted summation on the first translation training loss value and the second translation training loss value to obtain a first joint training loss value; When adjusting the parameters of the first translation model based on the fidelity training loss value, the method further includes: Based on the first joint training loss value, parameters of the first translation model and the first translation auxiliary model are adjusted.

8. The method according to any one of claims 2 to 6, characterized in that: Before performing multiple iterations of training on the first translation model to be trained, the method further includes: Based on the second sample training data set, the second translation model to be trained and the second translation auxiliary model to be trained are subjected to multiple rounds of joint pre-training until the second model convergence condition is met, the second translation model that meets the second preset condition in the multiple rounds of joint pre-training is used as the first translation model, and the second translation auxiliary model that meets the third preset condition in the multiple rounds of joint pre-training is used as the first translation auxiliary model, each second sample training data includes a second sample sentence to be translated and a corresponding second sample translation sentence, and each round of joint pre-training includes the following operations: Selecting second sample training data from the second sample training data set; Input the second sample training data into the second translation model to obtain third probabilities corresponding to respective output positions, wherein the third probability corresponding to each output position is the output probability of the second target element at the corresponding output position; Inputting the second sample translation sentence in the second sample training data into the second translation auxiliary model to obtain a fourth probability, wherein the fourth probability of each output position is the output probability of the second target element at the corresponding output position; Determining a third translation training loss value of the second translation model based on the third probabilities corresponding to the respective output positions, wherein the third translation training loss value represents an error between the predicted translation sentence and the second sample translation sentence; Determining a fourth translation training loss value of the second translation auxiliary model based on the fourth probabilities corresponding to the respective output positions, wherein the fourth translation training loss value represents an error between the predicted translation sentence and the second sample translation sentence; Performing a weighted summation on the third translation training loss value and the fourth translation training loss value to determine a second joint training loss value; Based on the second joint training loss value, parameters of the second translation model and the second translation auxiliary model are adjusted respectively.

9. The method according to claim 8, characterized in that The second translation model at least includes the same structure as the second translation auxiliary model, and the second translation model shares parameters with the same structure as the second translation auxiliary model.

10. A sentence translation device, characterized in that: include: A sentence acquisition module, used to obtain a target sentence to be translated; A sentence translation module, used for inputting the target sentence into a trained target translation model to obtain a target translation sentence; The target translation model is obtained by adjusting the parameters of the first translation model to be trained based on a fidelity training loss value in the process of training the first translation model to be trained using a first sample training data set, each first sample training data includes a first sample sentence to be translated and a corresponding first sample translation sentence, the fidelity training loss value is obtained based on the difference between the first probability and the second probability corresponding to each output position, the first probability corresponding to each output position is the output probability of the first target element obtained based on the first translation model and the corresponding first sample training data, and the second probability corresponding to each output position is the output probability of the first target element obtained based on the first translation auxiliary model and the corresponding first sample translation sentence; The model training module is used to obtain the fidelity training loss value through the following steps: Determine an element-level training loss value based on a probability difference between a first probability and a second probability corresponding to each of the output positions, wherein the element-level training loss value is negatively correlated with the probability difference of each of the output positions; Based on the probability difference between the first probability and the second probability corresponding to each of the output positions, a sentence-level training loss value is determined, wherein if the first ratio is greater than a ratio threshold, the sentence-level training loss value is zero, and if the first ratio is less than or equal to the ratio threshold, the sentence-level training loss value is negatively correlated with the probability difference of each output position, and the first ratio is: based on the probability difference of each output position, the ratio of the number of elements having a probability difference less than the difference threshold to the total number of each element in the first sample translation sentence; The fidelity training loss value is obtained based on at least one of the element-level training loss value and the sentence-level training loss value.

11. The device according to claim 10, characterized in that The model training module is specifically used to obtain the target translation model by training in the following manner: Based on the first sample training data set, the first translation model to be trained is trained multiple times until a first model convergence condition is met, and the first translation model that meets the first preset condition in the multiple times of iterative training is used as the target translation model, wherein one iterative training includes the following operations: Selecting first sample training data from the first sample training data set; Inputting the first sample training data into the first translation model to obtain first probabilities corresponding to respective output positions, wherein the first probability corresponding to each output position is determined based on the first sample sentence to be translated in the first sample training data and the historical translation text associated with the corresponding output position, wherein each element in the first sample translation sentence in the first sample training data corresponds to each output position one by one, and the historical translation text is the text content in the first sample translation sentence that is located before the element corresponding to the corresponding output position; Inputting the first sample translation sentence into the first translation auxiliary model to obtain second probabilities corresponding to the respective output positions, wherein the second probability corresponding to each output position is determined based on the historical translation text associated with the corresponding output position; Obtaining a fidelity training loss value based on a difference between the first probability and the second probability corresponding to each of the output positions; Based on the fidelity training loss value, parameters of the first translation model are adjusted.

12. A computer device, characterized in that: include: at least one processor, and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the method according to any one of claims 1 to 9 by executing the instructions stored in the memory.

13. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer executes the method according to any one of claims 1 to 9.

Images