A translation model training method, translation method, device, equipment and medium

Abstract

The present invention discloses a translation model training method, a translation method, an apparatus, a device and a medium. The translation model training method comprises: inputting the source text in the training sample into the encoder of a preset model to obtain an encoding result, decoding each word segmentation vector in the encoding result through the first decoder of the preset model and arranging and combining to output the initial text, masking the initial text to obtain the masked text, inputting the masked text into the second decoder of the preset model, decoding and outputting the predicted text through the second decoder of the preset model, constructing a negative candidate word list corresponding to each position in the predicted text, optimizing the parameters of the preset model according to the predicted text, the negative candidate word list and the translated text, and obtaining a trained preset model. The translation method comprises: inputting the text to be translated into the trained preset model and outputting the target text. The training method of the model is optimized, and the translation quality is improved while ensuring the translation efficiency of the trained model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a translation model training method, a translation method, a device, a device, and a medium. Background Art

[0002] With the increase in global medical communication, the need for medical information translation in multilingual environments is becoming increasingly urgent. To address this issue, the industry has begun exploring automated translation systems based on intelligent models. These systems, through trained translation models, can automatically translate between corresponding languages. However, existing intelligent models still have significant shortcomings in handling complex language structures and cultural differences. In particular, when dealing with multiple languages, it is often difficult to simultaneously ensure both efficiency and accuracy. For example, language models based on non-autoregressive neural machine translation have significant advantages in translation efficiency. These models attempt to generate translation results for entire sentences at once, significantly reducing translation time. However, due to the lack of fine-tuning on a word-by-word or sentence-by-sentence basis, they often lack translation accuracy. In the medical field, such translation inaccuracies can have serious consequences. Misdiagnosis, inappropriate treatment plans, or misinterpreted patient information can lead to worsening of a patient's health or even endanger their lives.

[0003] Therefore, how to optimize the model training method to improve the translation quality while ensuring the translation efficiency of the trained model has become an urgent problem to be solved. Summary of the Invention

[0004] Based on this, a translation model training method, translation method, device, equipment and medium are provided to solve the problem of how to optimize the model training method to improve the translation quality while ensuring the translation efficiency of the trained model.

[0005] In a first aspect, an embodiment of the present invention provides a translation model training method, comprising:

[0006] Obtaining a training sample, wherein the training sample includes a source text and a translated text;

[0007] Input the source text into an encoder of a preset model to obtain an encoding result, wherein the encoding result includes a word segmentation vector corresponding to each word in the source text, decode each word segmentation vector in the encoding result through a first decoder of the preset model to obtain an initial word segmentation sequence, and permutate and combine the initial word segmentation sequence to output an initial text;

[0008] Masking all elements in the initial text to obtain masked text, inputting the masked text into the second decoder of the preset model, and decoding the second decoder of the preset model to output predicted text;

[0009] According to the predicted text, a negative candidate word list corresponding to each position in the predicted text is determined; according to the predicted text, the negative candidate word list and the translated text, parameters of the preset model are optimized to obtain a trained preset model.

[0010] In a second aspect, an embodiment of the present invention provides a translation method, comprising:

[0011] Obtaining a trained preset model obtained by the translation model training method of the first aspect above;

[0012] Obtain the text to be translated, input the text to be translated into the trained preset model, and output the target text.

[0013] In a third aspect, an embodiment of the present invention provides a translation model training device, comprising:

[0014] A sample acquisition module, configured to acquire a training sample, wherein the training sample includes a source text and a translated text;

[0015] A first training module is configured to input the source text into an encoder of a preset model to obtain an encoding result, wherein the encoding result includes a word segmentation vector corresponding to each word in the source text, decode each word segmentation vector in the encoding result through a first decoder of the preset model to obtain an initial word segmentation sequence, and permutate and combine the initial word segmentation sequence to output an initial text;

[0016] A second training module is configured to mask all elements in the initial text to obtain masked text, input the masked text into a second decoder of the preset model, and decode the masked text through the second decoder of the preset model to output a predicted text;

[0017] The first optimization module is used to determine a negative candidate word list corresponding to each position in the predicted text according to the predicted text, and optimize the parameters of the preset model according to the predicted text, the negative candidate word list and the translated text to obtain a trained preset model.

[0018] In a fourth aspect, an embodiment of the present invention provides a translation device, comprising:

[0019] A model acquisition module, configured to acquire a trained preset model obtained by the translation model training method of the first aspect;

[0020] The first translation module is used to obtain a text to be translated, input the text to be translated into the trained preset model, and output a target text.

[0021] In a fifth aspect, an embodiment of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the translation model training method of the first aspect or the translation method of the second aspect is implemented.

[0022] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the translation model training method of the first aspect or the translation method of the second aspect is implemented.

[0023] The present invention is different from the technical effects achieved by the existing technical solutions: the training samples of the translation model training method of the present invention include source text and translation text, the source text is input into the encoder of the preset model to obtain an encoding result, the encoding result includes the word segmentation vector corresponding to each word in the source text, each word segmentation vector in the encoding result is decoded by the first decoder of the preset model to obtain an initial word segmentation sequence, and the initial word segmentation sequence is arranged and combined to output the initial text, all elements in the initial text are masked to obtain masked text, the masked text is input into the second decoder of the preset model, and the second decoder of the preset model is decoded to output a predicted text, according to the predicted text, the negative candidate word list corresponding to each position in the predicted text is determined, according to the predicted text, the negative candidate word list and the translation text, the preset model is parameter optimized to obtain a trained preset model. When translating according to the trained preset model, the text to be translated is input into the trained preset model, and the target text is output.

[0024] Among them, the preset model adopts a dual decoder (first decoder and second decoder) architecture. The first decoder directly acts on the word segmentation vector corresponding to each word in the encoding result, performs independent decoding and permutations and combinations. This process does not require in-depth consideration of contextual semantic information, thereby significantly shortening the translation processing time and improving translation efficiency. The second decoder further corrects and optimizes the initial text on this basis to generate the final predicted text. This staged decoding process helps the model to gradually refine the translation results. While ensuring the translation efficiency of the trained model, it also improves the translation accuracy. By constructing a negative candidate word list and optimizing the model parameters based on the negative candidate word list, it helps the model avoid selecting inappropriate words when generating translations, prompting the model to learn how to avoid wrong choices, thereby further improving the translation accuracy of the trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 1 is a schematic diagram of an application environment of a translation model training method and a translation method provided in the first embodiment of the present invention;

[0027] Figure 2 1 is a flowchart of a translation model training method provided in the second embodiment of the present invention;

[0028] Figure 3 1 is a flowchart of a translation model training method provided in Example 3 of the present invention;

[0029] Figure 4 1 is a flowchart of a translation model training method provided in a fourth embodiment of the present invention;

[0030] Figure 5 1 is a flowchart of a translation model training method provided in Example 5 of the present invention;

[0031] Figure 6 This is a flowchart of a translation method provided in Example 6 of the present invention;

[0032] Figure 7 This is a flowchart of a translation method provided in Example 7 of the present invention;

[0033] Figure 8 Schematic diagram of the structure of a translation model training device provided by Embodiment 8 of the present invention;

[0034] Figure 9 This is a schematic diagram of the structure of a translation device provided by Embodiment 9 of the present invention;

[0035] Figure 10 It is a structural diagram of a computer device provided in the tenth embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0037] It should be understood that the order of execution of the steps in the following embodiments does not necessarily mean the order in which they are executed. The order in which each process is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0038] like Figure 1 The figure shows a schematic diagram of an application environment for a translation model training method and translation method provided in a first embodiment of the present invention. A server communicates with a client, with the server providing translation model training and translation functions. The client sends training samples to the server for model training, and the client sends text to be translated to the server for translation. Clients include, but are not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The computer device corresponding to the server can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0039] like Figure 2 FIG. 1 is a flow chart of a translation model training method provided in Embodiment 2 of the present invention, comprising the following steps:

[0040] Step S201: Obtain training samples.

[0041] Step S202: Input the source text into the encoder of the preset model to obtain an encoding result, decode each word segmentation vector in the encoding result through the first decoder of the preset model to obtain an initial word segmentation sequence, and arrange and combine the initial word segmentation sequence to output the initial text.

[0042] In this embodiment, the training sample includes source text and translated text. The source text may refer to the original text data that has not been translated, and the translated text may refer to the text data translated from the source text. The encoding result includes the word segmentation vector corresponding to each word segment in the source text. The word segmentation may refer to the word or phrase obtained by dividing the source text. The word segmentation vector may refer to the vector representation of the word segmentation. The preset model may refer to the pre-set initial model for model training. The initial text may refer to the possible noisy translation text obtained by translating the source text by the first decoder of the preset model during the training process. The encoder may refer to the encoder that vectorizes the source text. The first decoder is used to decode the vector representation generated by the encoder to generate the initial text corresponding to the source text.

[0043] Specifically, in the process of training the preset model, the source text is input into the embedding layer input end of the encoder of the preset model, the source text is converted into a continuous vector representation through the embedding layer of the encoder of the preset model, the continuous vector representation of the source text is encoded by the encoder of the preset model to obtain the word segmentation vector corresponding to each word in the source text, the word segmentation vector corresponding to each word in the source text is input into the first decoder of the preset model, each word segmentation vector in the encoding result is decoded by the first decoder of the preset model to obtain an initial word segmentation sequence, and the initial word segmentation sequence is arranged and combined to output the initial text.

[0044] Step S203: Mask all elements in the initial text to obtain masked text, input the masked text into the second decoder of the preset model, and decode the second decoder of the preset model to output the predicted text.

[0045] In this embodiment, the masked text may refer to a text in which each element (e.g., words and characters) in the initial text is replaced with a specific form of representation or marking, the predicted text may refer to the training prediction result of the second encoder of the preset model translating the source text during the training process, and the second decoder may refer to generating predicted text from the masked text.

[0046] Specifically, all words or characters in the initial text are replaced with a specific form of representation to obtain masked text, the masked text is input into the embedding layer input end of the second decoder of the preset model, the masked text is converted into a continuous vector representation through the embedding layer of the second decoder of the preset model, the continuous vector representation of the masked text is input into the second decoder of the preset model, and the predicted text is decoded and output by the second decoder of the preset model.

[0047] Step S204: According to the predicted text, a negative candidate word list corresponding to each position in the predicted text is determined, and parameters of the preset model are optimized according to the predicted text, the negative candidate word list and the translated text to obtain a trained preset model.

[0048] In this embodiment, the negative candidate word list may include all words in the positions before and after the current position of the predicted text, which is used to indicate that words with low probability need to be assigned when the current position is generated. The trained preset model may refer to a preset model that meets the model training termination condition. The training termination condition may be that the loss function value drops below a preset threshold. At this time, it can be considered that the model has achieved sufficient accuracy and the training can be terminated.

[0049] Specifically, for any position in the predicted text, a negative candidate word list corresponding to the position is constructed based on the elements in the predicted text except the element corresponding to the position. The loss penalty value is calculated based on the elements at each position in the predicted text and the negative candidate word list at the corresponding position. According to the preset loss function and loss penalty value, the error value between the predicted text and the translated text is calculated. The parameters of the preset model are optimized based on the calculated error value to obtain a trained model.

[0050] The above-mentioned preset model can be a model formed by an architecture such as a neural network, a deep learning network, or a machine learning network, for example, a translation model based on natural language processing, a long short-term memory network model, and a sequence-to-sequence model.

[0051] In this embodiment, the preset model adopts a dual-decoder (first decoder and second decoder) architecture. The first decoder directly acts on the word segmentation vector corresponding to each word in the encoding result, performs independent decoding and permutations and combinations. This process does not require in-depth consideration of contextual semantic information, thereby significantly shortening the translation processing time and improving translation efficiency. The second decoder further corrects and optimizes the initial text on this basis to generate the final predicted text. This staged decoding process helps the model to gradually refine the translation results, while ensuring the translation efficiency of the trained model, it also improves the translation accuracy. By constructing a negative candidate word list and optimizing the model parameters based on the negative candidate word list, the model is helped to avoid selecting inappropriate words when generating translations, prompting the model to learn how to avoid wrong choices, thereby further improving the translation accuracy of the trained model.

[0052] like Figure 3 FIG. 1 is a flow chart of a translation model training method provided in a third embodiment of the present invention. In step S204 of the second embodiment above, a negative candidate word list corresponding to each position in the predicted text is determined based on the predicted text. Parameters of the preset model are optimized based on the predicted text, the negative candidate word list, and the translated text to obtain a trained preset model. The method may include the following steps:

[0053] Step S301: for any position in the predicted text, determine a negative candidate word list corresponding to the position based on elements in the predicted text except for the element corresponding to the position.

[0054] Step S302: Calculate the loss penalty value based on the elements at each position of the predicted text and the negative candidate word list at the corresponding position.

[0055] Step S303: Optimize the parameters of the preset model according to the loss penalty value, the predicted text, and the translated text to obtain a trained preset model.

[0056] In this embodiment, the loss penalty value may refer to a loss function penalty value calculated based on a negative candidate word list corresponding to a position in the predicted text.

[0057] Specifically, for any position in the predicted text, detect whether the element corresponding to the position is the same as any element in the negative candidate word list corresponding to the position. If they are the same, it means that a duplicate word is generated at that position. The loss penalty value is calculated based on the number of duplicate words appearing in all positions of the predicted text. Based on the loss penalty value, the loss value between the predicted text and the translated text is calculated. The parameters of the preset model are optimized based on the loss value to obtain a trained model.

[0058] Optionally, in addition to including all words at the positions before and after the current position of the predicted text, the negative candidate word list may also include words outside the predicted text that need to be assigned a low probability when the current position is generated.

[0059] For example, if the predicted text is "The patient is preliminarily diagnosed with lung cancer with a high probability, and further CT examination is recommended for confirmation", the negative candidate word list constructed for the position of the word "lung cancer" may include words such as patient, preliminary, diagnosis, possibility and high in the corresponding positions in the predicted text. It may also include words outside the predicted text that need to be assigned a low probability when generating the position of "lung cancer", such as cold, cough and vomiting, which are names of diseases that are highly correlated with "lung cancer".

[0060] In this embodiment, by maintaining a negative candidate word list, repeated words generated in the predicted text are detected based on the negative candidate word list, and a corresponding loss penalty value is obtained based on the detection result. Then, according to the loss penalty value, the parameters of the preset model are optimized, thereby reducing the probability of generating repeated words during the model training process and improving the translation accuracy of the trained preset model.

[0061] like Figure 4 As shown, a flowchart of a translation model training method provided by the fourth embodiment of the present invention is provided. In step S303 of the above-mentioned third embodiment, the parameters of the preset model are optimized according to the loss penalty value, the predicted text, and the translated text to obtain a trained preset model. The method may include the following steps:

[0062] Step S401: Calculate the loss of the predicted text and the translated text using a preset loss function and loss penalty value to obtain a loss value.

[0063] Step S402: Optimize the parameters of the preset model according to the loss value to obtain a trained preset model.

[0064] In this embodiment, the preset loss function may refer to a preset function for measuring the difference between the predicted text and the translated text, and the loss value may refer to the error value between the predicted text and the translated text calculated based on the preset loss function and the loss penalty value.

[0065] Specifically, the loss penalty value is added to the preset loss function to obtain an updated loss function. The loss value between each element in the predicted text and the element at the corresponding position in the translated text is calculated through the updated loss function. The loss values ​​calculated between all elements in the predicted text and all elements in the translated text are summed to obtain the loss value. According to the loss value, the parameters of the encoder, the first decoder and the second decoder of the preset model are optimized to obtain a trained preset model.

[0066] In this embodiment, the loss value between the predicted text and the translated text is calculated based on the loss penalty value and the preset loss function to optimize the parameters of the preset model, thereby improving the accuracy of the trained preset model.

[0067] like Figure 5 FIG. 1 is a flow chart of a translation model training method provided in Embodiment 5 of the present invention. The translation model training method further includes the following steps:

[0068] Step S501: input the source text into the encoder of the preset model, and decode it through the length prediction module of the first decoder of the preset model to output the predicted length.

[0069] Step S502: Optimize the parameters of the first decoder of the preset model according to the predicted length and the translation text length to obtain a pre-trained first decoder of the preset model.

[0070] In this embodiment, the length prediction module may refer to a module that predicts the length of the text after the source text is translated, and the predicted length may refer to the length of the translated text corresponding to the source text predicted by the length prediction module of the first decoder of the preset model during the training process.

[0071] Specifically, according to a preset loss function, an error value between the predicted length and the translated text length is calculated, and parameters of the first decoder of the preset model are optimized according to the calculated error value to obtain a pre-trained first decoder of the preset model.

[0072] In this embodiment, by training the length prediction module of the first decoder of the preset model, it helps the model to better control the output length during the generation process, avoid generating translations that are too long or too short, and thus improve the accuracy of the trained preset model.

[0073] like Figure 6FIG. 1 is a flow chart of a translation method provided in Embodiment 6 of the present invention, which may include the following steps:

[0074] Step S601: Obtain a trained preset model obtained by a translation model training method.

[0075] Step S602: Obtain the text to be translated, input the text to be translated into a trained preset model, and output the target text.

[0076] In this embodiment, the text to be translated may refer to the text to be translated, and the target text may refer to the text after the text to be translated is translated.

[0077] Specifically, the text to be translated is input into a trained preset model, and is processed by the encoder, the first decoder, and the second decoder of the trained preset model to output the target text.

[0078] For example, in a healthcare scenario, a large international hospital has many foreign patients, whose medical records often contain extensive technical terms and complex descriptions. To ensure the accuracy and efficiency of medical services, the hospital needs to translate these records into multiple languages ​​for both doctors and patients to understand accurately. To do this, the medical records requiring translation can be extracted from the patient's medical record system. This extracted medical record text serves as the target text for translation, which is then fed into a trained model. The trained model then outputs the target text, the translated version of the target text.

[0079] In this embodiment, the preset model trained by the model training method is used to translate the text to obtain the target text, thereby improving the efficiency and accuracy of the translation.

[0080] like Figure 7 FIG. 1 is a flow chart of a translation method provided in a seventh embodiment of the present invention. In step S602 of the sixth embodiment above, the text to be translated is input into a trained preset model, and the target text is output. The method may include the following steps:

[0081] Step S701: input the text to be translated into the encoder of the trained preset model, and decode it through the first decoder of the trained preset model to output the intermediate text and the target length.

[0082] Step S702: Mask all elements in the intermediate text to obtain the masked intermediate text, input the masked intermediate text into the second decoder of the trained preset model, and output the target text.

[0083] In this embodiment, the intermediate text may refer to a possible noisy translation text obtained by translating the text to be translated using a trained preset model. The target length may refer to the length prediction module of the first decoder of the trained preset model, the length of the translated text corresponding to the text to be translated predicted, and the target text may refer to the text after the text to be translated is translated using a trained preset model.

[0084] Specifically, the text to be translated is input into the embedding layer input end of the encoder of the trained preset model, the text to be translated is converted into a continuous vector representation by the embedding layer of the encoder of the trained preset model, the continuous vector representation of the text to be translated is encoded by the encoder of the trained preset model to obtain the word segmentation vector corresponding to each word in the text to be translated, the word segmentation vector corresponding to each word in the text to be translated is input into the first decoder of the trained preset model, the word segmentation vector corresponding to each word in the text to be translated is decoded by the first decoder of the trained preset model, and the intermediate text and the target length are output in an arranged and combined manner. All words or characters in the intermediate text are replaced with a specific form representation to obtain a masked intermediate text, the masked intermediate text is input into the embedding layer input end of the second decoder of the trained preset model, the masked intermediate text is converted into a continuous vector representation by the embedding layer of the second decoder of the trained preset model, the continuous vector representation of the masked intermediate text is input into the second decoder of the trained preset model, and the target text is decoded by the second decoder of the trained preset model.

[0085] In this embodiment, the preset model trained by the model training method is used to translate the text to obtain the target text, thereby improving the efficiency and accuracy of the translation.

[0086] like Figure 8 As shown, a translation model training device provided in Example 8 of the present invention corresponds one-to-one to the translation model training method in the above-mentioned embodiment. The translation model training device includes a sample acquisition module 81, a first training module 82, a second training module 83, and a first optimization module 84. Each functional module is described in detail as follows:

[0087] A sample acquisition module 81 is used to acquire a training sample, wherein the training sample includes a source text and a translated text;

[0088] A first training module 82 is configured to input the source text into an encoder of a preset model to obtain an encoding result, wherein the encoding result includes a word segmentation vector corresponding to each word in the source text, decode each word segmentation vector in the encoding result through a first decoder of the preset model to obtain an initial word segmentation sequence, and permutate and combine the initial word segmentation sequence to output an initial text;

[0089] A second training module 83 is configured to mask all elements in the initial text to obtain masked text, input the masked text into a second decoder of the preset model, and decode the masked text through the second decoder of the preset model to output a predicted text;

[0090] The first optimization module 84 is used to determine a negative candidate word list corresponding to each position in the predicted text according to the predicted text, and optimize the parameters of the preset model according to the predicted text, the negative candidate word list and the translated text to obtain a trained preset model.

[0091] Optionally, the first optimization module 84 includes:

[0092] a list building unit, configured to determine, for any position in the predicted text, a negative candidate word list corresponding to the position based on elements in the predicted text other than the element corresponding to the position;

[0093] a penalty value calculation unit, configured to calculate a loss penalty value based on an element at each position of the predicted text and a negative candidate word list at the corresponding position;

[0094] The second optimization unit is used to optimize the parameters of the preset model according to the loss penalty value, the predicted text and the translated text to obtain a trained preset model.

[0095] Optionally, the second optimization unit includes:

[0096] a loss calculation subunit, configured to perform loss calculation on the predicted text and the translated text using a preset loss function and the loss penalty value to obtain a loss value;

[0097] The third optimization subunit is used to optimize the parameters of the preset model according to the loss value to obtain the trained preset model.

[0098] Optionally, the model training device further includes:

[0099] A third training module is configured to input the source text into the encoder of the preset model, and decode the source text through the length prediction module of the first decoder of the preset model to output a predicted length;

[0100] The third optimization module is used to optimize the parameters of the first decoder of the preset model according to the predicted length and the translation text length to obtain a pre-trained first decoder of the preset model.

[0101] The specific definition of the translation model training device can be found in the definition of the translation model training method above and will not be repeated here. The various modules in the above-mentioned translation model training device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0102] like Figure 9 As shown, a translation device provided in the ninth embodiment of the present invention corresponds to the translation method in the above embodiment. The translation device includes a model acquisition module 91 and a first translation module 92. The functional modules are described in detail as follows:

[0103] A model acquisition module 91 is used to acquire a trained preset model obtained by a translation model training method;

[0104] The first translation module 92 is configured to obtain a text to be translated, input the text to be translated into the trained preset model, and output a target text.

[0105] Optionally, the first translation module 92 includes:

[0106] A second translation unit is configured to input the text to be translated into the encoder of the trained preset model, and decode the text through the first decoder of the trained preset model to output the intermediate text and the target length;

[0107] The third translation unit is used to mask all elements in the intermediate text to obtain the masked intermediate text, input the masked intermediate text into the second decoder of the trained preset model, and output the target text.

[0108] The specific definition of the translation device can be found in the definition of the translation method above and will not be repeated here. The various modules in the above-mentioned translation device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0109] Figure 10 This is a schematic diagram of the structure of a computer device provided by the tenth embodiment of the present invention. Figure 10 As shown, the computer device of this embodiment includes: at least one processor ( Figure 10Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, the steps of any of the above-mentioned translation model training method embodiments or the steps of any of the above-mentioned translation method embodiments are implemented.

[0110] The computer device may include, but is not limited to, a processor and a memory. It will be understood by those skilled in the art that Figure 10 The above is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0111] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0112] Memory includes readable storage media, internal memory, and the like. Internal memory can be the internal memory of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage medium. The readable storage medium can be the computer device's hard drive. In other embodiments, it can also be an external storage device, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, or a flash memory card. Furthermore, memory can include both the computer device's internal storage unit and external storage devices. Memory is used to store the operating system, application programs, boot loaders, data, and other programs, such as the program code of computer programs. Memory can also be used to temporarily store data that has been output or is about to be output.

[0113] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. Computer-readable media can include at least: any entity or device capable of carrying computer program code, recording media, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunications signals, and software distribution media. Examples include USB flash drives, removable hard drives, magnetic disks, or optical disks. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunications signals.

[0114] The present invention may implement all or part of the processes in the above-mentioned method embodiments, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0115] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0116] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0117] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0118] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0119] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein, and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention. The non-company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A translation model training method, characterized in that: include: Obtaining a training sample, wherein the training sample includes a source text and a translated text; Input the source text into an encoder of a preset model to obtain an encoding result, wherein the encoding result includes a word segmentation vector corresponding to each word in the source text, decode each word segmentation vector in the encoding result through a first decoder of the preset model to obtain an initial word segmentation sequence, and permutate and combine the initial word segmentation sequence to output an initial text; Masking all elements in the initial text to obtain masked text, inputting the masked text into the second decoder of the preset model, and decoding the second decoder of the preset model to output predicted text; According to the predicted text, a negative candidate word list corresponding to each position in the predicted text is determined, and according to the predicted text, the negative candidate word list and the translated text, the parameters of the preset model are optimized to obtain a trained preset model, wherein, for any position in the predicted text, a negative candidate word list corresponding to the position is determined based on elements in the predicted text except the elements corresponding to the position, and the negative candidate word list includes all elements of the positions before and after the position corresponding to the position in the predicted text, which are used to indicate that elements with a probability lower than a preset probability need to be assigned when the position is generated.

2. The translation model training method according to claim 1, wherein: Optimizing parameters of the preset model according to the predicted text, the negative candidate word list, and the translated text to obtain a trained preset model includes: Calculating a loss penalty value based on the elements at each position of the predicted text and the negative candidate word list at the corresponding position; Parameters of the preset model are optimized according to the loss penalty value, the predicted text, and the translated text to obtain a trained preset model.

3. The translation model training method according to claim 2, wherein: Optimizing parameters of the preset model according to the loss penalty value, the predicted text, and the translated text to obtain a trained preset model includes: Performing loss calculation on the predicted text and the translated text using a preset loss function and the loss penalty value to obtain a loss value; According to the loss value, the parameters of the preset model are optimized to obtain the trained preset model.

4. The translation model training method according to claim 1, wherein: The training sample also includes the length of the translated text, and the translation model training method further includes: Input the source text into the encoder of the preset model, and decode it through the length prediction module of the first decoder of the preset model to output the predicted length; According to the predicted length and the translated text length, parameters of the first decoder of the preset model are optimized to obtain a pre-trained first decoder of the preset model.

5. A translation method, characterized in that: include: Obtaining a trained preset model obtained by the translation model training method according to any one of claims 1 to 4; Obtain the text to be translated, input the text to be translated into the trained preset model, and output the target text.

6. The translation method according to claim 5, wherein: Inputting the text to be translated into the trained preset model and outputting the target text includes: Inputting the text to be translated into the encoder of the trained preset model, and decoding the text using the first decoder of the trained preset model to output the intermediate text and the target length; All elements in the intermediate text are masked to obtain the masked intermediate text, the masked intermediate text is input into the second decoder of the trained preset model, and the target text is output.

7. A translation model training device, characterized in that: include: A sample acquisition module, configured to acquire a training sample, wherein the training sample includes a source text and a translated text; A first training module is configured to input the source text into an encoder of a preset model to obtain an encoding result, wherein the encoding result includes a word segmentation vector corresponding to each word in the source text, decode each word segmentation vector in the encoding result through a first decoder of the preset model to obtain an initial word segmentation sequence, and permutate and combine the initial word segmentation sequence to output an initial text; A second training module is configured to mask all elements in the initial text to obtain masked text, input the masked text into a second decoder of the preset model, and decode the masked text through the second decoder of the preset model to output a predicted text; A first optimization module is used to determine, based on the predicted text, a negative candidate word list corresponding to each position in the predicted text, and optimize the parameters of the preset model based on the predicted text, the negative candidate word list and the translated text to obtain a trained preset model, wherein, for any position in the predicted text, a negative candidate word list corresponding to the position is determined based on elements in the predicted text except the elements corresponding to the position, and the negative candidate word list includes all elements of the positions before and after the position corresponding to the position in the predicted text, which are used to indicate that an element with a probability lower than a preset probability needs to be assigned when the position is generated.

8. A translation device, characterized in that: include: A model acquisition module, configured to acquire a trained preset model obtained by the translation model training method according to any one of claims 1 to 4; The first translation module is used to obtain a text to be translated, input the text to be translated into the trained preset model, and output a target text.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the translation model training method according to any one of claims 1 to 4 or the translation method according to any one of claims 5 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the translation model training method according to any one of claims 1 to 4 or the translation method according to any one of claims 5 to 6 is implemented.

Images