Translation model training method and device, translation method and device, equipment and medium

Abstract

The invention discloses a translation model training method and device, a translation method and device, equipment and a medium, and the translation model training method comprises the steps: inputting a source text in a training sample into an encoder of a preset model, and obtaining an encoding result, and decoding each word segmentation vector in the coding result through a first decoder of a preset model, performing permutation and combination to output an initial text, masking the initial text to obtain a masked text, inputting the masked text into a second decoder of the preset model, and decoding through the second decoder of the preset model to output a prediction text. And constructing a negative candidate word list corresponding to each position in the predicted text, and performing parameter optimization on the preset model according to the predicted text, the negative candidate word list and the translated text to obtain a trained preset model. The translation method comprises the steps of inputting a to-be-translated text into a trained preset model, and outputting a target text. The training mode of the model is optimized, and the translation quality is improved while the translation efficiency of the trained model is ensured.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a translation model training method, a translation method, a device, equipment and a medium. Background Art

[0002] With the increase in global medical exchanges, the need for medical information translation in a multilingual environment is becoming increasingly urgent. In order to solve this problem, the industry has begun to explore automated translation systems based on intelligent models. These systems can automatically translate the corresponding languages ​​through trained translation models. However, existing intelligent models still have obvious deficiencies in dealing with complex language structures and cultural differences, especially when facing multiple languages. The processing efficiency and accuracy of the model are often difficult to guarantee at the same time. For example, the language model based on non-autoregressive neural machine translation has significant advantages in translation efficiency. The model attempts to generate the translation results of the entire sentence at one time, which greatly shortens the translation time. However, due to the lack of fine-tuning word by word or sentence by sentence, it often lacks translation accuracy. In the medical field, such inaccurate translation may have serious consequences. Wrong diagnosis, inappropriate treatment plan or misunderstood patient information may lead to worsening of the patient's health or even endanger his life.

[0003] Therefore, how to optimize the model training method to improve the translation quality while ensuring the translation efficiency of the trained model has become an urgent problem to be solved. Summary of the invention

[0004] Based on this, a translation model training method, a translation method, an apparatus, a device and a medium are provided to solve the problem of how to optimize the training method of the model so as to improve the quality of translation while ensuring the translation efficiency of the trained model.

[0005] In a first aspect, an embodiment of the present invention provides a translation model training method, comprising: Acquire a training sample, wherein the training sample includes a source text and a translated text; Input the source text into an encoder of a preset model to obtain an encoding result, wherein the encoding result includes a word segmentation vector corresponding to each word segmentation in the source text, decode each word segmentation vector in the encoding result through a first decoder of the preset model to obtain an initial word segmentation sequence, and arrange and combine the initial word segmentation sequence to output an initial text; Masking all elements in the initial text to obtain masked text, inputting the masked text into the second decoder of the preset model, and decoding and outputting predicted text through the second decoder of the preset model; According to the predicted text, a negative candidate word list corresponding to each position in the predicted text is determined; according to the predicted text, the negative candidate word list and the translated text, parameters of the preset model are optimized to obtain a trained preset model.

[0006] In a second aspect, an embodiment of the present invention provides a translation method, comprising: Acquire a trained preset model obtained by the translation model training method of the first aspect above; The text to be translated is obtained, the text to be translated is input into the trained preset model, and the target text is output.

[0007] In a third aspect, an embodiment of the present invention provides a translation model training device, comprising: A sample acquisition module, used to acquire training samples, wherein the training samples include source text and translated text; A first training module is used to input the source text into an encoder of a preset model to obtain an encoding result, wherein the encoding result includes a word segmentation vector corresponding to each word segmentation in the source text, decode each word segmentation vector in the encoding result through a first decoder of the preset model to obtain an initial word segmentation sequence, and arrange and combine the initial word segmentation sequence to output an initial text; A second training module is used to mask all elements in the initial text to obtain masked text, input the masked text into a second decoder of the preset model, and decode and output a predicted text through the second decoder of the preset model; The first optimization module is used to determine the negative candidate word list corresponding to each position in the predicted text according to the predicted text, and optimize the parameters of the preset model according to the predicted text, the negative candidate word list and the translated text to obtain a trained preset model.

[0008] In a fourth aspect, an embodiment of the present invention provides a translation device, including: A model acquisition module, used to acquire a trained preset model obtained by the translation model training method of the first aspect above; The first translation module is used to obtain a text to be translated, input the text to be translated into the trained preset model, and output a target text.

[0009] In a fifth aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the translation model training method of the first aspect or the translation method of the second aspect is implemented.

[0010] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the translation model training method of the first aspect or the translation method of the second aspect is implemented.

[0011] The present invention is different from the technical effects achieved by the prior art solutions: the training samples of the translation model training method of the present invention include source text and translation text, the source text is input into the encoder of the preset model to obtain the encoding result, the encoding result includes the word segmentation vector corresponding to each word segmentation in the source text, each word segmentation vector in the encoding result is decoded by the first decoder of the preset model to obtain the initial word segmentation sequence, and the initial word segmentation sequence is arranged and combined to output the initial text, all elements in the initial text are masked to obtain the masked text, the masked text is input into the second decoder of the preset model, the second decoder of the preset model is decoded to output the predicted text, according to the predicted text, the negative candidate word list corresponding to each position in the predicted text is determined, according to the predicted text, the negative candidate word list and the translation text, the preset model is parameter optimized to obtain the trained preset model. When translating according to the trained preset model, the text to be translated is input into the trained preset model, and the target text is output.

[0012] Among them, the preset model adopts the architecture of dual decoders (first decoder and second decoder). The first decoder directly acts on the word vector corresponding to each word in the encoding result, performs independent decoding and permutation and combination. This process does not need to deeply consider the contextual semantic information, thereby significantly shortening the processing time of translation and improving the efficiency of translation. On this basis, the second decoder further corrects and optimizes the initial text to generate the final predicted text. This staged decoding process helps the model to gradually refine the translation results. While ensuring the translation efficiency of the trained model, it also improves the translation accuracy. By constructing a negative candidate word list and optimizing the model parameters according to the negative candidate word list, the model is helped to avoid selecting inappropriate vocabulary when generating translations, prompting the model to learn how to avoid wrong choices, thereby further improving the translation accuracy of the trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0014] Figure 1 It is a schematic diagram of an application environment of a translation model training method and a translation method provided in the first embodiment of the present invention; Figure 2 It is a flowchart of a translation model training method provided in Embodiment 2 of the present invention; Figure 3 It is a flowchart of a translation model training method provided in Embodiment 3 of the present invention; Figure 4 It is a flowchart of a translation model training method provided by Embodiment 4 of the present invention; Figure 5 It is a flowchart of a translation model training method provided in Embodiment 5 of the present invention; Figure 6 is a flowchart of a translation method provided by Embodiment 6 of the present invention; Figure 7 is a flowchart of a translation method provided by Embodiment 7 of the present invention; Figure 8 is a structural schematic diagram of a translation model training device provided by Embodiment 8 of the present invention; Fig. 9 is a structural schematic diagram of a translation device provided by Embodiment 9 of the present invention; Fig.10 It is a structural diagram of a computer device provided in Embodiment 10 of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] It should be understood that the order of execution of the steps in the following embodiments does not imply a precedence of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0017] like Figure 1As shown, it is a schematic diagram of an application environment of a translation model training method and a translation method provided in the first embodiment of the present invention, wherein the server communicates with the client, the server provides the functions of translation model training and translation, the client sends training samples to the server so that the server can perform model training, and the client sends text to be translated to the server so that the server can perform translation. The client includes but is not limited to PDAs, desktop computers, notebook computers, ultra-mobile personal computers (UMPC), netbooks, cloud computing devices, personal digital assistants (PDA) and other devices. The computer device corresponding to the server can be implemented with an independent server or a server cluster composed of multiple servers.

[0018] like Figure 2 FIG. 1 is a flow chart of a translation model training method provided in Embodiment 2 of the present invention, comprising the following steps: Step S201: Obtain training samples.

[0019] Step S202: input the source text into the encoder of the preset model to obtain an encoding result, decode each word segmentation vector in the encoding result through the first decoder of the preset model to obtain an initial word segmentation sequence, and arrange and combine the initial word segmentation sequence to output the initial text.

[0020] In this embodiment, the training sample includes a source text and a translated text. The source text may refer to original text data that has not been translated, and the translated text may refer to text data translated from the source text. The encoding result includes a word segmentation vector corresponding to each word segment in the source text. The word segmentation may refer to a word or phrase obtained by dividing the source text. The word segmentation vector may refer to a vector representation of the word segmentation. The preset model may refer to an initial model pre-set for model training. The initial text may refer to a possible translated text with noise obtained by translating the source text by the first decoder of the preset model during the training process. The encoder may refer to an encoder that vectorizes the source text. The first decoder is used to decode the vector representation generated by the encoder to generate an initial text corresponding to the source text.

[0021] Specifically, in the process of training the preset model, the source text is input into the embedding layer input end of the encoder of the preset model, the source text is converted into a continuous vector representation through the embedding layer of the encoder of the preset model, the continuous vector representation of the source text is encoded by the encoder of the preset model to obtain the word segmentation vector corresponding to each word in the source text, the word segmentation vector corresponding to each word in the source text is input into the first decoder of the preset model, each word segmentation vector in the encoding result is decoded by the first decoder of the preset model to obtain an initial word segmentation sequence, and the initial word segmentation sequence is arranged and combined to output the initial text.

[0022] Step S203: Mask all elements in the initial text to obtain masked text, input the masked text into the second decoder of the preset model, and decode and output the predicted text through the second decoder of the preset model.

[0023] In this embodiment, masked text may refer to text in which each element (e.g., words and characters, etc.) in the initial text is replaced with a specific form of representation or marking, predicted text may refer to the training prediction result of translating the source text by the second encoder of the preset model during the training process, and the second decoder may refer to generating predicted text from masked text.

[0024] Specifically, all words or characters in the initial text are replaced with a specific form of representation to obtain masked text, the masked text is input into the embedding layer input end of the second decoder of the preset model, the masked text is converted into a continuous vector representation through the embedding layer of the second decoder of the preset model, the continuous vector representation of the masked text is input into the second decoder of the preset model, and the predicted text is output by decoding the second decoder of the preset model.

[0025] Step S204: According to the predicted text, determine the negative candidate word list corresponding to each position in the predicted text, and optimize the parameters of the preset model according to the predicted text, the negative candidate word list and the translated text to obtain a trained preset model.

[0026] In this embodiment, the negative candidate word list may include all words in the positions before and after the current position of the predicted text, which is used to indicate that words with low probability need to be assigned when the current position is generated. The trained preset model may refer to a preset model that meets the model training termination condition. The training termination condition may be that the loss function value drops below a preset threshold. At this time, it can be considered that the model has achieved sufficient accuracy and the training can be terminated.

[0027] Specifically, for any position in the predicted text, a negative candidate word list corresponding to the position is constructed based on the elements in the predicted text except the element corresponding to the position, and a loss penalty value is calculated based on the elements of each position in the predicted text and the negative candidate word list of the corresponding position. According to the preset loss function and the loss penalty value, the error value between the predicted text and the translated text is calculated, and the parameters of the preset model are optimized according to the calculated error value to obtain a trained model.

[0028] The above-mentioned preset model can be a model formed by a neural network, a deep learning network, a machine learning architecture, etc., for example, a translation model based on natural language processing, a long short-term memory network model, and a sequence-to-sequence model.

[0029] In this embodiment, the preset model adopts a dual decoder (a first decoder and a second decoder) architecture. The first decoder directly acts on the word segmentation vector corresponding to each word segmentation in the encoding result, performs independent decoding and performs permutations and combinations. This process does not require in-depth consideration of contextual semantic information, thereby significantly shortening the processing time of translation and improving the efficiency of translation. The second decoder further corrects and optimizes the initial text on this basis to generate the final predicted text. This staged decoding process helps the model to gradually refine the translation results, while ensuring the translation efficiency of the trained model, it also improves the translation accuracy, and by constructing a negative candidate word list, according to the negative candidate word list, the model parameters are optimized to help the model avoid selecting inappropriate vocabulary when generating translations, prompting the model to learn how to avoid wrong choices, thereby further improving the translation accuracy of the trained model.

[0030] like Figure 3 As shown, it is a flow chart of a translation model training method provided in the third embodiment of the present invention. In step S204 of the second embodiment, according to the predicted text, a negative candidate word list corresponding to each position in the predicted text is determined, and according to the predicted text, the negative candidate word list and the translated text, the parameters of the preset model are optimized to obtain a trained preset model, which may include the following steps: Step S301: for any position in the predicted text, determine a negative candidate word list corresponding to the position based on elements in the predicted text except the element corresponding to the position.

[0031] Step S302: Calculate the loss penalty value according to the elements at each position of the predicted text and the negative candidate word list at the corresponding position.

[0032] Step S303: Optimize the parameters of the preset model according to the loss penalty value, the predicted text and the translated text to obtain a trained preset model.

[0033] In this embodiment, the loss penalty value may refer to a loss function penalty value calculated based on a negative candidate word list corresponding to a position in the predicted text.

[0034] Specifically, for any position in the predicted text, detect whether the element corresponding to the position is the same as any element in the negative candidate word list corresponding to the position. If they are the same, it means that a repeated word is generated at this position. The loss penalty value is calculated based on the number of repeated words appearing in all positions of the predicted text. Based on the loss penalty value, the loss value between the predicted text and the translated text is calculated. The parameters of the preset model are optimized according to the loss value to obtain a trained model.

[0035] Optionally, in addition to including all words in the previous and subsequent positions corresponding to the current position of the predicted text, the negative candidate word list may also include words outside the predicted text that need to be assigned a low probability when the current position is generated.

[0036] For example, if the predicted text is "The patient is preliminarily diagnosed with lung cancer with a high probability, and further CT examination is recommended for confirmation", the negative candidate word list constructed for the position of the word "lung cancer" may include words such as patient, preliminary, diagnosis, possibility and high in the corresponding positions in the predicted text. It may also include words outside the predicted text that need to be assigned a low probability when generating the "lung cancer" position, such as cold, cough and vomiting, which are names of diseases that are highly correlated with "lung cancer".

[0037] In this embodiment, by maintaining a negative candidate word list, repeated words generated in the predicted text are detected based on the negative candidate word list, and a corresponding loss penalty value is obtained based on the detection result, so that the parameters of the preset model are optimized based on the loss penalty value, thereby reducing the probability of generating repeated words during the model training process and improving the translation accuracy of the trained preset model.

[0038] like Figure 4 As shown, a flowchart of a translation model training method provided by Embodiment 4 of the present invention is provided. In step S303 of the above Embodiment 3, the parameters of the preset model are optimized according to the loss penalty value, the predicted text and the translated text to obtain a trained preset model, which may include the following steps: Step S401: By using a preset loss function and loss penalty value, loss calculation is performed on the predicted text and the translated text to obtain a loss value.

[0039] Step S402: Optimize the parameters of the preset model according to the loss value to obtain a trained preset model.

[0040] In this embodiment, the preset loss function may refer to a preset function for measuring the difference between the predicted text and the translated text, and the loss value may refer to the error value between the predicted text and the translated text calculated according to the preset loss function and the loss penalty value.

[0041] Specifically, the loss penalty value is added to the preset loss function to obtain an updated loss function, and the loss value between each element in the predicted text and the element at the corresponding position in the translated text is calculated through the updated loss function, and the loss values ​​between all elements in the predicted text and all elements in the translated text are summed to obtain the loss value, and according to the loss value, the parameters of the encoder, the first decoder and the second decoder of the preset model are optimized to obtain the trained preset model.

[0042] In this embodiment, the loss value between the predicted text and the translated text is calculated according to the loss penalty value and the preset loss function to optimize the parameters of the preset model, thereby improving the accuracy of the trained preset model.

[0043] like Figure 5 FIG. 1 is a flow chart of a translation model training method provided in Embodiment 5 of the present invention. The translation model training method further includes the following steps: Step S501: input the source text into the encoder of the preset model, and decode and output the predicted length through the length prediction module of the first decoder of the preset model.

[0044] Step S502: Optimize the parameters of the first decoder of the preset model according to the predicted length and the length of the translated text to obtain a pre-trained first decoder of the preset model.

[0045] In this embodiment, the length prediction module may refer to a module that predicts the length of the text after the source text is translated, and the predicted length may refer to the length of the translated text corresponding to the source text predicted by the length prediction module of the first decoder of the preset model during the training process.

[0046] Specifically, according to a preset loss function, an error value between the predicted length and the translated text length is calculated, and parameters of a first decoder of the preset model are optimized according to the calculated error value to obtain a pre-trained first decoder of the preset model.

[0047] In this embodiment, by training the length prediction module of the first decoder of the preset model, it helps the model to better control the output length during the generation process, avoid generating too long or too short translations, and thus improve the accuracy of the trained preset model.

[0048] like Figure 6 As shown, it is a flowchart of a translation method provided in Embodiment 6 of the present invention, which may include the following steps: Step S601: Obtain a trained preset model obtained by a translation model training method.

[0049] Step S602: Obtain the text to be translated, input the text to be translated into a trained preset model, and output the target text.

[0050] In this embodiment, the text to be translated may refer to the text to be translated, and the target text may refer to the text after the text to be translated is translated.

[0051] Specifically, the text to be translated is input into a trained preset model, and is processed by an encoder, a first decoder, and a second decoder of the trained preset model to output a target text.

[0052] For example, in a medical business scenario, a large international hospital has many foreign patients, and the medical records of these patients often contain a lot of professional terms and complex descriptions. In order to ensure the accuracy and efficiency of medical services, the hospital needs to translate these medical records into multiple languages ​​so that doctors and patients can understand them accurately. The medical record text that needs to be translated can be extracted from the patient's medical record system, and the extracted medical record text that needs to be translated is used as the text to be translated. The case text to be translated is input into the trained model, and the trained model is used to output the target text after the text to be translated is translated.

[0053] In this embodiment, the preset model trained by the model training method is used to translate the text to be translated to obtain the target text, thereby improving the efficiency and accuracy of the translation.

[0054] like Figure 7 As shown, it is a flowchart of a translation method provided by Embodiment 7 of the present invention. In Step S602 of Embodiment 6 above, the text to be translated is input into a trained preset model, and the target text is output, which may include the following steps: Step S701: input the text to be translated into the encoder of the trained preset model, and decode and output the intermediate text and the target length through the first decoder of the trained preset model.

[0055] Step S702: Mask all elements in the intermediate text to obtain the masked intermediate text, input the masked intermediate text into the second decoder of the trained preset model, and output the target text.

[0056] In this embodiment, the intermediate text may refer to a possible translated text with noise obtained by translating the text to be translated using a trained preset model, and the target length may refer to a length prediction module of the first decoder of the trained preset model, which predicts the length of the translated text corresponding to the text to be translated, and the target text may refer to a text after the text to be translated is translated using a trained preset model.

[0057] Specifically, the text to be translated is input into the embedding layer input end of the encoder of the trained preset model, the text to be translated is converted into a continuous vector representation through the embedding layer of the encoder of the trained preset model, the continuous vector representation of the text to be translated is encoded through the encoder of the trained preset model to obtain the word segmentation vector corresponding to each word in the text to be translated, the word segmentation vector corresponding to each word in the text to be translated is input into the first decoder of the trained preset model, the word segmentation vector corresponding to each word in the text to be translated is decoded through the first decoder of the trained preset model, and the intermediate text and the target length are output by arrangement and combination. All words or characters in the intermediate text are replaced with a specific form representation to obtain the masked intermediate text, the masked intermediate text is input into the embedding layer input end of the second decoder of the trained preset model, the masked intermediate text is converted into a continuous vector representation through the embedding layer of the second decoder of the trained preset model, the continuous vector representation of the masked intermediate text is input into the second decoder of the trained preset model, and the target text is decoded and output by the second decoder of the trained preset model.

[0058] In this embodiment, the preset model trained by the model training method is used to translate the text to be translated to obtain the target text, thereby improving the efficiency and accuracy of the translation.

[0059] like Figure 8 As shown, a translation model training device provided in Embodiment 8 of the present invention corresponds to the translation model training method in the above-mentioned embodiment. The translation model training device includes a sample acquisition module 81, a first training module 82, a second training module 83 and a first optimization module 84. Each functional module is described in detail as follows: A sample acquisition module 81 is used to acquire a training sample, wherein the training sample includes a source text and a translation text; A first training module 82 is used to input the source text into an encoder of a preset model to obtain an encoding result, wherein the encoding result includes a word segmentation vector corresponding to each word segmentation in the source text, decode each word segmentation vector in the encoding result through a first decoder of the preset model to obtain an initial word segmentation sequence, and arrange and combine the initial word segmentation sequence to output an initial text; The second training module 83 is used to mask all elements in the initial text to obtain masked text, input the masked text into the second decoder of the preset model, and decode and output the predicted text through the second decoder of the preset model; The first optimization module 84 is used to determine the negative candidate word list corresponding to each position in the predicted text according to the predicted text, and optimize the parameters of the preset model according to the predicted text, the negative candidate word list and the translated text to obtain a trained preset model.

[0060] Optionally, the first optimization module 84 includes: A list construction unit, for determining, for any position in the predicted text, a negative candidate word list corresponding to the position according to elements in the predicted text except elements corresponding to the position; A penalty value calculation unit, used to calculate a loss penalty value according to an element at each position of the predicted text and a negative candidate word list at a corresponding position; The second optimization unit is used to optimize the parameters of the preset model according to the loss penalty value, the predicted text and the translated text to obtain a trained preset model.

[0061] Optionally, the second optimization unit includes: A loss calculation subunit, configured to perform loss calculation on the predicted text and the translated text by using a preset loss function and the loss penalty value to obtain a loss value; The third optimization subunit is used to optimize the parameters of the preset model according to the loss value to obtain the trained preset model.

[0062] Optionally, the model training device further includes: A third training module is used to input the source text into the encoder of the preset model, and decode and output a predicted length through the length prediction module of the first decoder of the preset model; The third optimization module is used to optimize the parameters of the first decoder of the preset model according to the predicted length and the translated text length to obtain a pre-trained first decoder of the preset model.

[0063] For the specific definition of the translation model training device, please refer to the definition of the translation model training method above, which will not be repeated here. Each module in the above-mentioned translation model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0064] like Fig. 9 As shown, a translation device provided in the ninth embodiment of the present invention corresponds to the translation method in the above embodiment. The translation device includes a model acquisition module 91 and a first translation module 92. The functional modules are described in detail as follows: A model acquisition module 91 is used to acquire a trained preset model obtained by a translation model training method; The first translation module 92 is used to obtain a text to be translated, input the text to be translated into the trained preset model, and output a target text.

[0065] Optionally, the first translation module 92 includes: A second translation unit is used to input the text to be translated into the encoder of the trained preset model, and decode and output the intermediate text and the target length through the first decoder of the trained preset model; The third translation unit is used to mask all elements in the intermediate text to obtain the masked intermediate text, input the masked intermediate text into the second decoder of the trained preset model, and output the target text.

[0066] The specific definition of the translation device can be found in the definition of the translation method above, which will not be repeated here. Each module in the above-mentioned translation device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0067] Fig.10 This is a schematic diagram of the structure of a computer device provided by Embodiment 10 of the present invention. Fig.10 As shown, the computer device of this embodiment includes: at least one processor ( Fig.10 Only one is shown in the figure), a memory, and a computer program stored in the memory and executable on at least one processor, wherein when the processor executes the computer program, the steps in any of the above-mentioned translation model training method embodiments or the steps in any of the above-mentioned translation method embodiments are implemented.

[0068] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Fig.10 This is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0069] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0070] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments, it may also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory may also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0071] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0072] The present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing.

[0073] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0074] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0075] In the embodiments provided by the present invention, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0076] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0077] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein by equivalents, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention. The non-company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A translation model training method, characterized in that: include: Acquire a training sample, wherein the training sample includes a source text and a translated text; Input the source text into an encoder of a preset model to obtain an encoding result, wherein the encoding result includes a word segmentation vector corresponding to each word segmentation in the source text, decode each word segmentation vector in the encoding result through a first decoder of the preset model to obtain an initial word segmentation sequence, and arrange and combine the initial word segmentation sequence to output an initial text; Masking all elements in the initial text to obtain masked text, inputting the masked text into the second decoder of the preset model, and decoding and outputting predicted text through the second decoder of the preset model; According to the predicted text, a negative candidate word list corresponding to each position in the predicted text is determined; according to the predicted text, the negative candidate word list and the translated text, parameters of the preset model are optimized to obtain a trained preset model.

2. The translation model training method according to claim 1, characterized in that: The method further comprises: determining a negative candidate word list corresponding to each position in the predicted text according to the predicted text, and optimizing parameters of the preset model according to the predicted text, the negative candidate word list and the translated text to obtain a trained preset model, including: For any position in the predicted text, determine a negative candidate word list corresponding to the position according to elements in the predicted text except elements corresponding to the position; Calculating a loss penalty value according to the elements at each position of the predicted text and the negative candidate word list at the corresponding position; According to the loss penalty value, the predicted text and the translated text, the parameters of the preset model are optimized to obtain a trained preset model.

3. The translation model training method according to claim 2, characterized in that: The step of optimizing parameters of the preset model according to the loss penalty value, the predicted text, and the translated text to obtain a trained preset model includes: By using a preset loss function and the loss penalty value, loss calculation is performed on the predicted text and the translated text to obtain a loss value; According to the loss value, the parameters of the preset model are optimized to obtain the trained preset model.

4. The translation model training method according to claim 1, characterized in that: The training sample also includes the length of the translated text, and the translation model training method also includes: Input the source text into the encoder of the preset model, and decode and output the predicted length through the length prediction module of the first decoder of the preset model; According to the predicted length and the translated text length, parameters of the first decoder of the preset model are optimized to obtain a pre-trained first decoder of the preset model.

5. A translation method, characterized in that: include: Obtain a trained preset model obtained by the translation model training method according to any one of claims 1 to 4; The text to be translated is obtained, the text to be translated is input into the trained preset model, and the target text is output.

6. The translation method according to claim 5, characterized in that: The step of inputting the text to be translated into the trained preset model and outputting a target text comprises: Input the text to be translated into the encoder of the trained preset model, and decode the first decoder of the trained preset model to output the intermediate text and the target length; All elements in the intermediate text are masked to obtain the masked intermediate text, the masked intermediate text is input into the second decoder of the trained preset model, and the target text is output.

7. A translation model training device, characterized in that: include: A sample acquisition module, used to acquire training samples, wherein the training samples include source text and translated text; A first training module is used to input the source text into an encoder of a preset model to obtain an encoding result, wherein the encoding result includes a word segmentation vector corresponding to each word segmentation in the source text, decode each word segmentation vector in the encoding result through a first decoder of the preset model to obtain an initial word segmentation sequence, and arrange and combine the initial word segmentation sequence to output an initial text; A second training module is used to mask all elements in the initial text to obtain masked text, input the masked text into a second decoder of the preset model, and decode and output a predicted text through the second decoder of the preset model; The first optimization module is used to determine the negative candidate word list corresponding to each position in the predicted text according to the predicted text, and optimize the parameters of the preset model according to the predicted text, the negative candidate word list and the translated text to obtain a trained preset model.

8. A translation device, characterized in that: include: A model acquisition module, used to acquire a trained preset model obtained by the translation model training method according to any one of claims 1 to 4; The first translation module is used to obtain a text to be translated, input the text to be translated into the trained preset model, and output a target text.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the translation model training method according to any one of claims 1 to 4 or the translation method according to any one of claims 5 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the translation model training method according to any one of claims 1 to 4 or the translation method according to any one of claims 5 to 6 is implemented.

Images