Machine translation model training methods and machine translation methods

Abstract

The present disclosure provides a training method and apparatus for a machine translation model. The training method includes: obtaining a text to be translated and an expected translation of the text to be translated for a specific user; obtaining a first coded expression of the text to be translated, and obtaining a second coded expression for characterizing the language characteristics of the specific user; obtaining a third coded expression input to the machine translation model and a translation result of the text to be translated based on the first coded expression and the second coded expression; calculating a loss value based on the translation result of the text to be translated and the expected translation of the text to be translated; if the loss value is greater than or equal to a preset threshold, adjusting the weight parameters of the machine translation model based on the loss value; and stopping the training method if the loss value is less than the preset threshold. In this method, the second coded expression characterizing the language characteristics of the specific user is used to adjust the coded expression input to the machine translation model to provide personalized translation results.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to a training method for a machine translation model and a machine translation method. Background Art

[0002] Machine translation is the process of converting one natural language (source language) into another (target language) using a computer program. Currently, many machine translation implementations utilize machine translation models based on neural network technology. However, with the widespread application of machine translation, researchers have discovered that even for the same semantics, different users have different textual expressions. Although currently improved machine translation models can control translation results based on factors such as place of origin, occupation, and gender, this approach is not truly personalized translation. Essentially, it controls translation results based on group characteristics and does not achieve the goal of providing translation results based on the personalized language characteristics of a single individual. Summary of the Invention

[0003] In view of this, the present disclosure aims to provide a method for training a machine translation model and a method for implementing the machine translation model, wherein the implementation of the machine translation model can achieve the purpose of providing translation results based on individual personalized language features.

[0004] According to a first aspect of the present disclosure, a method for training a machine translation model is provided, comprising:

[0005] Obtaining a text to be translated and a desired translation of the text to be translated from a specific user;

[0006] obtaining a first coded representation of the text to be translated, and

[0007] Obtaining a second coded expression for characterizing a language feature of the specific user;

[0008] Obtaining, according to the first coded expression and the second coded expression, a third coded expression input to the machine translation model and a translation result of the text to be translated;

[0009] Calculating a loss value based on a translation result of the text to be translated and an expected translation of the text to be translated;

[0010] If the loss value is greater than or equal to a preset threshold, adjusting the weight parameters of the machine translation model according to the loss value;

[0011] If the loss value is less than a preset threshold, the training method is stopped.

[0012] In some embodiments, obtaining a second coded expression for characterizing the language characteristics of the specific user includes:

[0013] Determine whether the historical input of the specific user exists;

[0014] Keywords are extracted based on the historical input of the specific user, and the second coded expression is obtained accordingly.

[0015] In some embodiments, extracting keywords based on the historical input of the specific user and obtaining the second coded expression accordingly includes:

[0016] Obtaining contextual keywords based on the text to be translated and the historical input of the specific user;

[0017] Obtaining topic keywords based on the text to be translated and all users' historical inputs;

[0018] Obtain corresponding coding expressions based on the context keywords and the subject keywords respectively; and

[0019] The second coded expression is obtained by weighted summation calculation based on the coded expressions corresponding to the context keyword and the topic keyword, wherein the weights in the weighted summation calculation are a learnable parameter matrix.

[0020] In some embodiments, the step of obtaining the second coded expression by weighted summation based on the coded expressions corresponding to the context keyword and the topic keyword includes:

[0021] Perform matrix multiplication on the encoding expressions corresponding to the context keyword and the topic keyword respectively with a parameter matrix and then calculate the sum, and substitute the calculation results into an activation function to obtain a first value less than 1;

[0022] multiplying the first value by the coded expression corresponding to the subject keyword to obtain a first product;

[0023] Multiplying the difference between the value 1 and the first value by the encoded expression corresponding to the context keyword to obtain a second product;

[0024] The sum of the first product and the second product is expressed as the second code.

[0025] In some embodiments, the method further includes: using the loss value to adjust the respective weights of the coding expressions corresponding to the context keywords and the topic keywords in the weight summation calculation.

[0026] In some embodiments, obtaining a second coded expression for characterizing the language characteristics of the specific user further includes:

[0027] If the historical input of the specific user does not exist, the most similar user to the specific user is determined based on TF-IDF cosine similarity, and the historical input of the most similar user is used as the historical input of the specific user to obtain the second encoded expression.

[0028] In some embodiments, the context keywords and the topic keywords are obtained by combining a TD-IDF algorithm.

[0029] In some embodiments, the loss function used to calculate the loss value is composed of the maximum likelihood loss between the translation result of the text to be translated and the expected translation of the text to be translated, and the comparative loss of the historical translations of the specific user, the most similar user, and any selected dissimilar users.

[0030] According to a second aspect of the present disclosure, a method for training a machine translation model is provided, comprising:

[0031] a preprocessing module configured to receive a text to be translated from a specific user and a desired translation of the text to be translated, obtain a first coded representation of the text to be translated, obtain a second coded representation representing language characteristics of the specific user, and obtain a third coded representation based on the first coded representation and the second coded representation;

[0032] a loss calculation module, configured to input the third encoded expression into the machine translation model to obtain a translation result of the text to be translated, and calculate a loss value between the translation result of the text to be translated and an expected translation of the text to be translated;

[0033] A parameter updating module is used to adjust the weight parameters of the machine translation model based on the loss value.

[0034] According to a third aspect of the present disclosure, a computer device is provided, comprising:

[0035] a memory for storing computer executable code;

[0036] The processor is configured to execute the computer executable code to implement the above training method.

[0037] According to a third aspect of the present disclosure, a computer-readable medium is provided, comprising a computer-executable code, wherein the computer-executable code performs the above-mentioned training method when executed by a processor.

[0038] The training method provided by the embodiment of the present disclosure uses a second coded expression that represents the language characteristics of a specific user to adjust the first coded expression of the text to be translated input into the machine translation model, and adjusts the weight parameters of the machine translation model accordingly. This modeling method can achieve the purpose of using the user's personalized language characteristics to influence the translation results.

[0039] In addition, the TD-IDF algorithm is generally used for information mining and text processing. However, the embodiment of the present disclosure uses it to determine contextual keywords and topic keywords, and obtains a second coded expression based on the contextual keywords and topic keywords through weighted summation calculation, and continuously adjusts the parameter matrix in the weighted summation calculation through training to form a second coded expression that can better characterize the language characteristics of a specific user. This method can make the second coded expression more capable of characterizing the language characteristics of a specific user. And for a new user, the most similar user of the new user can be determined based on the TF-IDF cosine similarity, and the historical input of the most similar user is used instead of the historical input of the new user to determine the second coded expression of the new user. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and other objects, features and advantages of the present disclosure will become more apparent through description of the embodiments of the present disclosure with reference to the following drawings, in which:

[0041] Figure 1 A schematic diagram showing a scenario in which the training method for a machine translation model according to an embodiment of the present disclosure is applied;

[0042] Figure 2 A structural diagram of a server that can serve as a training device and an execution device is shown;

[0043] Figure 3 A flowchart of a method for training a machine translation model according to an embodiment of the present disclosure is shown;

[0044] Figure 4 Shown Figure 3 An exemplary specific flow chart of step S303 in FIG.

[0045] Figure 5 A flowchart of a method for training a machine translation model according to another embodiment of the present disclosure is shown;

[0046] Figure 6 A flowchart of a machine translation method according to an embodiment of the present disclosure is shown;

[0047] Figure 7 Shown Figure 6 Flowchart of a specific embodiment of step S603 in FIG.

[0048] Figure 8 4 is a structural diagram of a training device for a machine translation model according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] The present disclosure is described below based on examples, but the present disclosure is not limited to these examples. Certain specific details are described in detail in the detailed description of the present disclosure below. Those skilled in the art will appreciate that the present disclosure is fully understood without these details. To avoid obscuring the essence of the present disclosure, well-known methods, processes, and procedures have not been described in detail. The accompanying drawings are not necessarily drawn to scale.

[0050] Application scenarios and application architecture of the present disclosure

[0051] Figure 1 A schematic diagram of a scenario in which a method for training a machine translation model according to an embodiment of the present disclosure is applied is shown. The schematic diagram includes a training device 110 and an execution device 120. The training device 110 includes a machine translation model 111 and a model training apparatus 112, and the execution device 120 is used to run multiple personalized machine translation models 121.

[0052] As we all know, model training from scratch is performed on a neural network structure with weight parameters as initial values. This process requires a lot of resources, including but not limited to a large number of training samples, a large amount of computer resources and a long training time. However, not all model training processes need to start from scratch. Some training can start with a pre-trained model. The pre-trained model has obtained a better set of weight parameters due to prior training, but before applying it to a specific task, the weight parameters need to be fine-tuned (Fine-tune). Fine-tuning includes modifying the input and output layers, collecting training samples under specific tasks for incremental training, etc. The purpose of fine-tuning is to make the output model better adaptable to specific tasks. Figure 1 In the illustrated embodiment, the machine translation model 111 may be a pre-trained model with multi-language translation capabilities, and the model training device 112 may be a virtual device for performing multiple independent incremental trainings on the machine translation model using training samples from multiple different users to obtain multiple personalized machine translation models 121 suitable for multiple users.

[0053] Once the personalized machine translation model 121 is ready, it can be used in actual machine translation scenarios. For example, a user logs into the machine translation system running on the execution device 120 and enters text to be translated. The machine translation system will then invoke the personalized machine translation model 121 adapted for the user to perform the translation and provide the translation results back to the user.

[0054] Figure 2FIG2 is a schematic diagram of an exemplary server 200. The server 200 can function as a training device 110 or an execution device 120. As shown in the figure, the server 200 includes a scheduler 201, a storage unit 203, an I / O interface 204, and multiple model acceleration units 202, all coupled via a bus 205.

[0055] The storage unit 203 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache memory unit. The storage unit 203 may also include a readable medium in the form of a non-volatile storage unit, such as a read-only memory unit (ROM), a flash memory, and various disk memories.

[0056] The storage unit 203 can store various program modules and data. Various program modules include operating systems, applications that provide functions such as text processing, video playback, software editing and compiling. The executable code of these applications is read out and executed by the scheduler 201 from the storage unit 203 to realize the predetermined operation of these program modules. The scheduler 201 is generally a processor (CPU). In this example, the program modules and data stored in the storage unit 203 include machine translation model 111 and / or personalized machine translation model 121. Machine translation model 111 and personalized machine translation model 121 generally exist in the form of static files or dynamic code files. In addition, training samples are also stored in the storage unit 203.

[0057] Bus 205 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0058] The server 200 can communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the server 200, and / or any device that enables the server 200 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication can be performed through an input / output (I / O) interface 204. Furthermore, the server 200 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter (not shown). For example, through a network adapter, Figure 1 The terminal 103 in the figure can access the server 200. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used based on the server 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0059] As shown in the figure, the server 200 includes a plurality of model acceleration units 202. The traditional processor architecture design is very effective in terms of logic control, but is not efficient enough in terms of large-scale parallel computing, so it is not efficient for model calculation. For this reason, a model acceleration unit has been developed, and different models can be adapted to different model acceleration units. The model acceleration unit is, for example, a neural network acceleration unit (NPU). The NPU adopts a data-driven parallel computing architecture, and is a processing unit for processing a large number of operations (such as convolution, pooling, etc.) of each neural network node. Or it can be a graphics processing unit (GPU), which is used to perform image and graphics-related computing work. Since the graphics processing unit uses a large number of computing units specifically for graphics calculations, the graphics card reduces its dependence on the CPU and takes on some of the computationally intensive graphics and image processing work originally undertaken by the CPU, thereby greatly improving the processing efficiency of image data.

[0060] The multiple model acceleration units 202 are controlled by the scheduler 201. The scheduler 201 controls the multiple model acceleration units 202 to collaboratively execute the executable code of the model and read the training samples from the storage unit. In this example, if the training device 110 adopts Figure 2 If the execution device 120 adopts the following method, the translation result of the text to be translated can be fed back to the scheduler 201, which calculates the loss value and adjusts the weight parameters in the model. Figure 2 The translation result can be fed back to the machine translation system executed by the scheduler 201 to be presented to the user.

[0061] Training method of machine translation model according to an embodiment of the present disclosure

[0062] According to one embodiment of the present disclosure, a method for training a machine translation model is provided. The method may be performed by a training device 110, which may have a Figure 2 The hardware structure shown in Figure 1 is as follows. Figure 3 As shown, the method includes the following steps.

[0063] In step S301 , a text to be translated and an expected translation of the text to be translated of a specific user are obtained.

[0064] In step S302, a first coded representation of the text to be translated is obtained.

[0065] In step S303, a second coded expression for characterizing the language features of the specific user is obtained.

[0066] In step S304, a third coded expression is obtained according to the first coded expression and the second coded expression.

[0067] In step S305 , the third coded expression is input into a machine translation model to obtain a translation result of the text to be translated.

[0068] In step S306, a loss function is used to calculate a loss value based on the translation result of the text to be translated and the expected translation of the text to be translated, and the weight parameters of the machine translation model are adjusted based on the loss value.

[0069] In step S307, it is determined whether the loss value is less than the set threshold. If so, step S308 is executed, that is, training is stopped and the personalized machine translation model is output. If not, step S305 is jumped to continue calculating the translation result of the text to be translated based on the adjusted weight parameters.

[0070] The above steps are described in detail below.

[0071] To train a personalized machine translation model for a specific user, training samples must first be prepared (i.e., step S301). Multiple training samples can be constructed, each consisting of a specific user's text to be translated and the expected translation. It should be understood that different users may have different expected translations for the same text to be translated, so the expected translation is specific to a specific user.

[0072] Then, step S302 is to encode the text to be translated using an encoder for each training sample, and encode the text to be translated into a first encoded expression. Then, step S303 is to obtain a second encoded expression for characterizing the language features of a specific user.

[0073] Before step S302, each sentence in the text to be translated is segmented to obtain multiple minimum semantic units (tokens). For example, the sentence "How can you prove it" is segmented into multiple tokens, "How", "can", "you", "prove", and "it". Then, in step S302, each word is encoded to obtain a corresponding word vector, and then a sentence encoding expression composed of word vectors is obtained. Step S303 obtains multiple keywords that can represent the language characteristics of the specific user from the historical input or vocabulary of the specific user, and then obtains a second encoding expression for characterizing the language characteristics of the specific user based on these keywords.

[0074] Then in steps S304 to S308, a third coded expression is obtained based on the first coded expression and the second coded expression. For example, the first coded expression and the second coded expression are added to obtain the third coded expression. The third coded expression is input into the machine translation model to obtain the translation result of the text to be translated. A loss function is used to calculate the loss value based on the translation result of the text to be translated and the expected translation of the text to be translated. The weight parameters of the machine translation model are adjusted based on the loss value. When the loss value is greater than the set threshold, the process continues to jump to step S305 for execution. Otherwise, the loop is exited, and the current weight parameters are used as the weight parameters of the personalized machine translation model.

[0075] In some embodiments, the step of obtaining a second encoded expression comprises Figure 4 Steps S3031 to S3033 in the above example are performed. In step S3031, it is determined whether the specific user's historical input exists. If so, step S3032 is executed, which extracts keywords based on the specific user's historical input and obtains a second encoded expression accordingly. If not, step S3033 is executed, which determines the specific user's most similar users based on TF-IDF cosine similarity, extracts keywords from the most similar users' historical input, and obtains a second encoded expression accordingly.

[0076] In one embodiment, Figure 4 Step S3032 is implemented based on the following concept: if a segmented word appears less frequently in the historical input of all users but more frequently in the historical input of the current user, it indicates that the segmented word has strong discriminability and is therefore more important. Based on this concept, the frequency of occurrence of each segmented word in the text to be translated in the historical input of all users and in the historical input of the current user can be counted separately. The segmented words that appear more frequently in the historical input of the current user but less frequently in the historical input of all users are used as keywords representing the language characteristics of the current user. Based on the keywords, a second coded expression representing the language characteristics of the current user is obtained.

[0077] In one embodiment, step S3033 of "determining the most similar user to a specific user based on TF-IDF cosine similarity" includes: constructing a word frequency vector for the specific user based on the text to be translated, and then constructing word frequency vectors for other users based on historical inputs of other users, calculating the cosine value of the angle similarity between the word frequency vector of the specific user and the word frequency vector of each other user, the smaller the angle, the larger the cosine value, the more similar, and finding the user corresponding to the most similar word frequency vector as the most similar user. Then, any user is selected from the users with a cosine value of 0 as a dissimilar user. Here, the word frequency of each word in the word frequency vector is not the word frequency of each word in the text to be translated, but the word frequency of each word in the vocabulary corresponding to the text to be translated. Even if the word is not in the text to be translated, the word frequency in the vocabulary will also be calculated. In this way, the two word frequency vectors are of equal length and the cosine similarity calculation can be performed.

[0078] For example, user A inputs the text to be translated, "How can you prove it," and the word set is {How," "can," "you," "prove," "it"}. User B's historical input word set is {"can," "you," "prove," "it"}. Assume the vocabulary is {"it," "then," "How," "can," "you," "prove," "it"}. The corresponding word frequency vector for Zhang San is [0, 0, 1, 1, 1, 1], and the corresponding word frequency vector for Li Si is [0, 0, 0, 1, 1, 1, 1]. Then, calculate the cosine of the angle between these two word frequency vectors.

[0079] Figure 5 A flow chart of a method for training a machine translation model according to an embodiment of the present disclosure is shown. Figure 5 The difference lies in step S353, step S3561 and step S357.

[0080] In step S353, the topic keywords and context keywords of the text to be translated are obtained through TF-IDF, and the topic keywords and context keywords are weighted and summed to obtain user keywords. A second coded expression is then obtained based on the user keywords. The topic keywords are calculated by using the specific user's text to be translated as the document and the historical input of all users as the corpus for TF-IDF calculation. The context keywords are calculated by using the specific user's text to be translated as the document. If the specific user's historical input exists, all of the specific user's historical data is used as the corpus for TF-IDF calculation. If the specific user's historical input does not exist, the historical input of the most similar user is used as the specific user's historical input. The method for determining the most similar user has been described in detail above and will not be repeated here.

[0081] In addition, since the parameter matrix used as weights in the weight summation calculation of step S353 is a learnable parameter and needs to be continuously adjusted according to the loss value through the training process, step S3561 is added to the flowchart for adjusting the weights in the weight summation calculation of step S353 based on the loss value, and when step S357 determines that the loss value is greater than or equal to the set threshold, it jumps to step S353 to continue execution.

[0082] In one embodiment, the weight sum calculation formula of step S353 is as follows:

[0083]

[0084]

[0085]

[0086]

[0087] in, It is the topic keyword of user u. is the context keyword of user u, is the encoded expression of the topic keyword of user u, is the encoded expression of the contextual keywords of user u, W t and W r is the learnable parameter matrix, r (u) is the second encoding expression. α is the intermediate value obtained by weighted summation, and Sigmoid is the Sigmoid function.

[0088] In neural networks, the loss function reflects the performance of an algorithm by calculating the difference between the model's predicted value and the true value. Therefore, a high-performance model can be obtained by training the model to find a set of weight parameters that minimizes the loss function. Based on this, the loss function used to calculate the loss value in the above embodiment can use the maximum likelihood loss between the translation result of the text to be translated and the expected translation of the text to be translated.

[0089] In another embodiment, the loss function used to calculate the loss value in the above embodiment is composed of the maximum likelihood loss between the translation result of the text to be translated and the expected translation of the text to be translated, and the comparison loss of the historical translations of the specific user, the most similar user, and the dissimilar users. The details are as follows.

[0090] L=L mle +L cl (5)

[0091] The maximum likelihood loss is as follows:

[0092]

[0093] Among which X (u) represents the current input of user u, represents the (<i)-th generated translation of the historical input of user u, H (u) represents the historical input of user u, θ represents the model parameters, represents the i-th word of the translation currently generated for user u, both i and <i represent serial numbers.

[0094] The contrast loss is as follows:

[0095]

[0096] -d(X (u) , Y (u) , H (u) , H (u-) ), 0] (7)

[0097] Among which, d(X (u) , Y (u) , H (u) , H (u+) ) =

[0098]

[0099]

[0100]

[0101] Among which u, u + , u - respectively represent the current user, the most similar user and the dissimilar user; X and Y respectively represent the text to be translated as input and the translation result output by the model, H represents the historical input, d(*) represents the L2 distance, η represents the hyperparameter, which is set to 2 in this method.

[0102] It should be understood that the smaller the loss value, the higher the prediction accuracy of the machine translation model trained for the text to be translated. Therefore, the personalized training of the machine translation model can be stopped when the loss value is less than the set threshold.

[0103] It should be understood that when calculating the loss value, the training loss of contrastive learning can be added, pulling closer the translation results of similar users for the same sentence and pulling away the translation results of dissimilar users, so that the model generates different translations for different users. Different from the related technologies, this method can model the fine-grained features of a large number of users and generate personalized translations for new users under zero-resource conditions.

[0104] Figure 6This is a flow chart of a machine translation method provided by an embodiment of the present disclosure. The method may be executed by an execution device 120, which may have the following features: Figure 2 The hardware structure shown in FIG. The method includes the following steps. Specifically, the method includes the following steps.

[0105] In step S601, a text to be translated of a specific user is obtained.

[0106] In step S602, a first coded representation of the text to be translated is obtained.

[0107] In step S603, a second coded expression for characterizing the language features of the specific user is obtained.

[0108] In step S604, a third coded expression is obtained according to the first coded expression and the second coded expression.

[0109] In step S605 , the third coded expression is input into a machine translation model matched with the specific user to obtain a translation result of the text to be translated.

[0110] This embodiment uses the personalized machine translation model obtained through the above training process to perform machine translation. In this step, for each specific user, the machine translation model matching the specific user is used to translate the text to be translated input by the user and the translation result is provided to the specific user.

[0111] In one embodiment, step S603 includes: Figure 7 Follow the steps below.

[0112] In step S701, it is determined whether a specific user's historical input exists. If yes, step S702 is executed; if not, step S703 is executed.

[0113] In step S702 , context keywords are obtained based on the text to be translated and the historical input of a specific user.

[0114] In step S703, context keywords are obtained based on the text to be translated and the historical input of the most similar user. In step S704, topic keywords are obtained based on the text to be translated and the historical input of all users.

[0115] In step S705, the corresponding coding expressions are obtained based on the context keywords and the subject keywords.

[0116] In step S706, a second coded expression is obtained by weighted summation calculation based on the coded expressions corresponding to the context keyword and the topic keyword.

[0117] This embodiment is used to obtain a second coded expression that characterizes the linguistic characteristics of a specific user. For new users, since there is no historical input, the historical input of the most similar users can be used to obtain contextual keywords. For established users, the user's own historical input is used to obtain contextual keywords. coded expressions of the contextual keywords and the topic keywords are then obtained, and the second coded expression is derived based on these two coded expressions. The various steps of this embodiment have been described in detail above, so this description is relatively brief.

[0118] Figure 8 This is a block diagram of a machine translation model training device provided in one embodiment of the present disclosure. This training device is generally considered a virtual device and can be implemented using a programming language. As shown in the figure, the training device includes a preprocessing module 801, a loss calculation module 802, and a parameter update module 803.

[0119] The preprocessing module 801 is used to receive a text to be translated and an expected translation of the text to be translated from a specific user, and obtain a first coded expression of the text to be translated through a first coded expression calculation unit 8011, obtain a second coded expression representing the language characteristics of the specific user through a second coded expression calculation unit 8012, and obtain a third coded expression based on the first coded expression and the second coded expression through a third coded expression calculation unit 8013.

[0120] The loss calculation module 802 is used to input the third encoded expression into the machine translation model to obtain a translation result of the text to be translated, and calculate a loss value using a loss function based on the translation result of the text to be translated and the expected translation of the text to be translated. The loss value includes the maximum similarity loss 8021 and the contrast loss between the translation result and the expected translation. The contrast loss includes the contrast loss 8022 between the specific user and the most similar user and the contrast loss 8023 between the specific user and the least similar user, as shown in formulas (7) and (8).

[0121] Parameter update module 803 is used to adjust the weight parameters of machine translation model 900 based on the loss value. The loss value can be composed of one or both of maximum similarity loss 8021 and contrast loss. Loss calculation module 802 and parameter update module 803 can continue until the loss value falls below a set threshold.

[0122] Since the implementation details of the above-mentioned model training device have been described in the detailed description of the method embodiment above, they will not be described in detail to save space.

[0123] The commercial value of this disclosure

[0124] The disclosed embodiments provide a method for training a personalized machine translation model. Compared to existing technologies, this method can train models at a more granular level and produce machine translation models that better reflect the language characteristics of specific users. Therefore, this training method has broad application potential and market value.

[0125] It should be understood that the above description is merely a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, many variations of the embodiments of this specification are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included within the scope of protection of the present disclosure.

[0126] It should be understood that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments.

[0127] It should be understood that the foregoing description of this specification is based on specific embodiments. Other embodiments are within the scope of the claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0128] It should be understood that an element described herein in the singular or shown in the drawings as only one does not limit the number of the element to one. In addition, modules or elements described or shown herein as separate may be combined into a single module or element, and modules or elements described or shown herein as single may be split into multiple modules or elements.

[0129] It should also be understood that the terms and expressions used herein are for descriptive purposes only, and the one or more embodiments of this specification should not be limited to these terms and expressions. The use of these terms and expressions does not mean to exclude any equivalent features of the illustrations and descriptions (or portions thereof), and it should be recognized that various modifications that may exist should also be included in the scope of the claims. Other modifications, variations, and substitutions may also exist. Accordingly, the claims should be deemed to cover all such equivalents.

Claims

1. A method for training a machine translation model, comprising: Obtaining a text to be translated and a desired translation of the text to be translated from a specific user; obtaining a first coded representation of the text to be translated, and Obtaining a second coded expression for characterizing a language feature of the specific user; Obtaining, according to the first coded expression and the second coded expression, a third coded expression input to the machine translation model and a translation result of the text to be translated; Calculating a loss value based on a translation result of the text to be translated and an expected translation of the text to be translated; If the loss value is greater than or equal to a preset threshold, adjusting the weight parameters of the machine translation model according to the loss value; If the loss value is less than a preset threshold, stopping the training method; Wherein, obtaining a third coded expression input to the machine translation model based on the first coded expression and the second coded expression includes: adding the first coded expression and the second coded expression to obtain the third coded expression.

2. The training method according to claim 1, wherein: The obtaining of a second coded expression for characterizing the language feature of the specific user comprises: Determine whether the historical input of the specific user exists; If so, keywords are extracted based on the historical input of the specific user, and the second encoded expression is obtained based on the keywords.

3. The training method according to claim 2, wherein extracting keywords based on the historical input of the specific user and obtaining the second coded expression based on the keywords comprises: Obtaining contextual keywords based on the text to be translated and the historical input of the specific user; Obtaining topic keywords based on the text to be translated and all users' historical inputs; Obtain corresponding coding expressions based on the context keywords and the subject keywords respectively; And the second coding expression is obtained by weighted summation calculation based on the coding expressions corresponding to the context keywords and the topic keywords respectively, wherein the weights in the weighted summation calculation are a learnable parameter matrix.

4. The training method according to claim 3, wherein: The second coding expression is obtained by weighted summation calculation based on the coding expressions corresponding to the context keyword and the subject keyword, including: Perform matrix multiplication on the encoding expressions corresponding to the context keyword and the topic keyword respectively with a parameter matrix and then calculate the sum, and substitute the calculation results into an activation function to obtain a first value less than 1; multiplying the first value by the coded expression corresponding to the subject keyword to obtain a first product; Multiplying the difference between the value 1 and the first value by the encoded expression corresponding to the context keyword to obtain a second product; The sum of the first product and the second product is expressed as the second code.

5. The training method according to claim 3, further comprising: The respective weights of the coding expressions corresponding to the context keywords and the topic keywords in the weight summation calculation are adjusted based on the loss value.

6. The training method according to claim 3, wherein obtaining a second coded expression for characterizing the language characteristics of the specific user further comprises: If the historical input of the specific user does not exist, the most similar user to the specific user is determined based on TF-IDF cosine similarity, and the historical input of the most similar user is used as the historical input of the specific user to obtain the second encoded expression.

7. The training method according to claim 3, wherein: The context keywords and the topic keywords are obtained by combining the TD-IDF algorithm.

8. The training method according to claim 1, wherein: The loss function used to calculate the loss value is composed of the maximum likelihood loss between the translation result of the text to be translated and the expected translation of the text to be translated, and the comparison loss of the historical translations of the specific user, the most similar user and any selected dissimilar users.

9. A training device for a machine translation model, comprising: a preprocessing module configured to receive a text to be translated from a specific user and a desired translation of the text to be translated, obtain a first coded representation of the text to be translated, obtain a second coded representation representing language characteristics of the specific user, and obtain a third coded representation based on the first coded representation and the second coded representation; a loss calculation module, configured to input the third encoded expression into the machine translation model to obtain a translation result of the text to be translated, and calculate a loss value between the translation result of the text to be translated and an expected translation of the text to be translated; A parameter updating module, configured to adjust weight parameters of the machine translation model based on the loss value; The preprocessing module is further configured to add the first coding expression and the second coding expression to obtain the third coding expression.

10. A computer device comprising: a memory for storing computer executable code; A processor, configured to execute the computer executable code to implement the training method according to any one of claims 1 to 8.

11. A computer-readable medium comprising computer-executable codes, wherein when the computer-executable codes are executed by a processor, the training method according to any one of claims 1 to 8 is implemented.

Images