Neural machine translation method, device and storage medium for enhancing lexical consistency

Abstract

The present disclosure provides a neural machine translation method for enhancing vocabulary consistency, including: obtaining a standard sentence-level Transformer translation model through training, obtaining an initial "target-source" attention weight; for the passage input into the Transformer translation model, extracting a semantic context vector and a consistency context vector; using the consistency context vector to generate a corresponding unique consistency probability distribution for each group of repeated source words in the passage; using the consistency context vector and the semantic context vector to evaluate the confidence that the repeated source words need to be translated consistently; at the time of decoding the Transformer translation model, using the attention weight to calculate the final vocabulary prediction probability distribution at the current decoding time. The present disclosure also includes a corresponding neural machine electronic device and a readable storage medium for enhancing vocabulary consistency.

Description

Technical Field

[0001] The present disclosure relates to the field of language processing technology, and more particularly to a neural machine translation method, device and storage medium for enhancing vocabulary consistency. Background Art

[0002] Machine translation (MT) refers to the process of translating one natural language (usually called the source language) into another natural language (usually called the target language) with the help of a computer. It is an applied technology that crosses multiple disciplines such as linguistics, mathematics and computer science.

[0003] Although great progress has been made in the research of neural machine translation, the modeling and training of conventional neural machine translation systems are usually based on a strong independence assumption, that is, taking sentences as translation units. Therefore, when a document is input, the system can only translate sentence by sentence independently without considering the contextual relationship between sentences in the text. A fluent natural language document (or text) is not a simple pile of isolated sentences, but an organized and structured language unit, in which sentences are connected by coherent semantic relationships. Even if the translation of the neural machine translation may be completely correct at the sentence level, once placed in the context of the text, there may still be sentence-to-sentence disfluency or serious translation errors. Summary of the invention

[0004] In order to solve at least one of the above technical problems, the present disclosure provides a neural machine translation method, device and storage medium for enhancing vocabulary consistency.

[0005] According to one aspect of the present disclosure, a neural machine translation method for enhancing vocabulary consistency is provided, including: obtaining a standard sentence-level Transformer translation model through training to obtain initial "target-source" attention weights; for a passage input into the Transformer translation model, extracting a semantic context vector and a consistency context vector; using the consistency context vector to generate a corresponding unique consistency probability distribution for each group of repeated source words in the passage; using the consistency context vector and the semantic context vector to evaluate the confidence that repeated source words need to be translated consistently through a linear classifier; and at the decoding moment of the Transformer translation model, calculating the final vocabulary prediction probability distribution at the current decoding moment with the help of the attention weights.

[0006] According to the neural machine translation method for enhancing vocabulary consistency of at least one embodiment of the present disclosure, a standard sentence-level Transformer translation model is obtained by training with a training set and a development set.

[0007] According to the neural machine translation method for enhancing vocabulary consistency of at least one embodiment of the present disclosure, the Transformer translation model uses a 6-layer encoder and decoder, an 8-head attention mechanism, a hidden state dimension 3d=512, and a feedforward layer dimension size of 2048.

[0008] According to the neural machine translation method for enhancing vocabulary consistency of at least one embodiment of the present disclosure, the training set and the development set are constructed using translation data from three fields: news comments, TDE speeches, and movie subtitles.

[0009] According to the neural machine translation method for enhancing vocabulary consistency of at least one embodiment of the present disclosure, in the method of using the consistency context vector to generate a corresponding unique consistency probability distribution for each group of repeated source words in the passage, the consistency probability distribution of the nth group of repeated words is By global consistency of this group of words u n generate:

[0010]

[0011] Among them, W and b are the model parameters that need to be learned; W1 is a linear transformation matrix,

[0012] d is the latent vector dimension.

[0013] According to the neural machine translation method for enhancing vocabulary consistency of at least one embodiment of the present disclosure, in the process of evaluating the confidence level of repeated words at the source end that need to be translated consistently by a linear classifier, for any word x in the i-th source language sentence i,j , the calculation process is as follows:

[0014]

[0015] in is the network parameter to be learned; in the above formula, if x i,j Belong to<i,j> For repeated words, a two-layer linear network is used to calculate the confidence of translation consistency. The input of the network includes the encoding state vector h i,j , the corresponding consistency context vector u <i,j> , and the semantic context vector v of the sentence i ; if x i,j is a non-repeated word, the confidence that its translation is consistent is 0.

[0016] According to the neural machine translation method for enhancing vocabulary consistency in at least one embodiment of the present disclosure, in the final vocabulary prediction probability distribution at the current decoding time is calculated by the attention weight at the decoding time of the Transformer translation model, at the decoding time t of the i-th sentence, the calculation process of the vocabulary prediction probability distribution of the consistency constraint is as follows:

[0017]

[0018] in, is the probability distribution of the vocabulary predicted by the Transformer translation model at the decoding time t of the i-th sentence; a i，t，j The target word y i，t With source sentence X i ={x i，1 , …, x i，J The attention weight vector A of the words in} i，t ={a i，t，1 , …, a i，t，J} The elements in .

[0019] According to at least one embodiment of the neural machine translation method for enhancing vocabulary consistency of the present disclosure, a standard sentence-level Transformer translation model is obtained through training; a global context extraction module and a decoding-end enhancement module are added to the Transformer translation model; the global context extraction module is used to extract the semantic context vector and the consistency context vector of the input passage, and the decoding-end enhancement module uses the consistency context vector to generate a corresponding unique consistency probability distribution for each group of repeated source words in the passage, and uses the consistency context vector and the semantic context vector to evaluate the confidence that the repeated source words need to be translated consistently through a linear classifier; at the decoding moment of the Transformer translation model, the final vocabulary prediction probability distribution at the current decoding moment is calculated with the help of attention weights; the Transformer translation model to which the global context extraction module and the decoding-end enhancement module are added is continuously trained to optimize the model parameters.

[0020] According to another aspect of the present disclosure, there is provided an electronic device, including:

[0021] A memory storing execution instructions;

[0022] A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor performs the neural machine translation method for enhancing vocabulary consistency as described in any one of the above items.

[0023] According to another aspect of the present disclosure, a readable storage medium is provided, wherein the readable storage medium stores execution instructions, and when the execution instructions are executed by a processor, the neural machine translation method for enhancing vocabulary consistency as described in any one of the above items is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0025] Figure 1 It is a flowchart of a neural machine translation method for enhancing vocabulary consistency according to an embodiment of the present disclosure.

[0026] Figure 2 It is a schematic diagram of a decoding end enhancement module according to an embodiment of the present disclosure.

[0027] Figure 3 It is a structural diagram of a paragraph-level neural machine translation system according to an embodiment of the present disclosure.

[0028] Figure 4 It is a flowchart of a method for generating a neural machine translation model for enhancing vocabulary consistency according to an embodiment of the present disclosure.

[0029] Figure 5 It is a structural schematic diagram of an apparatus for generating a neural machine translation model for enhancing vocabulary consistency according to an embodiment of the present disclosure.

[0030] Reference numerals

[0031] 1001 Transformer translation model acquisition module

[0032] 1002 Transformer translation model extension module

[0033] 1003 Vocabulary prediction probability distribution determination module

[0034] 1004 Training and Optimizing Model Parameters Module

[0035] 1000 Neural Machine Translation Model

[0036] 1100 Bus

[0037] 1200 processor

[0038] 1300 Memory

[0039] 1400 Other circuits. DETAILED DESCRIPTION

[0040] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0041] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0043] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0044] When a component is referred to as being "on" or "over," "connected to," or "coupled to" another component, the component may be directly on, directly connected to, or directly coupled to the other component, or intervening components may be present. However, when a component is referred to as being "directly on," "directly connected to," or "directly coupled to" another component, there are no intervening components. For this purpose, the term "connected" may refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0045] For descriptive purposes, the present disclosure may use spatially relative terms such as "under," "beneath," "under," "down," "over," "upper," "above," "higher," and "side (e.g., as in "sidewall")," to describe the relationship of one component to another (other) component as shown in the accompanying drawings. The spatially relative terms are intended to encompass different orientations of the device in use, operation, and / or manufacture in addition to the orientation depicted in the accompanying drawings. For example, if the device in the accompanying drawings is turned over, components described as "under" or "beneath" other components or features would subsequently be positioned "over" the other components or features. Thus, the exemplary term "under" can encompass both the "above" and "below" orientations. Furthermore, the device may be otherwise oriented (e.g., rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein should be interpreted accordingly.

[0046] The terms used here are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used here, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used here, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0047] Figure 1 It is a flow chart of a neural machine translation method for enhancing vocabulary consistency according to an embodiment of the present invention, including: S1, obtaining a standard sentence-level Transformer translation model through training, and obtaining initial "target-source" attention weights; S2, extracting semantic context vectors and consistency context vectors for the passage input into the Transformer translation model; S3, using the consistency context vectors to generate a corresponding unique consistency probability distribution for each group of repeated source words in the passage; S4, using the consistency context vectors and the semantic context vectors, through a linear classifier to evaluate the confidence that repeated source words need to be translated consistently; S5, at the decoding moment of the Transformer translation model, using the attention weights to calculate the final vocabulary prediction probability distribution at the current decoding moment.

[0048] In step S1, a standard sentence-level Transformer translation model is obtained by training with a training set and a development set, and the initial "target-source" attention weight is obtained. The Transformer translation model is a translation model based on the self-attention mechanism proposed in 2017, which adopts an end-to-end encoder-decoder framework and a multi-layer stacked network structure based on the self-attention mechanism. This structure enables the model to not only effectively model the long-distance dependencies of words in a sequence, but also to perform calculations in a highly parallel manner, greatly improving the speed and performance of the translation system. Preferably, the training set and the development set are performed using translation data from three fields: news comments, TDE speeches, and movie subtitles. Further preferably, the trained Transformer translation model uses a 6-layer encoder and decoder, an 8-head attention mechanism, a hidden state dimension of 3d=512, and a feedforward layer dimension of 2048.

[0049] In step S2, for the passage input into the Transformer translation model, the semantic context vector and the consistency context vector are extracted; the semantic context vector provides context information of the entire passage to improve the translation quality of the sentence model and assist in the judgment of vocabulary consistency. The consistency context vector specifically uses repeated source language word information to impose consistency constraints on the decoding end.

[0050] In the present disclosure, the semantic context vector is extracted by adding a character tag " <doc>", using the self-attention mechanism of the Transformer multi-layer encoder, label" <doc>The state vector of " can encode the word information in the sentence; after completing the sentence encoding, the semantic context vector extraction module extracts the " <doc>"The label vector is used as the semantic representation of the sentence, thereby obtaining a set of sentence-level semantic vectors {h 1， <doc>< / doc> ,h 2， <doc>< / doc> ,…,h I， <doc>< / doc> }; This group of vectors is input into an encoding layer with the same encoder structure as the standard Transformer model to model the semantic associations between sentences, thereby generating a semantic context vector that incorporates global information.

[0051] In the present disclosure, the consistency context vector is extracted in the following way: each group of repeated words in the passage corresponds to a global consistency context vector. If the jth word x in the i-th sentence i,j belongs to the nth group of repeated words in the passage, then the consistent context vector extraction module will extract x i,j The encoding state vector h i,j Extract the corresponding set of repeated word vectors Therefore Contains the encoding state vectors of all the repeated words in the chapter belonging to the nth group; then, generates the global consistency context vector u of the nth group of repeated words n : Among them, the input of MaxPooling(·) is All elements in are pooled by dimension, and the output is a vector. Repeated words belonging to the same group in the passage share the same consistent context vector. The global consistent context vector set of the entire passage is u = {u1, ..., u n }.

[0052] In step S3, the consistency context vector is used to generate a corresponding unique consistency probability distribution for each group of repeated source words in the passage. By global consistency of this group of words u n generate:

[0053]

[0054] Among them, W and b are the model parameters that need to be learned; W1 is a linear transformation matrix, d is the latent vector dimension.

[0055] In step S4, the confidence that the repeated words at the source end need to be translated consistently is evaluated by a linear classifier, where for any word x in the i-th source language sentence i,j , the calculation process is as follows:

[0056]

[0057] in is the network parameter to be learned; in the above formula, if x i,j Belong to<i,j> For repeated words, a two-layer linear network is used to calculate the confidence of translation consistency. The input of the network includes the encoding state vector h i,j , the corresponding consistency context vector u <i,j> , and the semantic context vector v of the sentence i ; if x i,j is a non-repeated word, the confidence that its translation is consistent is 0.

[0058] In step S5, at the decoding time of the Transformer translation model, the final vocabulary prediction probability distribution at the current decoding time is calculated with the help of the attention weight. Among them, at the decoding time t of the i-th sentence, the calculation process of the vocabulary prediction probability distribution of the consistency constraint is as follows:

[0059]

[0060] in, is the probability distribution of the vocabulary predicted by the Transformer translation model at the decoding time t of the i-th sentence; a i，t，j The target word y i，t With source sentence X i ={x i，1 , …, x i，J The attention weight vector A of the words in} i，t ={a i，t，1 , …, a i，t，J } The elements in .

[0061] At each decoding moment, the probability distribution of the vocabulary with consistency constraints is calculated through steps S3-S5 to predict the final target end word. Among them, the calculation of steps S3 and S4 is independent of the current decoding moment t, so it only needs to be performed once in the entire decoding process, and can be calculated immediately after the global context vector is extracted. The following is a specific example to explain the calculation process of steps S3-S5.

[0062] Example: Assuming that a source language document contains two groups of repeated words, two consistency probability distributions can be generated after step S3: and . Suppose a source language sentence in a document has 5 words in, and Belongs to the first group of repeated words, belongs to the second group of repeated words, X2 and X5 are non-repeated words. After step S4, the translation consistency confidence corresponding to each word can be obtained, assuming that they are {0.45.0.0.65.0.78.0} respectively. In step S5, the attention weight A at the current decoding time t is t ={0.15.0.3.0.25.0.1.0.2}, the vocabulary prediction probability distribution of the original translation model is , then the final prediction probability distribution of the vocabulary of consistency constraints is:

[0063]

[0064] Preferably, in the implementation manner of the present disclosure, steps S3-S5 can be performed by a decoding end enhancement module. Figure 2 It is a schematic diagram of a decoding-end enhancement module according to an embodiment of the present disclosure. The decoding-end enhancement module uses the consistency context vector to generate a corresponding unique consistency probability distribution for each group of repeated source-end words in the passage, and uses the consistency context vector and the semantic context vector to evaluate the confidence that the repeated source-end words need to be translated consistently through a linear classifier; and calculates the final vocabulary prediction probability distribution at the current decoding moment with the help of attention weights. Among them, the method for implementing the above process by the decoding-end enhancement module refers to steps S3-S5 of the neural machine translation method for enhancing vocabulary consistency in the previous embodiment of the present disclosure.

[0065] The neural machine translation method for enhancing vocabulary consistency in the above-mentioned embodiment can be implemented by a neural machine translation model. Figure 3 A schematic diagram of the structure of a neural machine translation model for enhancing vocabulary consistency generated according to the above-mentioned embodiment of the present disclosure is shown, wherein the residual connection and layer normalization operation layer are not shown. In one embodiment of the present disclosure, a global context extraction module and a decoding-side enhancement module are added to the Transformer translation model. The global context extraction module is used to extract the semantic context vector and consistency context vector of the input passage. The decoding-side enhancement module constrains the vocabulary prediction probability distribution of repeated words on the source side during the decoding process.

[0066] Figure 3 The neural machine translation model shown can be generated by a method of enhancing a neural machine translation model for lexical consistency. Figure 4 It is a flow chart of a method for generating a neural machine translation model with enhanced vocabulary consistency according to an embodiment of the present disclosure, including: S10, obtaining a standard sentence-level Transformer translation model through training; S20, adding a global context extraction module and a decoding-end enhancement module to the Transformer translation model; S30, calculating the final vocabulary prediction probability distribution at the current decoding moment with the help of attention weights at the decoding moment of the Transformer translation model; S40, continuing to train the Transformer translation model to which the global context extraction module and the decoding-end enhancement module are added to optimize model parameters.

[0067] Figure 5 1 is a schematic diagram of the structure of an apparatus for generating a neural machine translation model 1000 for enhancing vocabulary consistency according to an embodiment of the present disclosure.

[0068] The device may include a corresponding module for performing each or several steps in the above flowchart. Therefore, each step or several steps in the above flowchart may be performed by a corresponding module, and the device may include one or more modules in these modules. The module may be one or more hardware modules specially configured to perform the corresponding steps, or implemented by a processor configured to perform the corresponding steps, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination.

[0069] The hardware structure can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various circuits including one or more processors 1200, memory 1300, and / or hardware modules together. The bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0070] The bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure only uses one connecting line, but does not mean that there is only one bus or one type of bus.

[0071] Figure 5 In the device for generating a neural machine translation model 1000 with enhanced vocabulary consistency, a Transformer translation model acquisition module 1001 is used to obtain a standard sentence-level Transformer translation model through training; a Transformer translation model extension module 1002 is used to add a global context extraction module and a decoding-end enhancement module to the Transformer translation model; a vocabulary prediction probability distribution determination module 1003 is used to calculate the final vocabulary prediction probability distribution at the current decoding moment with the help of attention weights at the Transformer translation model decoding moment; and a training and optimization model parameter module 1004 is used to continue training and optimizing model parameters of the Transformer translation model to which the global context extraction module and the decoding-end enhancement module are added.

[0072] Any process or method description in the flowchart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes other implementations, in which the functions may not be performed in the order shown or discussed, including performing the functions in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the present disclosure. The processor performs the various methods and processes described above. For example, the method implementation in the present disclosure can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).

[0073] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions on an instruction execution system, apparatus, or device), or in combination with such instruction execution systems, apparatuses, or devices.

[0074] For the purposes of this specification, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program may be printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a memory.

[0075] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0076] A person skilled in the art may understand that all or part of the steps of implementing the above-mentioned implementation method may be completed by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the implementation method.

[0077] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a readable storage medium. The storage medium may be a read-only memory, a disk or an optical disk, etc.

[0078] The present disclosure also provides an electronic device, including: a memory, the memory storing execution instructions; and a processor or other hardware module, the processor or other hardware module executes the execution instructions stored in the memory, so that the processor or other hardware module executes the above method.

[0079] The present disclosure also provides a readable storage medium, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the above method.

[0080] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present application. In this specification, the schematic representations of the above terms are not necessarily the same embodiment / method or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments / methods or examples described in this specification and the features of the different embodiments / methods or examples, unless they are contradictory.

[0081] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0082] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.< / doc> < / doc> < / doc>

Claims

1. A neural machine translation method for enhancing lexical consistency, characterized in that: include: Through training, a standard sentence-level Transformer translation model is obtained to initialize the "target-source" attention weights; For the passages input into the Transformer translation model, the semantic context vector and the consistency context vector are extracted; the consistency context vector is used to generate a corresponding unique consistency probability distribution for each group of repeated source words in the passage; Using the consistency context vector and the semantic context vector, a linear classifier is used to evaluate the confidence that repeated words at the source end need to be translated consistently. At the decoding moment of the Transformer translation model, the final vocabulary prediction probability distribution at the current decoding moment is calculated with the help of the attention weight. In the method of using the consistency context vector to generate a corresponding unique consistency probability distribution for each group of repeated source words in the passage, the consistency probability distribution of the nth group of repeated words is By global consistency of this group of words u n generate: Among them, W and b are the model parameters that need to be learned; W1 is a linear transformation matrix, d is the latent vector dimension; In the above-mentioned evaluation of the confidence that repeated words at the source end need to be translated consistently by a linear classifier, for any word xi,j in the i-th source language sentence, the calculation process is as follows: in, and is the network parameter to be learned; in the above formula, if x i,j Belong to<i,j> For repeated words, a two-layer linear network is used to calculate the confidence of translation consistency. The input of the network includes the encoding state vector h i,j , the corresponding consistency context vector u<i,j> , and the semantic context vector vi of the sentence; if xi,j is a non-repeated word, the confidence of its translation consistency is 0.

2. The neural machine translation method for enhancing vocabulary consistency according to claim 1, characterized in that: The standard sentence-level Transformer translation model is obtained by training the training set and the development set.

3. The neural machine translation method for enhancing vocabulary consistency according to claim 1, characterized in that: The Transformer translation model uses a 6-layer encoder and decoder, an 8-head attention mechanism, a hidden state dimension of 3d=512, and a feedforward layer dimension of 2048.

4. The neural machine translation method for enhancing vocabulary consistency according to claim 1, characterized in that: The training set and development set use translation data from three fields: news commentary, TDE speeches, and movie subtitles.

5. The neural machine translation method for enhancing vocabulary consistency according to claim 1, characterized in that: In the Transformer translation model decoding time, the final word list prediction probability distribution at the current decoding time is calculated with the help of attention weights. At the decoding time t of the i-th sentence, the calculation process of the word list prediction probability distribution of the consistency constraint is as follows: in, is the probability distribution of the vocabulary predicted by the Transformer translation model at the decoding time t of the i-th sentence; a i，t，j The target word y i，t With source sentence X i ={x i，1 , …, x i，J The attention weight vector of the word in} The elements in .

6. A method for generating a neural machine translation model that enhances lexical consistency, characterized in that: include: Obtain a standard sentence-level Transformer translation model through training; Add a global context extraction module and a decoding-side enhancement module to the Transformer translation model; The global context extraction module is used to extract the semantic context vector and the consistency context vector of the input passage. The decoding end enhancement module uses the consistency context vector to generate a corresponding unique consistency probability distribution for each group of repeated source words in the passage. The consistency context vector and the semantic context vector are used to evaluate the confidence that the repeated source words need to be translated consistently through a linear classifier; at the decoding moment of the Transformer translation model, the final vocabulary prediction probability distribution at the current decoding moment is calculated with the help of the attention weight; the Transformer translation model with the global context extraction module and the decoding end enhancement module is continuously trained to optimize the model parameters; In the method of using the consistency context vector to generate a corresponding unique consistency probability distribution for each group of repeated source words in the passage, the consistency probability distribution of the nth group of repeated words is By global consistency of this group of words u n generate: Among them, W and b are the model parameters that need to be learned; W1 is a linear transformation matrix, d is the latent vector dimension; In the above-mentioned evaluation of the confidence that repeated words at the source end need to be translated consistently by a linear classifier, for any word xi,j in the i-th source language sentence, the calculation process is as follows: in, and is the network parameter to be learned; in the above formula, if xi,j belongs to the<i,j> For repeated words, a two-layer linear network is used to calculate the confidence of translation consistency. The input of the network includes the encoding state vector h i,j , the corresponding consistency context vector u<i,j> , and the semantic context vector vi of the sentence; if xi,j is a non-repeated word, the confidence of its translation consistency is 0.

7. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor performs the neural machine translation method for enhancing vocabulary consistency according to any one of claims 1 to 5.

8. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which, when executed by a processor, are used to implement the neural machine translation method for enhancing vocabulary consistency according to any one of claims 1 to 5.

Images