A translation method and device, electronic device, and computer-readable storage medium

By introducing cache update mechanism and importance score management in the neural machine translation system, dynamically updating the encoder status is solved, and the problem of insufficient cross-sentence information transmission in the neural machine translation system is improved, and the overall connection and coherence of the translation results are improved. .

CN113887194BActive Publication Date: 2024-11-29CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010627857.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-07-01
Publication Date
2024-11-29
Estimated Expiration
2040-07-01

AI Technical Summary

Technical Problem

In the chapter-level translation, the existing neural machine translation system cannot reasonably determine the context information of the sentences to be translated, resulting in poor convergence and coherence of the translation results, which affects the overall quality of the entire target language text.

Method used

By introducing a cache update mechanism in the neural machine translation model, the multi-head attention model and fully connected feedforward network are based on the multi-head attention model and the fully connected feedforward network, and the cache area is managed in combination with the importance score, dynamically update and replace the encoder state, ensuring cross-sentence information transmission and decoder state relevance.

Benefits of technology

It improves the overall convergence and coherence of the source language discourse, avoids the problems of mispropagation and missed translation, and enhances the logical coherence and accuracy of the translation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113887194B_ABST
    Figure CN113887194B_ABST
Patent Text Reader

Abstract

The present application discloses a translation method and device, the method comprising: determining a first decoder state of a first target sentence; the first target sentence is a sentence to be translated in a source language sentence, and the first decoder state is a decoder state obtained by translating the first target sentence using the neural machine translation model; obtaining a first encoder state from a target region; the first encoder state refers to the encoder state of a set of translated target sentences in the source language sentence; the target region is used to determine the set of target sentences based on a cache update mechanism, and save the encoder state of the set of target sentences; determining a second decoder state of the first target sentence based on the first encoder state and the first decoder state. The present application utilizes the encoder state of the translated sentence to influence the decoder state of the sentence to be translated, realizes the transmission of cross-sentence information, and improves the coherence and coherence of the target language discourse.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of neural machine translation, and in particular to a translation method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Among the many research directions of Natural Language Processing (NLP), Machine Translation (MT) has always been a hot topic. With the introduction and in-depth development of the Neural Machine Translation (NMT) model, the translation quality of the machine translation system has made a breakthrough, and the research and application of neural machine translation has become the mainstream in the field of machine translation. However, the sentence-based neural machine translation still has the problem of lack of coherence and coherence between sentences in the text translation results in the paragraph-level translation. For example, when translating the fragment "NPC deputies are still not satisfied with the results of judicial system reform,..., although the reaction of people's deputies to the people's congress yesterday was more than that of last year" in the source language text, the current baseline neural machine translation system gives the translation result: "NPC deputies not satisfied with judicial system reform,..., although the reaction of people's deputies to the people's congress yesterday was more than that of last year". The second “NPC representative” in the above text should be translated as “npc deputies” or “thedeputies”, but it is translated as “people's deputies” in the translation produced by the baseline system, which reduces the overall coherence and consistency of the source language text translation. The reason for similar problems is that the baseline neural machine translation model only uses the intra-sentence information of sentences in the text while ignoring the inter-sentence information of different sentences in the text when translating the source language text.

[0003] In one approach, when translating a source language text, a fixed-dimensional global semantic information is generated by using a deep representation learning model to represent the context of the entire source language text and is transferred between sentences during the translation process. Representative works include training a variational auto-encoder (VAE) structure to perform variational inference on the encoder and decoder states to obtain global semantic information; or using a memory network to learn the context information of the source language text and the target language text. This approach can give full play to the representation learning advantages of deep learning and perform higher-dimensional representation learning of text information. However, since this method uses a fixed structure to represent global semantics, usually a fixed-dimensional vector is used to represent the context information of the text, it is difficult to effectively handle fine-grained information transfer.

[0004] In another solution, a cache is introduced to memorize the decoding state of the target language and pass it between sentences. In the process of translating the source language text, after a sentence is translated, part of the translated information will be stored in the cache. In the process of translating subsequent sentences, the cache content can be referenced through certain mechanisms (reading or retrieving, etc.) and introduced into the decoder as the context information of the source language text. This implementation method can keep the translation of the same component in the target language consistent, thereby improving the lexical cohesion of the translation result. However, during the translation process, if an erroneous decoding state occurs, the erroneous decoding state will also be stored in the cache, causing error propagation in the decoding process; at the same time, since only the decoding state of the target end is memorized during the translation process, when a missing translation problem occurs, the lost source information cannot be restored.

[0005] In addition, in one solution, the source text is directly modeled across sentences through a fixed window, and a fixed window is used to select k sentences or n words before the current sentence to be translated as context, and a certain deep structure is used to directly model it, and then the learned contextual semantic information is integrated into the translation model. A typical work is to model the first k sentences of the source sentence to be translated with the help of a hierarchical attention network. This implementation method uses a fixed window to obtain the source text as context, but in actual discourse, the transmission distance of context information between sentences is often not fixed, and in some cases it can span a very long distance (measured by the number of sentences in between). Therefore, when using this implementation method, the length of the window is difficult to determine, and either the required context cannot be effectively covered or too much noise is introduced.

[0006] In summary, among the existing solutions, since the context information required for translating the sentence to be translated cannot be reasonably determined, it is impossible to use reasonable context information to guide the translation process of the sentence to be translated, resulting in poor coherence and consistency between the translation result of the sentence to be translated and the translated sentence, which ultimately affects the overall coherence and consistency of the entire target language text obtained by translation. Summary of the invention

[0007] In order to solve the above-mentioned problem that the context information required for translating the sentence to be translated cannot be reasonably determined, resulting in the inability to use reasonable context information to guide the translation process of the sentence to be translated, resulting in poor coherence and consistency between the translation result of the sentence to be translated and the translated sentence, which ultimately affects the overall coherence and coherence of the entire target language text obtained by translation, the present application provides a translation method and device, an electronic device, and a computer-readable storage medium.

[0008] The present application provides a translation method, which is applied to a neural machine translation model, wherein the model includes an encoder and a decoder, and the method includes:

[0009] Determining a first decoder state of a first target sentence; the first target sentence is a sentence to be translated in a source language sentence, and the first decoder state is a decoder state obtained by translating the first target sentence using the neural machine translation model;

[0010] Acquire a first encoder state from the target region; the first encoder state refers to an encoder state of a target sentence set that has been translated from the source language sentence; wherein the target region is used to determine the target sentence set based on a cache update mechanism and save the encoder state of the target sentence set;

[0011] A second decoder state for the first target sentence is determined based on the first encoder state and the first decoder state.

[0012] In an optional implementation manner of the present application, before determining the first decoder state of the first target sentence, the method further includes:

[0013] Determining an encoder state of a second target sentence; the second target sentence is a sentence in the source language sentence that precedes the first target sentence;

[0014] If there is remaining storage space in the target area, saving the encoder state of the second target sentence to the target area, the target sentence set including at least the second target sentence;

[0015] If there is no remaining storage space in the target area, it is determined whether to replace one of the encoder states stored in the target area with the encoder state of the second target sentence according to the importance score of the second target sentence.

[0016] In an optional implementation manner of the present application, determining whether to replace one of the encoder states stored in the target area with the encoder state of the second target sentence according to the importance score of the second target sentence includes:

[0017] If the importance score of the second target sentence is less than all encoder states stored in the target region, maintaining all encoder states stored in the target region unchanged;

[0018] If the importance score of the second target sentence is greater than the partial encoder states stored in the target region, one of the encoder states in the partial encoder states is replaced with the encoder state of the second target sentence.

[0019] In an optional implementation manner of the present application, the method further includes:

[0020] For each sentence in the source language sentence, the importance score of the sentence is determined based on the subject-statement structure information of the sentence.

[0021] In an optional implementation manner of the present application, determining the importance score of the sentence based on the subject-statement structure information of the sentence includes:

[0022] The importance score of the sentence is obtained by performing linear transformation and logistic regression processing on the subject-statement structure information of the sentence.

[0023] In an optional implementation manner of the present application, determining the second decoder state of the first target sentence based on the first encoder state and the first decoder state includes:

[0024] A correlation between the first decoder state and the first encoder state is determined, and the second decoder state is determined based on the correlation and the first decoder state.

[0025] In an optional implementation manner of the present application, determining the correlation between the first decoder state and the first encoder state, and determining the second decoder state based on the correlation and the first decoder state includes:

[0026] Inputting the first encoder state and the first decoder state into a multi-head attention model, and obtaining the correlation between the first decoder state and the first encoder state through the multi-head attention model;

[0027] Inputting the association degree into a fully connected feed-forward network, and obtaining a target vector through the fully connected feed-forward network;

[0028] The target vector is weighted with the first decoder state to obtain a second decoder state of the first target sentence.

[0029] In an optional implementation of the present application, a regularization layer is provided after the multi-head attention model and the fully connected feed-forward network.

[0030] The present application also provides a translation device, which is applied to a neural machine translation model, wherein the model includes an encoder and a decoder, and the device includes:

[0031] A first determining unit is used to determine a first decoder state of a first target sentence; the first target sentence is a sentence to be translated in a source language sentence, and the first decoder state is a decoder state obtained by translating the first target sentence using the neural machine translation model;

[0032] an acquisition unit, configured to acquire a first encoder state from a target region; the first encoder state refers to an encoder state of a target sentence set that has been translated from a source language sentence; wherein the target region is used to determine the target sentence set based on a cache update mechanism and save the encoder state of the target sentence set;

[0033] A second determining unit is used to determine a second decoder state of the first target sentence based on the first encoder state and the first decoder state.

[0034] In an optional implementation manner of the present application, before determining the first decoder state of the first target sentence, the apparatus further includes:

[0035] A third determining unit is used to determine the encoder state of a second target sentence; the second target sentence is a sentence in the source language sentence that is located before the first target sentence;

[0036] a saving unit, configured to save the encoder state of the second target sentence to the target region if there is remaining storage space in the target region, the target sentence set including at least the second target sentence;

[0037] A fourth determination unit is used to determine whether to replace one of the encoder states stored in the target area with the encoder state of the second target sentence according to the importance score of the second target sentence if there is no remaining storage space in the target area.

[0038] In an optional embodiment of the present application, the fourth determination unit is specifically used to: if the importance score of the second target sentence is less than all encoder states stored in the target area, then maintain all encoder states stored in the target area unchanged; if the importance score of the second target sentence is greater than some encoder states stored in the target area, then replace one of the encoder states in the partial encoder states with the encoder state of the second target sentence.

[0039] In an optional implementation manner of the present application, the device further includes:

[0040] The fifth determining unit is used to determine, for each sentence in the source language sentence, the importance score of the sentence based on the subject-statement structure information of the sentence.

[0041] In an optional implementation of the present application, the fifth determination unit is specifically used to obtain an importance score of the sentence by performing linear transformation and logistic regression processing on the subject-statement structure information of the sentence.

[0042] In an optional implementation manner of the present application, the second determination unit is specifically used to: determine the correlation between the first decoder state and the first encoder state, and determine the second decoder state based on the correlation and the first decoder state.

[0043] In an optional embodiment of the present application, the second determination unit is further specifically used to: input the first encoder state and the first decoder state into a multi-head attention model, and obtain the correlation between the first decoder state and the first encoder state through the multi-head attention model; input the correlation into a fully connected feedforward network, and obtain a target vector through the fully connected feedforward network; weight the target vector and the first decoder state to obtain a second decoder state of the first target sentence.

[0044] In an optional implementation of the present application, a regularization layer is provided after the multi-head attention model and the fully connected feed-forward network.

[0045] The present application also provides an electronic device, comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the above-mentioned translation method.

[0046] The present application also provides a computer-readable storage medium, characterized in that it is used to store a computer program, and the computer program enables a computer to execute the above translation method.

[0047] The above one or more technical solutions of the present application have at least one or more of the following technical effects:

[0048] 1. A translation method and device provided by the present application, by determining a first decoder state of a first target sentence; the first target sentence is a sentence to be translated in a source language sentence, and the first decoder state is a decoder state obtained by translating the first target sentence using the neural machine translation model; obtaining a first encoder state from a target region; the first encoder state refers to the encoder state of a set of translated target sentences in a source language sentence; wherein the target region is used to determine the set of target sentences based on a cache update mechanism, and save the encoder state of the set of target sentences; and determining a second decoder state of the first target sentence based on the first encoder state and the first decoder state. In this way, the first encoder state corresponding to the set of translated target sentences can be determined based on the cache update mechanism of the target region, and the first encoder state can be used to influence the decoder state of the sentence to be translated in the source language sentence, thereby realizing the transmission of cross-sentence information in the source language sentence and improving the overall coherence and coherence of the entire source language text.

[0049] 2. A translation method and device provided by the present application, before determining the first decoder state of the first target sentence, determine the encoder state of the second target sentence; the second target sentence is the sentence in the source language sentence that is located before the first target sentence; if there is remaining storage space in the target area, the encoder state of the second target sentence is saved in the target area, and the target sentence set includes at least the second target sentence; if there is no remaining storage space in the target area, determine whether to replace one of the encoder states stored in the target area with the encoder state of the second target sentence according to the importance score of the second target sentence. Based on this, when the storage space in the target area is insufficient, according to the importance score of the translated sentences in the text, the encoder state of the translated sentences with higher importance scores can be stored in the target area, and the encoder state of the translated sentences with lower importance scores can be replaced from the target area, so that the encoder state information of the source language sentences with higher importance scores can be saved in the target area.

[0050] 3. The present application provides a translation method and device, if the importance score of the second target sentence is less than all encoder states stored in the target area, then all encoder states stored in the target area are maintained unchanged; if the importance score of the second target sentence is greater than some encoder states stored in the target area, then one of the encoder states in the partial encoder states is replaced with the encoder state of the second target sentence. Based on this, according to the importance scores of the translated sentences in the text, the encoder states of the translated sentences with higher importance scores can be stored in the target area, so that the encoder state information of the source language sentences with higher importance scores can be saved in the target area.

[0051] 4. The present application provides a translation method and device, for each sentence in the source language sentence, determining the importance score of the sentence based on the subject-statement structure information of the sentence. Based on this, the subject-statement structure information can be introduced to effectively determine the important information in the text.

[0052] 5. The present application provides a translation method and device, which obtains the importance score of a sentence by performing linear transformation and logistic regression on the subject-statement structure information of the sentence. Based on this, the importance of the sentence can be reflected in the form of a score.

[0053] 6. The present application provides a translation method and apparatus, which determines the correlation between the first decoder state and the first encoder state, and determines the second decoder state based on the correlation and the first decoder state. Based on this, the state of the second decoder that generates the sentence to be translated can be guided based on the important information stored in the target area.

[0054] 7. A translation method and device provided by the present application, inputting the first encoder state and the first decoder state into a multi-head attention model, obtaining the correlation between the first decoder state and the first encoder state through the multi-head attention model; inputting the correlation into a fully connected feedforward network, obtaining a target vector through the fully connected feedforward network; weighting the target vector with the first decoder state to obtain the second decoder state of the first target sentence. Based on this, the multi-head attention model can be used to obtain the correlation between the first decoder state and the first encoder state, and the fully connected feedforward network can be used to obtain the second decoder state of the first target sentence.

[0055] 8. The present application provides a translation method and device, wherein the multi-head attention model and the fully connected feedforward network are both provided with a regularization layer. Based on this, the problem of overfitting that may occur in the process of determining the correlation between the first decoder state and the first encoder state, and in the process of obtaining the second decoder state of the first target sentence based on the correlation between the first decoder state and the first encoder state and the first decoder state can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flowchart of a translation method provided in an embodiment of the present application;

[0057] Figure 2 A schematic diagram of a neural machine translation model provided in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of a cache information screening submodule provided in an embodiment of the present application;

[0059] Figure 4 A schematic diagram of a method for determining the importance score of a sentence provided in an embodiment of the present application;

[0060] Figure 5 A schematic diagram of a cache information utilization submodule provided in an embodiment of the present application;

[0061] Figure 6 A schematic diagram of the structure of the translation device provided in the embodiment of the present application;

[0062] Figure 7 A schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0063] Figure 8 A schematic structural diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to enable a more detailed understanding of the features and technical contents of the present application, the implementation of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the present application.

[0065] Figure 1 A flowchart of a translation method provided in an embodiment of the present application is provided, wherein the method is applied to a neural machine translation model, wherein the model includes an encoder and a decoder, such as Figure 1 As shown, the method comprises the following steps:

[0066] Step 101: Determine a first decoder state of a first target sentence; the first target sentence is a sentence to be translated in a source language sentence, and the first decoder state is a decoder state obtained by translating the first target sentence using the neural machine translation model.

[0067] The technical solution of the embodiment of the present application can use a neural machine translation model to translate a source language sentence into a target language, wherein the neural machine translation model has an "encoder-decoder" architecture, and the encoder is used to convert a source sentence in the source language sentence into a "meaning" vector, and then the "meaning" vector is converted into a target sentence for final translation through the decoder. In the process of translating a sentence in the source language sentence into a sentence in the target language using the neural machine translation model, the encoder state and the decoder state when translating the source language sentence can be obtained.

[0068] Step 102: Acquire a first encoder state from the target region; the first encoder state refers to the encoder state of the target sentence set that has been translated from the source language sentence; wherein the target region is used to determine the target sentence set based on a cache update mechanism and save the encoder state of the target sentence set;

[0069] In the embodiment of the present application, the target area is a cache area, and the encoder states of the target sentence set that have been translated in the source language sentence are stored in the cache area. Here, the encoder states of the translated target sentence set stored in the cache area can be the encoder states of all the translated sentences, or the encoder states of some of the translated sentences, that is, the target sentence set can be all the sentences in the translated source language sentence, or some of the translated source language sentence. Figure 2 A schematic diagram of a neural machine translation model provided in an embodiment of the present application, such as Figure 2 As shown, in addition to the encoder and decoder, the embodiment of the present application introduces a cache model on the basis of the traditional neural machine translation model to form the neural machine translation model of the embodiment of the present application. The information stored in the cache can be determined through the information storage and update mechanism (i.e., cache update mechanism) inside the cache model.

[0070] In an optional implementation manner of the present application, before determining the first decoder state of the first target sentence, the method further includes:

[0071] Determining an encoder state of a second target sentence; the second target sentence is a sentence in the source language sentence that precedes the first target sentence;

[0072] If there is remaining storage space in the target area, saving the encoder state of the second target sentence to the target area, the target sentence set including at least the second target sentence;

[0073] If there is no remaining storage space in the target area, it is determined whether to replace one of the encoder states stored in the target area with the encoder state of the second target sentence according to the importance score of the second target sentence.

[0074] Specifically, based on Figure 2 In the neural machine translation model shown, step 1 and step 2 are respectively the translation process of two adjacent sentences in the source language text, and step 1 is set to be the translation process of sentence 1 (i.e., the second target sentence), and step 2 is the translation process of sentence 2 (i.e., the first target sentence), and sentence 1 is the sentence translated before sentence 2. During the operation of the neural machine translation model in the embodiment of the present application, the encoder state of sentence 1 will be input into the cache model, and the information storage and update mechanism inside the cache model will be used to determine whether to save the encoder information of sentence 1 to the cache. Among them, if sentence 1 is the first sentence in the source language text currently being translated, the state of the decoder of sentence 1 is only affected by the encoder state of sentence 1; if sentence 1 is a sentence in the middle of the source language text currently being translated (that is, the first sentence in a non-source language text), the cache model can save the encoder state of sentence 1 to the cache if there is still remaining storage space in the cache, and if there is no remaining storage space in the cache, the cache model can determine whether to replace the encoder state of one of the multiple translated sentences in the cache with the encoder state of sentence 1 according to the importance score of sentence 1, wherein the one sentence is a sentence translated before sentence 1.

[0075] In an optional implementation manner of the present application, determining whether to replace one of the encoder states stored in the target area with the encoder state of the second target sentence according to the importance score of the second target sentence includes:

[0076] If the importance score of the second target sentence is less than all encoder states stored in the target region, maintaining all encoder states stored in the target region unchanged;

[0077] If the importance score of the second target sentence is greater than the partial encoder states stored in the target region, one of the encoder states in the partial encoder states is replaced with the encoder state of the second target sentence.

[0078] Specifically, Figure 3 This is a schematic diagram of a cache information screening submodule provided in an embodiment of the present application, and the cache information screening submodule is part of the cache model. Figure 3As shown, each unit in the cache of the embodiment of the present application is a four-tuple of the form (score, key, value, id), and each unit stores the encoder state of a translated sentence in the source language sentence, wherein the key and value represent the encoder state of a sentence, the id represents the sentence index of the sentence in the entire source language discourse, and the score represents the importance score of the sentence.

[0079] In one embodiment, the size of the cache is fixed. When there is space left in the cache, new cache entries (i.e., encoder states of translated sentences) will be directly recorded in the cache; and when the storage space in the cache is full, the cache entries will be replaced and updated according to the value of score. In the embodiment of the present application, the encoder states stored in the cache area are the encoder states of the sentences with higher importance scores among the sentences that have been translated in the source language. Exemplarily, based on Figure 2 In the neural machine translation model shown, when there is no remaining space in the cache, if the encoder states of sentence 1, sentence 2, sentence 3, sentence 4, and sentence 5 are stored in the cache, after translating sentence 6 (sentence 6 is translated after sentence 1, sentence 2, sentence 3, sentence 4, and sentence 5), if the score of sentence 6 is lower than the score of any sentence from sentence 1 to sentence 5, the cache information screening submodule will not store the encoder state of sentence 6 in the cache. If the score of sentence 6 is higher than the score of sentence 3 among sentences 1 to sentence 5 (sentence 3 is the lowest score among sentences 1 to sentence 5), the cache information screening submodule will replace the encoder state of sentence 3 with the encoder state of sentence 6.

[0080] The cache information screening mechanism, i.e., the cache update mechanism, of the embodiment of the present application can overcome the problems in the traditional scheme of using a fixed window to select context information, such as the difficulty in determining the length of the window, the inability to effectively cover the required context, or the introduction of too much noise. By using the cache, when translating a sentence in the source language text, longer distance information can be used as reference information for cross-sentence translation.

[0081] In the embodiment of the present application, the calculation of the score is the key. In an optional implementation manner of the present application, for each sentence in the source language sentence, the importance score of the sentence is determined based on the subject-statement structure information of the sentence.

[0082] Here, Theme and Rheme are concepts in systemic functional linguistics. Systemic functional linguistics is used for discourse analysis, and the subject-verb structure information including the theme, the rheme and the relationship between them is the main method to describe discourse function in systemic functional linguistics.

[0083] In one implementation, determining the importance score of the sentence based on the subject-statement structure information of the sentence includes:

[0084] The importance score of the sentence is obtained by performing linear transformation and logistic regression processing on the subject-statement structure information of the sentence.

[0085] Specifically, Figure 4 A schematic diagram of a method for determining the importance score of a sentence provided in an embodiment of the present application; Figure 4 As shown, for a source language containing multiple sentences, for each sentence in the source language sentence, the subject-statement structure of each given source language sentence can be automatically identified through a subject-statement annotation network based on a bidirectional long short-term memory network (LSTM), wherein the subject-statement annotation network based on the bidirectional LSTM is trained on a manually annotated subject-statement annotated text corpus. Afterwards, the result of the automatic identification of the subject-statement structure of each sentence in the given source language sentence is used as a subject-statement vector and then processed by a logistic regression model after passing through a linear transformation layer to finally obtain the importance score of each sentence in the given source language sentence, and the value range of the importance score is between 0 and 1.

[0086] Here, the memory and forgetting of information stored in the traditional cache are based on the principle of "first in, first out". Simply put, when the cache space is full, the new information will directly replace the earliest stored information. The technical solution of the embodiment of the present application introduces subject-statement structure information, determines the subject-statement structure information of each sentence in the source language text, determines the importance score of each sentence in the source language text based on the subject-statement structure information, and effectively screens the encoder information of the translated source language text stored in the cache through the importance score of the sentence, thereby retaining the important information in the source language text in the cache, affecting the decoding process of the translated sentences in the subsequent text.

[0087] Step 103: Determine a second decoder state of the first target sentence based on the first encoder state and the first decoder state.

[0088] In the embodiment of the present application, determining the second decoder state of the first target sentence based on the first encoder state and the first decoder state includes:

[0089] A correlation between the first decoder state and the first encoder state is determined, and the second decoder state is determined based on the correlation and the first decoder state.

[0090] Specifically, as an optional implementation manner, determining the correlation between the first decoder state and the first encoder state, and determining the second decoder state based on the correlation and the first decoder state includes:

[0091] Inputting the first encoder state and the first decoder state into a multi-head attention model, and obtaining the correlation between the first decoder state and the first encoder state through the multi-head attention model;

[0092] Inputting the association degree into a fully connected feed-forward network, and obtaining a target vector through the fully connected feed-forward network;

[0093] The target vector is weighted with the first decoder state to obtain a second decoder state of the first target sentence.

[0094] Specifically, Figure 5 A schematic diagram of a cache information utilization submodule provided in an embodiment of the present application, wherein the cache information utilization submodule is a part of a cache model, such as Figure 5 As shown in Figure 2, the multi-head attention model in the cache information utilization submodule can measure the decoder state s at a certain step in the neural machine translation model. t and the context information in the source language sentence in the cache h i The correlation between them is as follows:

[0095] d t =FFN(MultiHead(s t ,h i ,h i )) (1)

[0096] Specifically, the multi-head attention model can calculate the correlation between three inputs Q, K, and V, where Q is a request, which corresponds to the state s of the current decoder in the neural machine translation model. t ; K is the matrix used to constrain Q, and V is the state that actually needs to be correlated with Q. In the neural machine translation model, K and V are both the context information h in the source language sentence in the cache. i .

[0097] Then, referring to the Transformer model, the cache information utilization submodule inputs the operation results of the multi-head attention model into the fully connected feed-forward network (Position-wise Feed-Forward Networks), and then obtains the vector d containing the context information in the cache tHere, similar to the Transformer model, as a preferred implementation, a normalization layer (NormalizationLayer) is provided after the multi-head attention model and the fully connected feed-forward network.

[0098] Finally, the cache information uses the gating structure in the submodule to convert the vector d containing the context information in the cache t and the current decoder state s in the neural machine translation model t The vector is weighted to obtain a new decoder state c that contains the source language context information. t , where the new decoder state c t The calculation formula is as follows:

[0099] c t =α×s t +(1-α)×d t (2)

[0100] Here, the encoder state of the sentence in the translated source language text is introduced into the decoder, thereby affecting the decoder state of the sentence to be translated, thereby realizing the transmission of cross-sentence information during the translation process.

[0101] The technical solution of the embodiment of the present application, when translating the source language text, introduces the subject-statement structure information as a guide for the inter-sentence cache screening, and with the help of the structural information of the source language text, can more effectively screen and retain the important information in the source language text in the cache, and because the encoder information of the translated sentences is memorized in the cache, it can prevent the propagation of the wrong decoder information translated in the source language text and overcome the problem of context loss during missed translation. In addition, the embodiment of the present application uses the cache to break the defect of the fixed window of context selection in the traditional technical solution, and can make use of longer distance information when translating a sentence in the source language text.

[0102] Figure 6 A schematic diagram of the structure of a translation device provided in an embodiment of the present application, wherein the device is applied to a neural machine translation model, wherein the model includes an encoder and a decoder, such as Figure 6 As shown, the device comprises:

[0103] A first determining unit 601 is used to determine a first decoder state of a first target sentence; the first target sentence is a sentence to be translated in a source language sentence, and the first decoder state is a decoder state obtained by translating the first target sentence using the neural machine translation model;

[0104] The acquisition unit 602 is used to acquire a first encoder state from the target region; the first encoder state refers to the encoder state of the target sentence set that has been translated from the source language sentence; wherein the target region is used to determine the target sentence set based on a cache update mechanism and save the encoder state of the target sentence set;

[0105] The second determining unit 603 is used to determine a second decoder state of the first target sentence based on the first encoder state and the first decoder state.

[0106] In an optional implementation manner of the present application, before determining the first decoder state of the first target sentence, the apparatus further includes:

[0107] A third determining unit 604 is used to determine the encoder state of a second target sentence; the second target sentence is a sentence in the source language sentence that is located before the first target sentence;

[0108] A saving unit 605 is configured to save the encoder state of the second target sentence to the target region if there is remaining storage space in the target region, and the target sentence set includes at least the second target sentence;

[0109] The fourth determination unit 606 is used to determine whether to replace one of the encoder states stored in the target area with the encoder state of the second target sentence according to the importance score of the second target sentence if there is no remaining storage space in the target area.

[0110] In an optional implementation of the present application, the fourth determination unit 606 is specifically used to: if the importance score of the second target sentence is less than all encoder states stored in the target area, maintain all encoder states stored in the target area unchanged; if the importance score of the second target sentence is greater than some encoder states stored in the target area, replace one of the encoder states in the partial encoder states with the encoder state of the second target sentence.

[0111] In an optional implementation manner of the present application, the device further includes:

[0112] The fifth determining unit 607 is configured to determine, for each sentence in the source language sentence, an importance score of the sentence based on the subject-statement structure information of the sentence.

[0113] In an optional implementation of the present application, the fifth determination unit 607 is specifically used to obtain the importance score of the sentence by performing linear transformation and logistic regression processing on the subject-statement structure information of the sentence.

[0114] In an optional implementation manner of the present application, the second determination unit 603 is specifically used to: determine the correlation between the first decoder state and the first encoder state, and determine the second decoder state based on the correlation and the first decoder state.

[0115] In an optional implementation of the present application, the second determination unit 603 is further specifically used to: input the first encoder state and the first decoder state into a multi-head attention model, and obtain the correlation between the first decoder state and the first encoder state through the multi-head attention model; input the correlation into a fully connected feedforward network, and obtain a target vector through the fully connected feedforward network; weight the target vector and the first decoder state to obtain a second decoder state of the first target sentence.

[0116] In an optional implementation of the present application, a regularization layer is provided after the multi-head attention model and the fully connected feed-forward network.

[0117] Those skilled in the art should understand that Figure 6 The functions implemented by each unit in the illustrated translation device may be understood by referring to the related description of the aforementioned translation method. Figure 6 The functions of each unit in the illustrated translation device may be implemented by a program running on a processor, or may be implemented by a specific logic circuit.

[0118] Figure 7 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. The electronic device may be a server, and the electronic device includes Figure 6 The translation device shown, Figure 7 The electronic device 700 shown includes a processor 701, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0119] Alternatively, if Figure 7 As shown, the electronic device 700 may further include a memory 702. The processor 701 may call and run a computer program from the memory 702 to implement the method in the embodiment of the present application.

[0120] The memory 702 may be a separate device independent of the processor 701 , or may be integrated into the processor 701 .

[0121] Alternatively, if Figure 7 As shown, the electronic device 700 may further include a transceiver 703, and the processor 701 may control the transceiver 703 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices.

[0122] The transceiver 703 may include a transmitter and a receiver. The transceiver 703 may further include an antenna, and the number of antennas may be one or more.

[0123] Optionally, the electronic device 700 may specifically be a network device in an embodiment of the present application, and the electronic device 700 may implement corresponding processes implemented by the network device in various methods in the embodiments of the present application, which will not be described in detail here for the sake of brevity.

[0124] Optionally, the electronic device 700 may specifically be a mobile terminal / terminal device of an embodiment of the present application, and the electronic device 700 may implement the corresponding processes implemented by the mobile terminal / terminal device in each method of the embodiment of the present application, which will not be described in detail here for the sake of brevity.

[0125] Figure 8 It is a schematic structural diagram of the chip of an embodiment of the present application. Figure 8 The chip 800 shown includes a processor 801, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0126] Alternatively, if Figure 8 As shown, the chip 800 may further include a memory 802. The processor 801 may call and run a computer program from the memory 802 to implement the method in the embodiment of the present application.

[0127] The memory 802 may be a separate device independent of the processor 801 , or may be integrated into the processor 801 .

[0128] Optionally, the chip 800 may further include an input interface 803. The processor 801 may control the input interface 803 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0129] Optionally, the chip 800 may further include an output interface 804. The processor 801 may control the output interface 804 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0130] Optionally, the chip can be applied to the network device in the embodiments of the present application, and the chip can implement the corresponding processes implemented by the network device in each method of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0131] Optionally, the chip can be applied to the mobile terminal / terminal device in the embodiments of the present application, and the chip can implement the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0132] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0133] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0134] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0135] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0136] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0137] Optionally, the computer-readable storage medium can be applied to the network device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the network device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0138] Optionally, the computer-readable storage medium can be applied to the mobile terminal / terminal device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0139] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0140] Optionally, the computer program product can be applied to the network device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the network device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0141] Optionally, the computer program product can be applied to the mobile terminal / terminal device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0142] The embodiment of the present application also provides a computer program.

[0143] Optionally, the computer program can be applied to the network device in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes implemented by the network device in the various methods in the embodiments of the present application. For the sake of brevity, they are not described here.

[0144] Optionally, the computer program can be applied to the mobile terminal / terminal device in the embodiments of the present application. When the computer program runs on the computer, the computer executes the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0145] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0146] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0147] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0150] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0151] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A translation method, characterized in that: The method is applied to a neural machine translation model, the model comprising an encoder and a decoder, and the method comprises: Determining a first decoder state of a first target sentence; the first target sentence is a sentence to be translated in a source language sentence, and the first decoder state is a decoder state obtained by translating the first target sentence using the neural machine translation model; Acquire a first encoder state from the target region; the first encoder state refers to an encoder state of a target sentence set that has been translated from the source language sentence; wherein the target region is used to determine the target sentence set based on a cache update mechanism and save the encoder state of the target sentence set; determining a second decoder state for the first target sentence based on the first encoder state and the first decoder state; Wherein, before determining the first decoder state of the first target sentence, the method includes: Determining an encoder state of a second target sentence; the second target sentence is a sentence in the source language sentence that precedes the first target sentence; If there is remaining storage space in the target area, saving the encoder state of the second target sentence to the target area, the target sentence set including at least the second target sentence; If there is no remaining storage space in the target area, it is determined whether to replace one of the encoder states stored in the target area with the encoder state of the second target sentence according to the importance score of the second target sentence.

2. The method according to claim 1, characterized in that The determining, according to the importance score of the second target sentence, whether to replace one of the encoder states stored in the target area with the encoder state of the second target sentence comprises: If the importance score of the second target sentence is less than all encoder states stored in the target region, maintaining all encoder states stored in the target region unchanged; If the importance score of the second target sentence is greater than the partial encoder states stored in the target region, one of the encoder states in the partial encoder states is replaced with the encoder state of the second target sentence.

3. The method according to claim 1, characterized in that The method further comprises: For each sentence in the source language sentence, the importance score of the sentence is determined based on the subject-statement structure information of the sentence.

4. The method according to claim 3, characterized in that The determining the importance score of the sentence based on the subject-statement structure information of the sentence includes: The importance score of the sentence is obtained by performing linear transformation and logistic regression processing on the subject-statement structure information of the sentence.

5. The method according to any one of claims 1 to 4, characterized in that The determining a second decoder state of the first target sentence based on the first encoder state and the first decoder state comprises: A correlation between the first decoder state and the first encoder state is determined, and the second decoder state is determined based on the correlation and the first decoder state.

6. The method according to claim 5, characterized in that The determining of the correlation between the first decoder state and the first encoder state, and determining the second decoder state based on the correlation and the first decoder state, comprises: Inputting the first encoder state and the first decoder state into a multi-head attention model, and obtaining the correlation between the first decoder state and the first encoder state through the multi-head attention model; Inputting the association degree into a fully connected feed-forward network, and obtaining a target vector through the fully connected feed-forward network; The target vector is weighted with the first decoder state to obtain a second decoder state of the first target sentence.

7. The method according to claim 6, characterized in that The multi-head attention model and the fully connected feed-forward network are both provided with a regularization layer.

8. A translation device, characterized in that: The device is applied to a neural machine translation model, the model includes an encoder and a decoder, and the device includes: A first determining unit is used to determine a first decoder state of a first target sentence; the first target sentence is a sentence to be translated in a source language sentence, and the first decoder state is a decoder state obtained by translating the first target sentence using the neural machine translation model; an acquisition unit, configured to acquire a first encoder state from a target region; the first encoder state refers to an encoder state of a target sentence set that has been translated from a source language sentence; wherein the target region is used to determine the target sentence set based on a cache update mechanism and save the encoder state of the target sentence set; A second determining unit, configured to determine a second decoder state of the first target sentence based on the first encoder state and the first decoder state; Before determining the first decoder state of the first target sentence, the apparatus further includes: A third determining unit is used to determine the encoder state of a second target sentence; the second target sentence is a sentence in the source language sentence that is located before the first target sentence; a saving unit, configured to save the encoder state of the second target sentence to the target region if there is remaining storage space in the target region, the target sentence set including at least the second target sentence; A fourth determination unit is used to determine whether to replace one of the encoder states stored in the target area with the encoder state of the second target sentence according to the importance score of the second target sentence if there is no remaining storage space in the target area.

9. The device according to claim 8, characterized in that The device also includes: The fifth determining unit is used to determine, for each sentence in the source language sentence, the importance score of the sentence based on the subject-statement structure information of the sentence.

10. The device according to any one of claims 8 to 9, characterized in that The second determining unit is specifically configured to: determine the correlation between the first decoder state and the first encoder state, and determine the second decoder state based on the correlation and the first decoder state.

11. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 7.

12. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 7.