Text translation method, device, equipment and medium based on attention mechanism

Abstract

The embodiment of the present invention discloses a text translation method, device, equipment and medium based on an attention mechanism, the method comprising: receiving a translation request for a text; obtaining the text according to the translation request and preprocessing the text; encoding the preprocessed text according to a preset encoder to obtain a text vector of the text; combining words in the word vectors in the text vector according to the pre-attention of a preset decoder to obtain a combination vector; decoding the text vector according to the weight of the combination vector to obtain a translated text of the text. The present invention is based on natural language processing technology, combines words in the encoded text through an attention mechanism, and determines the optimal translation position of the text according to the weight of the combined words to decode the encoded text, which greatly improves the accuracy of text translation.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a text translation method, device, equipment and medium based on an attention mechanism. Background Art

[0002] Machine Translation (MT) refers to the process of converting one natural language into another natural language. In the prior art, machine translation usually uses an encoder-decoder containing an Attention Transformer to translate text. However, when this translation method is used for text translation, the translation machine cannot guarantee the quality of the translated text, especially when translating a whole sentence in the text, the accuracy of the translation cannot be guaranteed. For example, when there is "I went to the park at 10 am" in the text, the translation machine cannot determine the best position for translation during the translation of the sentence, and will translate "morning" and "ten o'clock" into "morning" and "10 o'clock" respectively, resulting in errors in the translation of the sentence, and the sentence cannot be accurately translated into "I went to the park at 10 am". Summary of the invention

[0003] In response to the above technical problems, the embodiments of the present invention provide a text translation method, apparatus, device and medium based on an attention mechanism, aiming to solve the problem in the prior art that the quality of the translated text cannot be guaranteed when translating the text.

[0004] In a first aspect, an embodiment of the present invention provides a text translation method based on an attention mechanism, which includes:

[0005] receiving translation requests for texts;

[0006] Acquire the text according to the translation request and pre-process the text;

[0007] Encoding the preprocessed text according to a preset encoder to obtain a text vector of the text;

[0008] Combining words on the word vectors in the text vector according to the pre-attention of the preset decoder to obtain a combined vector;

[0009] The text vector is decoded according to the weight of the combined vector to obtain a translation of the text.

[0010] In a second aspect, an embodiment of the present invention provides a text translation device based on an attention mechanism, which includes:

[0011] A receiving unit, configured to receive a translation request for a text;

[0012] A preprocessing unit, configured to obtain the text according to the translation request and preprocess the text;

[0013] An encoding unit, used for encoding the preprocessed text according to a preset encoder to obtain a text vector of the text;

[0014] A combining unit, used for combining words in the word vectors of the text vector according to the pre-attention of the preset decoder to obtain a combined vector;

[0015] The first decoding unit is used to decode the text vector according to the weight of the combined vector to obtain a translation text of the text.

[0016] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the attention-based text translation method as described in the first aspect above when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the text translation method based on the attention mechanism described in the first aspect above.

[0018] The embodiment of the present invention provides a text translation method, device, equipment and medium based on an attention mechanism. The method receives a translation request for a text, obtains the text according to the translation request and preprocesses the text, then encodes the preprocessed text and combines words between word vectors in the text vector obtained after encoding based on the attention mechanism to obtain a combination vector; finally, the encoded text vector is decoded according to the weight of the combination vector to obtain a translated text of the text. Before decoding the encoded text, the embodiment of the present invention combines the words in the encoded text by using an attention mechanism, and then determines the best translation position of the text according to the weight of the combined words to decode the encoded text, thereby completing the translation of the text, greatly improving the quality of text translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0020] Figure 1 A schematic diagram of an application scenario of a text translation method based on an attention mechanism provided in an embodiment of the present invention;

[0021] Figure 2 A flowchart of a text translation method based on an attention mechanism provided in an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of a sub-process of a text translation method based on an attention mechanism provided in an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of another sub-process of the text translation method based on the attention mechanism provided in an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of another sub-process of the text translation method based on the attention mechanism provided in an embodiment of the present invention;

[0025] Figure 6 Another flowchart of a text translation method based on an attention mechanism provided by an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of another sub-process of the text translation method based on the attention mechanism provided in an embodiment of the present invention;

[0027] Figure 8 A schematic block diagram of a text translation device based on an attention mechanism provided by an embodiment of the present invention;

[0028] Fig. 9 A schematic block diagram of subunits of a text translation device based on an attention mechanism provided in an embodiment of the present invention;

[0029] Fig.10 A schematic block diagram of another subunit of the text translation device based on the attention mechanism provided in an embodiment of the present invention;

[0030] Fig.11 A schematic block diagram of another subunit of the text translation device based on the attention mechanism provided in an embodiment of the present invention;

[0031] Fig.12 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0034] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0035] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0036] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of a text translation method based on an attention mechanism provided in an embodiment of the present invention; Figure 2 A flowchart of a text translation method based on an attention mechanism provided in an embodiment of the present invention. The text translation method based on an attention mechanism in an embodiment of the present invention is applied to a management server 20. After receiving a translation request for a preset text from a user terminal 10, the management server 20 obtains the text and translates the text. The method is executed by application software installed in the management server 20. The user terminal 10 is a terminal device with an Internet access function, such as a desktop computer, a laptop computer, a tablet computer, or a mobile phone.

[0037] The text translation method based on the attention mechanism is described in detail below. It should be noted that the text in the embodiment of the present invention is a Chinese text and the Chinese text is to be translated into an English text, that is, the translated text is an English text.

[0038] like Figure 2 As shown, the method includes the following steps S110~S150.

[0039] S110: Receive a text translation request.

[0040] Specifically, the translation request is instruction information in which the user terminal 10 requests the management server 20 to translate the text input by the user at the user terminal 10. After receiving the translation request sent by the user terminal 10, the management server 20 can obtain the text from the user terminal 10 according to the translation request and translate the text accordingly.

[0041] S120: Acquire the text according to the translation request and pre-process the text.

[0042] Specifically, the management server 20 cannot directly understand the words in the text. Therefore, before the management server 20 translates the text to be translated, it needs to pre-process the text so that the management server 20 can execute the translation program on the text. In this embodiment, after obtaining the text, the management server 20 performs special character removal, word segmentation, text annotation and other steps on the text, so that the management server 20 can more easily translate the text, thereby improving the efficiency of the management server 20 in translating the text.

[0043] In another embodiment, if Figure 3 As shown, step S120 includes sub-steps S121 and S122.

[0044] S121, performing word segmentation processing on the text to obtain words of the text.

[0045] Specifically, in the process of word segmentation of the text, there are usually three methods for word segmentation of the sentences in the text, including: a word segmentation method based on string matching, a word segmentation method based on statistics, and a word segmentation method based on understanding, wherein the word segmentation method based on string matching is a word segmentation method that matches a Chinese character string with an entry in a dictionary; the word segmentation method based on statistics counts the frequency of the combination of adjacent co-occurring characters in the corpus, calculates the probability of co-occurrence of adjacent characters, and thus determines whether adjacent characters can form a word; the word segmentation method based on understanding achieves the effect of identifying words by letting the computer simulate a person's understanding of the text. In an embodiment of the present invention, before word segmentation of the text, a preset regular expression is used to remove special characters in the text, and then the text after removal is subjected to word segmentation using the reverse maximum matching method in the word segmentation method based on string matching, and the word segmentation process is as follows: the number of Chinese characters contained in the longest entry in the preset dictionary is set to L, and processing is started from the end of the string of the text. At the beginning of each loop, the last L characters of the string are taken as the processing object and searched in the dictionary. If such an L-character word exists in the dictionary, the match is successful and the processing object is segmented as a word; if it is unsuccessful, the first Chinese character of the processing object is removed and the remaining string is used as a new processing object and matched again until the segmentation is successful, that is, a round of matching is completed and a word is segmented. This cycle repeats until all the words in the text are segmented.

[0046] For example, the length of the longest word in the dictionary is 6. For the text string "I went to the park at 10 am", first take the 6 words "I went to the park" as the string to be processed. There is no such word in the dictionary, so the match fails; remove the first word, and the remaining "I went to the park" is used as the new string to be processed, and the match fails again; continue in this way, and finally take "park" as the matching field. If there is such a word in the dictionary, the match is successful, and the first word "park" is split. Then take the remaining string "I went to the park at 10 am" in the text, and split it at the second word "trip". This cycle repeats, and the final segmentation result is: "morning", "ten o'clock", "I", "go", "yes", "trip", "park".

[0047] S122: Perform part-of-speech tagging on the words in the text to obtain the part-of-speech tagged words.

[0048] Specifically, the part-of-speech tagging refers to the process of marking a correct part of speech for the words, that is, the process of determining whether the words are nouns, verbs, adjectives or other parts of speech. The part of speech is the grammatical attribute of the words, which is determined based on the grammatical function of the words in the combination. The grammatical attributes of words in Chinese include nouns, verbs, adjectives, numerals, quantifiers, pronouns, distinguishing words, adverbs, prepositions, conjunctions, auxiliary words, interjections, modal particles, and onomatopoeia, a total of fourteen types of attributes. In this embodiment, in the process of part-of-speech tagging of the words in the text, the text is firstly subjected to syntactic analysis to determine the positional relationship of the words in the text in the sentence, and then the words in the text are tagged with parts of speech using a preset HMM hidden Markov model under the condition of a preset corpus, so that the words after the part-of-speech tagging can be obtained.

[0049] S130 . Encode the preprocessed text according to a preset encoder to obtain a text vector of the text.

[0050] In this embodiment, when the management server 20 translates the text, it adopts the Encoder-Decoder model framework, that is, the encoding-decoding model framework, wherein the Encoder model in the Encoder-Decoder model framework is used to decode the preprocessed text to convert the text into a vector of a fixed length, and the Decoder model in the Encoder-Decoder model is used to encode the text vector generated by the Encoder model to convert it into the translated text of the text.

[0051] In another embodiment, if Figure 4 As shown, step S130 includes sub-steps S131, S132 and S133.

[0052] S131, performing word embedding processing on the words after the part-of-speech tagging to obtain word vectors containing the part-of-speech tag.

[0053] Specifically, word embedding is a method of converting words in a text into digital vectors. After the tagged words are processed by word embedding, standard machine learning algorithms can be used to indirectly analyze the text. Word embedding mainly embeds a high-dimensional space with a dimension equal to the number of all words into a continuous vector space with a much lower dimension. Each word or phrase is mapped to a vector on the real number field, thereby generating a vector of the word after the part-of-speech tagging. In this embodiment, the word after the part-of-speech tagging is processed by word embedding using the embedding of a single word and the binary Bi-Gram in the N-Gram model, thereby effectively increasing the expressive power of the word after the part-of-speech tagging.

[0054] S132. Input the word vector containing the part-of-speech tag into the self-attention layer of the encoder for calculation to obtain a word vector containing a weight.

[0055] S133, inputting the weighted word vector into the feedforward fully connected neural network layer of the encoder for operation to obtain the text vector.

[0056] In this embodiment, the self-attention layer in the encoder is used to capture the attention of each word in the text to the words at other positions in the text so that the feedforward fully connected neural network layer in the encoder can better encode the text, and the attention is the weight of each word in the text. After receiving the word vector containing weights output by the self-attention layer, the feedforward fully connected neural network layer operates on the word vector containing weights to obtain a text vector of fixed length, that is, the text vector of the text. The position information of each word vector in the text vector is saved in the text vector, thereby ensuring that when generating each word of the target language during the decoding process, relevant information can be directly selected from the text vector as an auxiliary, greatly shortening the distance of information transmission. Among them, the self-attention layer uses an attention mechanism to capture the connection between each word in the text and words at other positions in the text from the text. The attention mechanism is a data processing method in machine learning, which is widely used in various types of machine learning tasks such as natural language processing, image recognition and speech recognition.

[0057] S140, combining words in the word vectors in the text vector according to the pre-attention of the preset decoder to obtain a combined vector.

[0058] Specifically, the decoder is a pre-attention decoder and is used to decode the text vector of the text to generate a translation text of the text. After the encoder generates the text vector of the text, the text vector is input into the decoder and the pre-attention of the decoder is used to combine the word vectors in the text vector in the order of the characters in the text to obtain the combination vector. For example, when the text is: I went to the park at 10 am, the words after the text is segmented and annotated are: morning, ten o'clock, I, go, have, have, park, and after the encoder encodes the words of the text, the word vectors in the obtained text vector respectively represent the words "morning", "ten o'clock", "I", "go", "have", "have", "park", and the decoder combines the word vectors representing the words in the text vector from left to right in the order of the characters in the text through pre-attention in the form of character combination, so as to obtain a combination vector, which respectively represents "ten o'clock in the morning", "I go", "have", and "park" in the text.

[0059] In another embodiment, if Figure 5 As shown, step S140 also includes steps S140a and S140b before step S140.

[0060] S140a. Determine whether the word vector with the maximum weight in the text vector is located in the first column of the text vector.

[0061] S140b: If it is located in the first column of the text vector, decode the text vector according to the weight of the word vector in the text vector to obtain the translated text.

[0062] Specifically, when the decoder decodes the text vector generated by the encoder after encoding, it usually decodes the text vector according to the arrangement order of the characters in the text, and then completes the translation of the text. In this embodiment, the decoder is a pre-attention decoder, and the pre-attention decoder usually translates the text according to the weight of the words in the text. However, when the word with the largest weight in the text is not at the first position of the text, the decoder cannot accurately decode the remaining words in the text after decoding the word with the largest weight in the text. Therefore, after the decoder obtains the weight of each word vector in the text vector through pre-attention, it is necessary to determine whether the word vector with the largest weight in the text vector is located in the first column of the text vector. If it is located in the first column of the text vector, the text vector is decoded according to the size of the weight of the word vector in the text vector, and the translated text can be obtained. For example, when the words in the text are "I", "use", "computer", "write", and "code", after the decoder decodes the text vector, the translated text obtained is "I use a computer to write code"; if it is not located in the first column of the present vector, step S140 is executed. For example, when the words in the text are "morning", "ten o'clock", "I", "go", "have", "trip", and "park", the weights of the word vectors in the text vector are "I", "go", "have", "trip", "park", "morning", and "ten o'clock" from large to small. After the decoder continues to decode the text vector according to the size of the weight of the word vector in the text vector, "I went to the park morning 10 o'clock" is obtained, which leads to low translation accuracy. Therefore, when the word vector with the largest weight in the text vector is not located in the first column of the text vector, step S140 needs to be executed.

[0063] S150: Decode the text vector according to the weight of the combined vector to obtain a translation of the text.

[0064] Specifically, the word vectors in the text vector are combined in the order of characters in the text through the pre-attention of the decoder to obtain the combined vector, and then the weight of the combined vector is calculated based on the weight of the word vector in the text vector to obtain the weight of the combined vector. Finally, the corresponding combined vectors are decoded in turn according to the size of the weight of the combined vector to complete the decoding of the text vector and obtain the translated text of the text.

[0065] In another embodiment, if Figure 6 As shown, step S150a is included before step S150.

[0066] S150a, calling a preset Argmax function to perform weight calculation on the combination vector to obtain the weight of the combination vector.

[0067] In this embodiment, after the word vectors in the text vector are combined from left to right in the order of characters in the text by the decoder's front-end attention, a preset Argmax function is called to perform weight calculation on the combined vector to obtain the weight of the combined vector. The Argmax function is used to calculate the maximum value of the weight of each combined vector.

[0068] In another embodiment, if Figure 7 As shown, step S150 includes sub-steps S151, S152 and S153.

[0069] S151, obtaining the combination vector with the maximum weight and the weights of the first word vector and the second word vector adjacent to the position of the combination vector with the maximum weight; wherein the first word vector and the second word vector are both word vectors without word combination.

[0070] S152. Determine whether the weight of the first word vector is equal to the weight of the second word vector.

[0071] S153: If they are not equal, the combined vector with the maximum weight is combined with the second word vector again, and the text vector is decoded according to the weight of the recombined vector to obtain the translated text.

[0072] In this embodiment, after calculating the weight of the combined vector by invoking the Argmax function to obtain the weight of the combined vector, the combined vector with the maximum weight is obtained from the combined vector, and the first word vector and the second word vector adjacent to the combined vector are obtained from the text vector according to the combined vector, where both the first word vector and the second word vector are word vectors without word combination. The first word vector is the word vector at the previous position of the combined vector with the maximum weight, and the second word vector is the word vector at the next position of the combined vector with the maximum weight. Then, it is judged whether the weight of the first word vector is equal to the weight of the second word vector. If they are equal, the combined vectors are decoded in descending order according to the weight of the combined vector to obtain the translation text; if they are not equal, the combined vector and the second word vector are combined again in the order of characters from left to right in the text in the form of character combination, and then it is judged whether the weights of the word vectors on both sides of the recombined vector are equal. The above steps are repeated until the word vectors on both sides of the combined vector are equal and then decoded. Among them, when there is no word vector at the next position of the combined vector, the last combined vector is directly decoded, and the word vectors at the previous position after the last combination are decoded in the order of the weights of the first combined vector until the translation text is obtained. For example, when the text is: I went to the park at ten o'clock in the morning, the words represented by the word vectors in the text vector are: morning, ten o'clock, I, go, to, park. After combining the word vectors in the text vector, the combined vectors represent "ten o'clock in the morning", "I go", "to the park" in the text respectively. The combined vector with the maximum weight calculated is "I go". At the same time, it is judged whether the weight of the word vector "ten o'clock" is equal to the weight of the word vector "to". If they are not equal, the combined vector with the maximum weight is combined with the second word vector, and the recombined vector "I went to" can be obtained. Then, it is judged whether the weights of the word vectors before and after the recombined vector are equal until the finally obtained combined vectors are "I went to the park" and "ten o'clock in the morning". Finally, the finally obtained combined vectors are decoded according to the weights of the finally obtained combined vectors, and the translation text can be obtained.

[0073] In the text translation method based on the attention mechanism provided in the embodiment of the present invention, a translation request for a text is received; the text is obtained and preprocessed according to the translation request; the preprocessed text is encoded according to a preset encoder to obtain a text vector of the text; the word vectors in the text vector are combined according to the pre-attention of a preset decoder to obtain a combined vector; the text vector is decoded according to the weight of the combined vector to obtain a translated text of the text. Before decoding the encoded text, the embodiment of the present invention combines the words in the encoded text by using an attention mechanism, and then determines the best translation position of the text according to the weight of the combined words to decode the encoded text, thereby completing the translation of the text, which greatly improves the accuracy of text translation.

[0074] The embodiment of the present invention further provides a text translation device 100 based on an attention mechanism, which is used to execute any embodiment of the aforementioned text translation method based on the attention mechanism.

[0075] Specifically, see Figure 8 , Figure 8 is a schematic block diagram of a text translation device 100 based on an attention mechanism provided in an embodiment of the present invention.

[0076] like Figure 8 As shown, the text translation device 100 based on the attention mechanism includes: a receiving unit 110, a preprocessing unit 120, an encoding unit 130, a combining unit 140 and a first decoding unit 150.

[0077] The receiving unit 110 is configured to receive a text translation request.

[0078] The preprocessing unit 120 is used to obtain the text according to the translation request and preprocess the text.

[0079] In other inventive embodiments, Fig. 9 As shown, the preprocessing unit 120 includes: a word segmentation unit 121 and a part-of-speech tagging unit 122.

[0080] The word segmentation unit 121 is used to perform word segmentation processing on the text to obtain words in the text; the part-of-speech tagging unit 122 is used to perform part-of-speech tagging on the words in the text to obtain the part-of-speech tagged words.

[0081] The encoding unit 130 is used to encode the preprocessed text according to a preset encoder to obtain a text vector of the text.

[0082] In other inventive embodiments, Fig.10As shown, the encoding unit 130 includes: a word embedding unit 131, a first input unit 132 and a second input unit 133.

[0083] The word embedding unit 131 is used to perform word embedding processing on the words after part-of-speech tagging to obtain word vectors containing part-of-speech tags; the first input unit 132 is used to input the word vectors containing part-of-speech tags into the self-attention layer of the encoder for calculation to obtain word vectors containing weights; the second input unit 133 is used to input the word vectors containing weights into the feedforward fully connected neural network layer of the encoder for calculation to obtain the text vector.

[0084] The combining unit 140 is used to combine words in the word vectors in the text vector according to the pre-attention of the preset decoder to obtain a combined vector.

[0085] In other inventive embodiments, Figure 8 As shown, the text translation device 100 based on the attention mechanism further includes: a first judgment unit 140a and a second decoding unit 140b.

[0086] The first judging unit 140a is used to judge whether the word vector with the maximum weight in the text vector is located in the first column of the text vector.

[0087] The second decoding unit 140b is used to decode the text vector according to the weight of the word vector in the text vector to obtain the translated text if it is located in the first column of the text vector.

[0088] The first decoding unit 150 is used to decode the text vector according to the weight of the combined vector to obtain a translation text of the text.

[0089] In other inventive embodiments, Figure 8 As shown, the text translation device 100 based on the attention mechanism further includes: a weight calculation unit 150a.

[0090] The weight calculation unit 150a is used to call a preset Argmax function to perform weight calculation on the combination vector to obtain the weight of the combination vector.

[0091] In other inventive embodiments, Fig.11 As shown, the first decoding unit 150 includes: an acquisition unit 151, a second judgment unit 152 and a third decoding unit 153.

[0092] The acquisition unit 151 is used to obtain the combination vector with the maximum weight and the weights of the first word vector and the second word vector adjacent to the position of the combination vector with the maximum weight; wherein the first word vector and the second word vector are both word vectors without word combination; the second judgment unit 152 is used to judge whether the weight of the first word vector is equal to the weight of the second word vector; the third decoding unit 153 is used to combine the combination vector with the maximum weight with the second word vector again if they are not equal, and decode the text vector according to the weight of the recombined vector to obtain the translated text.

[0093] The text translation device 100 based on the attention mechanism provided in the embodiment of the present invention is used to execute the above-mentioned translation request of the received text; obtain the text according to the translation request and preprocess the text; encode the preprocessed text according to a preset encoder to obtain a text vector of the text; combine the words in the word vectors in the text vector according to the pre-attention of a preset decoder to obtain a combination vector; decode the text vector according to the weight of the combination vector to obtain a translated text of the text.

[0094] See also Fig.12 , Fig.12 It is a schematic block diagram of a computer device provided by an embodiment of the present invention.

[0095] See also Fig.12 The device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0096] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 may execute a text translation method based on an attention mechanism.

[0097] The processor 502 is used to provide computing and control capabilities to support the operation of the entire device 500.

[0098] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a text translation method based on an attention mechanism.

[0099] The network interface 505 is used for network communication, such as providing data information transmission, etc. Those skilled in the art will understand that Fig.12The structure shown in the figure is merely a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the device 500 to which the solution of the present invention is applied. The specific device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0100] The processor 502 is used to run a computer program 5032 stored in a memory to implement the following functions: receiving a text translation request; obtaining the text according to the translation request and preprocessing the text; encoding the preprocessed text according to a preset encoder to obtain a text vector of the text; combining words in the word vectors in the text vector according to the pre-attention of a preset decoder to obtain a combined vector; decoding the text vector according to the weight of the combined vector to obtain a translated text of the text.

[0101] Those skilled in the art will understand that Fig.12 The embodiment of the device 500 shown in the figure does not constitute a limitation on the specific structure of the device 500. In other embodiments, the device 500 may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the device 500 may only include a memory and a processor 502. In such an embodiment, the structure and function of the memory and the processor 502 are the same as those of the embodiment of the present invention. Fig.12 The embodiments shown are consistent and will not be described again here.

[0102] It should be understood that in the embodiment of the present invention, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors 502, digital signal processors 502 (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor 502 may be a microprocessor 502 or the processor 502 may also be any conventional processor 502, etc.

[0103] In another embodiment of the present invention, a computer storage medium is provided. The storage medium may be a non-volatile computer-readable storage medium. The storage medium stores a computer program 5032, wherein the computer program 5032 implements the following steps when executed by the processor 502: receiving a translation request for a text; obtaining the text according to the translation request and preprocessing the text; encoding the preprocessed text according to a preset encoder to obtain a text vector of the text; combining words in the word vectors in the text vector according to the pre-attention of a preset decoder to obtain a combined vector; decoding the text vector according to the weight of the combined vector to obtain a translated text of the text.

[0104] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the above-described equipment, devices and units can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0105] In the several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. Units with the same function may also be combined into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be an electrical, mechanical or other form of connection.

[0106] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0107] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0108] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a device 500 (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk.

[0109] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A text translation method based on attention mechanism, characterized in that: The following steps are involved: receiving translation requests for texts; Acquire the text according to the translation request and pre-process the text; Encoding the preprocessed text according to a preset encoder to obtain a text vector of the text; Combining words on the word vectors in the text vector according to the pre-attention of the preset decoder to obtain a combined vector; Decoding the text vector according to the weight of the combined vector to obtain a translation of the text; Before combining the word vectors in the text vector according to the pre-attention of the preset decoder to obtain the combined vector, the method further includes: Determine whether the word vector with the maximum weight in the text vector is located in the first column of the text vector; If it is located in the first column of the text vector, decoding the text vector according to the weight of the word vector in the text vector to obtain the translated text; The decoding process of the text vector according to the weight of the combined vector to obtain the translation text of the text includes: Obtaining a combination vector with a maximum weight and weights of a first word vector and a second word vector adjacent to the combination vector with the maximum weight; wherein the first word vector and the second word vector are both word vectors without word combination; Determine whether the weight of the first word vector is equal to the weight of the second word vector; If they are not equal, the combined vector with the maximum weight is combined with the second word vector again, and the text vector is decoded according to the weight of the recombined vector to obtain the translated text.

2. The text translation method based on the attention mechanism according to claim 1, characterized in that: The preprocessing of the text comprises: Perform word segmentation on the text to obtain words of the text; Part-of-speech tagging is performed on the words in the text to obtain the part-of-speech tagged words.

3. The text translation method based on the attention mechanism according to claim 2, characterized in that: The encoding process of the preprocessed text according to the preset encoder to obtain the text vector of the text includes: Perform word embedding processing on the words after the part-of-speech tag to obtain word vectors containing the part-of-speech tag; Inputting the word vector containing the part-of-speech tag into the self-attention layer of the encoder for operation to obtain a word vector containing a weight; The word vector containing the weight is input into the feedforward fully connected neural network layer of the encoder for operation to obtain the text vector.

4. The text translation method based on the attention mechanism according to claim 1, characterized in that: The word vectors in the text vector are combined according to the pre-attention of the preset decoder to obtain a combined vector, including: Based on the pre-attention of the decoder, the word vectors in the text vector are combined according to the order of character arrangement in the text to obtain the combined vector.

5. The text translation method based on the attention mechanism according to claim 1, characterized in that: Before decoding the text vector according to the weight of the combined vector to obtain the translation text of the text, the method further includes: The preset Argmax function is called to perform weight calculation on the combination vector to obtain the weight of the combination vector.

6. A text translation device based on an attention mechanism, characterized in that: include: A receiving unit, configured to receive a translation request for a text; A preprocessing unit, configured to obtain the text according to the translation request and preprocess the text; An encoding unit, used for encoding the preprocessed text according to a preset encoder to obtain a text vector of the text; A combining unit, used for combining words in the word vectors of the text vector according to the pre-attention of the preset decoder to obtain a combined vector; A first decoding unit, configured to decode the text vector according to the weight of the combined vector to obtain a translation text of the text; The text translation device based on the attention mechanism also includes: a first judgment unit and a second decoding unit; A first judging unit, used to judge whether the word vector with the maximum weight in the text vector is located in the first column of the text vector; A second decoding unit, configured to decode the text vector according to the weight of the word vector in the text vector to obtain the translated text if the word vector is located in the first column of the text vector; The first decoding unit includes: an acquisition unit, a second judgment unit and a third decoding unit; The acquisition unit is used to acquire the combination vector with the maximum weight and the weights of the first word vector and the second word vector adjacent to the combination vector with the maximum weight; wherein the first word vector and the second word vector are both word vectors without word combination; The second judging unit is used to judge whether the weight of the first word vector is equal to the weight of the second word vector; The third decoding unit is used to combine the combined vector with the maximum weight with the second word vector again if they are not equal, and decode the text vector according to the weight of the recombined vector to obtain the translated text.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the attention mechanism-based text translation method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the text translation method based on the attention mechanism according to any one of claims 1 to 5.

Images