Sentence translation method, device, computer equipment and storage medium

Abstract

The present application relates to a sentence translation method, apparatus, computer equipment and storage medium. The method comprises: obtaining a source language text, the source language text comprising a sentence to be translated and associated sentences of the sentence to be translated; determining feature information of each word in the source language text, the feature information of each word comprising the meaning of each word, the position of each word in the source language text and the sentence to which each word belongs; generating a target language text of the sentence to be translated based on the feature information of each word in the source language text. The present application can distinguish between auxiliary information and main information, thereby reducing the noise generated by context sentences and improving the translation accuracy of the sentence to be translated.

Description

Technical Field

[0001] This application relates to the technical field of machine translation, and particularly to a sentence translation method, apparatus, computer device, and storage medium. Background Art

[0002] Currently, machine translation is performed by a translation model obtained through sample training, which translates a source language into a target language. Some sentences are relatively short and ambiguous. For example, for the Chinese sentence "I want to be quiet", it is difficult to know whether its meaning is "leave me alone" or "I miss Jingjing" just based on this sentence. Therefore, it is necessary to make the translation result more accurate by combining the context sentences. Generally, a sentence and its context sentences are input into the machine translation model together. Although this can play a certain role in eliminating ambiguity, the accuracy of the translation result still needs to be improved. Summary of the Invention

[0003] To solve the above technical problems or at least partially solve the above technical problems, this application provides a sentence translation method, apparatus, computer device, and storage medium.

[0004] In a first aspect, this application provides a sentence translation method, including:

[0005] Obtain a source language text, where the source language text includes a sentence to be translated and associated sentences of the sentence to be translated;

[0006] Determine the feature information of each word in the source language text, where the feature information of each word includes the meaning of each word, the position of each word in the source language text, and the sentence to which each word belongs;

[0007] Generate a target language text of the sentence to be translated according to the feature information of each word in the source language text.

[0008] In a second aspect, this application provides a sentence translation apparatus, including:

[0009] A first acquisition module, configured to obtain a source language text, where the source language text includes a sentence to be translated and associated sentences of the sentence to be translated;

[0010] A feature determination module, configured to determine the feature information of each word in the source language text, where the feature information of each word includes the meaning of each word, the position of each word in the source language text, and the sentence to which each word belongs;

[0011] A target generation module, configured to generate a target language text of the sentence to be translated according to the feature information of each word in the source language text.

[0012] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0014] In the present application, the source language text is obtained, and the characteristic information of each word in the source language text is determined. The characteristic information of each word includes the meaning of each word, the position of each word in the source language text, and the sentence to which each word belongs. According to the sentence to which each word belongs, it can be known whether the sentence to which the word belongs is the main information or auxiliary information. Moreover, according to the position of each word in the source language text, the distance between the word and the words in the sentence to be translated can be determined. The closer the word is, the higher its reference value. According to the sentence to which each word belongs and the position of each word, and combined with the meaning of each word, the auxiliary information can be effectively used to understand the main information, obtain the target language text with appropriate translation, reduce the noise generated by the context sentence, and improve the translation accuracy of the sentence to be translated. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1 A flowchart of a sentence translation method provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of the structure of a sentence translation device provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0021] In the first aspect, the present application provides a sentence translation method, which can be applied to a variety of scenarios where text translation is required, such as long text scenarios of lines, short text scenarios of lines, and dialogue scenarios. Figure 1 As shown, the method comprises the following steps:

[0022] S110, obtaining a source language text, wherein the source language text includes a sentence to be translated and associated sentences of the sentence to be translated;

[0023] It is understandable that the specific meanings of the above-mentioned related sentences may be different in different application scenarios, but no matter in which scenario, the related sentences usually refer to sentences that have a certain interpretation effect on the sentences to be translated. For example, in the scenario of translating lines, the related sentences of the sentences to be translated can be the previous sentences of the sentences to be translated, the following sentences of the sentences to be translated, or the previous sentences and the following sentences of the sentences to be translated. Of course, in addition to the context sentences, the related sentences can also be the relevant annotation texts, annotation texts, and annotation texts of the sentences to be translated.

[0024] In specific implementation, the specific process of obtaining the source language text in S110 includes:

[0025] S111, obtaining the sentence to be translated and the associated sentence;

[0026] In a specific implementation, if the associated sentence is a context sentence of the sentence to be translated (referring to the preceding sentence and / or the following sentence), the process of obtaining the associated sentence may include:

[0027] S111a, setting a position index for each sentence in the source text corresponding to the sentence to be translated, wherein the position index is used to represent the position of each sentence in the source text;

[0028] For example, if the sentence to be translated is a line from a TV series, then the source text is a line library formed by all the lines from the TV series. Of course, the sentence to be translated can be not only the lines from a TV series, but also the lines from a movie, the sentences in an article, or the sentences in a book, so there are many types of corresponding source texts.

[0029] Assuming that a machine translation model is mainly used to translate lines in TV dramas, then this machine translation model can translate lines in many TV dramas. In order to distinguish different TV dramas, a source identifier is set for each TV drama, and different TV dramas correspond to different source identifiers; then a position index is set for all the lines in the TV drama corresponding to a source identifier to represent the position of the line in all the lines in the TV drama.

[0030] S111b, acquiring the context sentence in the source text according to the position index of the sentence to be translated;

[0031] Since each sentence has a position index, the corresponding context sentence can be found in the source text according to the position index.

[0032] S112: Concatenate the sentence to be translated and the associated sentence to obtain a concatenated text, and use the concatenated text as the source language text.

[0033] Among them, the sentences to be translated and the associated sentences can be spliced ​​in a preset order. The preset order can be set as needed, for example, the previous sentence, the following sentence and the sentence to be translated, that is, the previous sentence is at the front, the following sentence is in the middle, and the sentence to be translated is at the end. Of course, it can also be other orders, for example, the previous sentence, the sentence to be translated and the following sentence. Among them, the number of context sentences can be selected as needed, and the number of context sentences required for a machine translation model is generally certain. For example, the number of contexts n corresponding to a machine translation model is 1, that is, the number of previous sentences is 1, and the number of following sentences is 1. If n is 2, the number of previous sentences is 2, and the number of following sentences is 2.

[0034] In a specific implementation, when splicing, in order to distinguish each sentence, a separator can be set after each sentence. For example, if n is 1, the sentence to be translated is b, the previous sentence is a, and the next sentence is c, then the sentence spliced ​​from the three sentences is a[sep]c[sep]b[sep], and the sep in [sep] can specify a specific separator. For another example, if n is 2, the sentence to be translated is c, the previous sentences are a and b, and the next sentences are d and e, then the sentence spliced ​​from these five sentences is: a[sep]b[sep]d[sep]e[sep]c[sep].

[0035] That is to say, the method provided by the present application may further include: when performing splicing, adding a separator after each sentence. The significance of setting the separator is that each sentence can be distinguished more clearly. Each separator is used as a word. Since the function of each separator is to separate sentences, the meaning of each separator is the same, that is, each separator corresponds to the same first vector, and the first vector will be explained later.

[0036] S120, determining feature information of each word in the source language text, wherein the feature information of each word includes the meaning of each word, the position of each word in the source language text, and the sentence to which each word belongs;

[0037] It is understandable that a word is the smallest unit in a sentence and cannot be further divided. A word can include one character or multiple indivisible characters.

[0038] It is understandable that, taking context sentences as associated sentences as an example, the sentence to which each word belongs refers to whether the sentence in which a word is located is a sentence to be translated, a previous sentence, or a following sentence. If the number of previous sentences or following sentences is greater than 1, it also specifically refers to which previous sentence or which following sentence, because the closer the context sentence is to the sentence to be translated, the higher the reference value.

[0039] It is understandable that the position of each word in the source language text refers to the position of a word in the entire source language text, for example, the position in the above concatenated text. If the concatenated text has 50 words in total, and a word is the 20th word in the concatenated text, then the position of the word in the source language text is 20, and 20 can be specifically represented by a vector, namely the second vector mentioned later.

[0040] In a specific implementation, based on the above S111 and S112, the above S120 may specifically include:

[0041] S121, performing word segmentation processing on the concatenated text to obtain a plurality of words;

[0042] S122. Perform vectorization processing on each word to obtain a feature vector of each word, where the feature vector is used to represent feature information of each word.

[0043] In other words, the feature information of each word is represented by a feature vector.

[0044] In a specific implementation, the feature vector of each word may include a first vector, a second vector and a third vector. The first vector is used to represent the meaning of each word, the second vector is used to represent the position of each word in the concatenated text, and the third vector is used to represent the sentence to which each word belongs. All words in the same sentence correspond to the same third vector, and words in different sentences correspond to different third vectors.

[0045] It is understandable that vectors, i.e., embeddings, are used to represent the relevant information of a word. Each word corresponds to three vectors, i.e., three embeddings. The first vector is used to represent the meaning of the word. For example, if a concatenated text has 50 words (including separators), the concatenated text will correspond to 50 first vectors, and one word corresponds to one first vector. The second vector is used to represent the position of the word in the concatenated text. For example, if a concatenated text has 50 words (including separators), the concatenated text will correspond to 50 second vectors, and each word corresponds to a second vector to represent the position of the word among the 50 words. For example, if a concatenated text has 50 words (including separators), the concatenated text corresponds to 50 third vectors, and one word corresponds to one third vector. Since the third vector represents the sentence in which a word is located, the third vectors corresponding to all words in the sentence are the same.

[0046] It is understandable that since all words in the same sentence correspond to the same third vector, and words in different sentences correspond to different third vectors, the third vector can be used to distinguish whether the sentence is a previous sentence, a following sentence, or a sentence to be translated.

[0047] It is understandable that although the separator has no actual meaning, it can be used as a separate word, and as the last word in a sentence immediately before the separator, the separator, as part of the sentence, also participates in the steps of word segmentation and setting three vectors. For example, the role of the separator is to segment each sentence, and it has no actual meaning, so all separators correspond to the same first vector, and the separator also occupies a position in the concatenated text, so the second vector of the separator also indicates the position of the separator in the concatenated text, and the third vector of the separator is also used to indicate which sentence it is in.

[0048] S130, generating a target language text of the sentence to be translated according to feature information of each word in the source language text;

[0049] In specific implementation, the first vector, the second vector and the third vector may be fused to obtain the fused vector corresponding to each word. For example, there are 50 words in a spliced ​​text, and each word corresponds to a first vector, a second vector and a third vector. Then the spliced ​​text corresponds to 50 first vectors, 50 second vectors and 50 third vectors. Assuming that each vector is a 3-dimensional vector, for each word, the three vectors of the word are spliced ​​to obtain a 9-dimensional fused vector. In this way, the spliced ​​text finally corresponds to 50 fused vectors.

[0050] In a specific implementation, the process of generating the target language text of the sentence to be translated based on the feature information of each word can be to input the feature information of each word into a pre-trained machine learning network, and use the machine learning network to generate the target language text of the sentence to be translated. Specifically, the three vectors of each word in the source language text can be input into the machine learning network, or the fusion vectors of each word in the source language text can be input into the machine learning network.

[0051] It is understandable that this machine learning network can know which sentence each word comes from, specifically whether it comes from the sentence to be translated, from a related sentence, and from which sentence in the related sentence, etc., based on the sentence to which each word belongs (for example, the third vector above or the element corresponding to the third vector in the fusion vector), and thus know whether the sentence in which the word is located is the main information or auxiliary information. Moreover, based on the position of each word in the source language text, the distance between the word and the word in the sentence to be translated can be determined. The closer the word is, the higher its reference value. Based on the sentence in which each word is located and the position of each word, and combined with the meaning of each word, auxiliary information can be effectively used to understand the main information and obtain a properly translated target language text.

[0052] It is understandable that the process of machine learning network translation is actually a process of encoding and decoding the input information. Specifically, the semantic information of the sentence to be translated is first obtained by encoding, and then by decoding, the semantic information can be translated into the target language text.

[0053] The machine learning network is obtained by training the training samples. After a certain number of iterations, the loss function reaches convergence, indicating that the training of the machine learning network is completed. The training samples include sample pairs formed by the source language text and the target language text, wherein the source language text includes the sentences to be translated and the related sentences, and the target language text is the target language text of the sentences to be translated.

[0054] For example, if the source language text includes a sentence to be translated and a preceding sentence of the sentence to be translated, by executing the above method provided by the present application, the target language text of the sentence to be translated can be obtained. For another example, if the source language text includes a sentence to be translated and a following sentence of the sentence to be translated, by executing the above method provided by the present application, the target language text of the sentence to be translated can be obtained. For another example, if the source language text includes a sentence to be translated and a preceding sentence and a following sentence of the sentence to be translated, by executing the above method provided by the present application, the target language text of the sentence to be translated can be obtained.

[0055] In a specific implementation, while generating the target language text of the sentence to be translated, the target language text of the associated sentence can also be generated at the same time. That is to say, the method provided by the present application can also include: generating the target language text of the associated sentence based on the feature information of each word in the source language text, so that the user can also understand the target language text of the sentence to be translated in combination with the target language text of the associated sentence.

[0056] For example, if the source language text includes a sentence to be translated and a previous sentence of the sentence to be translated, by executing the above method provided by the present application, the target language text of the sentence to be translated and the target language text of the previous sentence can be obtained; for another example, if the source language text includes a sentence to be translated and a following sentence of the sentence to be translated, by executing the above method provided by the present application, the target language text of the sentence to be translated and the target language text of the following sentence can be obtained; for another example, if the source language text includes a sentence to be translated and a previous sentence and a following sentence of the sentence to be translated, by executing the above method provided by the present application, the target language text of the sentence to be translated, the previous sentence and the following sentence can be obtained.

[0057] For example, the source language is Chinese and the target language is Indonesian. 10,000 texts are translated using the translation method provided by the prior art, and the 10,000 texts are also translated using the translation method provided by this application, and the two translation methods are evaluated using the parameter BLEU. Among them, BLEU is called Bilingual Evaluation Understudy, which means bilingual evaluation substitute, that is, evaluating the translation results on behalf of people. The higher the value, the higher the translation quality of the model. Through calculation, the BLEU of the prior art is 22.59, and the BLEU of this application is 23.35. It can be seen that the parameter BLEU of this application has been improved, so the translation quality of this application is higher than that of the prior art.

[0058] The sentence translation method provided by the present application obtains the source language text and determines the characteristic information of each word in the source language text. The characteristic information of each word includes the meaning of each word, the position of each word in the source language text and the sentence to which each word belongs. According to the sentence to which each word belongs, it can be known whether the sentence to which the word belongs is the main information or auxiliary information. Moreover, according to the position of each word in the source language text, the distance between the word and the words of the sentence to be translated can be determined. The closer the distance is, the higher the reference value is. According to the sentence to which each word belongs and the position of each word, and combined with the meaning of each word, the auxiliary information can be effectively used to understand the main information, obtain the target language text with appropriate translation, reduce the noise generated by the context sentence, eliminate ambiguity, and is particularly suitable for sentences with short sentences and ambiguous sentences, thereby improving the translation accuracy of the sentence to be translated.

[0059] In a second aspect, the present application also provides a sentence translation device, such as Figure 2 As shown, the device 100 includes:

[0060] A first acquisition module 110 is used to acquire a source language text, wherein the source language text includes a sentence to be translated and associated sentences of the sentence to be translated;

[0061] A feature determination module 120 is used to determine feature information of each word in the source language text, wherein the feature information of each word includes the meaning of each word, the position of each word in the source language text, and the sentence to which each word belongs;

[0062] The target generation module 130 is used to generate the target language text of the sentence to be translated according to the feature information of each word in the source language text.

[0063] It is understandable that the explanations, examples, beneficial effects, etc. of the relevant contents of the sentence translation device provided in the embodiment of the present application can refer to the corresponding parts in the first aspect and will not be repeated here.

[0064] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the sentence translation method provided in the first aspect when executing the computer program.

[0065] Figure 3 FIG. 2 shows an internal structure diagram of a computer device in one embodiment. Figure 3As shown, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and can also store a computer program. When the computer program is executed by the processor, the processor can implement the sentence translation method. The internal memory can also store a computer program. When the computer program is executed by the processor, the processor can execute the sentence translation method. The display screen of the computer device can be a liquid crystal display or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse.

[0066] Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0067] In one embodiment, the sentence translation device provided by the present application can be implemented in the form of a computer program. The computer program can be used in Figure 3 The computer device shown in the figure is run. The memory of the computer device can store various program modules constituting the translation device, such as: Figure 2 The first acquisition module 110, feature determination module 120, target generation module 130, and encoding and decoding module 140 are shown. The computer program composed of various program modules enables the processor to execute the steps of the sentence translation method of each embodiment of the present application described in this specification.

[0068] For example, Figure 3 The computer device shown can be Figure 2 The first acquisition module 110 in the sentence translation device shown is executed to acquire a source language text, wherein the source language text includes a sentence to be translated and associated sentences of the sentence to be translated; the computer device can determine the feature information of each word in the source language text through a feature determination module 120, wherein the feature information of each word includes the meaning of each word, the position of each word in the source language text, and the sentence to which each word belongs; the computer device can generate a target language text of the sentence to be translated according to the feature information of each word in the source language text through a target generation module 130.

[0069] It is understandable that the explanations, examples, beneficial effects, etc. of the computer device provided in the embodiments of the present application can refer to the corresponding parts in the first aspect and will not be repeated here.

[0070] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the sentence translation method provided in the first aspect are implemented.

[0071] It is understandable that the computer-readable storage medium provided in the embodiments of the present application, the explanations, examples, beneficial effects and other parts of the relevant contents can refer to the corresponding parts in the first aspect and will not be repeated here.

[0072] It is understood that any reference to memory, storage, database or other media used in the embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0073] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0074] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A sentence translation method, characterized in that: include: Acquire a source language text, wherein the source language text includes a sentence to be translated and associated sentences of the sentence to be translated; The obtaining of the source language text includes: obtaining the sentence to be translated and the related sentence; splicing the sentence to be translated and the related sentence to obtain a spliced ​​text, and using the spliced ​​text as the source language text; Determine the feature information of each word in the source language text, wherein the feature information of each word includes the meaning of each word, the position of each word in the source language text, and the sentence to which each word belongs; the determining the feature information of each word in the source language text includes: performing word segmentation processing on the concatenated text to obtain multiple words; performing vectorization processing on each word to obtain a feature vector of each word, wherein the feature vector is used to characterize the feature information of each word; the feature vector of each word includes a first vector, a second vector, and a third vector, wherein the first vector is used to characterize the meaning of each word, the second vector is used to characterize the position of each word in the concatenated text, and the third vector is used to characterize the sentence to which each word belongs, all words in the same sentence correspond to the same third vector, and words in different sentences correspond to different third vectors; The target language text of the sentence to be translated is generated according to the feature information of each word in the source language text.

2. The method according to claim 1, characterized in that The associated sentence is the context sentence of the sentence to be translated; the associated sentence is obtained in the following manner: Setting a position index for each sentence in the source text corresponding to the sentence to be translated, wherein the position index is used to represent the position of each sentence in the source text; The context sentence is obtained from the source text according to the position index of the sentence to be translated.

3. The method according to claim 1, characterized in that Also includes: The target language text of the associated sentence is generated according to the feature information of each word in the source language text.

4. A sentence translation device, characterized in that: include: A first acquisition module is used to acquire a source language text, wherein the source language text includes a sentence to be translated and associated sentences of the sentence to be translated; The obtaining of the source language text includes: obtaining the sentence to be translated and the related sentence; splicing the sentence to be translated and the related sentence to obtain a spliced ​​text, and using the spliced ​​text as the source language text; A feature determination module is used to determine the feature information of each word in the source language text, wherein the feature information of each word includes the meaning of each word, the position of each word in the source language text, and the sentence to which each word belongs; the determining the feature information of each word in the source language text includes: performing word segmentation processing on the concatenated text to obtain multiple words; performing vectorization processing on each word to obtain a feature vector of each word, wherein the feature vector is used to characterize the feature information of each word; the feature vector of each word includes a first vector, a second vector, and a third vector, wherein the first vector is used to characterize the meaning of each word, the second vector is used to characterize the position of each word in the concatenated text, and the third vector is used to characterize the sentence to which each word belongs, all words in the same sentence correspond to the same third vector, and words in different sentences correspond to different third vectors; The target generation module is used to generate the target language text of the sentence to be translated according to the feature information of each word in the source language text.

5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images