Text acquisition method and device, computer device and storage medium

Abstract

The application discloses a text acquisition method and device, computer equipment and a storage medium, and belongs to the technical field of computers. The application obtains a plurality of candidate texts in a second language by translating original texts in a first language, and screens a target candidate text according to a quality coefficient and an importance coefficient of each candidate text. Since the translation quality of the candidate text is considered by the quality coefficient and the importance of the information carried by the candidate text is considered by the importance coefficient, the target candidate text that takes into account the translation quality and the importance of the carried information can be screened under the action of the two coefficients, which is beneficial to generating the target candidate text with high translation quality and high importance coefficient. The target candidate text screened in this way can play a greater role when put into the training stage of the positive NMT model, thereby being beneficial to maximizing the improvement of the translation performance of the positive NMT model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a text acquisition method and device, computer equipment and storage medium. BACKGROUND

[0002] With the development of machine translation technology, back translation (BT) as a means to improve the performance of neural machine translation (NMT) model has been widely used.

[0003] Back translation includes two steps: 1) generating pseudo corpus text using a back translation model; 2) training a forward NMT model using the pseudo corpus text. For example, first train a back translation model from German to English, then select a large amount of German monolingual corpus, use the back translation model to translate the German monolingual corpus into English corpus, and then use the original German monolingual corpus and the translated English corpus as pseudo bilingual corpus, and mix it with real English-German bilingual corpus to train a forward English-German NMT model on the training corpus set composed of real bilingual corpus and pseudo bilingual corpus.

[0004] At present, although it has been proved that the pseudo corpus text obtained by back translation can improve the performance of the forward NMT model, it is still unclear how the pseudo corpus text affects the performance of the NMT model. Therefore, how to obtain the pseudo corpus text that can maximize the improvement of the performance of the NMT model has gradually become a hot spot that needs to be studied. SUMMARY

[0005] The embodiments of the present application provide a text acquisition method and device, computer equipment and storage medium, which can obtain pseudo corpus text that can maximize the improvement of the translation performance of the forward NMT model. The technical solution is as follows:

[0006] On the one hand, a text acquisition method is provided, which comprises:

[0007] Based on any original text in a first language, a plurality of candidate texts in a second language are translated, and the semantics of the original text and the candidate texts are associated;

[0008] Obtain the quality coefficient and the importance coefficient of each of the plurality of candidate texts, wherein the quality coefficient represents the translation quality of the candidate text, and the importance coefficient represents the importance of the information carried by the candidate text;

[0009] Based on the quality coefficient and the importance coefficient, a target candidate text is selected from the plurality of candidate texts.

[0010] On the one hand, a text acquisition device is provided, which comprises:

[0011] a translation module, configured to translate a plurality of candidate texts in a second language from any original text in a first language, the original text being associated with semantics of the candidate texts;

[0012] an acquisition module, configured to acquire a quality coefficient and an importance coefficient of each of the candidate texts, the quality coefficient representing a translation quality of the candidate text, and the importance coefficient representing an importance degree of information carried by the candidate text;

[0013] a screening module, configured to screen a target candidate text from the plurality of candidate texts based on the quality coefficient and the importance coefficient.

[0014] In some embodiments, the screening module comprises:

[0015] a determination unit, configured to determine a selection weight coefficient of each of the candidate texts based on the quality coefficient and the importance coefficient, the selection weight coefficient being a screening index balancing the quality coefficient and the importance coefficient;

[0016] a screening unit, configured to screen the target candidate text from the plurality of candidate texts based on the selection weight coefficient.

[0017] In some embodiments, the determination unit comprises:

[0018] a first determination sub-unit, configured to determine, for any candidate text, a first value obtained by multiplying a first adjustment factor and the importance coefficient of the candidate text;

[0019] The first determination sub-unit is further configured to determine a second value obtained by multiplying a second adjustment factor and the quality coefficient of the candidate text, the second adjustment factor and the first adjustment factor adding up to 1.

[0020] an adding unit, configured to add the first value and the second value to obtain a third value;

[0021] a second determination sub-unit, configured to determine the selection weight coefficient of the candidate text based on the third value of the candidate text and third values of other candidate texts, the other candidate texts being candidate texts obtained by translating the original text except for the candidate text.

[0022] In some embodiments, the second determination sub-unit is configured to:

[0023] obtain a first exponent of a natural base number by taking the third value of the candidate text as the first exponent;

[0024] for any one of the other candidate texts, obtaining the second exponent power of the natural base with the third value of the other candidate text as the second exponent;

[0025] adding the first exponent power of the natural base and the second exponent powers of the natural base to obtain a target sum value;

[0026] dividing the first exponent power of the natural base by the target sum value to obtain the selection weight coefficient of the candidate text.

[0027] In some embodiments, the screening unit is configured to:

[0028] from the plurality of candidate texts, determining the candidate text with the largest selection weight coefficient as the target candidate text.

[0029] In some embodiments, the screening unit is configured to:

[0030] from the plurality of candidate texts, randomly sampling the target candidate text according to a probability distribution to which the selection weight coefficient is subject.

[0031] In some embodiments, the obtaining module comprises:

[0032] a first obtaining unit, configured to, for any one of the candidate texts, obtain an initial quality coefficient of the candidate text;

[0033] a first normalization unit, configured to normalize the initial quality coefficient based on a text length of the candidate text to obtain the quality coefficient, the text length representing a number of characters contained in the candidate text.

[0034] In some embodiments, the first normalization unit is configured to:

[0035] obtaining a first mean value and a first variance of the logarithmic quality coefficients based on the logarithmic quality coefficients of the plurality of candidate texts, the logarithmic quality coefficient being a logarithmic value of the initial quality coefficient;

[0036] dividing the logarithmic quality coefficient of the candidate text by the text length of the candidate text to obtain a normalized quality coefficient;

[0037] dividing the value obtained by subtracting the normalized quality coefficient from the first mean value by the first variance to obtain the quality coefficient.

[0038] In some embodiments, the obtaining module comprises:

[0039] a second obtaining unit, configured to, for any one of the candidate texts, obtain an initial importance coefficient of the candidate text;

[0040] a second normalization unit, configured to normalize the initial importance coefficient based on a text length of the candidate text, to obtain the importance coefficient, the text length representing a number of characters contained in the candidate text.

[0041] In some embodiments, the second normalization unit is configured to:

[0042] obtain a second mean and a second variance of the log importance coefficients based on the initial importance coefficients of the plurality of candidate texts, the log importance coefficient being a logarithmic value of the initial importance coefficient;

[0043] divide the log importance coefficient of the candidate text by a text length of the candidate text to obtain a normalized importance coefficient;

[0044] divide a value obtained by subtracting the normalized importance coefficient from the second mean by the second variance to obtain the importance coefficient.

[0045] In some embodiments, the translation module is configured to:

[0046] input the original text into a back translation model, and translate the original text into the plurality of candidate texts in a random sampling manner through the back translation model, the back translation model being configured to translate input text from the first language to the second language.

[0047] In some embodiments, the original text and the target candidate text constitute a corpus pair for training a forward translation model, the forward translation model being configured to translate input text from the second language to the first language.

[0048] In an aspect, a computer device is provided, which includes one or more processors and one or more memories, the one or more memories storing at least one computer program, which is loaded and executed by the one or more processors to implement the above-described text obtaining method.

[0049] In an aspect, a storage medium is provided, which stores at least one computer program, which is loaded and executed by a processor to implement the above-described text obtaining method.

[0050] In an aspect, a computer program product is provided, which includes one or more computer programs stored in a computer readable storage medium. One or more processors of a computer device can read the one or more computer programs from the computer readable storage medium, and execute the one or more computer programs, so that the computer device can perform the above-mentioned text acquisition method.

[0051] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects:

[0052] The original text in the first language is translated to obtain a plurality of candidate texts in the second language, and the target candidate text is obtained by screening according to the quality coefficient and the importance coefficient of each candidate text. Since the translation quality of the candidate text is considered by the quality coefficient, and the importance of the information carried by the candidate text is considered by the importance coefficient, the target candidate text that takes into account both the translation quality and the importance of the information carried can be screened under the action of the two coefficients, which is beneficial to generate the target candidate text with high translation quality and high importance coefficient. These screened target candidate texts can play a greater role when put into the training stage of the positive NMT model, thereby being beneficial to maximize the improvement of the translation performance of the positive NMT model. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical schemes in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0054] Figure 1 is a schematic diagram of an implementation environment of a text acquisition method provided by an embodiment of the present application;

[0055] Figure 2 is a flowchart of a text acquisition method provided by an embodiment of the present application;

[0056] Figure 3 is a flowchart of a text acquisition method provided by an embodiment of the present application;

[0057] Figure 4 is a structural schematic diagram of a text acquisition device provided by an embodiment of the present application;

[0058] Figure 5 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0060] The terms "first", "second" and the like in the present application are used to distinguish the same items or similar items with substantially the same function, and it should be understood that there is no logical or time sequence relationship between "first", "second" and "nth", and the number and execution order are not limited.

[0061] The term "at least one" in the present application means one or more, and the meaning of "multiple" is two or more, for example, multiple candidate texts refer to two or more candidate texts.

[0062] The term "including at least one of A or B" in the present application relates to the following cases: including A only, including B only, and including both A and B.

[0063] The user-related information (including but not limited to user device information, personal information, behavior information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application, when the method of the embodiments of the present application is applied to specific products or technologies, are all with the permission, consent, authorization of the user or with the full authorization of all parties, and the collection, use and processing of the relevant information, data and signals need to comply with the relevant laws, regulations and standards of the country and region. For example, the original text involved in the present application is obtained under full authorization.

[0064] Artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0065] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes audio processing technology, computer vision technology, natural language processing technology, machine learning / deep learning, automatic driving and intelligent transportation and other fields.

[0066] With the research and progress of artificial intelligence technology, artificial intelligence technology is researched and applied in many fields, such as common smart home, smart wearable device, virtual assistant, smart speaker, smart marketing, unmanned driving, autonomous driving, unmanned aerial vehicle, robot, smart medical treatment, smart customer service, Internet of Vehicles, autonomous driving, intelligent transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0067] Letting computers can hear, can see, can speak, can feel, is the development direction of future human-computer interaction. Among them, natural language processing (NLP) is an important direction in the field of computer science and artificial intelligence, and is a branch discipline of artificial intelligence and linguistics, aiming to explore how to process and use natural language. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language used in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes machine translation, text preprocessing, semantic understanding, robot question and answer, knowledge graph and other technologies.

[0068] In the field of NLP, machine translation refers to the process of converting one natural language (source language) into another natural language (target language) using a computer, which is a branch of computational linguistics and one of the ultimate goals of artificial intelligence, and has important scientific research value. At the same time, machine translation also has important practical value. With the rapid development of economic globalization and the Internet, machine translation technology plays an increasingly important role in promoting economic and cultural exchanges. In machine translation, neural machine translation has achieved good success. The technical solutions provided by the embodiments of the present application involve neural machine translation and other technologies of artificial intelligence, which will be described in detail below.

[0069] In the following, the terms involved in the embodiments of the present application will be explained:

[0070] Deep learning (DL): is a branch of machine learning, which is an algorithm that tries to use multiple processing layers containing complex structures or composed of multiple nonlinear transformations to abstract data at a high level.

[0071] Neural network (NN): a deep learning model that simulates the structure and function of biological neural networks in the field of machine learning and cognitive science.

[0072] Machine Translation (MT): Also known as automatic translation, it is the use of electronic computers and other devices to automatically translate one natural language text into another natural language text.

[0073] Neural Machine Translation (NMT): A machine translation technology based on neural networks.

[0074] Back Translation (BT): A method of generating pseudo-bilingual corpus from monolingual corpus, which combines pseudo-bilingual corpus and real bilingual corpus to train NMT model, and can simply and effectively improve the performance of NMT model.

[0075] Sequence to Sequence Model (Seq2Seq): A deep neural network model with sequence as input and output.

[0076] Transformer: Also known as transducer, a machine translation model of neural network framework. Transformer model uses encoder-decoder framework based on self-attention neural network (SAN), and belongs to the model structure of sequence to sequence (Seq2Seq) generation.

[0077] Tokens: The smallest semantic unit of machine translation, i.e. word segmentation. In English, a token refers to a word (i.e. vocabulary) or an English punctuation mark, and in Chinese, a token refers to a Chinese word (a word may include one or more Chinese characters) or a Chinese punctuation mark.

[0078] Bilingual Evaluation Understudy (BLEU): An evaluation index of machine translation model, the higher the BLEU value, the better the translation effect of machine translation model.

[0079] Checkpoint: The parameter state information of NMT model at the midway of training. By setting Checkpoint during the training process of NMT model, the model parameters of NMT model at the midway of training can be saved, so as to evaluate when the NMT model reaches the optimal performance by longitudinal comparison. For example, during the process of iterating 1000 steps for a certain NMT model, a Checkpoint is set every 100 steps, i.e. the model parameters of NMT model at the midway of training are saved every 100 steps.

[0080] Since the birth of NMT technology, back-translation has become one of the important research focuses in the field of NMT research as a simple and effective means to improve the performance of NMT model. Moreover, back-translation is also a key optimization target of unsupervised NMT model, and has a very important position in other NLP research fields such as pre-training paraphrase, translation stylization, etc.

[0081] Back-translation includes two steps: 1) generating pseudo corpus text using a back-NMT model; 2) training a forward NMT model using the pseudo corpus text. Before back-translation begins, a back-NMT model needs to be trained using real bilingual corpus, for example, a back-NMT model (usually a Transformer model) is trained; then, a large amount of monolingual corpus is translated using the trained back-NMT model to obtain pseudo bilingual corpus; then, the pseudo bilingual corpus is mixed with the real bilingual corpus to obtain training bilingual corpus, and the forward NMT model is trained on the training bilingual corpus, so that the forward NMT model trained in this way has better translation performance than the forward NMT model trained only using real bilingual corpus.

[0082] In an exemplary scenario, taking the forward NMT model for English-German translation direction as an example, a German-English back-NMT model is first trained using English-German bilingual parallel corpus until the back-NMT model is fitted, and the Checkpoint technology is used to test the model parameters saved in the middle of training on the validation set to obtain the translation performance of the back-NMT model under each model parameter on the validation set, and the Checkpoint model parameter with the best translation performance on the validation set is selected as the back-NMT model for subsequent back-translation process. Then, a large amount of German monolingual corpus is selected, and the back-NMT model is used to translate the German monolingual corpus into English corpus, and the original German monolingual corpus and the translated English corpus are used as pseudo bilingual corpus. Then, the pseudo bilingual corpus is mixed with the real English-German bilingual corpus to obtain the training corpus set for training, and the English-German forward NMT model is trained on the training corpus set. Optionally, some data enhancement methods can be used to improve the performance of the model during the training process of the forward NMT model, such as up-sampling real corpus, gradient accumulation, etc. The data enhancement method used when training the forward NMT model is not limited in the embodiments of the present application.

[0083] Since the reverse translation was proposed, the reverse translation has been greatly developed, but in the process of using the reverse translation to improve the performance of the forward NMT model, the default reverse translation method is used to generate the pseudo corpus text, but it is not clear how the pseudo corpus text obtained by the reverse translation affects the translation performance of the NMT model. For example, some studies show that the quality of the pseudo corpus text is crucial to promote the translation performance of the NMT model, while another study finds that low-quality pseudo corpus text can help the NMT model achieve higher test performance. The above contradictory experimental results show that the quality of the pseudo corpus text is not the only factor that determines the translation performance of the NMT model, therefore, the fundamental problem is that what kind of pseudo corpus text can maximize the translation performance of the forward NMT model? This problem is still a research hotspot that the reverse translation needs to answer urgently.

[0084] Therefore, the embodiment of the present application provides a text acquisition method, which is committed to exploring the characteristics of the pseudo corpus text itself, and generating the pseudo corpus text most beneficial to the forward NMT model by adjusting the generation strategy. This acquisition method of the pseudo corpus text does not need iterative pseudo corpus text generation or model training, and also does not need to label the pseudo corpus text, so as to avoid the difference between the training and test data distribution.

[0085] Next, the system architecture of the embodiment of the present application is described.

[0086] Figure 1 FIG. 1 is a schematic diagram of an implementation environment of a text acquisition method provided by the embodiment of the present application. Referring to FIG. 1, Figure 1 In the implementation environment, a terminal 101 and a server 102 are included.

[0087] The terminal 101 is configured to provide a text to be translated. The text to be translated usually includes one or more sentences, for example, the text to be translated is a sentence. The terminal 101 is installed and runs an application supporting text translation, for example, the application includes a translation application, a social application, a word search application, a search engine, an intelligent assistant, a content sharing application, a live broadcast application, a conference application, a book reading application, etc. The application type is not limited in the embodiment of the present application.

[0088] In some embodiments, after the user logs in the application on the terminal 101, the user inputs the text to be translated in the application. The input method of the text to be translated includes, but is not limited to, manual input, voice instruction, gesture instruction, shortcut option, and preset text selection. The embodiments of the present application do not specifically limit the input method of the text to be translated. Then, the user triggers the terminal 101 to send a text translation request carrying the text to be translated to the server 102. The text translation request is used to request a translation text associated with the text to be translated in terms of semantics but different in language. The language of the translation text can be specified in the text translation request, or the language of the translation text can not be specified. The embodiments of the present application do not specifically limit this. The server 102 responds to the text translation request and translates the text to be translated using the pre-trained forward NMT model, that is, the translation text of the text to be translated is obtained, and the translation text is returned to the terminal 101.

[0089] The terminal 101 and the server 102 can be directly or indirectly connected through wired or wireless communication. The present application does not make any limitation here.

[0090] The server 102 is used to provide machine translation services. The server 102 can include at least one of a server, multiple servers, a cloud computing platform, or a virtualization center. Optionally, the server 102 undertakes the main translation calculation work, and the terminal 101 undertakes the secondary translation calculation work. Alternatively, the server 102 undertakes the secondary translation calculation work, and the terminal 101 undertakes the main translation calculation work. Alternatively, the terminal 101 and the server 102 use a distributed computing architecture to cooperatively calculate the translation.

[0091] In some embodiments, the server 102 receives the text translation request of the terminal 101, analyzes the text translation request to obtain the text to be translated (or the text to be translated and the specified translation text language), selects a default translation text language if the text translation request does not specify the translation text language, then finds a pre-trained forward NMT model for translation from the language of the text to be translated to the language of the translation text, converts the text to be translated from the original language to the language of the translation text using the found forward NMT model to obtain the translation text of the text to be translated, and then returns the translation text to the terminal 101. The translation text and the text to be translated have a semantic association relationship but are different in language.

[0092] In some embodiments, on the server 102 side, the pseudo-corpus text is obtained by using the text acquisition method provided by the embodiments of the present application, the pseudo-corpus text is mixed with the real corpus text to obtain a training corpus set, and the above-mentioned forward NMT model is trained on the training corpus set.

[0093] In the above embodiments, the server 102 maintains the forward NMT model on its own server, which can migrate the translation process to the server 102, and save the processing resources and computing resources of the terminal 101. In some other embodiments, after the server 102 trains the forward NMT model, the server 102 sends the forward NMT model to the terminal 101, so that the terminal 101 can locally call the forward NMT model to translate the text to be translated, and obtain the translation of the text to be translated. In this case, the terminal 101 does not need to send a translation request to the server 102 every time, which can reduce the communication overhead between the terminal 101 and the server 102. In some other embodiments, the terminal 101 can also train the forward NMT model locally, and call the forward NMT model locally to translate the text to be translated, and obtain the translation of the text to be translated. Similarly, the communication overhead between the terminal 101 and the server 102 can be reduced.

[0094] It should be noted that each text to be translated and the translation obtained by translation and other text data involved in the embodiments of the present application can be saved on the blockchain.

[0095] The server 102 described above can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms, etc. basic cloud computing services.

[0096] The terminal 101 described above can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, an e-book reader, etc., but is not limited thereto.

[0097] Those skilled in the art can know that the terminal 101 can generally refer to one of a plurality of terminals, and the number of the terminals described above can be more or less. For example, the terminals described above can be only one, or the terminals described above can be dozens or hundreds, or more. The number and type of the terminal are not limited in the embodiments of the present application.

[0098] In the following, the core idea of the embodiments of the present application will be described:

[0099] NMT uses a neural network-based translation model p(y|x; θ) to construct a translation mapping, x represents the original text to be translated, y represents the translated text, θ represents the model parameter set of the NMT model, and p(y|x; θ) represents the conditional probability of translating the original text x into the translated text y using the NMT model θ, that is, represents the translation mapping of the NMT model from the original text x to the translated text y. This translation mapping can be understood as: mapping the original text x of the source language to the translated text y of the target language. In the machine translation process, the source language side can be referred to as the source end (referring to the language before translation), and the target language side can be referred to as the target end (referring to the language after translation).

[0100] In the traditional NMT training mode, a bilingual corpus represents a pair of different languages but semantically related bilingual corpus (i.e. corpus pair), x i represents the i-th sample corpus of the source end, y i represents the i-th sample corpus of the target end which is different in language but semantically related to x i . In the bilingual corpus , the NMT model is trained on the training corpus set, and the training target is to minimize the objective function shown in the following formula

[0101]

[0102] wherein, represents the objective function (or loss function) in the training stage, the objective function is a function value related to the NMT model θ and the training corpus set , (x i ; y i ) represents the i-th corpus pair in the training corpus set , x i is the source text in the i-th corpus pair, y i is the target text in the i-th corpus pair, x i and y i are different in language but have the same or similar semantics, p(y i |x i ; θ) represents the conditional probability of mapping the source text x i to the target text y i by the NMT model θ, and logp(y i |x i ; θ) represents the logarithmic value of the above conditional probability.

[0103] In the NMT training approach based on back translation, two stages are involved: pseudo-corpus generation and parameter training. In the pseudo-corpus generation stage, a back NMT model p(x|y;π) is first trained. The back NMT model is used to translate the target text y to the source text x. π represents the set of model parameters of the back NMT model, and p(x|y;π) represents the conditional probability of translating the target text y to the source text x using the back NMT model. It also represents the translation mapping from the target text y to the source text x by the back NMT model. Then, using the back NMT model p(x|y;π), the target monolingual corpus y∈M is back-translated to obtain the source pseudo-corpus. For example, the j-th monolingual corpus y in the target corpus M j ∈M, pseudo-corpus is obtained by back-translation using the inverse NMT model p(x|y;π). Source pseudo-corpus and target corpus y j ∈M can form a pair of pseudo-bilingual corpora. Next, repeat the above operations to obtain multiple pairs of pseudo-bilingual corpora. Then, these pseudo-bilingual corpora are combined with real bilingual corpora to construct a training corpus. A forward NMT model is trained on the training corpus, which contains both pseudo-bilingual and real bilingual corpora. The training objective is to minimize the objective function shown in the following formula:

[0104]

[0105] in, This refers to the objective function of the traditional NMT training method. This represents the additional loss term considered in the objective function due to the reverse translation training method, indicating the impact of using the forward NMT model θ on the source pseudo-corpus. The conditional probability mapped to the target real corpus y, where M represents the corpus set composed of the target real monolingual corpus.

[0106] Furthermore, in unsupervised learning, maximizing marginal probability is a crucial principle. Using this principle to re-examine the aforementioned NMT training process based on back-translation, the marginal probability for each target monolingual corpus y can be derived from Bayes' theorem and Jensen's inequality. Combining this marginal probability with the objective function of the aforementioned NMT training process based on back-translation, we arrive at the following inequality relationship:

[0107]

[0108] where p(x) is the distribution of the source-side corpus x, p(y; θ) is the distribution of the target-side corpus y translated from the source-side corpus x by the forward NMT model θ, p(x|y) is the backward NMT model, p(y|x; θ) is the forward NMT model, and log represents the logarithm operation. represents the mathematical expectation of the expression in the curly braces with respect to the source-side corpus x obeying the distribution p(·|y).

[0109] In the above inequality, the represents the lower bound of the objective function of the forward NMT model under the principle of maximizing the marginal probability.

[0110] It can be seen that the logarithmic term logp(y|x; θ) is the same as the objective function in the conventional NMT training manner, and the difference is that an additional multiplication factor is applied to the logarithmic term logp(y|x; θ). The multiplication factor is the importance coefficient involved in each of the embodiments below, and the logarithmic term logp(y|x; θ) is the quality coefficient involved in each of the embodiments below.

[0111] Based on the above analysis, assume that the symbol is used to represent the importance coefficient, then the importance coefficient is defined as follows:

[0112]

[0113] Therefore, there are two coefficients that can be used as evaluation indicators of the pseudo corpus of the forward NMT model to control whether the pseudo corpus can provide a better effect on the training of the forward NMT model, one is the importance coefficient and the other is the quality coefficient logp(y|x; θ).

[0114] In theory, if the translation quality of the pseudo corpus is higher, it can provide more abundant training signals, so the quality coefficient will be higher; if the importance of the information carried by the pseudo corpus is higher, the pseudo corpus may provide sufficient important information even if the translation quality is not high, so the importance coefficient will be higher. By balancing the above quality coefficient and the importance coefficient This can help improve the translation performance of the trained forward NMT model during the process of training the forward NMT model based on reverse translation.

[0115] In some embodiments, in order to measure pseudo-corpus quality coefficient Due to pseudo-corpus The higher the translation quality, the higher the quality score. The higher the value, the better the translation performance of the inverse NMT model. The pseudo-corpus translated using the inverse NMT model... The higher the translation quality, the more likely pseudo-corpora can be used. Estimate pseudo-corpora using their own BLEU score or the BLEU score of the inverse NMT model. quality coefficient Among them, the BLEU value is a metric used to evaluate the translation performance of NMT models. A larger BLEU value indicates better translation performance, while a smaller BLEU value indicates worse translation performance.

[0116] In some embodiments, in order to measure pseudo-corpus Importance coefficient The importance coefficients mentioned above can be estimated using the likelihood value of the monolingual model p(x; ω). The numerator p(x) is given by ω, where ω represents the set of model parameters of the monolingual language model of the source monolingual corpus x, and the importance coefficients mentioned above are estimated using the likelihood value of the inverse NMT model p(x|y;π). The denominator p(x|y) is therefore, for the entire importance coefficient The estimation process can be approximated as follows:

[0117]

[0118] In an exemplary scenario, an NMT training method based on reverse translation is adopted to train a forward NMT model for English-to-German translation. It can be seen that during the training process, a reverse NMT model for German-to-English translation is used to generate English pseudo-corpus. The aforementioned reverse NMT model was tested on the WMT14 validation set, resulting in pseudo-corpus. Its own Test performance of the inverse NMT model (Test BLEU) and log-likelihood value of the inverse NMT model. Furthermore, use Estimating the pseudo-corpus generated by the inverse NMT model quality coefficient Using log-likelihood Estimating the importance coefficient of the inverse NMT model

[0119] By the above training and estimating manners, the reverse NMT model in the German-English direction is tested on the WMT14 validation set, and the indexes obtained by the test are shown in Table 1:

[0120] Table 1

[0121]

[0122]

[0123] Among them, different translation manners can be used when the reverse NMT model generates pseudo corpus The above Table 1 involves the following translation manners: A) random sampling, that is, a target corpus y is input to the reverse NMT model, and a source pseudo corpus is randomly sampled from the candidate translation given by the reverse NMT model B) beam search, that is, a target corpus y is input to the reverse NMT model, and a source pseudo corpus with higher context matching degree is found from the candidate translation given by the reverse NMT model using the beam search manner Generally, the source pseudo corpus generated by the beam search has higher translation performance; C) beam search with noise addition, that is, in the process of generating the source pseudo corpus by the beam search in the above B), random noise is added to the token of the source pseudo corpus

[0124] Among them, characterizes the translation quality of the source pseudo corpus , the logarithmic likelihood value characterizes the denominator of the importance coefficient Imp., and Imp. characterizes the importance coefficient of the pseudo corpus, and Test BLEU characterizes the test performance of the reverse NMT model.

[0125] As can be seen from Table 1, for the reverse NMT model, the of the beam search manner is much higher than the of the random sampling manner, but the Imp. of the beam search manner is much lower than the Imp. of the random sampling manner, which means that the pseudo corpus generated by the beam search manner always has good translation quality, but its importance coefficient Imp. is usually low; Correspondingly, the pseudo corpus generated by the random sampling manner always has poor translation quality, but it usually has a good importance coefficient Imp.

[0126] ​​​​​​Similarly, the reverse NMT models for the English-to-German and Russian-to-English directions are also tested on the WMT14 validation set in a similar way, and the obtained indicators are shown in Table 2.

[0127] Table 2

[0128]

[0129]

[0130] From Table 2, it can be seen that for the reverse NMT model, whether in the English-to-German direction or in the Russian-to-English direction, it has similar characteristics as summarized for the German-to-English direction, that is, the pseudo corpus generated by the beam search method always has good translation quality, but its importance coefficient Imp. is usually low; in contrast, the pseudo corpus generated by the random sampling method always has poor translation quality, but it usually has a good importance coefficient Imp..

[0131] In some embodiments, the beam search method can generate pseudo corpus with high translation quality, while the random sampling method can generate pseudo corpus with high importance coefficient. Assuming that by controlling the proportion of beam search and random sampling in the pseudo corpus to achieve the goal of generating pseudo corpus with both high translation quality and high importance weight. In view of this, a data integration method is involved, which is to control the first 0-k monolingual corpus y to generate respective pseudo corpus by the beam search method and control the k- |M| monolingual corpus y to generate respective pseudo corpus by the random sampling method where |M| represents the sample size of the target corpus set M. Illustratively, the data integration method is modeled by the following formula:

[0132]

[0133]

[0134] where M c represents the pseudo bilingual corpus pair obtained by the data integration method, represents the pseudo bilingual corpus pair obtained by the beam search method, y i represents the i-th monolingual corpus in the target corpus set M, represents the pseudo corpus generated by the reverse NMT model for y i using the beam search method, i is an integer greater than or equal to 0 and less than k, characterizing the pseudo bilingual corpus pair obtained by the random sampling method, y i characterizing the i-th monolingual corpus in the target corpus set M, characterizing the pseudo corpus y i obtained by the reverse NMT model, i is an integer greater than or equal to k and less than or equal to |M|, |M| represents the sample size of the target corpus set M (also equal to the total number of pseudo corpora generated), k is the product of the hyperparameter γ and |M| rounded down, characterizing the down rounding operation on γ|M|, that is, obtaining the maximum integer less than or equal to γ|M|.

[0135] It can be seen that when the data integration method is used, by controlling the hyperparameter γ in the above modeled formula, the proportion of the quality coefficient and the importance coefficient in each pseudo corpus generated can be easily controlled.

[0136] It should be noted that although the data integration method is easier to implement, since the pseudo corpus is generated by integrating the beam search and random sampling methods, the average quality coefficient of the pseudo corpus obtained by the data integration method must be lower than that of the beam search method, and similarly, the average importance coefficient must also be lower than that of the random sampling method.

[0137] In summary, by starting from the marginal probability of semi-supervised learning, the lower bound of the objective function is derived, and based on the derived lower bound of the objective function, the two coefficients that control the effect of the pseudo corpus pair model training are found from a theoretical point of view: the quality coefficient and the importance coefficient. The quality coefficient and the importance coefficient are crucial for maximizing the lower bound of the objective function. Since the quality coefficient and the importance coefficient are mutually exclusive, if only one of them is considered to be improved, the other will inevitably decrease. In view of this, the present application embodiment provides an interpolation method to balance the quality coefficient and the importance coefficient, so that the pseudo corpus generated by the reverse NMT model can have as high a quality coefficient and as high an importance coefficient as possible. Then, after training the forward NMT model using the pseudo corpus generated in this way, the translation performance of the forward NMT model can be maximized. The process of obtaining the pseudo corpus by the interpolation method will be described in detail below.

[0138] Figure 2 is a flowchart of a text obtaining method provided by an embodiment of the present application. Referring to Figure 2 , the embodiment is executed by a computer device, which can be the terminal 101 or the server 102 in the above implementation environment. Taking the computer device as the server 102 for example, the embodiment includes the following steps:

[0139] 201、the server translates a plurality of candidate texts in the second language based on any original text in the first language, the original text and the candidate texts being semantically associated.

[0140] wherein the original text refers to a text to be translated, the candidate text refers to any candidate translation of the original text, the original text and the candidate text have the same or similar semantics but belong to different languages, and here the original text is taken as the first language and the candidate text is taken as the second language for illustration, the first language and the second language are different languages.

[0141] Illustratively, in the NMT training scenario based on back translation, since a pre-trained back translation NMT model is needed to generate pseudo corpus before training a forward NMT model, the text acquisition method involved in the embodiments of the present application can be regarded as a process of acquiring pseudo corpus by using a back translation NMT model, in which process, the first language refers to the target language (Target, target end), and the second language refers to the source language (Source, source end), the forward NMT model is used to translate input text from the second language to the first language (from the source end to the target end), and the back translation NMT model is used to translate input text from the first language to the second language (from the target end to the source end).

[0142] In some embodiments, the server acquires a monolingual corpus set in the first language from a corpus sample library, acquires any original text in the first language from the monolingual corpus set, for example, randomly selects an English original text from an English monolingual corpus set. Then, the server inputs the original text into a back translation model, translates the original text from the first language to the second language through the back translation model, and generates a plurality of selectable candidate texts, which are all translations having a semantic association with the original text. It should be noted that the semantic association means that the semantics of the original text and the candidate text are the same or similar, for example, the semantic similarity of the original text and the candidate text is greater than a similarity threshold, or the semantics of the original text and the candidate text meet certain similarity conditions, which are not limited by the embodiments of the present application.

[0143] In some embodiments, the above-mentioned original text is acquired from a locally stored corpus sample library, or the above-mentioned original text is acquired from a cloud-stored corpus sample library, and the source of the original text is not limited by the embodiments of the present application.

[0144] In some embodiments, after obtaining the original text in the first language, the server inputs the original text into a back-translation model, i.e., a back-NMT model, and performs encoding-decoding processing on the original text through an encoder-decoder framework in the back-NMT model to predict a plurality of candidate texts associated with the semantics of the original text but in a different language. The back-NMT model is used to translate the input text from the first language to the second language. The translation process of the back-NMT model will be described in detail in the next embodiment, which will not be repeated here.

[0145] 202. The server obtains a quality coefficient and an importance coefficient of each of the plurality of candidate texts, wherein the quality coefficient represents the translation quality of the candidate text, and the importance coefficient represents the importance of the information carried by the candidate text.

[0146] The quality coefficient involved in the embodiments of the present application is a quantitative indicator for measuring the translation quality of the candidate text. Illustratively, the quality coefficient is positively correlated with the translation quality, for example, the higher the value of the quality coefficient, the higher the translation quality of the candidate text translated from the original text, which also reflects that the matching degree or semantic similarity between the candidate text and the original text is higher, or the context connection of each character in the candidate text is tighter, or the number of grammatical errors in the candidate text is smaller, etc. Conversely, the lower the value of the quality coefficient, the lower the translation quality of the candidate text translated from the original text, which also reflects that the matching degree or semantic similarity between the candidate text and the original text is lower, or the context connection of each character in the candidate text is looser, or the number of grammatical errors in the candidate text is larger, etc.

[0147] The importance coefficient involved in the embodiments of the present application is a quantitative indicator for measuring the importance of the information carried by the candidate text. This importance can be used to indicate how much contribution the candidate text can make to the optimization and improvement of the performance of the model when it is used as pseudo corpus to train the forward NMT model. Illustratively, the importance coefficient is positively correlated with the importance of the information carried by the candidate text, for example, the higher the value of the importance coefficient, the higher the importance of the information carried by the candidate text, which reflects that the candidate text will play a more important role when it is used as pseudo corpus to train the forward NMT model. Conversely, the lower the value of the importance coefficient, the lower the importance of the information carried by the candidate text, which reflects that the candidate text will play a smaller role when it is used as pseudo corpus to train the forward NMT model.

[0148] In some embodiments, for each of the plurality of candidate texts, the server obtains the quality coefficient and the importance coefficient of the candidate text. The acquisition methods of the quality coefficient and the importance coefficient will be described below.

[0149] Optionally, in the process of obtaining the quality coefficient, since the higher the translation quality of the candidate text is, the higher the quality coefficient of the candidate text is, and the BLEU is a performance indicator for measuring the translation quality, the BLEU value of the candidate text can be used to represent the quality coefficient of the candidate text; or, since the better the translation performance of the reverse NMT model is, the higher the translation quality of the candidate text translated by the reverse NMT model is, the higher the quality coefficient of the candidate text is, the BLEU value of the reverse NMT model can also be used to represent the quality coefficient of the candidate text; or, since the likelihood value of the reverse NMT model represents the translation quality of the reverse NMT model, the larger the likelihood value of the reverse NMT model is, the better the translation quality of the candidate text translated by the reverse NMT model is, and the higher the quality coefficient of the candidate text is, the likelihood value of the reverse NMT model can also be used to represent the quality coefficient of the candidate text, and the representation manner of the quality coefficient is not limited in the embodiments of the application.

[0150] For example, for any candidate text , the BLEU value of the candidate text itself is used as the quality coefficient of the candidate text , or the test performance Test BLEU of the reverse NMT model is used as the quality coefficient of the candidate text , or the likelihood value or the logarithmic likelihood value of the reverse NMT model is used as the quality coefficient of the candidate text .

[0151] Optionally, in the process of obtaining the importance coefficient, the symbol is used to represent the importance coefficient, where represents the candidate text, y represents the original text, and x represents the annotated reference text of the second language associated with the semantics of the original text, and the importance coefficient is defined as follows:

[0152]

[0153] Optionally, the likelihood value of the monolingual model p(x; ω) is used to represent the numerator p(x) of the importance coefficient , where ω represents the model parameter set of the monolingual language model of the reference text x, and the likelihood value of the reverse NMT model p(x|y; π) is used to represent the denominator p(x|y) of the importance coefficient , so that the process of obtaining the entire importance coefficient can be represented as:

[0154] ​

[0155] In some embodiments, for any candidate text obtain the candidate text the likelihood value of the monolingual model p(x; w) and obtain the candidate text the likelihood value of the reverse NMT model p(x|y; p) divide by obtain the importance coefficient of the candidate text

[0156] In other embodiments, since taking the likelihood value or the log-likelihood value of the reverse NMT model p(x|y; p) does not affect the trend of the entire fraction of the importance coefficient for any candidate text obtain the candidate text the likelihood value of the monolingual model p(x; w) and obtain the candidate text the log-likelihood value of the reverse NMT model p(x|y; p) divide by obtain the importance coefficient of the candidate text

[0157] 203、The server screens the target candidate text from the plurality of candidate texts based on the quality coefficient and the importance coefficient.

[0158] In some embodiments, when using the data integration manner, the server can determine the corpus sequence number i of the original text y in the monolingual corpus set of the first language. Assuming that the monolingual corpus set of the first language is represented as M, the sample capacity of the monolingual corpus set M, i.e., the number of original texts y contained in the monolingual corpus set M, is |M|, then in the case where the technician has pre-set the hyperparameter g, the product of the sample capacity |M| and the hyperparameter g can be rounded down to obtain the demarcation value k. When the corpus sequence number i of the original text y is greater than or equal to 0 and less than k, the candidate text with the largest quality coefficient is selected as the target candidate text. When the corpus sequence number i of the original text y is greater than or equal to k and less than or equal to |M|, the candidate text with the largest importance coefficient is selected as the target candidate text.

[0159] ​​​​​In some embodiments, in addition to the data integration manner, the embodiments of the present application also provide a manner of screening target candidate texts based on Gamma interpolation, which can balance the quality coefficient and the importance coefficient, thereby significantly improving the sample quality of the target candidate texts screened, playing a stronger role in training the forward NMT model using the corpus pair composed of the target candidate texts and the original texts, and further significantly improving the translation performance of the forward NMT model. The manner of screening the target candidate texts based on Gamma interpolation will be described in detail in the next embodiment, and will not be described here.

[0160] All the optional technical solutions described above can be combined to form optional embodiments of the present disclosure, which will not be described here.

[0161] The method provided by the embodiments of the present application translates the original text of the first language into multiple candidate texts of the second language, and screens target candidate texts according to the quality coefficient and the importance coefficient of each candidate text. Since the translation quality of the candidate text is considered by the quality coefficient and the importance of the information carried by the candidate text is considered by the importance coefficient, the target candidate text that takes into account both the translation quality and the importance of the information carried can be screened under the action of the two coefficients, which is beneficial to generate target candidate texts with high translation quality and high importance coefficient. These screened target candidate texts can play a greater role when they are put into the training phase of the forward NMT model, thereby facilitating the maximization of the translation performance of the forward NMT model.

[0162] In the embodiments of the present application, the manner of screening the target candidate texts provides a general method for obtaining pseudo-corpus (i.e., target candidate texts) under the training manner of NMT based on reverse translation, which can well balance the quality coefficient and the importance coefficient of the generated pseudo-corpus, thereby providing theoretical guidance for subsequent training of the forward NMT model using the pseudo-corpus, and the pseudo-corpus can also be used to train other NLP applications that require reverse translation, such as translation stylization, NLP paraphrase training, etc. The embodiments of the present application do not make specific limitations on this.

[0163] In the above embodiments, the processing flow of the text acquisition method related by the embodiments of the present application is simply introduced, and in the embodiments of the present application, how to translate each candidate text and how to screen the target candidate text will be described in detail as follows.

[0164] Figure 3 is a flowchart of a text acquisition method provided by the embodiments of the present application. Referring to Figure 3The embodiment is executed by a computer device, which can be the terminal 101 or the server 102 in the implementation environment described above, and is described by taking the server 102 as an example. The embodiment includes the following steps.

[0165] 301. The server inputs any original text in the first language into the reverse translation model, and translates a plurality of candidate texts in the second language in a random sampling manner through the reverse translation model, the original text and the candidate text being associated with the same semantics.

[0166] The original text refers to a text to be translated, and the candidate text refers to any candidate translation of the original text. The original text and the candidate text have the same or similar semantics but belong to different languages. Here, the original text is in the first language, and the candidate text is in the second language. The first language and the second language are different languages.

[0167] In some embodiments, the server obtains a monolingual corpus set in the first language from a corpus sample library, and obtains any original text in the first language from the monolingual corpus set, for example, an English original text is randomly selected from an English monolingual corpus set.

[0168] In some embodiments, the original text is obtained from a locally stored corpus sample library, or the original text is obtained from a cloud-stored corpus sample library. The source of the original text is not limited in the embodiment.

[0169] In some embodiments, the reverse translation model is used to translate input text from the first language to the second language. The reverse translation model can be any neural network-based NMT model for translating from the first language to the second language, which is also referred to as a reverse NMT model in the subsequent steps. Correspondingly, the forward translation model is used to translate input text from the second language to the first language. The forward translation model can be any neural network-based NMT model for translating from the second language to the first language, which is also referred to as a forward NMT model in the subsequent steps.

[0170] Illustratively, in the NMT training scenario based on back translation, since a pre-trained back NMT model needs to be used to generate pseudo corpus before training a forward NMT model, the text acquisition method disclosed in the embodiments of the present application can be regarded as a process of using a back NMT model to acquire pseudo corpus, in which the first language refers to the target language (Target), and the second language refers to the source language (Source), the forward NMT model is used to translate input text from the second language to the first language (from the source end to the target end), and the back NMT model is used to translate input text from the first language to the second language (from the target end to the source end).

[0171] In some embodiments, after obtaining the original text in the first language, the server inputs the original text into a back translation model, i.e., a back NMT model, and performs encoding and then decoding on the original text through an encoder-decoder framework in the back NMT model to predict a plurality of candidate texts associated with the semantics of the original text but different in language. These candidate texts are all translations having a semantic association with the original text. It should be noted that having a semantic association means that the semantics of the original text and the candidate text are the same or similar, for example, the semantic similarity of the original text and the candidate text is greater than a similarity threshold, or the semantics of the original text and the candidate text meet certain similarity conditions, which are not limited in the embodiments of the present application.

[0172] In some embodiments, the back NMT model is taken as a Transformer model for illustration. The Transformer translation model adopts an encoder-decoder framework based on a SAN network, including an embedding (Embedding) module, an encoding (Encoder) module, a decoding (Decoder) module and an output module, and belongs to the mainstream sequence-to-sequence generation model structure.

[0173] The server first performs Tokenize processing on the original text to obtain a character sequence of the original text, inputs the character sequence of the original text into the embedding module, and performs embedding processing on the character sequence of the original text through the embedding module to obtain an embedding vector sequence of the original text. In addition, the position information of each character in the character sequence of the original text is encoded to obtain a position encoding vector of the original text, wherein the position encoding vector of the original text is used to represent the position sequence information of each character in the original text. The embedding vector sequence of the original text and the position encoding vector are concatenated (Concat) and input into the encoding module.

[0174] The encoding module is composed of N encoders in cascade (for example, N = 6 encoders in cascade), and each encoder internally includes a multi-head attention layer and a feed forward neural network layer. The multi-head attention layer is used to comprehensively extract the correlation between characters in the original text from multiple expression subspaces, and the feed forward neural network layer is used to fully connect the feature vectors output by the multi-head attention layer. Residual structures are arranged after the multi-head attention layer and the feed forward neural network layer, that is, the input and output of the current layer are connected in residual connection (i.e., splicing) and normalized before being input into the next layer. Wherein, N represents the number of cascaded encoders.

[0175] After the embedding vector sequence of the original text and the position encoding vector are input into the encoding module after splicing (Concat), the input vectors are encoded by the N encoders in the encoding module, and the semantic vector sequence of the original text is output by the last encoder.

[0176] The decoding module is composed of N decoders in cascade (for example, N = 6 decoders in cascade), and each decoder internally includes a masked multi-head attention layer, a fusion multi-head attention layer and a feed forward neural network layer. The masked multi-head attention layer is similar to the multi-head attention layer, but only focuses on the translation results before the current time, so the translation results after the current time need to be Masked. The fusion multi-head attention layer is also similar to the multi-head attention layer, but the fusion multi-head attention layer takes the output (i.e., the result after residual connection and normalization) of the feed forward neural network layer of the corresponding encoder as input in addition to the output of the masked multi-head attention layer of the decoder. This design is used to focus on the encoding information of the encoder. In other words, the decoder predicts the next character in the candidate text by viewing the output of the encoder and self-attention on its own output. The feed forward neural network layer of the decoder is similar to the feed forward neural network layer of the encoder, which will not be described here. Residual structures are also arranged after the masked multi-head attention layer, the fusion multi-head attention layer and the feed forward neural network layer of the decoder, that is, the input and output of the current layer are connected in residual connection (i.e., splicing) and normalized before being input into the next layer. Wherein, N represents the number of cascaded decoders, that is, the number of cascaded encoders and the number of cascaded decoders need to be consistent.

[0177] After the semantic vector sequence of the original text output by the last encoder in the encoding module is input into the decoding module, the semantic vector sequence of the original text is decoded by the N decoders in the decoding module, and the final hidden vector sequence is output by the last decoder.

[0178] The output module includes a linear layer and a softmax layer, and is capable of mapping each hidden vector in the hidden vector sequence output by the decoding module to each character in the candidate text, and the characters predicted in sequence will form the candidate text.

[0179] The linear layer is a fully connected layer, and the dimension of the fully connected layer is the number of all characters contained in the second language vocabulary. In the softmax layer, each hidden vector in the hidden vector sequence is mapped to a matching probability of the hidden vector with each character in the second language vocabulary, and the matching probability represents the possibility of the hidden vector at the current time being mapped to each character in the second language vocabulary.

[0180] In some embodiments, after obtaining the matching probability of each hidden vector at the current time with each character in the second language vocabulary through the softmax layer, the candidate text can be obtained through various sampling methods, each of which can obtain one or more candidate texts. The sampling methods include random sampling, top K sampling, greedy algorithm, beam search, etc., which are not limited in the embodiments of the present application.

[0181] Optionally, in the random sampling method, the hidden vector at each time is randomly sampled under the probability distribution of each matching probability to obtain the character output at the corresponding time in the candidate text. The above operation is repeatedly performed until all characters in the candidate text are output. For example, assuming that the matching probability of the hidden vector at t=0 with character A is 80%, with character B is 10%, with character C is 5%, with character D is 5%, and with the remaining characters is 0%, then in the random sampling process, there is an 80% probability that the character at t=0 in the candidate text is output as character A, a 10% probability that the character at t=0 in the candidate text is output as character B, and a 5% probability that the character at t=0 in the candidate text is output as character C or D.

[0182] Optionally, in the top K sampling method, the top K characters with the largest matching probability are selected for each hidden vector at each time, and one of the top K characters is randomly selected as the character output at the corresponding time in the candidate text. The above operation is repeatedly performed until all characters in the candidate text are output. K is a value greater than or equal to 1 that is set in advance.

[0183] Optionally, in the greedy algorithm manner, the character with the largest matching probability is always selected as the output of the corresponding moment in the candidate text, at this time, the reverse NMT model will output a deterministic candidate text for the given original text, and each character in the candidate text is the character with the largest matching probability with the hidden vector at the corresponding moment.

[0184] Optionally, in the beam search manner, for the hidden vector of each moment, the M (M is referred to as the beam width, which is a hyperparameter) characters with the largest matching probability are filtered out, and the characters that do not match the context with the historical output are removed from the M characters, and the above operation is repeatedly performed until all characters in the candidate text are output, and the characters will have a relatively close context relationship.

[0185] In the embodiments of the present application, the reverse NMT model is used to obtain multiple candidate texts by using the random sampling manner, that is, the step 301 provides a possible implementation manner of obtaining multiple candidate texts of the second language by translating any original text of the first language, in other embodiments, one or more sampling manners (such as the random sampling manner, the top K sampling manner, the greedy algorithm manner, the beam search manner, etc.) can be used to obtain multiple candidate texts, each sampling manner can generate one or more candidate texts, and multiple candidate texts obtained by all sampling manners are obtained, that is, multiple candidate texts translated by the reverse NMT model are obtained, and the embodiments of the present application do not specifically limit the manner of obtaining the candidate texts.

[0186] 302. For any candidate text in the multiple candidate texts, the server obtains an initial quality coefficient and an initial importance coefficient of the candidate text.

[0187] In some embodiments, for any candidate text, the server obtains the initial quality coefficient and the initial importance coefficient by using the manner introduced in the step 202 in the above embodiments, and it should be noted that the step 202 is to obtain the initial quality coefficient and the initial importance coefficient as the final quality coefficient and the final importance coefficient respectively, and in the embodiments of the present application, the initial quality coefficient is normalized to obtain the quality coefficient by the following step 303, and similarly, the initial importance coefficient is normalized to obtain the importance coefficient by the following step 304.

[0188] Illustratively, for any candidate text the BLEU value of the candidate text itself is used as the initial quality coefficient of the candidate text , or the test performance Test BLEU of the reverse NMT model is used as the initial quality coefficient of the candidate text . The initial quality coefficients, or the likelihood values ​​using the inverse NMT model. or log-likelihood value to be used as candidate text The initial quality coefficient.

[0189] In this embodiment of the application, the likelihood value is used with the inverse NMT model. as candidate text Let's take the initial quality coefficient as an example for illustration.

[0190] Indicatively, for any candidate text Get candidate text The likelihood value of the monolingual model p(x; ω) And obtain candidate text The likelihood value of the inverse NMT model p(x|y;π) Will Divide by Obtain candidate text Initial importance coefficient Alternatively, obtain candidate text. The likelihood value of the monolingual model p(x; ω) And obtain candidate text The log-likelihood value of the inverse NMT model p(x|y;π) Will Divide by Obtain candidate text Initial importance coefficient

[0191] 303. Based on the text length of the candidate text, the server normalizes the initial quality coefficient to obtain the quality coefficient of the candidate text, which represents the translation quality of the candidate text.

[0192] The text length represents the number of characters contained in the candidate text.

[0193] In some embodiments, the server can obtain the initial quality coefficient of each candidate text through step 302; then, by taking the logarithm of the initial quality coefficient of each candidate text, the logarithmic quality coefficient of the initial quality coefficient of each candidate text can be obtained. In other words, the logarithmic quality coefficient is the logarithm of the initial quality coefficient; then, based on the logarithmic quality coefficient of the initial quality coefficient of each of the multiple candidate texts, the first mean and the first variance of the logarithmic quality coefficient are obtained.

[0194] To illustrate, assume that random sampling is used to generate N candidate texts in the second language for each original text y in the first language. After obtaining the initial quality coefficient for each candidate text Next, the log-quality coefficient of each candidate text is obtained. Furthermore, the log-quality coefficients of each of the N candidate texts are... Calculate the arithmetic mean to obtain the first mean μ. p Log-quality coefficients for each of the N candidate texts Calculate the variance to obtain the first variance σ. p .

[0195] Optionally, the first mean μ p and the first variance σ p The expressions are as follows:

[0196]

[0197]

[0198] Where i is an integer greater than or equal to 1 and less than or equal to N, and N is an integer greater than or equal to 2.

[0199] That is, the first mean μ p Equal to the log quality coefficient of each of the N candidate texts The sum of the values ​​divided by N is the first variance σ. p Equal to the logarithmic quality coefficient of each candidate text With the first mean μ p The value obtained by dividing the sum of squares of the differences by (N-1).

[0200] In some embodiments, after obtaining the logarithmic quality coefficient of each candidate text, the text length of each candidate text is also obtained. Then, for each candidate text, the logarithmic quality coefficient of the candidate text is divided by the text length of the candidate text to obtain the normalized quality coefficient of the candidate text.

[0201] Schematic, for the i-th (1≤i≤N) candidate text Get candidate text logarithmic mass coefficient and text length Next, the logarithmic mass coefficient Divide by text length Obtain the candidate text Normalized quality coefficient

[0202] In some embodiments, after obtaining the first mean, the first variance, and the normalized quality coefficient of each candidate text, for each candidate text, the value obtained by subtracting the normalized quality coefficient of the candidate text from the first mean is divided by the first variance to obtain the final quality coefficient of the candidate text.

[0203] Illustratively, the i-th (1≤i≤N) candidate text The normalized quality coefficient of the candidate text is obtained. Then, the normalized quality coefficient is subtracted by the first mean μ p , to obtain a difference value Then, the difference value is divided by the first variance σ p , to obtain the final quality coefficient of the candidate text The expression of the quality coefficient is as follows:

[0204]

[0205] In the above process, a way of normalizing the initial quality coefficient by using the text length of the candidate text is provided. The quality coefficient obtained after normalization can be put into the process of obtaining the selection weight coefficient, so that the selection weight coefficient can form a probability distribution, so as to facilitate subsequent random sampling on the probability distribution constituted by the selection weight coefficient. In other embodiments, if random sampling on the probability distribution constituted by the selection weight coefficient is not required, the quality coefficient can also not be normalized, and the initial quality coefficient can be directly used to obtain the selection weight coefficient. The embodiments of the present application do not make specific limitations in this regard.

[0206] 304. The server normalizes the initial importance coefficient based on the text length of the candidate text, to obtain an importance coefficient of the candidate text, which represents the importance degree of the information carried by the candidate text.

[0207] The text length represents the number of characters contained in the candidate text.

[0208] In some embodiments, the server can obtain the initial importance coefficient of each candidate text through step 302; then, the logarithmic importance coefficient of the initial importance coefficient of each candidate text is obtained by taking the logarithm value of the initial importance coefficient, in other words, the logarithmic importance coefficient is the logarithm value of the initial importance coefficient; then, the second mean and the second variance of the logarithmic importance coefficient are obtained based on the logarithmic importance coefficient of the initial importance coefficient of each candidate text.

[0209] Illustratively, assuming that the random sampling method is used, N candidate texts In some embodiments, the server can obtain the initial importance coefficient of each candidate text through step 302; then, the logarithmic importance coefficient of the initial importance coefficient of each candidate text is obtained by taking the logarithm value of the initial importance coefficient, in other words, the logarithmic importance coefficient is the logarithm value of the initial importance coefficient; then, the second mean and the second variance of the logarithmic importance coefficient are obtained based on the logarithmic importance coefficient of the initial importance coefficient of each candidate text. ​​Next, the log-importance coefficient of each candidate text is obtained. Furthermore, the log-importance coefficients of each of the N candidate texts... Calculate the arithmetic mean to obtain the second mean μ. Imp Log importance coefficients for each of the N candidate texts Calculate the variance to obtain the second variance σ. Imp .

[0210] Optionally, the second mean μ Imp Second variance σ Imp The expressions are as follows:

[0211]

[0212]

[0213] Where i is an integer greater than or equal to 1 and less than or equal to N, and N is an integer greater than or equal to 2.

[0214] That is, the second mean μ Imp Equal to the log importance coefficient of each of the N candidate texts The sum of the values ​​divided by N is the second variance σ. Imp Equal to the log importance coefficient of each candidate text With the second mean μ Imp The value obtained by dividing the sum of squares of the differences by (N-1).

[0215] In some embodiments, after obtaining the log importance coefficient of each candidate text, the text length of each candidate text is also obtained. Then, for each candidate text, the log importance coefficient of the candidate text is divided by the text length of the candidate text to obtain the normalized importance coefficient of the candidate text.

[0216] Schematic, for the i-th (1≤i≤N) candidate text Get candidate text log importance coefficient and text length Next, the logarithmic importance coefficients will be... Divide by text length Obtain the candidate text Normalized importance coefficient

[0217] In some embodiments, after obtaining the second mean, the second variance, and the normalized importance coefficient of each candidate text, for each candidate text, the value obtained by subtracting the normalized importance coefficient of the candidate text from the second mean is divided by the second variance to obtain the final importance coefficient of the candidate text.

[0218] Schematic, for the i-th (1≤i≤N) candidate text Get candidate text Normalized importance coefficient Then, the normalized importance coefficient is... With the second mean μ Imp Subtract them to get a difference. Then take the difference Divide by the second variance σ Imp Candidate texts are obtained. Final importance coefficient Importance coefficient The expression is as follows:

[0219]

[0220] In the above process, a method is provided to normalize the initial importance coefficient using the text length of the candidate text. The importance coefficient obtained after normalization can be used in the process of obtaining the selection weight coefficient, so that the selection weight coefficient can form a probability distribution, so that random sampling can be performed on the probability distribution formed by the selection weight coefficient in the future. In some embodiments, if it is not necessary to perform random sampling on the probability distribution formed by the selection weight coefficient, the importance coefficient can be obtained directly without normalizing the importance coefficient. This application does not specifically limit this.

[0221] In steps 303-304 above, one possible implementation for obtaining the quality coefficient and importance coefficient of each of the at least one candidate text is provided. That is, by using the text length of the candidate text, the initial quality coefficient and initial importance coefficient of each candidate text can be normalized to obtain the quality coefficient and importance coefficient of each candidate text. In other embodiments, the method involved in step 202 of the previous embodiment can be used, that is, without using the text length for normalization, the initial quality coefficient and initial importance coefficient are directly used as the final quality coefficient and importance coefficient in the subsequent processing flow. This application embodiment does not specifically limit whether to perform normalization operation on the initial quality coefficient and initial importance coefficient.

[0222] 305. The server determines the first value obtained by multiplying the first adjustment factor by the importance coefficient of the candidate text.

[0223] The first adjustment factor is a hyperparameter preset by the technician on the server side. For example, the first adjustment factor is denoted as γ. Since the second adjustment factor plus the first adjustment factor equals 1, the second adjustment factor involved in the following step 306 can be obtained from the first adjustment factor, that is, the second adjustment factor is 1 minus the first adjustment factor, and the second adjustment factor is denoted as (1-γ).

[0224] In some embodiments, the server obtains a first number by multiplying the first adjustment factor γ and the importance coefficient of the i th (1≤i≤N) candidate text The server obtains a first number by multiplying the first adjustment factor γ and the importance coefficient of the i th (1≤i≤N) candidate text

[0225] 306. The server determines a second number by multiplying the second adjustment factor and the quality coefficient of the candidate text.

[0226] The second adjustment factor can be a hyperparameter preset by the technician on the server side (provided that the sum of the first adjustment factor and the second adjustment factor is 1), or the second adjustment factor is a value automatically configured based on the first adjustment factor preset by the technician. For example, the technician presets the first adjustment factor γ, and then obtains the second adjustment factor (1-γ) by subtracting 1 from the first adjustment factor γ. The source of the second adjustment factor is not limited in the embodiments of the present application.

[0227] In some embodiments, the server obtains a first number by multiplying the first adjustment factor γ and the importance coefficient of the i th (1≤i≤N) candidate text The server obtains a first number by multiplying the first adjustment factor γ and the importance coefficient of the i th (1≤i≤N) candidate text

[0228] 307. The server adds the first number and the second number to obtain a third number.

[0229] In some embodiments, the server obtains a first number for each candidate text through the above step 305, obtains a second number through the above step 306, and adds the first number and the second number of the candidate text to obtain a third number of the candidate text.

[0230] In some embodiments, the server obtains a first number by multiplying the first adjustment factor γ and the importance coefficient of the i th (1≤i≤N) candidate text The server obtains a first number by multiplying the first adjustment factor γ and the importance coefficient of the i th (1≤i≤N) candidate text ​​​​the second value obtained in step 306 adding the third value

[0231] 308、the server determines a selection weight coefficient of the candidate text based on the third value of the candidate text and the third values of other candidate texts, the selection weight coefficient being a screening index balancing the quality coefficient and the importance coefficient.

[0232] The other candidate texts are candidate texts obtained by translating the original text except for the candidate text.

[0233] In some embodiments, the server obtains the first index power of the natural base e with the third value of the candidate text as the first index.

[0234] Illustratively, for the i(th) (1≤i≤N) candidate text the server obtains the third value of the candidate text obtains the second index power of the natural base e with the third value of the candidate text as the second index.

[0235] In some embodiments, the server obtains the second index power of the natural base e with the third value of any other candidate text as the second index.

[0236] Illustratively, for any other candidate text except for the i(th) candidate text 1≤j≤N and j≠i, the third value of the other candidate text can also be obtained through steps 302-307 above obtains the second index power of the natural base e with the third value of the candidate text as the second index.

[0237] In some embodiments, the server adds the first index power of the natural base and the second index powers of the natural base to obtain a target sum.

[0238] Illustratively, the server adds the first index power of the candidate text and the second index powers of each other candidate text to obtain a target sum Here, j can be equal to i.

[0239] Alternatively, the server adds the first index power of the candidate text or the second index powers of other candidate texts The server will obtain the third power of the natural base after obtaining the third value through step 307, and obtain the target sum value by adding the third power of the natural base of all candidate texts (including candidate texts and each other candidate text ).

[0240] In some embodiments, the server divides the first power of the natural base by the target sum value to obtain the selection weight coefficient of the candidate text, which is a screening index balancing the quality coefficient and the importance coefficient.

[0241] Illustratively, the server divides the first power of the candidate text by the target sum value to obtain the selection weight coefficient of the candidate text. Assuming that the selection weight coefficient is represented by the symbol Γ(x i ; ω, π), the expression of the selection weight coefficient Γ(x i ; ω, π) is as follows:

[0242]

[0243] The above selection weight coefficient Γ(x i ; ω, π) is also called the Gamma coefficient. Since the first adjustment factor γ and the second adjustment factor (1-γ) are both pre-set hyperparameters, the server can obtain a selection weight coefficient Γ(x i ; ω, π) for each candidate text by the above method. This selection weight coefficient Γ(x i ; ω, π) can be used in the following step 309 to decide which candidate text to select as the target candidate text.

[0244] In the above steps 305-308, a possible implementation is provided for determining the selection weight coefficient of each candidate text based on the quality coefficient and the importance coefficient. Since the selection weight coefficient takes into account both the quality coefficient and the importance coefficient, the selection weight coefficient can balance the quality coefficient and the importance coefficient of the candidate text at the same time.

[0245] It should be noted that since the above Gamma coefficient can balance the quality coefficient and the importance coefficient of the candidate text, under the guidance of the Gamma coefficient, it is beneficial to screen the target candidate text with high translation quality and high importance coefficient. This process of evaluating the comprehensive quality of the target candidate text through the Gamma coefficient can be represented by the following formula:

[0246]

[0247] Further, since the quality coefficient and the importance coefficient are essentially mutually exclusive, and the reverse NMT model also has the problem of calibration, considering the quality coefficient alone or considering the importance coefficient alone will produce contradictory conclusions, and neither can filter out the target candidate text with the best overall quality. Therefore, the Gamma coefficient proposed by the embodiments of the present application can balance the importance coefficient and the quality coefficient in a Gamma interpolation manner, so as to simply and effectively filter out the target candidate text with the best overall quality, and the target candidate text filtered out will play a role of maximizing the translation performance of the NMT model when used as pseudo corpus in the training stage of the forward NMT model.

[0248] 309. The server filters a target candidate text from the plurality of candidate texts based on the selection weight coefficients of the plurality of candidate texts.

[0249] In some embodiments, the server determines the candidate text with the largest selection weight coefficient as the target candidate text from the plurality of candidate texts, which is equivalent to providing a deterministic selection method to select the target candidate text, that is, for each original text, a target candidate text with the largest selection weight coefficient is selected. Since the selection weight coefficient is also called the Gamma coefficient, this deterministic selection method based on the Gamma coefficient is also called Gamma selection.

[0250] In other embodiments, after the server obtains the selection weight coefficient of each candidate text through step 308, since the selection weight coefficient itself is equivalent to a screening indicator after Softmax exponential normalization, it represents that the selection weight coefficients of all candidate texts at this time add up to 1, which is equivalent to the selection weight coefficients of all candidate texts being subject to a certain probability distribution, and the selection weight coefficient of each candidate text represents the selection probability of the candidate text. Therefore, the server can randomly sample the target candidate text from the plurality of candidate texts according to the probability distribution to which the selection weight coefficient is subject.

[0251] For example, assuming that the original text y of a certain first language is output by the reverse NMT model N candidate texts of a second language The selection weight coefficients of the N candidate texts are subject to a certain and known probability distribution, so the final target candidate text can be obtained by random sampling based on the selection weight coefficient according to the probability distribution. For example, assuming that the selection weight coefficient of the i(th, 1≤i≤N) candidate text is 90%, then when filtering the target candidate text, there is a 90% probability of selecting the i(th) candidate text As the target candidate text, there will also be a small probability event to select other candidate texts, which makes the sampling mode of the entire target candidate text have a certain randomness.

[0252] In the above process, a random selection method is provided to select the target candidate text, that is, for each original text, there is a greater probability to select a candidate text with a larger selection weight coefficient as the target candidate text, but there is still a small probability event to select a candidate text with a smaller selection weight coefficient as the target candidate text, which can introduce a certain randomness in the process of screening the target candidate text. Since the selection weight coefficient is also called the Gamma coefficient, this random selection method based on the Gamma coefficient is also called Gamma sampling.

[0253] In steps 305-309 described above, a possible implementation of screening the target candidate text from the plurality of candidate texts based on the quality coefficient and the importance coefficient is provided, that is, after obtaining the Gamma coefficient, the target candidate text can be screened by Gamma selection or Gamma sampling. In other embodiments, the target candidate text can also be screened by the description of step 203 in the previous embodiment, and the screening method of the target candidate text in the embodiments of the present application is not specifically limited.

[0254] In some embodiments, the original text and the target candidate text screened by step 309 described above can constitute a corpus pair, and repeated execution of steps 301-309 described above can obtain a plurality of corpus pairs. Since the candidate texts in these corpus pairs are not real corpus annotated by artificial, but pseudo corpus obtained by machine translation by the reverse NMT model, the plurality of corpus pairs can be used as pseudo corpus texts in the training process of the forward translation model, that is, the forward NMT model. In other words, the plurality of corpus pairs can be used to train the forward NMT model, wherein the forward NMT model is used to translate input texts from the second language to the first language.

[0255] All optional technical solutions described above can be combined to form optional embodiments of the present disclosure, which will not be repeated here.

[0256] The method provided in the embodiments of the present application can obtain a plurality of candidate texts in a second language by translating an original text in a first language, and can obtain a target candidate text according to a quality coefficient and an importance coefficient of each candidate text. Since the translation quality of the candidate text is considered by the quality coefficient and the importance of the information carried by the candidate text is considered by the importance coefficient, the target candidate text that takes into account both the translation quality and the importance of the information carried can be obtained under the action of the two coefficients, which is beneficial to generating the target candidate text with high translation quality and high importance coefficient. The target candidate text obtained through the screening can play a greater role when it is put into the training phase of the forward NMT model, thereby being beneficial to maximizing the improvement of the translation performance of the forward NMT model.

[0257] In the embodiments of the present application, the screening method of the target candidate text provides a general method for obtaining pseudo corpus (i.e., target candidate text) in the NMT training method based on reverse translation, which can well balance the quality coefficient and the importance coefficient of the generated pseudo corpus, thereby providing a theoretical guidance for subsequent training of the forward NMT model using the pseudo corpus, and the pseudo corpus can also be used to train other NLP applications that require reverse translation, such as translation stylization, NLP paraphrase training, etc. The embodiments of the present application do not make specific limitations on this.

[0258] Next, the test performance of the NMT model trained by the pseudo corpus generated based on the data integration (DM) method, the Gamma selection method and the Gamma sampling method involved in the embodiments of the present application will be respectively demonstrated.

[0259] For the data integration method, the test is performed on the validation set of WMT14 De-En direction. The test involves the forward NMT model based on the traditional NMT training method and the forward NMT model based on the NMT training method based on reverse translation. According to the different generation methods of the pseudo corpus, the forward NMT model based on the NMT training method based on reverse translation involves the beam search method for generating pseudo corpus, the random sampling method for generating pseudo corpus and the data integration method for generating pseudo corpus. Assuming that the forward NMT model is a Transformer model, the test results of the Transformer model obtained by the above four different training methods on the validation set of WMT14 De-En direction are shown in Table 3:

[0260] Table 3

[0261]

[0262] wherein, w / o bitext represents the case of not using real bilingual corpus and only using pseudo bilingual corpus as training corpus set, and w bitext represents the case of using real bilingual corpus and pseudo bilingual corpus as training corpus set.

[0263] In Table 3, the first row of Transformer refers to the BLEU value of the Transformer model trained based on the traditional NMT training mode, i.e., the Transformer model trained using only real bilingual corpus, and the column of w / o bitext is empty because the traditional NMT training mode does not use the reverse translation to generate pseudo bilingual corpus; the second row of Beam BT refers to the BLEU value of the Transformer model trained in the case of using the beam search method to generate reverse translation pseudo bilingual corpus; the third row of Sampling BT refers to the BLEU value of the Transformer model trained in the case of using the random sampling method to generate reverse translation pseudo bilingual corpus; and the fourth row of DM refers to the BLEU value of the Transformer model trained in the case of using the data integration method to generate reverse translation pseudo bilingual corpus.

[0264] It can be seen that, in the case of not using real bilingual corpus, compared with the beam search method and the random sampling method, the data integration method can significantly improve the translation performance of the NMT model, and obtain a BLEU value slightly lower than that of the Transformer model using only real bilingual corpus, but in the case of using real bilingual corpus and pseudo bilingual corpus, the data integration method cannot bring obvious improvement, and can only obtain additional benefits similar to the random sampling method.

[0265] Further, on the basis of the above tests, additional experimental groups of the Gamma selection method and the Gamma sampling method for generating pseudo corpus are added, and the tests are performed again on the WMT14 German-English validation set, and the test results are shown in Table 4.

[0266] Table 4

[0267] Training approach BLEU values Transformer 32.1 Beam BT 32.7 Sampling BT 34.1 DM+bitext 34.2 Gamma sampling BT 35.0* Gamma selection BT 34.7*

[0268] In Table 4, the first to fourth rows are consistent with the BLEU values of the four training modes under the column of "w bitext" in Table 3, the fifth row Gamma sampling BT represents the BLEU value of the Transformer model trained in the case of using the Gamma sampling mode to generate the pseudo bilingual corpus and mixing the pseudo bilingual corpus with the real bilingual corpus, and the sixth row Gamma selection BT represents the BLEU value of the Transformer model trained in the case of using the Gamma selection mode to generate the pseudo bilingual corpus and mixing the pseudo bilingual corpus with the real bilingual corpus.

[0269] It can be seen that the pseudo corpus generated by the Gamma selection mode and the Gamma sampling mode involved in the embodiments of the present application can effectively improve the translation performance of the forward NMT model after being put into the training stage of the forward NMT model, and greater benefits than the original beam search, random sampling and data integration mode are obtained, and compared with the random sampling mode, the Gamma sampling mode can at most improve the BLEU score by 0.9 points.

[0270] Further, the same test scheme as that for the English-to-German direction is adopted, and the test is performed again on the validation set of the WMT14 English-to-German direction and the English-to-Russian direction, and the test result is shown in Table 5.

[0271] Table 5

[0272] Training approach En-De Ru-En Transformer 27.4 34.1 Beam BT 29.7 35.9 Sampling BT 30.0 35.6 Gamma sampling BT 31.0* 36.1* Gamma selection BT 30.9* 36.3*

[0273] It can be seen from Table 5 that the pseudo corpus generated by the Gamma selection mode and the Gamma sampling mode involved in the embodiments of the present application can also obtain effects better than the beam search mode and the random sampling mode in the English-to-German (En-De) direction and the English-to-Russian (Ru-En) direction after being put into the training stage of the forward NMT model.

[0274] In summary, by performing the test on the validation set of the English-to-German, English-to-German and English-to-Russian directions respectively, it is verified that the target candidate text selected by the text acquisition mode involved in the embodiments of the present application can significantly improve the translation performance of the NMT model trained by being put into the training stage of the forward NMT model compared with the beam search mode and the random sampling mode, so that the text acquisition mode involved in the embodiments of the present application can provide the pseudo corpus text that maximizes the improvement of the performance of the NMT model.

[0275] Figure 4 is a structural schematic diagram of a text acquisition device provided by the embodiments of the present application, please refer to Figure 4 The device comprises:

[0276] The translation module 401 is configured to translate a plurality of candidate texts in a second language from any original text in a first language, the original text and the candidate texts being associated with semantics;

[0277] The acquisition module 402 is configured to acquire a quality coefficient and an importance coefficient of each of the plurality of candidate texts, the quality coefficient representing translation quality of the candidate text, and the importance coefficient representing an importance degree of information carried by the candidate text;

[0278] The screening module 403 is configured to screen a target candidate text from the plurality of candidate texts based on the quality coefficient and the importance coefficient.

[0279] The device provided by the embodiments of the present application can translate a plurality of candidate texts in a second language from an original text in a first language, and screen a target candidate text according to a quality coefficient and an importance coefficient of each candidate text. Since the quality coefficient takes into account the translation quality of the candidate text, and the importance coefficient takes into account the importance degree of information carried by the candidate text, the target candidate text that takes into account both the translation quality and the importance degree of information carried can be screened under the action of the two coefficients, which is conducive to generating a target candidate text with high translation quality and high importance coefficient. The target candidate text screened can play a greater role when put into the training stage of the forward NMT model, thereby being conducive to maximizing the translation performance of the forward NMT model.

[0280] In some embodiments, the device is configured to screen the target candidate text from the plurality of candidate texts based on the quality coefficient and the importance coefficient. Figure 4 The screening module 403 includes:

[0281] The determination unit is configured to determine a selection weight coefficient of each of the plurality of candidate texts based on the quality coefficient and the importance coefficient, the selection weight coefficient being a screening index balancing the quality coefficient and the importance coefficient.

[0282] The screening unit is configured to screen the target candidate text from the plurality of candidate texts based on the selection weight coefficient.

[0283] In some embodiments, the device is configured to determine the selection weight coefficient of each of the plurality of candidate texts based on the quality coefficient and the importance coefficient. Figure 4 The determination unit includes:

[0284] The first determination sub-unit is configured to determine, for any candidate text, a first value obtained by multiplying a first adjustment factor and the importance coefficient of the candidate text.

[0285] The first determination sub-unit is further configured to determine a second value obtained by multiplying a second adjustment factor and the quality coefficient of the candidate text, the second adjustment factor and the first adjustment factor adding up to 1.

[0286] An adding unit, configured to add the first value and the second value to obtain a third value.

[0287] A second determining sub-unit, configured to determine the selection weight coefficient of the candidate text based on the third value of the candidate text and third values of other candidate texts, the other candidate texts being candidate texts obtained by translating the original text except for the candidate text.

[0288] In some embodiments, the second determining sub-unit is configured to:

[0289] obtain the first index power of the natural base with the third value of the candidate text as the first index;

[0290] obtain the second index power of the natural base with the third value of the other candidate text as the second index for any of the other candidate texts;

[0291] obtain a target sum value by adding the first index power of the natural base and the second index powers of the natural base;

[0292] obtain the selection weight coefficient of the candidate text by dividing the first index power of the natural base by the target sum value.

[0293] In some embodiments, the screening unit is configured to:

[0294] determine the candidate text with the largest selection weight coefficient from the plurality of candidate texts as the target candidate text.

[0295] In some embodiments, the screening unit is configured to:

[0296] randomly sample the target candidate text from the plurality of candidate texts according to a probability distribution to which the selection weight coefficient is subject.

[0297] In some embodiments, the apparatus is configured based on Figure 4 The obtaining module 402 includes:

[0298] A first obtaining unit, configured to obtain an initial quality coefficient of any of the candidate texts.

[0299] A first normalization unit, configured to normalize the initial quality coefficient based on a text length of the candidate text to obtain the quality coefficient, the text length representing a number of characters contained in the candidate text.

[0300] In some embodiments, the first normalization unit is configured to:

[0301] The first mean and the first variance of the logarithmic quality coefficient are obtained based on the logarithmic quality coefficient of the initial quality coefficient of each of the plurality of candidate texts, the logarithmic quality coefficient being a logarithmic value of the initial quality coefficient;

[0302] The logarithmic quality coefficient of the candidate text is divided by the text length of the candidate text to obtain a normalized quality coefficient;

[0303] The value obtained by subtracting the normalized quality coefficient from the first mean is divided by the first variance to obtain the quality coefficient.

[0304] In some embodiments, the device is configured based on Figure 4 The obtaining module 402 comprises:

[0305] The second obtaining unit is configured to obtain, for any of the candidate texts, an initial importance coefficient of the candidate text.

[0306] The second normalization unit is configured to normalize the initial importance coefficient based on a text length of the candidate text to obtain the importance coefficient, the text length representing a number of characters contained in the candidate text.

[0307] In some embodiments, the second normalization unit is configured to:

[0308] The second mean and the second variance of the logarithmic importance coefficient are obtained based on the logarithmic importance coefficient of the initial importance coefficient of each of the plurality of candidate texts, the logarithmic importance coefficient being a logarithmic value of the initial importance coefficient;

[0309] The logarithmic importance coefficient of the candidate text is divided by the text length of the candidate text to obtain a normalized importance coefficient.

[0310] The value obtained by subtracting the normalized importance coefficient from the second mean is divided by the second variance to obtain the importance coefficient.

[0311] In some embodiments, the translation module 401 is configured to:

[0312] The original text is input into a back-translation model, and the plurality of candidate texts are translated in a random sampling manner by the back-translation model, the back-translation model being configured to translate input text from the first language to the second language.

[0313] In some embodiments, the original text and the target candidate text constitute a corpus pair for training a forward translation model, the forward translation model being configured to translate input text from the second language to the first language.

[0314] All the optional technical solutions described above can be combined to form optional embodiments of the present disclosure, which will not be described again here.

[0315] It should be noted that the text acquisition apparatus provided in the above embodiments only takes the division of the above functional modules as an example when acquiring the target candidate text, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the text acquisition apparatus and the text acquisition method provided in the above embodiments belong to the same concept, and the specific implementation process is described in the text acquisition method, which will not be repeated here.

[0316] Figure 5 is a structural schematic diagram of a computer device provided in the embodiments of the present application. The computer device 500 can have great differences due to different configurations or performances. The computer device 500 includes one or more processors (Central Processing Units, CPU) 501 and one or more memories 502. The memory 502 stores at least one computer program, which is loaded and executed by the one or more processors 501 to implement the text acquisition method provided in the above embodiments. Optionally, the computer device 500 also has a wired or wireless network interface, a keyboard, an input and output interface, and the like, so as to perform input and output. The computer device 500 also includes other components for implementing device functions, which will not be described here.

[0317] In some embodiments, a computer readable storage medium is also provided, for example, a memory including at least one computer program. The at least one computer program can be executed by a processor in a terminal to complete the text acquisition method in the above embodiments. For example, the computer readable storage medium includes a ROM (Read-Only Memory), a RAM (Random-Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0318] In some embodiments, a computer program product is also provided, including one or more computer programs stored in a computer readable storage medium. One or more processors of a computer device can read the one or more computer programs from the computer readable storage medium, and the one or more processors execute the one or more computer programs, so that the computer device can be executed to complete the text acquisition method in the above embodiments.

[0319] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed to relevant hardware by a program. Optionally, the program is stored in a computer readable storage medium, and the storage medium mentioned above is a read-only memory, a magnetic disk or an optical disk and the like.

[0320] The above only describes optional embodiments of the present application and is not used to limit the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A text acquisition method, characterized in that, The method includes: Based on any original text in the first language, multiple candidate texts in the second language are translated, wherein the original text and the candidate texts are semantically related. The quality coefficient and importance coefficient of each of the multiple candidate texts are obtained. The quality coefficient represents the translation quality of the candidate text, and the importance coefficient is calculated using the likelihood value of the back translation model and the likelihood value of the monolingual model. It represents the importance of the information carried by the candidate text and reflects the role it plays when the candidate text is used as pseudo-corpus in the training process of the forward translation model. The back translation model is used to translate the input text from the first language to the second language, and the forward translation model is used to translate the input text from the second language to the first language. The quality coefficient and the importance coefficient are mutually exclusive. Based on the quality coefficient and the importance coefficient, the selection weight coefficients of each of the multiple candidate texts are determined, and the selection weight coefficients are screening indicators that balance the quality coefficient and the importance coefficient; based on the selection weight coefficients, target candidate texts are selected from the multiple candidate texts.

2. The method according to claim 1, characterized in that, The determination of the selection weight coefficients for each of the multiple candidate texts based on the quality coefficient and the importance coefficient includes: For any of the candidate texts, determine a first value obtained by multiplying the first adjustment factor by the importance coefficient of the candidate text; Determine a second value obtained by multiplying the second adjustment factor by the quality coefficient of the candidate text, wherein the second adjustment factor is added to the first adjustment factor and equals 1; Add the first value and the second value to get the third value; Based on the third value of the candidate text and the third value of other candidate texts, the selection weight coefficient of the candidate text is determined, wherein the other candidate texts are candidate texts other than the candidate texts obtained by translating the original text.

3. The method according to claim 2, characterized in that, The determination of the selection weight coefficient for the candidate text based on the third value of the candidate text and the third values ​​of other candidate texts includes: Using the third value of the candidate text as the first exponent, obtain the first exponent raised to the power of the natural base; For any of the other candidate texts, the third value of the other candidate text is used as the second exponent to obtain the second exponent raised to the power of the natural base; The target sum is obtained by adding the first power of the natural base to the second power of multiple natural bases. The selection weight coefficient of the candidate text is obtained by dividing the first exponent of the natural base by the target sum value.

4. The method according to claim 1, characterized in that, The step of filtering the target candidate text from the plurality of candidate texts based on the selection weight coefficient includes: From the plurality of candidate texts, the candidate text with the largest selection weight coefficient is determined as the target candidate text.

5. The method according to claim 1, characterized in that, The step of filtering the target candidate text from the plurality of candidate texts based on the selection weight coefficient includes: The target candidate text is obtained by randomly sampling from the plurality of candidate texts according to the probability distribution followed by the selection weight coefficients.

6. The method according to claim 1, characterized in that, The process of obtaining the quality coefficients of the multiple candidate texts includes: For any of the candidate texts, obtain the initial quality coefficient of the candidate text; Based on the text length of the candidate text, the initial quality coefficient is normalized to obtain the quality coefficient, where the text length represents the number of characters contained in the candidate text.

7. The method according to claim 6, characterized in that, The initial quality coefficient is normalized based on the text length of the candidate text to obtain the quality coefficient, which includes: Based on the logarithmic quality coefficient of the initial quality coefficient of each of the plurality of candidate texts, the first mean and the first variance of the logarithmic quality coefficient are obtained, wherein the logarithmic quality coefficient is the logarithmic value of the initial quality coefficient; The normalized quality coefficient is obtained by dividing the logarithmic quality coefficient of the candidate text by the length of the candidate text. The quality coefficient is obtained by subtracting the normalized quality coefficient from the first mean and dividing the result by the first variance.

8. The method according to claim 1, characterized in that, The process of obtaining the importance coefficients of the multiple candidate texts includes: For any of the candidate texts, obtain the initial importance coefficient of the candidate text; Based on the text length of the candidate text, the initial importance coefficient is normalized to obtain the importance coefficient, where the text length represents the number of characters contained in the candidate text.

9. The method according to claim 8, characterized in that, The initial importance coefficient is normalized based on the text length of the candidate text to obtain the importance coefficient, which includes: Based on the logarithmic importance coefficients of the initial importance coefficients of the plurality of candidate texts, the second mean and the second variance of the logarithmic importance coefficients are obtained, wherein the logarithmic importance coefficients are the logarithmic values ​​of the initial importance coefficients; The normalized importance coefficient is obtained by dividing the logarithmic importance coefficient of the candidate text by the text length of the candidate text. The importance coefficient is obtained by subtracting the normalized importance coefficient from the second mean and then dividing the result by the second variance.

10. The method according to claim 1, characterized in that, The multiple candidate texts in the second language obtained by translating any original text in the first language include: The original text is input into the reverse translation model, and the multiple candidate texts are obtained by random sampling through the reverse translation model.

11. The method according to claim 1, characterized in that, The corpus consisting of the original text and the target candidate text is used to train the forward translation model.

12. A text acquisition device, characterized in that, The device includes: A translation module is used to translate any original text in a first language into multiple candidate texts in a second language, wherein the original text and the candidate texts are semantically related. The acquisition module is used to acquire the quality coefficient and importance coefficient of each of the multiple candidate texts. The quality coefficient represents the translation quality of the candidate text, and the importance coefficient is calculated using the likelihood value of the back translation model and the likelihood value of the monolingual model. It represents the importance of the information carried by the candidate text and reflects the role it plays when the candidate text is used as pseudo-corpus in the training process of the forward translation model. The back translation model is used to translate the input text from the first language to the second language, and the forward translation model is used to translate the input text from the second language to the first language. The quality coefficient and the importance coefficient are mutually exclusive. The filtering module includes: a determining unit, configured to determine the selection weight coefficients of each of the plurality of candidate texts based on the quality coefficient and the importance coefficient, wherein the selection weight coefficients are filtering indicators that balance the quality coefficient and the importance coefficient; and a filtering unit, configured to filter target candidate texts from the plurality of candidate texts based on the selection weight coefficients.

13. The apparatus according to claim 12, characterized in that, The determining unit includes: The first determining subunit is used to determine, for any candidate text, a first value obtained by multiplying the first adjustment factor by the importance coefficient of the candidate text; The first determining subunit is further configured to determine a second value obtained by multiplying the second adjustment factor by the quality coefficient of the candidate text, wherein the second adjustment factor and the first adjustment factor are added together to equal 1; The addition unit is used to add the first value and the second value to obtain the third value; The second determining subunit is used to determine the selection weight coefficient of the candidate text based on the third value of the candidate text and the third values ​​of other candidate texts, wherein the other candidate texts are candidate texts other than the candidate texts obtained by translating the original text.

14. The apparatus according to claim 13, characterized in that, The second determining subunit is used for: Using the third value of the candidate text as the first exponent, obtain the first exponent raised to the power of the natural base; For any of the other candidate texts, the third value of the other candidate text is used as the second exponent to obtain the second exponent raised to the power of the natural base; The target sum is obtained by adding the first power of the natural base to the second power of multiple natural bases. The selection weight coefficient of the candidate text is obtained by dividing the first exponent of the natural base by the target sum value.

15. The apparatus according to claim 12, characterized in that, The filtering unit is used for: From the plurality of candidate texts, the candidate text with the largest selection weight coefficient is determined as the target candidate text.

16. The apparatus according to claim 12, characterized in that, The filtering unit is used for: The target candidate text is obtained by randomly sampling from the plurality of candidate texts according to the probability distribution followed by the selection weight coefficients.

17. The apparatus according to claim 12, characterized in that, The acquisition module includes: The first acquisition unit is used to acquire the initial quality coefficient of any candidate text. The first normalization unit is used to normalize the initial quality coefficient based on the text length of the candidate text to obtain the quality coefficient, wherein the text length represents the number of characters contained in the candidate text.

18. The apparatus according to claim 17, characterized in that, The first normalization unit is used for: Based on the logarithmic quality coefficient of the initial quality coefficient of each of the plurality of candidate texts, the first mean and the first variance of the logarithmic quality coefficient are obtained, wherein the logarithmic quality coefficient is the logarithmic value of the initial quality coefficient; The normalized quality coefficient is obtained by dividing the logarithmic quality coefficient of the candidate text by the length of the candidate text. The quality coefficient is obtained by subtracting the normalized quality coefficient from the first mean and dividing the result by the first variance.

19. The apparatus according to claim 12, characterized in that, The acquisition module includes: The second acquisition unit is used to acquire the initial importance coefficient of any candidate text. The second normalization unit is used to normalize the initial importance coefficient based on the text length of the candidate text to obtain the importance coefficient, wherein the text length represents the number of characters contained in the candidate text.

20. The apparatus according to claim 19, characterized in that, The second normalization unit is used for: Based on the logarithmic importance coefficients of the initial importance coefficients of the plurality of candidate texts, the second mean and the second variance of the logarithmic importance coefficients are obtained, wherein the logarithmic importance coefficients are the logarithmic values ​​of the initial importance coefficients; The normalized importance coefficient is obtained by dividing the logarithmic importance coefficient of the candidate text by the text length of the candidate text. The importance coefficient is obtained by subtracting the normalized importance coefficient from the second mean and then dividing the result by the second variance.

21. The apparatus according to claim 12, characterized in that, The translation module is used for: The original text is input into the reverse translation model, and the multiple candidate texts are obtained by random sampling through the reverse translation model.

22. The apparatus according to claim 12, characterized in that, The corpus consisting of the original text and the target candidate text is used to train the forward translation model.

23. A computer device, characterized in that, The computer device includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement the text acquisition method as described in any one of claims 1 to 11.

24. A storage medium, characterized in that, The storage medium stores at least one computer program, which is loaded and executed by a processor to implement the text acquisition method as described in any one of claims 1 to 11.

25. A computer program product, characterized in that, The computer program product includes at least one computer program, which is loaded and executed by a processor to implement the text acquisition method as described in any one of claims 1 to 11.

Images