Text translation method, device, computer device and storage medium

Abstract

The present application discloses a text translation method, apparatus, computer device, and storage medium, belonging to the field of computer technology. In the present application, a coarse-grained sample sentence is split into fine-grained sample phrases and even finer-grained sample characters, and the sample characters are used to enable the student model to first learn relatively simple and easy-to-learn character knowledge, then the sample phrases are used to enable the first translation model to learn more complex phrase knowledge, and finally the sample sentences are used to enable the second translation model to learn the most complex sentence knowledge. Since the target translation model supports parallel translation of input characters, the translation speed of the target translation model will not be affected, and without additionally introducing the amount of training data and model parameters, the translation accuracy of the target translation model is greatly improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a text translation method, apparatus, computer device, and storage medium. Background Art

[0002] With the development of computer technology, in the field of machine translation, machine translation methods based on deep learning have gradually become a hot topic. Currently, the mainstream neural machine translation model is the autoregressive machine translation (AT) model, that is, the generation of each translated character depends on the previous translation result. Therefore, the AT model can only generate the translation word by word, that is, it supports serial translation, and the translation speed is slow.

[0003] On this basis, the non-autoregressive machine translation (NAT) model came into being. The NAT model can parallelly decode the entire sentence translation, that is, it supports parallel translation, and can significantly improve the translation speed of the model. However, the cost is lower translation accuracy, mainly manifested as poor translation effects on long sentences, poor performance on languages with large word order differences, and lower translation accuracy on one-to-many translation tasks. Therefore, there is an urgent need for a method that can improve the translation accuracy of machine translation models that support parallel translation, such as the NAT model, without affecting the translation speed of machine translation models that support parallel translation, such as the NAT model. Summary of the Invention

[0004] The embodiments of this application provide a text translation method, apparatus, computer device, and storage medium, which can improve the translation accuracy of machine translation models that support parallel translation, such as the NAT model, without affecting the translation speed. The technical solution is as follows:

[0005] On the one hand, a text translation method is provided. The method includes:

[0006] Obtain sample characters and sample phrases in a sample sentence;

[0007] Based on the sample characters, adjust the parameters of the student model to obtain a first translation model, where the student model supports parallel translation processing of multiple input characters;

[0008] Based on the sample phrases, adjust the parameters of the first translation model to obtain a second translation model;

[0009] Based on the sample sentence, adjust the parameters of the second translation model to obtain a target translation model, where the target translation model is used to convert an input sentence into a translated sentence with the same semantics.

[0010] On the one hand, a text translation device is provided, which includes:

[0011] An acquisition module, configured to acquire sample characters and sample phrases in a sample statement;

[0012] A first adjustment module, configured to adjust parameters of a student model based on the sample characters to obtain a first translation model, where the student model supports parallel translation processing of multiple input characters;

[0013] A second adjustment module, configured to adjust parameters of the first translation model based on the sample phrases to obtain a second translation model;

[0014] A third adjustment module, configured to adjust parameters of the second translation model based on the sample statement to obtain a target translation model, where the target translation model is used to convert an input statement into a translated statement with the same semantics.

[0015] In a possible implementation manner, the first adjustment module is configured to:

[0016] Call the student model to translate the sample characters to obtain translated characters;

[0017] Determine a first loss value based on reference characters and the translated characters, where the reference characters have the same semantics as the sample characters;

[0018] Adjust parameters of the student model based on the first loss value to obtain the first translation model.

[0019] In a possible implementation manner, the device further includes:

[0020] A first translation module, configured to call a teacher model to translate the sample characters to obtain the reference characters, where the teacher model supports serial translation processing of multiple input characters.

[0021] In a possible implementation manner, the device further includes:

[0022] A second translation module, configured to call a teacher model to translate the sample statement to obtain a reference statement, where the teacher model supports serial translation processing of multiple input characters;

[0023] A first determination module, configured to determine character alignment information based on the sample statement and the reference statement, where the character alignment information is used to characterize whether characters in the sample statement and corresponding characters in the reference statement have the same semantics;

[0024] A second determination module, configured to determine the reference character from the reference statement based on the character alignment information.

[0025] In a possible implementation manner, the second adjustment module is configured to:

[0026] Call the first translation model to translate the sample phrase, and obtain a translated phrase;

[0027] Determine a second loss value based on the reference phrase and the translated phrase, where the reference phrase has the same semantics as the sample phrase;

[0028] Adjust the parameters of the first translation model based on the second loss value to obtain the second translation model.

[0029] In a possible implementation manner, the apparatus further includes:

[0030] A third translation module, configured to call a teacher model to translate the sample phrase, and obtain the reference phrase, where the teacher model supports serial translation processing of multiple input characters.

[0031] In a possible implementation manner, the apparatus further includes:

[0032] A second translation module, configured to call a teacher model to translate the sample statement, and obtain a reference statement, where the teacher model supports serial translation processing of multiple input characters;

[0033] A first determination module, configured to determine character alignment information based on the sample statement and the reference statement, where the character alignment information is used to characterize whether the characters in the sample statement and the corresponding characters in the reference statement have the same semantics;

[0034] A third determination module, configured to determine, from the reference statement, multiple reference characters that have the same semantics as multiple sample characters included in the sample phrase based on the character alignment information;

[0035] A fourth determination module, configured to determine the multiple reference characters as the reference phrase.

[0036] In a possible implementation manner, the third adjustment module is configured to:

[0037] Call the second translation model to translate the sample statement, and obtain a translated statement;

[0038] Determine a third loss value based on the reference statement and the translated statement, where the reference statement has the same semantics as the sample statement;

[0039] Based on the third loss value, adjust the parameters of the second translation model to obtain the target translation model.

[0040] In a possible implementation manner, the apparatus further includes:

[0041] A second translation module, configured to call a teacher model to translate the sample statement to obtain the reference statement, where the teacher model supports serial translation processing of multiple input characters.

[0042] In a possible implementation manner, the obtaining module includes:

[0043] A word segmentation processing unit, configured to perform word segmentation processing on the sample statement to obtain the sample characters;

[0044] A phrase extraction unit, configured to extract phrases from the sample statement to obtain the sample phrases.

[0045] In a possible implementation manner, the phrase extraction unit is configured to:

[0046] Obtain character alignment information between the sample statement and the reference statement, where the character alignment information is used to indicate whether the characters in the sample statement and the corresponding characters in the reference statement have the same semantics;

[0047] Based on the character alignment information, obtain the sample phrases that meet the target conditions, where the target conditions are that at least one character included in the sample phrase is indicated in the character alignment information as having the same semantics as the corresponding character in the reference statement.

[0048] On the one hand, a computer device is provided, which includes one or more processors and one or more memories. At least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement the text translation method as described above.

[0049] On the one hand, a storage medium is provided, in which at least one computer program is stored, and the at least one computer program is loaded and executed by a processor to implement the text translation method as described above.

[0050] On the one hand, a computer program product or a computer program is provided. The computer program product or the computer program includes one or more program codes, and the one or more program codes are stored in a computer-readable storage medium. One or more processors of a computer device can read the one or more program codes from the computer-readable storage medium, and the one or more processors execute the one or more program codes, so that the computer device can execute the text translation method as described above.

[0051] The beneficial effects brought by the technical solutions provided in the embodiments of the present application at least include:

[0052] By splitting the coarse-grained sample sentences into fine-grained sample phrases and even finer-grained sample characters, and using the sample characters to enable the student model to first learn relatively simple and easy-to-learn character knowledge, then using the sample phrases to enable the first translation model to learn more complex phrase knowledge, and finally using the sample sentences to enable the second translation model to learn the most complex sentence knowledge. Since the target translation model supports parallel translation of input characters, the translation speed of the target translation model will not be affected, and without introducing additional training data volume and model parameters, the translation accuracy of the target translation model is greatly improved. Description of the Drawings

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0054] Figure 1 It is a schematic diagram of the principle of an encoding and decoding model framework provided by the embodiments of the present application;

[0055] Figure 2 It is a comparison diagram between a SAN model and an RNN model provided by the embodiments of the present application;

[0056] Figure 3 It is a comparison diagram between an AT model and a NAT model provided by the embodiments of the present application;

[0057] Figure 4 It is a schematic diagram of the implementation environment of a text translation method provided by the embodiments of the present application;

[0058] Figure 5 It is a flowchart of a text translation method provided by the embodiments of the present application;

[0059] Figure 6 It is a flowchart of the training process of a translation model provided by the embodiments of the present application;

[0060] Figure 7 It is a schematic diagram of the principle of the framework of a teacher model provided by the embodiments of the present application;

[0061] Figure 8 It is a schematic diagram of the principle of the framework of a second translation model provided by the embodiments of the present application;

[0062] Figure 9It is a performance comparison chart of training a NAT model using the PMG training strategy and the traditional training strategy provided by an embodiment of the present application;

[0063] Figure 10 It is a schematic structural diagram of a text translation device provided by an embodiment of the present application;

[0064] Figure 11 It is a schematic structural diagram of a computer device provided by an embodiment of the present application;

[0065] Figure 12 It is a schematic structural diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0066] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0067] In the present application, terms such as "first" and "second" are used to distinguish identical or similar items with basically the same functions and effects. It should be understood that there is no logical or chronological dependency between "first", "second", and "nth", nor are the quantity and execution order limited.

[0068] In the present application, the term "at least one" means one or more, and the meaning of "multiple" means two or more. For example, multiple first positions mean two or more first positions.

[0069] Artificial Intelligence (AI) uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, and is a theory, method, technology, and application system that perceives the environment, acquires knowledge, and uses knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines to enable the machines to have the functions of perception, reasoning, and decision-making.

[0070] Artificial intelligence technology is an interdisciplinary subject involving a wide range of fields, including both hardware-level technologies and software-level technologies. Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technologies mainly include several major directions such as audio processing technology, computer vision technology, natural language processing technology, and machine learning / deep learning.

[0071] Enabling computers to listen, see, speak, and sense is the future development direction of human-computer interaction. Among them, Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, the research in this field will involve natural language, that is, the language people use in daily life, so it has a close connection with the research of linguistics. Natural language processing technologies usually include machine translation, text preprocessing, semantic understanding, robot question answering, knowledge graphs, and other technologies.

[0072] In the field of NLP, machine translation (MT, also known as automatic translation) refers to the process of using a computer to convert one natural language (source language) into another natural language (target language). It is a branch of computational linguistics and one of the ultimate goals of artificial intelligence, with important scientific research value. At the same time, machine translation also has important practical value. With the rapid development of economic globalization and the Internet, machine translation technology is playing an increasingly important role in promoting political, economic, and cultural exchanges. In machine translation, translation models based on deep learning have achieved good success.

[0073] The following will explain the terms involved in the embodiments of this application:

[0074] Deep Learning (DL): It is a branch of machine learning and an algorithm that attempts to use multiple processing layers containing complex structures or composed of multiple non-linear transformations to perform high-level abstractions on data.

[0075] Neural Network (NN): A deep learning model in the fields of machine learning and cognitive science that mimics the structure and function of biological neural networks.

[0076] Machine Translation (MT): Using devices such as electronic computers to automatically translate one natural language text into another natural language text.

[0077] Statistical Machine Translation (SMT): A traditional machine translation technology based on Bayesian probability statistics that can use statistical features to map and achieve text conversion at the character level or phrase level.

[0078] Neural Machine Translation (NMT): A machine translation technology based on neural networks.

[0079] Cross-Attention Mechanism: Also known as the Encoder-Decoder Attention mechanism, a method for modeling the dependency relationship between the hidden states of the encoder and decoder in a neural network, or a neural network structure that focuses on the encoder representation starting from the decoder.

[0080] Figure 1 It is a schematic diagram of an encoding and decoding model framework provided by an embodiment of this application. As Figure 1 shown, it shows an encoder-decoder model framework. Between the encoder 120 and the decoder 140, there is also a cross-attention module 130. The cross-attention module 130 is used to extract the dependency relationship between the hidden states of the encoder 120 and the decoder 140.

[0081] Self-Attention Network (SAN): A neural network structure model based on the self-attention mechanism.

[0082] Figure 2 It is a comparison diagram between a SAN model and an RNN model provided by an embodiment of this application. As Figure 2 shown, the left part 201 shows the data processing flow of the RNN (Recurrent Neural Network) model, and the right part 202 shows the data processing flow of the SAN model. It can be seen that the neurons in adjacent hidden layers in the SAN model are fully connected.

[0083] Sequence to Sequence Model (Seq2Seq): A deep neural network model where both the input and output are sequences.

[0084] Tokens: The smallest semantic unit in machine translation, i.e., word segmentation. In English, a token refers to a word (i.e., vocabulary) or an English punctuation mark. In Chinese, a token refers to a Chinese word (a word may include one or more Chinese characters) or a Chinese punctuation mark.

[0085] Phrase Table: A list used to represent the corresponding relationship between phrases in multiple languages.

[0086] Bilingual Evaluation Understudy (BLEU): It is an evaluation metric for machine translation models. The higher the BLEU value, the better the translation effect of the machine translation model.

[0087] In recent years, deep neural networks have become a core model in the field of artificial intelligence, widely used in fields such as speech recognition, computer vision, and natural language processing. And neural network models have performed excellently in various tasks, especially in the machine translation task. In the field of machine translation, the NMT model based on the encoder-decoder framework has developed by leaps and bounds. The performance of the NMT model far exceeds that of the traditional SMT model. However, as the depth of the NMT model increases and the data flow becomes larger, the decoding efficiency of the NMT model has gradually become one of the main problems in application.

[0088] The mainstream NMT model adopts the autoregressive translation (AT) mechanism. That is, the AT model generates the translation word by word. The generation of each translated character at each step depends on the previous translation result. That is, the AT model supports serial translation. Therefore, the translation speed of the AT model is relatively slow. On this basis, the non-autoregressive machine translation (NAT) model emerges as the times require. Since the NAT model independently models the generation of each translated character, it can parallelly decode the entire sentence translation. That is, the NAT model supports parallel translation and can significantly improve the translation speed of the model.

[0089] Figure 3 It is a comparison diagram of an AT model and a NAT model provided by an embodiment of the present application. As Figure 3 shown, the left part 301 shows the data processing flow of the AT model. It can be seen that the generation of each translated character at each step of the AT model depends on the previous translation result. The right part 302 shows the data processing flow of the NAT model. It can be seen that the NAT model independently models the generation of each translated character, so as to be able to generate the entire sentence translation simultaneously.

[0090] The standard AT model directly optimizes the conditional probability of its target-side sentence y = y1,..., y I under the given source-side sentence x = x1,..., x J :

[0091]

[0092] where θ is the parameter of the AT model, and y < j is y jThe previous translation above. The probability P(y|x) defines a neural network-based encoder-decoder framework. The training process of the parameters θ of the AT model is to maximize the likelihood of the training samples :

[0093]

[0094] where represents the loss value, and M represents the sample size.

[0095] Ideally, the conditional probability of the NAT model is free from the conditional dependence of y < j :

[0096]

[0097] It should be noted that the "source side" involved in the embodiments of the present application refers to the untranslated characters, phrases or sentences on the source language side, and the "target side" refers to the translated characters, phrases or sentences on the target language side. Therefore, the translation process is to convert the sentence to be translated from the source language into the target language.

[0098] It can be seen that during training, the encoders of the AT model and the NAT model are the same, while the decoders are different. Regarding the length of the sentence on the target side, the AT model implicitly predicts the sentence length by predicting the end symbol <eos>, while the NAT model explicitly predefines the output sentence length. The decoder input of the AT model requires the shift mask technique to mask the content after time t, while the NAT model does not.

[0099] The AT model mainly has two problems. One is that the decoding speed of the AT model will slow down as the complexity of the AT model increases. The other is that the dependent decoding method of the AT model is prone to the error propagation problem, that is, if the character prediction at a certain moment is incorrect, this error will be propagated to each moment after that moment.

[0100] As shown in Table 1, the decoding durations of the 6-layer NMT model and the 12-layer NMT model are respectively shown. It can be seen that compared with the 6-layer NMT model, the 12-layer NMT model increases the decoding duration by 68% for each sentence, that is, the decoding speed of the AT model will slow down as the complexity of the AT model increases.

[0101] Table 1

[0102] NMT model Decoding duration (ms / millisecond) Average sentence length 6 layers 210 4.76 12 layers 353 2.83

[0103] As shown in Table 2, the translation accuracy rates of the first 4 characters and the last 4 characters are respectively shown for the left-to-right (L2R) decoding method and the right-to-left (R2L) decoding method. It can be seen that the translation accuracy rate of the first 4 characters under the L2R decoding method is higher than that of the last 4 characters, indicating that the translation errors occurring in the other characters before the last 4 characters are passed word by word, resulting in a decrease in the translation accuracy rate. Similarly, the translation accuracy rate of the first 4 characters under the R2L decoding method is lower than that of the last 4 characters, indicating that the translation errors occurring in the other characters after the first 4 characters are passed word by word, resulting in a decrease in the translation accuracy rate.

[0104] Table 2

[0105] Decoding method Translation accuracy of the first 4 characters Translation accuracy of the last 4 characters L2R 40.21％ 35.10％ R2L 35.67％ 39.47％

[0106] The proposed NAT model can well alleviate the above two problems that occur in the AT model. Since the NAT model lacks the modeling of dependencies between target-side characters, there is a certain gap between the NAT model and the AT model in terms of translation performance. In other words, the cost of the NAT model significantly improving the translation speed is relatively low translation accuracy, which is mainly manifested as poor translation effects on long sentences, poor performance on languages with large word order differences, and low translation accuracy on one-to-many translation tasks, etc. In view of this, the embodiments of the present application relate to a method for improving the translation accuracy of a machine translation model that supports parallel translation, such as the NAT model, without affecting the translation speed of the machine translation model that supports parallel translation, such as the NAT model, and will be described in detail below.

[0107] Figure 4 is a schematic diagram of the implementation environment of a text translation method provided by an embodiment of the present application. Refer to Figure 4 , in this implementation environment, it includes a terminal 410 and a server 420, and both the terminal 410 and the server 420 are a type of computer device.

[0108] The terminal 410 can be used to provide the text to be translated. After the user imports the text to be translated on the terminal 410, the terminal 410 generates a translation request carrying the text to be translated, and sends the translation request to the server 420, so that the server 420 responds to the translation request, translates the text to be translated, obtains the translated text of the text to be translated, and returns the translated text to the terminal 410.

[0109] The terminal 410 and the server 420 can be directly or indirectly connected through wired or wireless communication means, and the present application does not make any restrictions here.

[0110] Server 420 can be used to provide machine translation services. Taking the teacher-student model training framework as an example, with the AT model as the teacher model and the NAT model as the student model, the server 420 uses sample corpora (including sample characters, sample phrases, and sample sentences) to progressively train the student model in a multi-granularity manner under the guidance of the teacher model. That is, the student model first learns characters, then phrases, and finally sentences. Since the sample characters and sample phrases are both derived from the sample sentences, it is possible to shorten the performance gap between the NAT model and the AT model without introducing additional data and parameters, making machine translation models that support parallel translation, such as the NAT model, more valuable. Here, the NAT model is only an example of a machine translation model that supports parallel translation, and the AT model is only an example of a machine translation model that supports serial translation. The trained student model can be called the target translation model, and the training process refers to adjusting the parameters of the student model (which can be abbreviated as "parameter tuning") to obtain the target translation model.

[0111] Optionally, the server 420 only maintains the target translation model on its own server side. When the server 420 receives a translation request from the terminal 410, it can call the target translation model to translate the text to be translated carried in the translation request, obtain the translated text of the text to be translated, and return the translated text to the terminal 410. By migrating the translation process to the server 420, it is possible to save the local processing resources and computing resources of the terminal 410.

[0112] Optionally, after training the target translation model, the server 420 sends the target translation model to the terminal 410 so that the terminal 410 can locally call the target translation model to translate the text to be translated by itself, obtain the translated text of the text to be translated, and there is no need to send a translation request to the server 420 every time, which can reduce the communication overhead between the terminal 410 and the server 420.

[0113] Optionally, the terminal 410 can also locally train the target translation model and then locally call the target translation model to translate the text to be translated by itself, obtain the translated text of the text to be translated, which can reduce the communication overhead between the terminal 410 and the server 420.

[0114] The server 420 can include at least one of a single server, multiple servers, a cloud computing platform, or a virtualization center. Optionally, the server 420 can undertake the main computing work, and the terminal 410 can undertake the secondary computing work; or, the server 420 undertakes the secondary computing work, and the terminal 410 undertakes the main computing work; or, the terminal 410 and the server 420 use a distributed computing architecture for collaborative computing.

[0115] The server 420 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0116] The terminal 410 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, an e-book reader, etc., but is not limited thereto.

[0117] Those skilled in the art can understand that the terminal 410 can generally refer to one of multiple terminals, and the number of the above terminals can be more or less. For example, the above terminal can be only one, or dozens or hundreds of the above terminals, or a larger number. The embodiments of the present application do not limit the number and device type of the terminals.

[0118] Hereinafter, the core concept of the embodiments of the present application will be described:

[0119] In the training process of a general translation model, the initial model is trained at the sentence level with sentences as units. However, in human translation practice, it is usually inclined to first translate relatively simple characters or phrases and then adjust the word order to translate the entire sentence. Inspired by this way of thinking, for the original sample sentence, the sample sentence can be split into sample characters and sample phrases, that is, the sentence-level training data is split into character-level training data and phrase-level training data. The sample characters, sample phrases, and sample sentences can form three levels of training data from fine-grained to coarse-grained, enriching the levels and expressive capabilities of the training data without additionally increasing the amount of training data.

[0120] It is assumed that fine-grained sample characters are easier for the NAT model (student model) to learn. As shown in Table 3, for the three granularities of characters, phrases, and sentences respectively, the AT model and the NAT model are used for translation to obtain the translation accuracy in the corresponding cases. The NAT model is further divided into the NAT model trained in the traditional (Raw) way and the NAT model trained based on knowledge distillation (KD). Among them, knowledge distillation refers to transferring the knowledge learned in a complex model (Teacher, teacher model) to a simple model (Student, student model), that is, training the student model with the synthetic data refined by the teacher model to reduce the data complexity. Generally speaking, the teacher model has powerful capabilities and performance, while the student model is very small in size. Through knowledge distillation, it is hoped that the student model can approximate or even exceed the performance of the teacher model as much as possible, so as to obtain similar prediction effects with less complexity and achieve model compression and quantization. Among them, the NAT model is only an example of a machine translation model that supports parallel translation, and the AT model is also only an example of a machine translation model that supports serial translation.

[0121] Table 3

[0122]

[0123] Among them, △ refers to the gap between the accuracy of the NAT model and the accuracy of the AT model.

[0124] It can be seen from Table 3 that although on the basis of the Raw NAT model, training the NAT model by knowledge distillation of the AT model can improve the translation performance at three granularities to a certain extent (manifested by the reduction of the absolute value of △ in the KD NAT model), there is still a certain gap between the translation performance of the KD NAT model and the AT model. In addition, since the absolute value of the accuracy gap △ at the character granularity is less than the absolute value of the accuracy gap △ at the phrase granularity, and the absolute value of the accuracy gap △ at the phrase granularity is less than the absolute value of the accuracy gap △ at the sentence granularity, that is, 0.8 < 1.8 < 2.2, it shows that finer-grained training data is easier for the NAT model to learn. In addition, there are still complex word orders and structures in the training data synthesized by knowledge distillation, which makes it still very difficult for the NAT model to learn when trained in the KD way.

[0125] By predicting the entire sentence translation, the NAT model can significantly accelerate the inference process of the NAT model, but it is very weak in learning multi-modal knowledge (such as one-to-many translation). Since bilingual knowledge is the core of sufficiency modeling, in the absence of autoregressive factorization, the NAT model also performs poorly when dealing with languages with long sequences and large word order differences.

[0126] On this basis, if the training patterns are divided into different granularities, machine translation models that support parallel translation, such as the NAT model, can learn from easy to difficult, and can truly simulate the human learning process of language from easy to difficult. Compared with the sentence level, machine translation models that support parallel translation, such as the NAT model, are more likely to learn fine-grained, low-mode knowledge, such as characters and phrases. Therefore, the original training corpus (i.e., sample sentences) can be decomposed into training data of three granularities: characters, phrases, and sentences, and the granularity can be gradually increased as the training progresses. By decomposing the high-mode at the sentence level into finer granularity, that is, low-mode data such as bilingual characters and phrases, it is easier for the model to better learn bilingual knowledge.

[0127] As shown in Table 4, some examples are selected to illustrate the difficulty of translation at different granularities. Sentence-level bilingual knowledge contains a variety of patterns, such as word alignment ("English" vs "English" / "English"), phrase translation ("be good at" vs "...very good at..." / "...high level"), and even reordering ("English" can be both subject and object). However, phrase-level bilingual knowledge generally has similar structures, so the patterns are much simpler than sentence-level bilingual knowledge, and character-level bilingual knowledge is a simpler one-to-one mapping. Generally speaking, the lower the granularity of bilingual knowledge, the easier it is for the NAT model to learn. The examples in Table 4 also reveal why the sentence-level difference between the NAT model and the AT model is more significant than the lexical and phrase levels in Table 3.

[0128] Table 4

[0129]

[0130] Based on the above analysis, it can be considered that directly training a machine translation model that supports parallel translation, such as a NAT model, based on sentence-level training data is not the optimal training strategy. In the embodiments of the present application, a simple and effective Progressive Multi Granularity (PMG) training strategy is adopted to improve the performance of a machine translation model that supports parallel translation, such as a NAT model, in processing sentences. The PMG training strategy is to first let a machine translation model that supports parallel translation, such as a NAT model, learn simple characters, then learn phrases, and finally learn sentence-level training data, enabling the machine translation model that supports parallel translation, such as a NAT model, to gradually learn from "low" to "high" granularity, that is, characters → phrases → sentences. Without introducing additional data and parameters, it can ultimately shorten the performance gap between a machine translation model that supports parallel translation, such as a NAT model, and a machine translation model that supports serial translation, such as an AT model, so as to improve the application value of the machine translation model that supports parallel translation, such as a NAT model. This training strategy can disassemble the original sentence-level training data from easy to difficult, enabling a machine translation model that supports parallel translation, such as a NAT model, to learn better phrase patterns and the boundaries of fine-grained bilingual phrases, thereby enhancing the phrase translation and reordering capabilities of the machine translation model that supports parallel translation, such as a NAT model, and ultimately bringing about an increase in translation performance.

[0131] Figure 5 It is a flowchart of a text translation method provided by an embodiment of the present application. Refer to Figure 5 This embodiment is applied to a computer device. Here, taking the server 420 in the above-mentioned implementation environment as an example of the computer device for illustration, this embodiment includes the following steps:

[0132] 501. The server obtains sample characters and sample phrases in the sample sentence.

[0133] In some embodiments, the server can obtain the sample sentence in the training corpus. The number of the sample sentences is at least one. The training corpus can be stored locally on the server or in other distributed storage systems. The embodiments of the present application do not specifically limit this.

[0134] In some embodiments, when obtaining the sample characters, the server can perform a tokenize process on the sample sentence to obtain the sample characters. In some embodiments, when obtaining the sample phrases, the server can perform phrase extraction on the sample sentence to obtain the sample phrases.

[0135] In the above process, by preprocessing the sample sentences, the sample characters and sample phrases in the sample sentences can be segmented. This segmentation process does not introduce additional training data, but only reprocesses the original training data, yet it can greatly enrich the representation levels and capabilities of the training data.

[0136] Optionally, when the server performs word segmentation, it can automatically implement word segmentation using a word segmentation tool, or it can also be manually segmented by technicians. The embodiments of the present application do not specifically limit the word segmentation method.

[0137] Optionally, when the server extracts phrases, it can obtain the character alignment information between the sample sentence and the reference sentence. This character alignment information is used to indicate whether the characters in the sample sentence have the same semantics as the corresponding characters in the reference sentence; based on this character alignment information, the sample phrase that meets the target condition is obtained, and the target condition is that at least one character included in the sample phrase is indicated in the character alignment information as having the same semantics as the corresponding character in the reference sentence.

[0138] Optionally, when obtaining the reference sentence, the server can call the teacher model to translate the sample sentence to obtain the reference sentence. The teacher model supports serial translation processing of multiple input characters, that is, the teacher model is used to translate the input characters word by word. The teacher model is essentially an AT model. In the above process, by inputting the sample sentence into the teacher model and translating the sample sentence through the teacher model to output the reference sentence, the efficiency of obtaining the reference sentence can be greatly improved. In some other embodiments, the reference sentence can also be manually marked by technicians. The embodiments of the present application do not specifically limit this. Optionally, the teacher model can be any AT model. For example, the teacher model is a Transformer translation model or a variant of the Transformer translation model.

[0139] In the above process, through the character alignment information, the server can accurately screen out the sample characters in the sample sentence that can form the sample phrase, so as to achieve automatic extraction of the sample phrase by the machine and improve the extraction efficiency of the sample phrase. In some other embodiments, technicians can also manually extract phrases. The embodiments of the present application do not specifically limit the phrase extraction method.

[0140] In an exemplary scenario, since the SMT model needs to perform word segmentation (i.e., preprocessing), word alignment, and construct a phrase table during translation, the SMT model can be used to obtain the sample characters and the sample phrases. That is, the word segmentation tool of the SMT model is used for word segmentation, and the script tool GIZA++ is used to obtain the word alignment bit matrix (i.e., character alignment information). The bit matrix is a bitmap matrix, and each element in the bit matrix is used to represent whether a sample character in the sample sentence is aligned with the corresponding reference character in the reference sentence (i.e., has the same semantics). For example, an element value of 1 represents alignment, and an element value of 2 represents non-alignment. Based on the bit matrix, sample phrases can be extracted. The sample phrases need to meet the target conditions, which can be simply summarized as closure and non-emptiness. Closure means that the aligned parties of the sample characters included in the sample phrase must be in the corresponding reference phrase without leakage; non-emptiness means that at least one sample character in the sample phrase is aligned with the reference character (with a value of 1 in the bit matrix). Optionally, when specifying the phrase length of the sample phrase, the bit matrix can be used to find all sample phrases that meet the target conditions. For example, when the phrase length is Len, the bit matrix is horizontally slid with Len as the window size, and each phrase formed by the sample characters within the sliding window is checked one by one to see if it meets the target conditions. If it meets the target conditions, it is determined that the sample characters within the window form a sample phrase; otherwise, it continues to slide to the right.

[0141] 502. The server adjusts the parameters of the student model based on the sample characters to obtain a first translation model. The student model supports parallel translation processing of multiple input characters.

[0142] Among them, the student model supporting parallel translation processing of multiple input characters means that the student model does not translate word by word, that is, the student model is a NAT model or a variant of the NAT model.

[0143] In some embodiments, the process of the server adjusting the parameters of the student model, that is, the process of training the student model, may include: calling the student model to translate the sample characters to obtain translated characters; determining a first loss value based on the reference characters and the translated characters, where the reference characters have the same semantics as the sample characters; and adjusting the parameters of the student model based on the first loss value to obtain the first translation model.

[0144] Optionally, the server may input the sample character into the student model, translate the sample character through the student model, and output the translated character. Optionally, obtain the cross-entropy loss between the reference character and the translated character as the first loss value. If the first stopping condition is not met, iteratively adjust the parameters of the student model based on the backpropagation algorithm until the first stopping condition is met, and then stop the iteration to obtain the first translation model. Optionally, the first stopping condition includes: the first loss value is less than or equal to the first loss threshold, or the number of iterations is greater than the first number threshold, where the first loss threshold may be any value greater than or equal to 0 and less than or equal to 1, and the first number threshold may be any integer greater than or equal to 1. For example, the first loss threshold is 0.2 and the first number threshold is 50000.

[0145] In some embodiments, when the server obtains the reference character, it may directly call the teacher model to translate the sample character, that is: the server calls the teacher model to translate the sample character to obtain the reference character. The teacher model supports serial translation processing of multiple input characters, that is, the teacher model is an AT model or a variant of the AT model.

[0146] In the above process, the teacher model is used to translate the sample character to obtain the reference character, which can improve the acquisition efficiency of the reference character. In some other embodiments, the reference character may also be manually labeled by a technician. The embodiments of the present application do not specifically limit the acquisition method of the reference character.

[0147] In some embodiments, when the server obtains the reference character, it may first call the teacher model to translate the sample sentence to obtain a reference sentence, and then find the reference character corresponding to the sample character in the reference sentence according to the character alignment information, that is: the server calls the teacher model to translate the sample sentence to obtain a reference sentence. The teacher model supports serial translation processing of multiple input characters; based on the sample sentence and the reference sentence, determine the character alignment information, which is used to represent whether the characters in the sample sentence and the corresponding characters in the reference sentence have the same semantics; based on the character alignment information, determine the reference character from the reference sentence.

[0148] In the above process, the server inputs the sample sentence into the teacher model instead of inputting the sample character into the teacher model. In this way, the teacher model can be used to translate the sample sentence only once to obtain the reference sentence. Based on the reference sentence, the reference character corresponding to the sample character can be found using the character alignment information, without having to repeatedly translate the sample character, thus greatly saving computing resources.

[0149] 503. The server adjusts the parameters of the first translation model based on the sample phrase to obtain a second translation model.

[0150] In some embodiments, the process of the server adjusting the parameters of the first translation model, that is, the process of training the first translation model, may include: invoking the first translation model to translate the sample phrase to obtain a translated phrase; determining a second loss value based on the reference phrase and the translated phrase, where the reference phrase has the same semantics as the sample phrase; adjusting the parameters of the first translation model based on the second loss value to obtain the second translation model.

[0151] Optionally, the server may input the sample phrase into the first translation model, translate the sample phrase through the first translation model, and output the translated phrase. Optionally, obtaining the cross-entropy loss between the reference phrase and the translated phrase as the second loss value. If the second stop condition is not satisfied, iteratively adjust the parameters of the first translation model based on the backpropagation algorithm until the second stop condition is met, and then stop the iteration to obtain the second translation model. Optionally, the second stop condition includes: the second loss value is less than or equal to the second loss threshold, or the number of iterations is greater than the second number threshold, where the second loss threshold may be any value greater than or equal to 0 and less than or equal to 1, and the second number threshold may be any integer greater than or equal to 1. For example, the second loss threshold is 0.2 and the second number threshold is 50000.

[0152] In some embodiments, when the server obtains the reference phrase, it may directly invoke the teacher model to translate the sample phrase, that is: the server invokes the teacher model to translate the sample phrase to obtain the reference phrase. The teacher model supports serial translation processing of multiple input characters, that is, the teacher model is an AT model.

[0153] In the above process, translating the sample phrase through the teacher model to obtain the reference phrase can improve the efficiency of obtaining the reference phrase. In some other embodiments, the reference phrase may also be manually annotated by a technician. The embodiments of the present application do not specifically limit the method of obtaining the reference phrase.

[0154] In some embodiments, when the server obtains a reference phrase, it may first call a teacher model to translate a sample statement to obtain a reference statement. Then, it determines each sample character included in the sample phrase, and based on the character alignment information, finds each reference character corresponding to each sample character in the reference statement. These reference characters can form the reference phrase. That is to say, the server calls the teacher model to translate the sample statement to obtain a reference statement, and the teacher model supports serial translation processing of multiple input characters; based on the sample statement and the reference statement, character alignment information is determined, and this character alignment information is used to represent whether the characters in the sample statement and the corresponding characters in the reference statement have the same semantics; based on the character alignment information, from the reference statement, multiple reference characters having the same semantics as the multiple sample characters included in the sample phrase are determined; and these multiple reference characters are determined as the reference phrase.

[0155] In the above process, the server inputs the sample statement into the teacher model instead of inputting the sample phrase into the teacher model. In this way, the sample statement can be translated only once by the teacher model to obtain the reference statement. Based on the reference statement and using the character alignment information, the reference characters corresponding to each sample character included in the sample phrase can be found, and each reference character is determined as the sample phrase, without the need to repeatedly translate the sample phrase. Thus, a large amount of computing resources can be saved.

[0156] 504. The server adjusts the parameters of the second translation model based on the sample statement to obtain a target translation model, and the target translation model is used to convert an input statement into a translated statement with the same semantics.

[0157] In some embodiments, the process of the server adjusting the parameters of the second translation model, that is, the process of training the second translation model, may include: calling the second translation model to translate the sample statement to obtain a translated statement; determining a third loss value based on the reference statement and the translated statement, where the reference statement has the same semantics as the sample statement; and adjusting the parameters of the second translation model based on the third loss value to obtain the target translation model.

[0158] Optionally, the server may input the sample statement into the second translation model, translate the sample statement through the second translation model, and output the translated statement. Optionally, obtaining the cross-entropy loss between the reference statement and the translated statement as the third loss value. If the third stopping condition is not met, iteratively adjust the parameters of the second translation model based on the backpropagation algorithm until the third stopping condition is met, and then stop the iteration to obtain the target translation model. Optionally, the third stopping condition includes: the third loss value is less than or equal to the third loss threshold, or the number of iterations is greater than the third number threshold, where the third loss threshold may be any value greater than or equal to 0 and less than or equal to 1, and the third number threshold may be any integer greater than or equal to 1. For example, the third loss threshold is 0.2, and the third number threshold is 200000.

[0159] In some embodiments, when the server obtains the reference statement, it may directly call the teacher model to translate the sample statement, that is: the server calls the teacher model to translate the sample statement to obtain the reference statement. The teacher model supports serial translation processing of multiple input characters, that is, the teacher model is an AT model.

[0160] In the above process, translating the sample statement through the teacher model to obtain the reference statement can improve the acquisition efficiency of the reference statement. In some other embodiments, the reference statement may also be manually marked by a technician. The embodiments of the present application do not specifically limit the acquisition method of the reference statement.

[0161] All of the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present disclosure, which will not be elaborated here one by one.

[0162] The method provided by the embodiments of the present application splits the coarse-grained sample statement into fine-grained sample phrases and even finer-grained sample characters, and uses the sample characters to enable the student model to first learn relatively simple and easy-to-learn character knowledge, then uses the sample phrases to enable the first translation model to learn more complex phrase knowledge, and finally uses the sample statement to enable the second translation model to learn the most complex statement knowledge. Since the target translation model supports parallel translation of input characters, the translation speed of the target translation model will not be affected, and without additionally introducing the amount of training data and model parameters, the translation accuracy of the target translation model is greatly improved.

[0163] Figure 6 It is a flowchart of a training process of a translation model provided by the embodiments of the present application. Please refer to Figure 6 , this embodiment is applied to a computer device. Here, taking the server 420 in the above implementation environment as an example for illustration, this embodiment includes the following steps:

[0164] 601. The server obtains a sample statement.

[0165] In some embodiments, the server may obtain the sample statement from a training corpus. The number of sample statements is at least one. The training corpus may be stored locally on the server or in other distributed storage systems. The embodiments of the present application do not specifically limit this.

[0166] In the embodiments of the present application, only the processing flow of a single sample statement is taken as an example for illustration, but it should not constitute a limitation on the number of sample statements. When the number of sample statements is greater than or equal to 2, a similar processing flow can be executed for each sample statement.

[0167] 602. The server invokes a teacher model to translate the sample statement to obtain a reference statement. The teacher model supports serial translation processing of multiple input characters, and the reference statement has the same semantics as the sample statement.

[0168] Among them, the teacher model supports serial translation processing of multiple input characters, that is, the teacher model is used to translate the input characters word by word. The teacher model is essentially an AT model.

[0169] In the above process, the server inputs the sample statement into the teacher model, and the teacher model translates the sample statement to output the reference statement, which can greatly improve the acquisition efficiency of the reference statement. In some other embodiments, the reference statement can also be manually marked by technicians. The embodiments of the present application do not specifically limit this. Optionally, the teacher model can be any AT model. For example, the teacher model is a Transformer translation model or a variant of the Transformer translation model.

[0170] The following will take the teacher model as a Transformer translation model as an example for illustration. Figure 7 is a framework schematic diagram of a teacher model provided by the embodiments of the present application. As Figure 7 shown, the Transformer translation model includes an embedding module 701, an encoding module 702, a decoding module 703, and an output module 704. It adopts an encoder-decoder framework based on the SAN network and belongs to the mainstream model structure for sequence-to-sequence generation. During the translation process, the sample statement is first input into the embedding module 701 for embedding processing to obtain an embedded statement. After splicing the embedded statement with the position encoding vector of the sample statement, it is input into the encoding module 702. Among them, the position encoding vector is used to represent the position sequence information of each sample character in the sample statement.

[0171] The encoding module 702 can be formed by cascading multiple encoders (for example, N = 6 encoders are cascaded). Each encoder internally includes a multi-head attention layer and a feed-forward neural network layer. The multi-head attention layer is used to comprehensively extract the correlation relationships between each sample character within the sample statement from multiple representation subspaces. The feed-forward neural network layer is used to perform a full connection on the feature vector output by the multi-head attention layer. Residual structures are set after both the multi-head attention layer and the feed-forward neural network layer, that is, the input and output of the current layer are connected (i.e., concatenated) and normalized and then input into the next layer. Here, N represents the number of cascaded encoders.

[0172] The decoding module 703 can be formed by cascading multiple decoders (for example, N = 6 decoders are cascaded). Each decoder internally includes a masked multi-head attention layer, a fused multi-head attention layer, and a feed-forward neural network layer. The masked multi-head attention layer is similar to the multi-head attention layer, but only focuses on the translation results before the current moment. Therefore, it is necessary to perform a mask (occlusion) process on the translation results after the current moment. The fused multi-head attention layer is also similar to the multi-head attention layer, but in addition to taking the output of the masked multi-head attention layer of this decoder as the input, it also takes the output of the feed-forward neural network layer of the encoder with the corresponding serial number (referring to the result after residual connection and normalization) as the input. This design is used to focus on the encoding information of the encoder. In other words, the decoder predicts the next character in the reference statement by looking at the output of the encoder and the self-attention on its own output. The feed-forward neural network layer of the decoder is similar to that of the encoder and will not be elaborated here. Similarly, residual structures are also set after the masked multi-head attention layer, the fused multi-head attention layer, and the feed-forward neural network layer of the decoder, that is, the input and output of the current layer are connected (i.e., concatenated) and normalized and then input into the next layer. Here, N represents the number of cascaded decoders, that is to say, the number of cascaded encoders needs to be consistent with the number of cascaded decoders.

[0173] The output module 704 includes a linear layer and a softmax layer, which can map the vector output by the decoding module 703 into the final respective reference characters, and finally form the reference statement. Among them, the linear layer is a full connection layer, and the dimension is the number of all characters included in the target language corpus of the translation. The softmax layer is used to obtain the probability of each sample character corresponding to each character in the target language, select the character with the highest probability as the reference character, and use the statement composed of each reference character as the reference statement.

[0174] In addition, the input of the decoding module 703 involves a shifted right mechanism. Since the learning goal of the model training stage is to predict what the character (Token) at the next position is, the Masking method is used here to cover the content on the right side of the current moment to ensure that the learning process is predicted from left to right character by character. The shifted right mechanism ensures that the masked multi-head attention layer can only focus on the translation results before the current moment. For example, assuming that the output sequence is "I love China.", which includes a total of 4 characters, when predicting "love", it is necessary to block out "China.", that is, "love" can only be predicted based on "I". Similarly, when predicting "China", it is necessary to block out ".", that is, it can only be predicted based on "I love".

[0175] 603. The server performs word segmentation processing on the sample sentence to obtain sample characters.

[0176] Optionally, when performing word segmentation processing, the server may use a word segmentation tool to automatically implement word segmentation, or a technician may perform manual word segmentation. The embodiment of the present application does not specifically limit the word segmentation method.

[0177] 604. The server determines character alignment information based on the sample sentence and the reference sentence, where the character alignment information is used to indicate whether characters in the sample sentence and corresponding characters in the reference sentence have the same semantics.

[0178] In some embodiments, the character alignment information may be a word alignment bit matrix, and the server may use the script tool GIZA++ to perform word alignment on the sample sentence and the reference sentence to obtain the word alignment bit matrix. The bit matrix is a bitmap matrix, and each element in the bit matrix is used to indicate whether a sample character in the sample sentence is aligned with the corresponding reference character in the reference sentence (i.e., has the same semantics). For example, an element value of 1 represents alignment, and an element value of 2 represents misalignment.

[0179] In other embodiments, the character alignment information may also be a bilingual character table consisting of sample characters and reference characters, where each column in the bilingual character table is used to represent a pair of sample characters and reference characters with the same semantics. The embodiment of the present application does not specifically limit the form of the character alignment information.

[0180] The above step 604 is an exemplary description of the server obtaining the character alignment information between the sample sentence and the reference sentence.

[0181] 605. The server determines, from the reference sentence based on the character alignment information, a reference character having the same semantics as the sample character.

[0182] In some embodiments, the character alignment information is a word alignment bit matrix. The server can query this bit matrix. If the element corresponding to the sample character in the bit matrix indicates alignment, the character at the corresponding position is obtained as the reference character. If the element corresponding to the sample character in the bit matrix indicates non - alignment, the bilingual character table is queried to obtain the reference character corresponding to the sample character.

[0183] In some embodiments, the server can also directly query, according to the bilingual phrase table, for the reference character that belongs to the same column as the sample character. Optionally, the bilingual phrase table can be stored in the form of a hash table, key - value pair, etc. For example, when stored in the form of key - value pairs, the sample character can be used as the key name and the reference character as the key value. The server can query the content stored in the key value corresponding to the sample character and obtain the stored content as the reference character.

[0184] In the above process, the server inputs the sample sentence into the teacher model, rather than inputting the sample character into the teacher model. In this way, the sample sentence can be translated only once by the teacher model to obtain the reference sentence. Based on the reference sentence, using the character alignment information, the reference character corresponding to the sample character can be found without having to translate the sample character repeatedly, thus greatly saving computing resources.

[0185] In some embodiments, the server can also obtain the reference character in the following way: The server calls the teacher model to translate the sample character to obtain the reference character. In the above process, translating the sample character through the teacher model to obtain the reference character can improve the efficiency of obtaining the reference character. In some other embodiments, the reference character can also be manually marked by a technician. The embodiments of the present application do not specifically limit the method of obtaining the reference character.

[0186] 606. The server, based on the character alignment information, obtains a sample phrase that meets the target condition. The target condition is that at least one character included in the sample phrase is indicated in the character alignment information as having the same semantics as the corresponding character in the reference sentence.

[0187] In some embodiments, since the SMT model needs to perform word segmentation (i.e., preprocessing), word alignment, and construct a phrase table during translation, the SMT model can be used to obtain the sample characters and the sample phrases. That is, the word segmentation tool of the SMT model is used for word segmentation processing, and the script tool GIZA++ is used to obtain the word alignment bit matrix. Based on the bit matrix, sample phrases can be extracted. The sample phrases need to meet the target conditions, which can be simply summarized as closure and non-emptiness. Closure means that the aligned parties of the sample characters included in the sample phrase must be in the corresponding reference phrase without leakage; non-emptiness means that at least one sample character in the sample phrase is aligned with the reference character (with a value of 1 in the bit matrix). Optionally, when specifying the phrase length of the sample phrase, the bit matrix can be used to find all sample phrases that meet the target conditions. For example, when the phrase length is Len, the bit matrix is horizontally slid with a window size of Len, and each phrase formed by the sample characters within the sliding window is checked one by one to see if it meets the target conditions. If it meets the target conditions, it is determined that the sample characters within the window form a sample phrase; otherwise, continue to slide to the right.

[0188] The above step 606 is an exemplary illustration of the server extracting sample phrases from the sample statement. In the above process, the server can accurately screen out each sample character in the sample statement that can form a sample phrase through the character alignment information, so as to automatically extract the sample phrase by machine and improve the extraction efficiency of the sample phrase. In some other embodiments, the technical personnel can also perform manual phrase extraction, and the embodiments of the present application do not specifically limit the phrase extraction method.

[0189] In the above steps 603 and 606, the server obtains the sample characters and sample phrases in the sample statement. In the above process, by preprocessing the sample statement, the sample characters and sample phrases in the sample statement can be segmented. This segmentation process does not introduce additional training data, but only reprocesses the original training data, but can greatly enrich the representation level and representation ability of the training data.

[0190] 607. The server determines a reference phrase with the same semantics as the sample phrase from the reference statement based on the character alignment information.

[0191] In some embodiments, the server can determine multiple reference characters with the same semantics as the multiple sample characters included in the sample phrase from the reference statement based on the character alignment information; and determine the multiple reference characters as the reference phrase.

[0192] In the above process, the server inputs the sample sentence into the teacher model instead of the sample phrase into the teacher model, so that the sample sentence can be translated only once using the teacher model to obtain the reference sentence. On the basis of the reference sentence, the character alignment information can be used to find the reference characters corresponding to the sample characters contained in the sample phrase, and each reference character can be determined as the sample phrase without repeatedly translating the sample phrase, which can greatly save computing resources.

[0193] In some embodiments, the server may also obtain the reference phrase by calling the teacher model to translate the sample phrase to obtain the reference phrase. In the above process, translating the sample phrase by the teacher model to obtain the reference phrase can improve the efficiency of obtaining the reference phrase. In other embodiments, the reference phrase may also be manually annotated by a technician. The embodiment of the present application does not specifically limit the method of obtaining the reference phrase.

[0194] In the above process, sentence-level bilingual data (sample sentences and reference sentences) are still used as input. Through fine-grained bilingual knowledge extraction, bilingual vocabulary mapping (sample characters and reference characters) and bilingual phrase mapping (sample phrases and reference phrases) can be obtained. When extracting sample characters and sample phrases, since the phrase table is an important component of the SMT model and is used to record the correspondence between bilingual phrases, the phrase table of the sample sentence can be obtained based on the SMT model, and all possible bilingual phrases in the sentence can be extracted from the phrase table. Taking the sample sentence "He is very good at English." in Table 4 as an example, the bilingual phrase pairs "good|||very good" and "good at English|||good at English" can be extracted from the original sample sentence, which can inspire the NAT student model to learn explicit phrase boundaries.

[0195] 608. The server adjusts parameters of the student model based on the sample character to obtain a first translation model, where the student model supports parallel translation processing of multiple input characters.

[0196] The student model supports parallel translation of multiple input characters, which means that the student model does not translate word by word, that is, the student model is a NAT model or a variant of the NAT model.

[0197] The above step 608 is similar to the above step 502 and will not be described in detail here.

[0198] 609. The server adjusts parameters of the first translation model based on the sample phrase to obtain a second translation model.

[0199] The above step 609 is similar to the above step 503, and will not be elaborated here.

[0200] 610. The server adjusts the parameters of the second translation model based on the sample sentence to obtain a target translation model, which is used to convert the input sentence into a translated sentence with the same semantics.

[0201] The above step 610 is similar to the above step 504, and will not be elaborated here.

[0202] The following takes the student model as the NAT model as an example for illustration. Since the second translation model is obtained by training the first translation model, and the first translation model is obtained by training the student model, the first translation model and the second translation model are also NAT models. Similarly, the target translation model obtained by training the second translation model is also a NAT model. It should be noted that the student model can be any machine translation model that supports parallel translation. In addition to the above example of the NAT model, the student model can also be a variant of the NAT model.

[0203] Figure 8 It is a framework schematic diagram of a second translation model provided by an embodiment of the present application. Please refer to Figure 8 , the NAT model includes an embedding module 801, an encoding module 802, a decoding module 803, and an output module 804. During the translation process, the sample sentence is first input into the embedding module 801, and the embedding module 801 performs embedding processing on the sample sentence to obtain an embedded sentence. After splicing the embedded sentence with the position encoding vector of the sample sentence, it is input into the encoding module 802. The encoding module 802 encodes the input vector and outputs the sample sentence feature vector and the character number indication information. Among them, the position encoding vector is used to represent the position sequence information of each sample character in the sample sentence. Among them, the process of performing embedding processing on the sample sentence is also the process of performing embedding processing on each sample character included in the sample sentence to obtain the embedding vector of each sample character. The embedding vector sequence composed of each embedding vector is the embedded sentence.

[0204] The encoding module 802 can be formed by cascading multiple encoders (for example, N = 6 encoders are cascaded). Each encoder internally includes a multi-head attention layer and an MLP (Multi-Layer Perceptron) layer. The multi-head attention layer is used to comprehensively extract the correlation relationships between each sample character in the sample statement from multiple expression subspaces. The MLP layer is used to perform weighted processing on the feature vectors output by the multi-head attention layer. Among them, the hidden layers of the MLP are also fully connected. Residual structures are set after both the multi-head attention layer and the MLP layer, that is, the input and output of the current layer are residually connected (i.e., concatenated) and normalized and then input into the next layer. Different from the AT model, since the AT model translates word by word and stops translation with a terminator, the NAT model translates in parallel, so it is necessary to introduce a character quantity indication information. This character quantity indication information is used to indicate the number of translated characters corresponding to each sample character. By setting an exponential normalization layer after the last encoder in the encoding module 802, this character quantity indication information can be extracted. This character quantity indication information can be a character quantity sequence, and the length of the character quantity sequence is equal to the length of the character sequence of the input sample statement. Each element in the character quantity sequence is used to represent how many translated characters are needed to translate the sample character at the corresponding position in the character sequence of the sample statement. For example, in the process of translating English into German, the sample statement is “We totally accept it.”, and its character sequence is [We, totally, accept, it, .], and the character quantity sequence is [1, 1, 2, 0, 1], which means that: the English word “We” needs to be represented by one German word, the English word “totally” needs to be represented by one German word, the English word “accept” needs to be represented by two German words, the English word “it” does not need a separate German word to represent after translation, and the punctuation mark “.” needs to be represented by one punctuation mark. Among them, N represents the number of cascaded encoders.

[0205] Based on the character quantity indication information output by the encoding module 802, the sample statement can be transformed so that the number of each sample character in the sample statement matches this character quantity indication information, obtaining a transformed statement. For example, according to the character quantity sequence [1, 1, 2, 0, 1], the sample statement “We totally accept it.” is transformed to obtain the transformed statement “We totally accept accept.” Then, the transformed statement is input into the embedding module 701 for embedding processing to obtain an embedded transformed statement. After concatenating the embedded transformed statement with the position encoding vector of the transformed statement, it is input into the decoding module 803. The decoding module 803 decodes the input vector and outputs a translated statement feature vector.

[0206] The decoding module 803 can be formed by cascading multiple decoders (for example, N = 6 decoders are cascaded). Each decoder internally includes a multi-head attention layer, a multi-head position attention layer, a multi-head interactive attention layer, and an MLP layer. Among them, the multi-head attention layer in the decoder is similar to the multi-head attention layer in the encoder, which will not be elaborated here. The multi-head position attention layer is used to introduce the position encoding vector again on the basis of the output result of the multi-head attention layer to emphasize the position order relationship of each character in the transformed sentence. In addition to taking the output of the multi-head position attention layer of this decoder as the input, the multi-head interactive attention layer also takes the output of the MLP layer of the encoder with the corresponding serial number (referring to the result after residual connection and normalization, that is, the sample sentence feature vector) as the input. This design is used to focus on the encoding information of the encoder. In other words, the decoder performs text translation by looking at the output of the encoder and the self-attention of its own output. The MLP layer of the decoder is similar to the MLP layer of the encoder, which will not be elaborated here. Similarly, residual structures are also set after the multi-head attention layer, multi-head position attention layer, multi-head interactive attention layer, and MLP layer of the decoder, that is, the input of the current layer is connected (i.e., concatenated) with the output for residual connection and then normalized before being input into the next layer. Among them, N represents the number of cascaded decoders. It can be seen that the number of cascaded encoders and the number of cascaded decoders need to be kept consistent.

[0207] The output module 804 includes an exponential normalization layer. The exponential normalization layer can map the translated sentence feature vector output by the last decoder in the decoding module 803 into the final translated sentence. In other words, the translated sentence feature vector is a sequence of feature vectors, and each feature vector in the sequence includes the feature vectors of each translated character. In the exponential normalization layer, the feature vector of each translated character can be mapped into multiple probabilities, and each probability is used to represent the possibility of the translated character corresponding to each character in the target language. The character with the highest probability is selected as the translated character, and the sentence composed of each translated character is used as the translated sentence.

[0208] In the embodiments of the present application, starting from the data level ingeniously, a progressive multi-granularity training strategy is introduced on the basis framework of knowledge distillation, that is, first learning the training data at the character granularity, then learning the training data at the phrase granularity, and finally learning the training data at the sentence granularity, which is effective and general for machine translation models that support parallel translation and include various NAT models. Experiments on Romanian-English, English-German, Chinese-English, and Japanese-English all show that the method provided by the embodiments of the present application greatly improves the translation accuracy of phrases and the reordering ability of the model, and can improve the translation quality of the model under strong NAT baseline conditions, indicating that the translation performance of the model can be significantly improved according to deterministic fine-grained knowledge.

[0209] Since the first translation model is trained based on the student model, the second translation model is trained based on the first translation model, and the target translation model is trained based on the second translation model, the overall training process can be regarded as using the PMG strategy to progressively train the target translation model in a multi-granularity manner based on the student model. Set the first number threshold as the training step from the student model to the first translation model (that is, the first stop condition is that the number of iterations is greater than the first number threshold), set the second number threshold as the training step from the first translation model to the second translation model (that is, the second stop condition is that the number of iterations is greater than the second number threshold), and set the third number threshold as the training step from the second translation model to the target translation model (that is, the third stop condition is that the number of iterations is greater than the third number threshold). For example, the first number threshold is 50000, the second number threshold is 50000, and the third number threshold is 2000000. Since the phrase granularity and character granularity are more deterministic than the sentence granularity, this process is also a curriculum learning method based on determinacy.

[0210] All the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present disclosure, which will not be elaborated here one by one.

[0211] The method provided by the embodiments of the present application splits the coarse-grained sample sentences into fine-grained sample phrases and even finer-grained sample characters, and uses the sample characters to enable the student model to first learn relatively simple and easy-to-learn character knowledge, then uses the sample phrases to enable the first translation model to learn more complex phrase knowledge, and finally uses the sample sentences to enable the second translation model to learn the most complex sentence knowledge. Since the target translation model supports parallel translation of input characters, the translation speed of the target translation model will not be affected, and without introducing additional training data volume and model parameters, the translation performance of the target translation model is greatly improved.

[0212] Based on the method provided by the embodiments of the present application, tests are conducted on 4 widely used translation datasets, including: the small-scale WMT16 dataset (Romanian-English, abbreviated as Ro-En), the medium-scale WMT14 dataset (English-German, abbreviated as En-De), the large-scale WMT17 dataset (Chinese-English, abbreviated as Zh-En), and the WAT17 dataset with different word orders (Japanese-English, abbreviated as Ja-En). The data scales are 0.6, 4.5, 20, and 2 million sentence pairs respectively. Tokenization is performed in the BPE (Byte Pair Encoding) manner, and the original training data (i.e., the union of the 4 translation datasets) is preprocessed using 32K (i.e., 32000) merge operations. In addition, for the trained target translation model, BLEU is used as the performance metric, and a statistical significance test is conducted.

[0213] During the testing process, the knowledge distillation method is used to simplify the training data, and the knowledge of the BASE (basic) Transformer teacher model is distilled using the same training data to obtain the BIG Transformer model. The training results in all directions except Ro-En of the BIG Transformer model are used as the final teacher model. Based on this teacher model and the original training data, it can be migrated to train various different structured NAT models or other machine translation models that support parallel translation. In the embodiments of the present application, taking the MaskPredict (mask prediction) NAT model and the Levenshtein (delete, insert, and substitute) NAT model as examples, they are trained and tested based on the training method provided by the embodiments of the present application. Among them, the training step size at the character granularity and the phrase granularity are both set to 50000 steps, and the training step size at the sentence granularity is set to 2000000 steps. Both of the above NAT models contain 6 encoders and 6 decoders. Among them, the decoder is trained in the conditional masked language model manner. The multi-head attention layer of the NAT model contains 8 heads (i.e., self-attention features are extracted for 8 expression subspaces), the dimension of the output vector of the multi-head attention layer is 512, the dimension of the output vector of the MLP layer is 2048, and the BLEU values of the average of the first three checkpoints are used to avoid random errors.

[0214] Table 5 lists the comparison of translation performance on the WMT16 Ro-En, WMT14 En-De, WMT17 Zh-En, and WAT17 Ja-En datasets with traditional NAT models. As can be seen from Table 4, on the four translation datasets for different source languages to target languages, after training the MaskPredict NAT model and the Levenshtein NAT model using the PMG training strategy, compared with the traditional training strategy, the translation performance of the NAT model can be improved (i.e., the BLEU value increases, and "↑" indicates that the statistical result has a statistically significant difference from the baseline system, i.e., the confidence level p < 0.05). More notably, after training the NAT model with the PMG training strategy provided in the embodiments of the present application, the translation performance is even slightly better than that of the teacher model (Transformer-Base) in three tasks (En-De, Zh-En, and Ja-En).

[0215] Table 5

[0216]

[0217]

[0218] Among them, Iterative NAT refers to the iterative NAT model, DisCo (Disentangled Context Transformer) refers to the context-disentangled NAT model, and Context-aware NAT refers to the context-aware NAT model.

[0219] Compared with the representative curriculum learning NMT methods, traditional learning methods can be divided into the discretized curriculum learning (DCL) and the continuous curriculum learning (CCL). The sentence length is the most important variable in multi-granularity data. Therefore, the sentence length (referring to the source side) standard can be used to achieve discretized and continuous CL (Curriculum Learning). In the DCL setting, the number of data bins is explicitly predefined, while in the CCL approach, shorter examples are continuously sampled as the training progresses. Here, a data bin refers to dividing the data into different piles (i.e., datasets) according to specific criteria.

[0220] For the DCL method, the training samples are split into a predefined number of parts (taking 5 parts as an example in this test). For the CCL method, a length curriculum and a square root completion function are adopted. It can be found that the DCL method is worse than the KD baseline value (-0.6 BLEU), while the CCL method is 0.3 BLEU points higher than the KD baseline value. The PMG training strategy provided by the embodiments of this application (+0.6 BLEU) is the most effective method with the optimal translation performance.

[0221] Furthermore, since the PMG training strategy hopes that the NAT model can pay more attention to bilingual phrases, thereby improving the accuracy of phrase translation. Therefore, Table 6 shows the improvements in multiple granularities in the N-gram grammar to evaluate the accuracy of phrase translation. The number of N-grams represents the number of characters contained in the phrase. It can be seen that the BLEU value of the PMG training strategy (NAT w / PMG) is always better than the baseline, which indicates that the PMG training strategy does significantly improve the ability of the NAT model to capture phrase patterns. Among them, NAT w / PMG refers to NAT with PMG, that is, the NAT model trained using the PMG strategy.

[0222] Table 6

[0223] N-gram 2 3 4 5 6 △BLEU 0.5 0.3 0.3 0.2 0.2

[0224] The bilingual phrase information (i.e., the phrase table) extracted by the SMT model can intuitively form bilingual phrase boundaries. After extracting the sample phrases and reference phrases, the NAT model (i.e., the target translation model) trained based on the PMG training strategy has better reordering ability. Optionally, RIBES (Rank-based Intuitive Bilingual Evaluation Score) is selected as the evaluation index for the reordering effect, and BLEU is selected as the evaluation index for the translation quality. The reordering abilities of the NAT models trained using the PMG training strategy and the traditional training strategy are compared.

[0225] Figure 9 is a performance comparison chart of the NAT model trained using the PMG training strategy and the traditional training strategy provided by the embodiments of this application. As Figure 9 shown, the full-scale test set is classified into multiple statistical nodes according to the sentence length. The difference in the translation performance BLEU between the two is shown in the left part 901. It can be seen that the translation performance of the PMG training strategy is significantly better. The difference in the reordering ability RIBES between the two is shown in the right part 902. It can be seen that the reordering ability of the PMG training strategy is also significantly better. This shows that the PMG training strategy can significantly improve the translation performance and reordering quality (i.e., reordering ability) of the NAT model.

[0226] Furthermore, the fine-grained bilingual knowledge extracted from the original sample sentences, namely word alignment and phrase table, still has a very large capacity after filtering. A third-party scorer - BERTScore (BERT scoring function, where BERT refers to Bidirectional Encoder Representations from Transformers, a translation model based on bidirectional encoding representations of Transformers) can be used to control the quality of the fine-grained bilingual knowledge to improve the certainty of the fine-grained bilingual knowledge. As shown in Table 7, it can be seen that high-quality bilingual knowledge (such as 50%) can further improve the performance of the PMG training strategy. That is, if sample phrases with a quality score above 50% are selected by the scorer for training, then the translation performance of the NAT model can be further significantly improved. Therefore, the PMG training strategy has great potential.

[0227] Table 7

[0228] Ratio 10％ 35％ 50％ 100％ △BLEU +0.3 +0.6 +0.7 +0.6

[0229] In the embodiments of the present application, a multi-granularity translation analysis method is proposed to quantify whether traditional translation models can have good translation performance at different granularities, thereby revealing the weaknesses of the NAT model. In addition, a training method (i.e., the PMG training strategy) that can simultaneously model bilingual knowledge at different granularities is proposed. Without changing the parameters of the model itself and without reducing the decoding rate advantage, it can significantly improve the translation effect (manifested as an increase in the BLEU value) of machine translation models that support parallel translation, taking the NAT model as an example. And it is found through analysis that the reordering ability and other abilities of machine translation models that support parallel translation, taking the NAT model as an example, are also further improved. In addition, the NAT model trained based on the PMG training strategy can be applied to various natural language generation tasks. For example, it can be applied in a machine translation system, which can help understand and improve the effect of the machine translation system to enhance the user experience. It can also provide an API (Application Programming Interface) interface externally to save external translation costs and improve translation efficiency.

[0230] Figure 10 is a schematic structural diagram of a text translation device provided by the embodiments of the present application, as Figure 10 shown. The device includes:

[0231] An acquisition module 1001, configured to acquire sample characters and sample phrases in the sample sentence;

[0232] The first adjustment module 1002 is configured to adjust the parameters of the student model based on the sample characters to obtain a first translation model, and the student model supports parallel translation processing of multiple input characters;

[0233] The second adjustment module 1003 is configured to adjust the parameters of the first translation model based on the sample phrase to obtain a second translation model;

[0234] The third adjustment module 1004 is configured to adjust the parameters of the second translation model based on the sample sentence to obtain a target translation model, and the target translation model is used to convert an input sentence into a translated sentence with the same semantics.

[0235] The device provided by the embodiment of the present application splits a coarse-grained sample sentence into a fine-grained sample phrase and a finer-grained sample character, and uses the sample character to enable the student model to first learn relatively simple and easy-to-learn character knowledge, and then uses the sample phrase to enable the first translation model to learn more complex phrase knowledge. Finally, the sample sentence is used to enable the second translation model to learn the most complex sentence knowledge. Since the target translation model supports parallel translation of input characters, the translation speed of the target translation model will not be affected, and without additionally introducing the amount of training data and model parameters, the translation accuracy of the target translation model is greatly improved.

[0236] In a possible implementation manner, the first adjustment module 1002 is configured to:

[0237] Call the student model to translate the sample characters to obtain translated characters;

[0238] Determine a first loss value based on the reference characters and the translated characters, and the reference characters have the same semantics as the sample characters;

[0239] Adjust the parameters of the student model based on the first loss value to obtain the first translation model.

[0240] In a possible implementation manner, based on Figure 10 the device composition, the device further includes:

[0241] The first translation module is configured to call the teacher model to translate the sample characters to obtain the reference characters, and the teacher model supports serial translation processing of multiple input characters.

[0242] In a possible implementation manner, based on Figure 10 the device composition, the device further includes:

[0243] The second translation module is configured to call the teacher model to translate the sample sentence to obtain a reference sentence, and the teacher model supports serial translation processing of multiple input characters;

[0244] A first determination module, configured to determine character alignment information based on the sample statement and the reference statement, where the character alignment information is used to characterize whether the characters in the sample statement have the same semantics as the corresponding characters in the reference statement;

[0245] A second determination module, configured to determine the reference character from the reference statement based on the character alignment information.

[0246] In a possible implementation manner, the second adjustment module 1003 is configured to:

[0247] Call the first translation model to translate the sample phrase to obtain a translated phrase;

[0248] Determine a second loss value based on the reference phrase and the translated phrase, where the reference phrase has the same semantics as the sample phrase;

[0249] Adjust the parameters of the first translation model based on the second loss value to obtain the second translation model.

[0250] In a possible implementation manner, based on Figure 10 the composition of the device, the device further includes:

[0251] A third translation module, configured to call a teacher model to translate the sample phrase to obtain the reference phrase, where the teacher model supports serial translation processing of multiple input characters.

[0252] In a possible implementation manner, based on Figure 10 the composition of the device, the device further includes:

[0253] A second translation module, configured to call a teacher model to translate the sample statement to obtain a reference statement, where the teacher model supports serial translation processing of multiple input characters;

[0254] A first determination module, configured to determine character alignment information based on the sample statement and the reference statement, where the character alignment information is used to characterize whether the characters in the sample statement have the same semantics as the corresponding characters in the reference statement;

[0255] A third determination module, configured to determine, from the reference statement, multiple reference characters that have the same semantics as multiple sample characters included in the sample phrase based on the character alignment information;

[0256] A fourth determination module, configured to determine the multiple reference characters as the reference phrase.

[0257] In a possible implementation manner, the third adjustment module 1004 is configured to:

[0258] Call the second translation model to translate the sample sentence to obtain a translated sentence;

[0259] Based on the reference sentence and the translated sentence, determine a third loss value, where the reference sentence has the same semantics as the sample sentence;

[0260] Based on the third loss value, adjust the parameters of the second translation model to obtain the target translation model.

[0261] In a possible implementation manner, based on Figure 10 the composition of the device, the device further includes:

[0262] A second translation module, configured to call a teacher model to translate the sample sentence to obtain the reference sentence, where the teacher model supports serial translation processing of multiple input characters.

[0263] In a possible implementation manner, based on Figure 10 the composition of the device, the acquisition module 1001 includes:

[0264] A word segmentation processing unit, configured to perform word segmentation processing on the sample sentence to obtain the sample characters;

[0265] A phrase extraction unit, configured to extract phrases from the sample sentence to obtain the sample phrases.

[0266] In a possible implementation manner, the phrase extraction unit is configured to:

[0267] Obtain character alignment information between the sample sentence and the reference sentence, where the character alignment information is used to indicate whether the characters in the sample sentence and the corresponding characters in the reference sentence have the same semantics;

[0268] Based on the character alignment information, obtain the sample phrases that meet the target condition, where the target condition is that at least one character included in the sample phrase is indicated in the character alignment information as having the same semantics as the corresponding character in the reference sentence.

[0269] All of the above optional technical solutions can be combined arbitrarily to form alternative embodiments of the present disclosure, which will not be elaborated herein one by one.

[0270] It should be noted that: when the text translation device provided in the above embodiments translates text, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the text translation device provided in the above embodiments and the text translation method embodiments belong to the same concept, and the specific implementation process thereof can be found in the text translation method embodiments, which will not be elaborated herein.

[0271] Figure 11 It is a schematic structural diagram of a computer device provided by an embodiment of the present application. Optionally, the device type of the terminal 1100 includes: smart phone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) player, laptop computer or desktop computer. The terminal 1100 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, etc.

[0272] Generally, the terminal 1100 includes a processor 1101 and a memory 1102.

[0273] Optionally, the processor 1101 includes one or more processing cores, such as a 4-core processor, an 8-core processor, etc. Optionally, the processor 1101 is implemented in at least one of the following hardware forms: DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), PLA (Programmable Logic Array). In some embodiments, the processor 1101 includes a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1101 integrates a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 further includes an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0274] In some embodiments, the memory 1102 includes one or more computer-readable storage media, optionally non-transitory. Optionally, the memory 1102 further includes high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one program code for being executed by the processor 1101 to implement the text translation method provided in various embodiments of the present application.

[0275] In some embodiments, the terminal 1100 may further optionally include: a peripheral device interface 1103 and at least one peripheral device. The processor 1101, the memory 1102, and the peripheral device interface 1103 can be connected through a bus or signal lines. Each peripheral device can be connected to the peripheral device interface 1103 through a bus, signal lines, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, an audio circuit 1107, a positioning component 1108, and a power supply 1109.

[0276] The peripheral device interface 1103 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1101 and the memory 1102. In some embodiments, the processor 1101, the memory 1102, and the peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1101, the memory 1102, and the peripheral device interface 1103 are implemented on a separate chip or circuit board, and the present embodiment does not limit this.

[0277] The radio frequency circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1104 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1104 converts an electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and so on. Optionally, the radio frequency circuit 1104 communicates with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: a metropolitan area network, generations of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1104 further includes a circuit related to NFC (Near Field Communication), which is not limited in this application.

[0278] The display screen 1105 is used to display a UI (User Interface). Optionally, the UI includes graphics, text, icons, videos, and any combination thereof. When the display screen 1105 is a touch display screen, the display screen 1105 also has the ability to collect touch signals on or above the surface of the display screen 1105. The touch signal can be input as a control signal to the processor 1101 for processing. Optionally, the display screen 1105 is further used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there is one display screen 1105, which is disposed on the front panel of the terminal 1100; in other embodiments, there are at least two display screens 1105, which are respectively disposed on different surfaces of the terminal 1100 or are in a foldable design; in still other embodiments, the display screen 1105 is a flexible display screen, which is disposed on a curved surface or a foldable surface of the terminal 1100. Even more optionally, the display screen 1105 is set to an irregular non-rectangular shape, that is, a special-shaped screen. Optionally, the display screen 1105 is prepared using materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0279] The camera assembly 1106 is used to capture images or videos. Optionally, the camera assembly 1106 includes a front camera and a rear camera. Generally, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth camera, a wide-angle camera, and a telephoto camera, so as to implement the background blurring function by fusing the main camera and the depth camera, the panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions by fusing the main camera and the wide-angle camera. In some embodiments, the camera assembly 1106 further includes a flash. Optionally, the flash is a single-color temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to the combination of a warm light flash and a cold light flash, which is used for light compensation under different color temperatures.

[0280] In some embodiments, the audio circuit 1107 includes a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals and input them to the processor 1101 for processing, or input them to the radio frequency circuit 1104 to implement voice communication. For the purpose of stereo collection or noise reduction, there are multiple microphones, which are respectively disposed at different parts of the terminal 1100. Optionally, the microphone is an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signal from the processor 1101 or the radio frequency circuit 1104 into sound waves. Optionally, the speaker is a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for uses such as ranging. In some embodiments, the audio circuit 1107 further includes a headphone jack.

[0281] The positioning component 1108 is used to locate the current geographical location of the terminal 1100 to implement navigation or LBS (Location Based Service). Optionally, the positioning component 1108 is a positioning component based on the GPS (Global Positioning System) of the United States, the Beidou system of China, the GLONASS system of Russia, or the Galileo system of the European Union.

[0282] The power supply 1109 is used to supply power to each component in the terminal 1100. Optionally, the power supply 1109 is alternating current, direct current, a disposable battery or a rechargeable battery. When the power supply 1109 includes a rechargeable battery, the rechargeable battery supports wired charging or wireless charging. The rechargeable battery is also used to support fast charging technology.

[0283] In some embodiments, the terminal 1100 further includes one or more sensors 1110. The one or more sensors 1110 include, but are not limited to: an acceleration sensor 1111, a gyroscope sensor 1112, a pressure sensor 1113, a fingerprint sensor 1114, an optical sensor 1115, and a proximity sensor 1116.

[0284] In some embodiments, the acceleration sensor 1111 detects the magnitudes of accelerations on the three coordinate axes of the coordinate system established with the terminal 1100. For example, the acceleration sensor 1111 is used to detect the components of the gravitational acceleration on the three coordinate axes. Optionally, the processor 1101 controls the display screen 1105 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 1111. The acceleration sensor 1111 is also used for collecting game or user motion data.

[0285] In some embodiments, the gyroscope sensor 1112 detects the body direction and rotation angle of the terminal 1100. The gyroscope sensor 1112 cooperates with the acceleration sensor 1111 to collect the 3D actions of the user on the terminal 1100. The processor 1101 realizes the following functions according to the data collected by the gyroscope sensor 1112: motion sensing (such as changing the UI according to the user's tilting operation), image stabilization during shooting, game control, and inertial navigation.

[0286] Optionally, the pressure sensor 1113 is disposed on the side frame of the terminal 1100 and / or the lower layer of the display screen 1105. When the pressure sensor 1113 is disposed on the side frame of the terminal 1100, it can detect the holding signal of the user on the terminal 1100, and the processor 1101 performs left / right hand recognition or shortcut operations according to the holding signal collected by the pressure sensor 1113. When the pressure sensor 1113 is disposed on the lower layer of the display screen 1105, the processor 1101 controls the operable controls on the UI interface according to the pressure operation of the user on the display screen 1105. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0287] The fingerprint sensor 1114 is used to collect the fingerprint of a user. The processor 1101 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 1114, or the fingerprint sensor 1114 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 1101 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings, etc. Optionally, the fingerprint sensor 1114 is disposed on the front, back, or side of the terminal 1100. When there are physical buttons or manufacturer Logos on the terminal 1100, the fingerprint sensor 1114 can be integrated with the physical buttons or manufacturer Logos.

[0288] The optical sensor 1115 is used to collect the ambient light intensity. In one embodiment, the processor 1101 controls the display brightness of the display screen 1105 according to the ambient light intensity collected by the optical sensor 1115. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1105 is increased; when the ambient light intensity is low, the display brightness of the display screen 1105 is decreased. In another embodiment, the processor 1101 also dynamically adjusts the shooting parameters of the camera module 1106 according to the ambient light intensity collected by the optical sensor 1115.

[0289] The proximity sensor 1116, also known as the distance sensor, is usually disposed on the front panel of the terminal 1100. The proximity sensor 1116 is used to collect the distance between the user and the front of the terminal 1100. In one embodiment, when the proximity sensor 1116 detects that the distance between the user and the front of the terminal 1100 is gradually decreasing, the processor 1101 controls the display screen 1105 to switch from the lit state to the off state; when the proximity sensor 1116 detects that the distance between the user and the front of the terminal 1100 is gradually increasing, the processor 1101 controls the display screen 1105 to switch from the off state to the lit state.

[0290] Those skilled in the art can understand that Figure 11 the structure shown in does not constitute a limitation on the terminal 1100, and it can include more or fewer components than shown in the figure, or combine some components, or adopt different component arrangements.

[0291] Figure 12It is a schematic structural diagram of a computer device provided by an embodiment of the present application. The computer device 1200 may vary greatly due to different configurations or performances. The computer device 1200 includes one or more processors (Central Processing Units, CPUs) 1201 and one or more memories 1202. Among them, at least one computer program is stored in the memory 1202, and the at least one computer program is loaded and executed by the one or more processors 1201 to implement the text translation method provided by each of the above embodiments. Optionally, the computer device 1200 further has components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output. The computer device 1200 further includes other components for implementing the functions of the device, which will not be elaborated here.

[0292] In an exemplary embodiment, a computer-readable storage medium is further provided, such as a memory including at least one computer program. The at least one computer program can be executed by a processor in a terminal to complete the text translation method in each of the above embodiments. For example, the computer-readable storage medium includes ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, and optical data storage devices, etc.

[0293] In an exemplary embodiment, a computer program product or a computer program is further provided, including one or more program codes. The one or more program codes are stored in a computer-readable storage medium. One or more processors of a computer device can read the one or more program codes from the computer-readable storage medium, and the one or more processors execute the one or more program codes, so that the computer device can execute to complete the text translation method in the above embodiments.

[0294] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware, and can also be completed by instructing relevant hardware through a program. Optionally, the program is stored in a computer-readable storage medium. Optionally, the above-mentioned storage medium is a read-only memory, a magnetic disk, or an optical disc, etc.

[0295] The above are only optional embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A text translation method, characterized in that, The method includes: Obtaining sample characters and sample phrases in a sample sentence; Based on the sample characters, adjusting parameters of a student model to obtain a first translation model, where the student model supports parallel translation processing of multiple input characters; Based on the sample phrases, adjusting parameters of the first translation model to obtain a second translation model; Based on the sample sentence, adjusting parameters of the second translation model to obtain a target translation model, where the target translation model is used to convert an input sentence into a translated sentence with the same semantics.

2. The method according to claim 1, wherein The adjusting parameters of the student model based on the sample characters to obtain the first translation model includes: Invoking the student model to translate the sample characters to obtain translated characters; Based on reference characters and the translated characters, determining a first loss value, where the reference characters have the same semantics as the sample characters; Based on the first loss value, adjusting parameters of the student model to obtain the first translation model.

3. The method according to claim 2, wherein Before determining the first loss value based on the reference characters and the translated characters, the method further includes: Invoking a teacher model to translate the sample characters to obtain the reference characters, where the teacher model supports serial translation processing of multiple input characters.

4. The method according to claim 2, wherein Before determining the first loss value based on the reference characters and the translated characters, the method further includes: Invoking a teacher model to translate the sample sentence to obtain a reference sentence, where the teacher model supports serial translation processing of multiple input characters; Based on the sample sentence and the reference sentence, determining character alignment information, where the character alignment information is used to characterize whether the characters in the sample sentence and the corresponding characters in the reference sentence have the same semantics; Based on the character alignment information, determining the reference characters from the reference sentence.

5. The method according to claim 1, wherein The adjusting parameters of the first translation model based on the sample phrases to obtain the second translation model includes: Invoking the first translation model to translate the sample phrases to obtain translated phrases; Based on reference phrases and the translated phrases, determining a second loss value, where the reference phrases have the same semantics as the sample phrases; Based on the second loss value, adjusting parameters of the first translation model to obtain the second translation model.

6. The method according to claim 5, characterized in that, Before determining the second loss value based on the reference phrases and the translated phrases, the method further includes: Invoking a teacher model to translate the sample phrases to obtain the reference phrases, where the teacher model supports serial translation processing of multiple input characters.

7. The method according to claim 5, wherein Before determining the second loss value based on the reference phrases and the translated phrases, the method further includes: Invoking a teacher model to translate the sample sentence to obtain a reference sentence, where the teacher model supports serial translation processing of multiple input characters; Based on the sample sentence and the reference sentence, determining character alignment information, where the character alignment information is used to characterize whether the characters in the sample sentence and the corresponding characters in the reference sentence have the same semantics; Based on the character alignment information, determine, from the reference sentence, multiple reference characters that have the same semantics as multiple sample characters included in the sample phrase; Determine the multiple reference characters as the reference phrase.

8. The method according to claim 1, characterized in that, The parameter adjustment of the second translation model based on the sample sentence to obtain the target translation model includes: Invoke the second translation model to translate the sample sentence to obtain a translated sentence; Based on the reference sentence and the translated sentence, determine a third loss value, where the reference sentence and the sample sentence have the same semantics; Based on the third loss value, perform parameter adjustment on the second translation model to obtain the target translation model.

9. The method according to claim 8, wherein Before determining the third loss value based on the reference sentence and the translated sentence, the method further includes: Invoke a teacher model to translate the sample sentence to obtain the reference sentence, where the teacher model supports serial translation processing of multiple input characters.

10. The method according to claim 1, wherein The obtaining of the sample characters and the sample phrase in the sample sentence includes: Perform word segmentation on the sample sentence to obtain the sample characters; Perform phrase extraction on the sample sentence to obtain the sample phrase.

11. The method according to claim 10, wherein The performing of phrase extraction on the sample sentence to obtain the sample phrase includes: Obtain character alignment information between the sample sentence and the reference sentence, where the character alignment information is used to characterize whether the characters in the sample sentence and the corresponding characters in the reference sentence have the same semantics; Based on the character alignment information, obtain the sample phrase that meets the target condition, where the target condition is that at least one character included in the sample phrase is indicated in the character alignment information as having the same semantics as the corresponding character in the reference sentence.

12. A text translation device, characterized in that, The apparatus includes: An acquisition module, configured to acquire sample characters and a sample phrase in a sample sentence; A first adjustment module, configured to perform parameter adjustment on a student model based on the sample characters to obtain a first translation model, where the student model supports parallel translation processing of multiple input characters; A second adjustment module, configured to perform parameter adjustment on the first translation model based on the sample phrase to obtain a second translation model; A third adjustment module, configured to perform parameter adjustment on the second translation model based on the sample sentence to obtain a target translation model, where the target translation model is used to convert an input sentence into a translated sentence with the same semantics.

13. The device according to claim 12, characterized in that, The first adjustment module is configured to: Invoke the student model to translate the sample characters to obtain translated characters; Based on reference characters and the translated characters, determine a first loss value, where the reference characters and the sample characters have the same semantics; Based on the first loss value, perform parameter adjustment on the student model to obtain the first translation model.

14. A computer device, characterized in that, The computer device includes one or more processors and one or more memories, and at least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement the text translation method according to any one of claims 1 to 11.

15. A storage medium, characterized in that, At least one computer program is stored in the storage medium, and the at least one computer program is loaded and executed by a processor to implement the text translation method according to any one of claims 1 to 11.

Images