Machine translation method, machine translation device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a machine translation method, a machine translation device, electronic equipment and a storage medium, and belongs to the technical field of financial science and technology. The method comprises the steps that a source language sentence is obtained, syntactic analysis is conducted on the source language sentence, a target syntactic structure is obtained, a probability context-independent grammar of the target syntactic structure is obtained, a target syntactic tree is generated according to the probability context-independent grammar, the target syntactic tree comprises syntactic nodes, and syntactic types of the syntactic nodes are obtained, and calling a word generation model according to the syntactic type to generate words to obtain target words of the syntactic type, and assembling the target words according to the target syntactic tree to obtain a target language sentence. The embodiment of the invention can improve the quality of machine translation.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to a machine translation method, a machine translation device, an electronic device and a storage medium. Background Art

[0002] Machine translation is the process of using computers to convert one natural language into another. Machine translation technology can achieve cross-language communication and exchange. For example, machine translation technology can enable multi-language financial services such as mobile payments and online banking services, improving the accessibility of web pages. In related technologies, the expressive power of machine translation models is limited, making it impossible to capture the complex dependencies between words in natural languages, resulting in low translation quality. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a machine translation method, a machine translation device, an electronic device and a storage medium, aiming to improve the quality of machine translation.

[0004] To achieve the above object, a first aspect of an embodiment of the present application proposes a machine translation method, the method comprising:

[0005] Obtain source language sentences;

[0006] Performing syntactic analysis on the source language sentence to obtain a target syntactic structure;

[0007] Obtaining a probabilistic context-free grammar of the target syntactic structure;

[0008] Generate a target syntax tree according to the probabilistic context-free grammar; wherein the target syntax tree includes syntax nodes;

[0009] Acquire the syntactic type of the syntactic node, call the word generation model to generate words according to the syntactic type, and obtain the target word of the syntactic type;

[0010] The target words are assembled according to the target syntax tree to obtain a target language sentence.

[0011] In some embodiments, generating a target syntax tree according to the probabilistic context-free grammar includes:

[0012] generating a candidate syntax tree according to the probabilistic context-free grammar;

[0013] The syntax tree scores of the candidate syntax trees are obtained, and the candidate syntax trees are screened according to the syntax tree scores to obtain the target syntax tree.

[0014] In some embodiments, the target syntactic structure includes the syntactic type, the source language sentence includes source words, the probabilistic context-free grammar includes a context-free grammar of the syntactic type and an original grammar probability of the context-free grammar, and generating a candidate syntactic tree according to the probabilistic context-free grammar includes:

[0015] Determining a target grammar from the syntactic type to the source word according to the context-free grammar of the syntactic type;

[0016] Acquire a target grammar probability of the target grammar according to the original grammar probability;

[0017] The target grammar with the highest target grammar probability is selected as the candidate syntax tree.

[0018] In some embodiments, the obtaining the syntax tree scores of the candidate syntax trees, screening the candidate syntax trees according to the syntax tree scores, and obtaining the target syntax tree includes:

[0019] Performing an attention transformation on the candidate syntax tree to obtain a score of the syntax tree;

[0020] The candidate syntax tree with the largest syntax tree score is selected as the target syntax tree.

[0021] In some embodiments, performing syntactic analysis on the source language sentence to obtain a target syntactic structure includes:

[0022] Acquire a target translation model, wherein the target translation model includes an encoder and a decoder;

[0023] Segmenting the source language sentence to obtain source words;

[0024] Encoding the source word by the encoder to obtain a source word vector of the source word;

[0025] The source word vector is decoded by the decoder to obtain the target syntactic structure.

[0026] In some embodiments, the target translation model is trained according to the following steps:

[0027] Obtaining a sample source language sentence and a sample target language sentence;

[0028] Translating the sample source language sentence using a preset translation model to obtain a first predicted target language sentence;

[0029] Translating the first predicted target language sentence using the preset translation model to obtain a predicted source language sentence;

[0030] Translating the predicted source language sentence using the preset translation model to obtain a second predicted target language sentence;

[0031] Perform loss calculation based on the sample source language sentence, the sample target language sentence, the first predicted target language sentence, and the second predicted target language sentence to obtain target loss data;

[0032] The model parameters of the preset translation model are adjusted according to the target loss data to obtain the target translation model.

[0033] In some embodiments, the performing loss calculation according to the sample source language sentence, the sample target language sentence, the first predicted target language sentence, and the second predicted target language sentence to obtain target loss data includes:

[0034] Performing loss calculation on the sample target language sentence and the first predicted target language sentence to obtain a first loss;

[0035] Performing loss calculation on the first predicted target language sentence and the second predicted target language sentence to obtain a second loss;

[0036] Segmenting the sample source language sentence to obtain sample source segments;

[0037] Segmenting the sample target language sentence to obtain sample target segments;

[0038] Randomly mixing the sample source segment and the sample target segment to obtain a reference source language sentence;

[0039] Translating the reference source language sentence using the preset translation model to obtain a third predicted target language sentence;

[0040] Performing loss calculation on the sample target language sentence and the third predicted target language sentence to obtain a third loss;

[0041] The first loss, the second loss and the third loss are summed to obtain the target loss data.

[0042] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a machine translation device, the device comprising:

[0043] A sentence acquisition module, used to acquire source language sentences;

[0044] A syntactic analysis module, used for performing syntactic analysis on the source language sentence to obtain a target syntactic structure;

[0045] A grammar acquisition module, used to acquire a probabilistic context-free grammar of the target syntactic structure;

[0046] A generating module, configured to generate a target syntax tree according to the probabilistic context-free grammar; wherein the target syntax tree comprises syntax nodes;

[0047] A syntactic type acquisition module, used to acquire the syntactic type of the syntactic node, call the word generation model to generate words according to the syntactic type, and obtain the target word of the syntactic type;

[0048] An assembling module is used to assemble the target words according to the target syntax tree to obtain a target language sentence.

[0049] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0050] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0051] The machine translation method, machine translation device, electronic device and computer-readable storage medium proposed in the embodiment of the present application, by obtaining the source language sentence, to perform the subsequent translation process based on the source language sentence, the source language sentence is converted into a target language sentence. The source language sentence is subjected to syntactic analysis to obtain the target syntactic structure, so as to understand the grammatical structure and semantic structure of the source language sentence based on the target syntactic structure. In order to avoid over-reliance on the syntactic structure of the source language sentence in the machine translation process, the probabilistic context-free grammar of the target syntactic structure is obtained, and the complex dependencies between the words of the target language sentence are captured according to the probabilistic context-free grammar, while improving the quality of machine translation, the computational burden brought by the complex syntactic structure can be avoided, and the efficiency of machine translation is improved. In order to clearly show the relationship between the words in the target language sentence, and then improve the language expression ability and language interpretability of the machine translation model, a target syntactic tree is generated according to the probabilistic context-free grammar, so as to determine the grammatical structure and semantic structure of the target language sentence based on the target syntactic tree. The target syntactic tree includes a syntactic node, the syntactic type of the syntactic node is obtained, and the word generation model is called according to the syntactic type to generate words, so as to lexicalize the syntactic node and obtain the target word of the syntactic type. In order to ensure the completeness and accuracy of machine translation, the target words are assembled according to the target syntax tree to obtain the target language sentence, which greatly improves the quality of machine translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1is a flow chart of a machine translation method provided by an embodiment of the present application;

[0053] Figure 2 yes Figure 1 Flow chart of step S120 in FIG.

[0054] Figure 3 is a flow chart of the training process of the target translation model provided in the embodiment of the present application;

[0055] Figure 4 yes Figure 3 Flow chart of step S350 in FIG.

[0056] Figure 5 yes Figure 1 Flow chart of step S140 in FIG.

[0057] Figure 6 yes Figure 5 Another flow chart of step S510 in FIG.

[0058] Figure 7 is a schematic diagram of a syntax tree provided in an embodiment of the present application;

[0059] Figure 8 yes Figure 5 Flow chart of step S520 in FIG.

[0060] Fig. 9 is a schematic diagram of the structure of a machine translation device provided in an embodiment of the present application;

[0061] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0065] Machine translation is the process of using computers to convert one natural language into another. Machine translation technology can achieve cross-language communication and exchange. For example, machine translation technology can enable multi-language financial services such as mobile payments and online banking services, improving the accessibility of web pages. In related technologies, the expressive power of machine translation models is limited, making it impossible to capture the complex dependencies between words in natural languages, resulting in low translation quality.

[0066] Based on this, the embodiments of the present application provide a machine translation method, a machine translation device, an electronic device and a storage medium, aiming to improve the quality of machine translation.

[0067] The machine translation method, machine translation device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the machine translation method in the embodiments of the present application is described.

[0068] The machine translation method provided in the embodiment of the present application relates to the field of financial technology. The machine translation method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the machine translation method, etc., but is not limited to the above forms.

[0069] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0070] Figure 1 is an optional flowchart of the machine translation method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S110 to S160.

[0071] Step S110, obtaining a source language sentence;

[0072] Step S120, performing syntactic analysis on the source language sentence to obtain a target syntactic structure;

[0073] Step S130, obtaining a probabilistic context-free grammar of the target syntactic structure;

[0074] Step S140, generating a target syntax tree according to the probabilistic context-free grammar; wherein the target syntax tree includes syntax nodes;

[0075] Step S150, obtaining the syntactic type of the syntactic node, calling the word generation model to generate words according to the syntactic type, and obtaining the target word of the syntactic type;

[0076] Step S160, assembling the target words according to the target syntax tree to obtain a target language sentence.

[0077] Steps S110 to S160 shown in the embodiment of the present application can capture the complex grammatical structure and semantic structure between natural language output words through probabilistic context-free syntax, so that the machine translation model does not rely on the syntactic structure of the source language sentence during the translation process. While improving the translation quality, it avoids the computational burden brought by the complex syntactic structure and improves the translation speed.

[0078] In step S110 of some embodiments, the banking system usually provides video customer service, voice customer service, and web online service, etc. These services are oriented to customer groups with different language backgrounds, and provide financial transactions, transaction inquiries and other services to customer groups. In order to improve the availability and accessibility of these services, and facilitate different customer groups to conduct financial transactions, query account details and transaction records, etc., it is necessary to convert the source language sentences in the service from the original language to the target language required by the customer through machine translation technology. The source language sentence is the sentence in the original language to be translated. The original language is the language translated by the machine translation model, and the target language is the language obtained after being translated by the machine translation model. The target language can be set by the client according to the needs to provide multi-language support. An automated tool integrated with optical character recognition technology and speech recognition technology can be used to extract sentences in the original language from video, voice or text to obtain source language sentences.

[0079] See also Figure 2 In some embodiments, step S120 may include but is not limited to steps S210 to S240:

[0080] Step S210, obtaining a target translation model, where the target translation model includes an encoder and a decoder;

[0081] Step S220, segmenting the source language sentence to obtain source words;

[0082] Step S230, encoding the source word through an encoder to obtain a source word vector of the source word;

[0083] Step S240, decoding the source word vector through a decoder to obtain a target syntactic structure.

[0084] In step S210 of some embodiments, the target translation model can be deployed in various services of the banking system so that each service can provide instant multilingual support for different customer groups. The target translation model is a pre-trained machine translation model, and the target translation model includes an encoder and a decoder. Both the encoder and the decoder are pre-trained models. The encoder is used to convert input data into a hidden coding representation, and the encoder can be a BERT model, a bidirectional long short-term memory network (Long Short-Term Memory, LSTM), etc. The decoder is used to generate an output sequence according to the hidden coding representation, and the decoder can be a long short-term memory network, a generative model (Generative Pre-trained Transformer, GPT), etc.

[0085] In step S220 of some embodiments, a word segmentation algorithm is called to segment the source language sentence, and the source language sentence is divided into words to obtain source words. The word segmentation algorithm includes a rule-based word segmentation method, a statistical word segmentation method, and a deep learning-based word segmentation method. The rule-based word segmentation method includes a forward maximum matching method, a reverse maximum matching method, a two-way matching method, etc., the statistical word segmentation method includes a hidden Markov model, a conditional random field, etc., and the deep learning-based word segmentation method includes a transformer model, a BERT model, etc.

[0086] In step S230 of some embodiments, the source word is encoded by an encoder, and the source word is converted into a hidden coding representation to obtain a source word vector of the source word. The source word vector contains semantic information of the source word, such as the literal meaning of the source word, the context represented by the source word, and the user intention implied by the source word. If the source word is a bank, its literal meaning and context can be a financial institution, and the user intention can be accessing financial transaction services.

[0087] In step S240 of some embodiments, the encoder and the decoder are combined by grafting to enhance the feature expression ability of the model, and the output of the encoder is used as the input of the decoder, that is, the decoder decodes the source word vectors of all source words output by the encoder, converts the source word vectors into an output sequence, and obtains a target syntactic structure. The target syntactic structure is the grammatical structure of the source language sentence, wherein the source language sentence includes source words, and the target syntactic structure includes a syntactic type. The syntactic type is used to describe the grammatical category of the source word, and can indicate the part of speech of the source word and the combination rules between the source words in the sentence. The syntactic type can be a noun phrase, a verb phrase, a noun, a preposition, etc. The part of speech can reflect the grammatical function of the source word in the source language sentence, and the part of speech includes adjectives, nouns, verbs, and prepositions. For example, the source language sentence is "The cat sat on the mat", and the source language sentence is subjected to syntactic analysis, and the part of speech of the source word The is a determiner, and the part of speech of the source word cat is a noun. The clause The Cat formed by the combination of these two source words is a noun phrase.

[0088] Through the above steps S210 to S240, the syntactic structure of the source language sentence can be obtained to express the deep semantics of the source language sentence based on the syntactic structure, and then generate a high-quality target language sentence according to the syntactic structure. Compared with the original encoder and decoder, the grafting of the pre-trained model can achieve better feature extraction performance.

[0089] See also Figure 3 In some embodiments, the training process of the target translation model may include but is not limited to steps S310 to S360:

[0090] Step S310, obtaining a sample source language sentence and a sample target language sentence;

[0091] Step S320, translating the sample source language sentence using a preset translation model to obtain a first predicted target language sentence;

[0092] Step S330, translating the first predicted target language sentence using a preset translation model to obtain a predicted source language sentence;

[0093] Step S340, translating the predicted source language sentence using a preset translation model to obtain a second predicted target language sentence;

[0094] Step S350, performing loss calculation based on the sample source language sentence, the sample target language sentence, the first predicted target language sentence, and the second predicted target language sentence to obtain target loss data;

[0095] Step S360, adjusting the model parameters of the preset translation model according to the target loss data to obtain a target translation model.

[0096] In step S310 of some embodiments, sample source language sentences and sample target language sentences are obtained from a training data set, the sample source language sentences and the sample target language sentences constitute a sample pair, the sample source language sentences are sentences in the original language used for model training, and the sample target language sentences are sentences in the target language used for model training and associated with the sample source language sentences.

[0097] In step S320 of some embodiments, the preset translation model includes an encoder, a decoder, and an output layer connected in series, and the target translation model is trained by the preset translation model. The sample source language sentence is input into the preset translation model for machine translation, and the sample source language sentence is converted from the source language to the target language to obtain a first predicted target language sentence.

[0098] In steps S330 to S340 of some embodiments, the reverse translation mode refers to a translation method from a target language to a source language. The reverse translation mode will produce sentences containing ambiguity and complex structures. In order to improve the generalization of the translation model and enable the banking system to provide financial business services for customer groups with different language backgrounds, the reverse translation mode is used to train the preset translation model to increase the diversity of training data, promote the translation model to learn the differences between different languages, and then improve the translation quality of the translation model for different samples, and improve the translation model's understanding of complex sentences. Specifically, first, the first predicted target language sentence is input into the preset translation model for machine translation, and the first predicted target language sentence is converted from the target language to the source language to obtain a predicted source language sentence. Then the predicted source language sentence is input into the preset translation model for machine translation, and the predicted source language sentence is converted from the source language to the target language to obtain a second predicted target language sentence.

[0099] In step S350 of some embodiments, loss calculation is performed on the sample source language sentence, the sample target language sentence, the first predicted target language sentence and the second predicted target language sentence through a loss function, and the output value of the loss function is used as the target loss data to evaluate the translation performance of the preset translation model on the training sample according to the target loss data.

[0100] In step S360 of some embodiments, during the training process, the model parameters of the preset translation model are adjusted according to an optimization algorithm such as a gradient descent method or an adaptive learning rate algorithm. When the target loss data reaches a minimum value, the training round reaches an upper limit of the training round, or the change value between the target loss data of two adjacent trainings is less than a preset change threshold, the training of the preset translation model is stopped and the preset translation model at this time is used as the target translation model.

[0101] Through the above steps S310 to S360, a target translation model with better translation performance can be obtained, and sentence translation is performed based on the target translation model, thereby improving the quality of machine translation.

[0102] See also Figure 4 In some embodiments, step S350 may include but is not limited to step S410 or step S480:

[0103] Step S410, calculating the loss of the sample target language sentence and the first predicted target language sentence to obtain a first loss;

[0104] Step S420, calculating the loss of the first predicted target language sentence and the second predicted target language sentence to obtain a second loss;

[0105] Step S430, segmenting the sample source language sentence to obtain sample source segments;

[0106] Step S440, segmenting the sample target language sentence to obtain sample target segments;

[0107] Step S450, randomly mixing the sample source segment and the sample target segment to obtain a reference source language sentence;

[0108] Step S460, translating the reference source language sentence using a preset translation model to obtain a third predicted target language sentence;

[0109] Step S470, calculating the loss of the sample target language sentence and the third predicted target language sentence to obtain a third loss;

[0110] Step S480, summing the first loss, the second loss and the third loss to obtain target loss data.

[0111] In step S410 of some embodiments, the preset translation model outputs the predicted probability of each word in the first predicted target language sentence. The one-hot encoding of each word in the sample target language sentence is obtained, and for each word, the predicted probability and the one-hot encoding are calculated by the cross entropy loss function to obtain the cross entropy loss of the word. The calculation formula of the cross entropy loss is as follows:

[0112] loss i =-y i P i ,

[0113] Among them, loss i represents the cross entropy loss of the i-th word; y i represents the unique hot encoding of the i-th word; P i It represents the probability that the output of the preset translation model at the ith position of the first predicted target language sentence is the ith word, that is, the predicted probability of the ith word.

[0114] The cross entropy losses of each word are added together, and the result of the addition is divided by the sentence length of the first predicted target language sentence to obtain the first loss. The sentence length is the number of words in the first predicted target language sentence. The first loss is used to measure the difference between the sample target language sentence and the first predicted target language sentence.

[0115] In step S420 of some embodiments, the second loss is used to measure the difference between the first predicted target language sentence and the second predicted target language sentence. Referring to step S410, the loss is calculated for the first predicted target language sentence and the second predicted target language sentence to obtain the second loss.

[0116] In step S430 of some embodiments, the sample source language sentence is segmented by a word segmentation algorithm or a word segmentation tool, and words are extracted from the sample source language sentence to obtain sample source segments. The sample source segments are single words or phrases composed of multiple words in the sample source language sentence.

[0117] In step S440 of some embodiments, the sample target language sentence is segmented by a word segmentation algorithm or a word segmentation tool, and words are extracted from the sample target language sentence to obtain a sample target segment. The sample target segment is a single word or a phrase composed of multiple words in the sample target language sentence. The sample target segment corresponds to the sample source segment one by one.

[0118] In step S450 of some embodiments, in order to promote cross-language feature learning of the translation model and enable the model to cope with complex translation scenarios where the source language and the target language are mixed, the sample source segments and the sample target segments are randomly mixed to obtain a baseline source language sentence, so as to improve the model's ability to understand the sentence and accurately establish the correspondence between different language segments.

[0119] Specifically, the number of sample source segments in the sample source language sentence is counted to obtain a first number, and the first number is equal to the number of sample target segments in the sample target language sentence. A second number of sample source segments are selected from the sample source language sentence, and the second number of sample source segments are replaced with corresponding sample target segments to obtain a reference source language sentence, and the second number is greater than zero and less than the first number. Alternatively, a third number of sample target segments are selected from the sample target language sentence, and the third number of sample target segments are replaced with corresponding sample source segments to obtain a reference source language sentence, and the third number is greater than zero and less than the first number.

[0120] In step S460 of some embodiments, the reference source language sentence is input into a preset translation model for machine translation to obtain a third predicted target language sentence.

[0121] In step S470 of some embodiments, referring to step S410, loss calculation is performed on the sample target language sentence and the third predicted target language sentence to obtain a third loss. The third loss is used to measure the difference between the sample target language sentence and the third predicted target language sentence.

[0122] In step S480 of some embodiments, the first loss, the second loss and the third loss are summed to obtain target loss data. In order to further improve the quality of sentence translation, the first score of the first predicted target language sentence, the second score of the second predicted target language sentence and the third score of the third predicted target language sentence are respectively obtained through the scoring model. The higher the score, the higher the quality of the sentence translation. The first loss, the second loss, the third loss, the negative of the first score, the negative of the second score and the negative of the third score are summed to obtain target loss data. The scoring model can be a bilingual evaluation understudy (BLEU) model, a cross-lingual optimized metric for evaluation of translation (COMET) model, etc.

[0123] Through the above steps S410 to S480, the target loss data can be improved, so as to optimize the model performance of the preset translation model based on the target loss data and improve the accuracy of machine translation.

[0124] In step S130 of some embodiments, the existing machine translation model has limited expressive power, cannot obtain accurate translation results, and occupies too many computing resources during the translation process. For example, the autoregressive model uses a word-by-word generation method to predict the next word of each word. When processing long sentences, it cannot capture the long-distance dependencies of the sentences, resulting in a decrease in the quality of the generated sentences and low translation efficiency. The Vanilla NAT model assumes that the relationship between the words in the source language sentence is independent of each other, allowing all words to be generated in parallel. Although the translation efficiency is improved by parallel prediction, the contextual dependencies between the target language sentences are ignored, resulting in rich expressiveness and diversity in the target language, causing model performance degradation and serious multimodal problems. The multimodal problem refers to the translation of a source language sentence into multiple target language sentences with the same semantics but different expressions.

[0125] In order to improve translation accuracy and translation efficiency and reduce multimodal problems, the present application embodiment obtains a probabilistic context-free grammar (PCFG) of a target syntactic structure. Probabilistic context-free grammar extends the grammar of context-free grammar (CFG) and assigns a probability value to the context-free grammar. Probabilistic context-free grammar includes the context-free grammar of the syntactic type in the target syntactic structure and the original grammar probability of the context-free grammar. Context-free grammar is used to describe the generation rules of syntactic types. For example, the syntactic type is a noun phrase, and the context-free grammar is a noun phrase composed of a qualifier and a noun. The same syntactic type may have multiple context-free grammars, and the original grammar probability is the probability of selecting a context-free grammar under a given syntactic type.

[0126] Probabilistic context-free grammars can be obtained by training large-scale syntactic treebanks such as Penn Treebank. Large-scale syntactic treebanks include a set of sentences that conform to the grammar rules of the target language. Syntactic types and context-free grammars of syntactic types are extracted from the sentence set through a grammatical analysis tool. For each syntactic type, the frequency of each context-free grammar of the syntactic type in the sentence set is counted, and the total frequency of all context-free grammars of the syntactic type in the sentence set is generated statistically. The probability of the context-free grammar is obtained based on the ratio of the frequency of each context-free grammar to the total frequency, and the syntactic type, the context-free grammar of the syntactic type and the probability of the context-free grammar are used as grammar rules. The grammar rules are searched according to the syntactic type of the target syntactic structure, and the context-free grammar that matches the syntactic type in the grammar rules and the probability of the context-free grammar are used as probabilistic context-free grammars.

[0127] Probabilistic context-free grammars can better capture the complex grammatical and semantic structures in the target language, so that the model does not rely on the syntactic structure of the source language sentence during the production translation process, but adjusts the sentence generation process according to different context-free grammars and the probabilities of context-free grammars, so that the target translation model has good flexibility and scalability, can adapt to different language types, and then generate target language sentences that conform to the grammatical rules of the target language through a single translation, thereby improving the quality of machine translation. At the same time, probabilistic context-free grammars can avoid the translation computational burden brought by complex syntactic structures and improve the translation speed, which is very beneficial for scenarios that require rapid generation of translation results, such as real-time conversation translation. Compared with complex autoregressive models, probabilistic context-free grammars can be quickly constructed according to grammatical rules, thereby improving the training efficiency of the translation model.

[0128] The target syntax tree is generated according to the probabilistic context-free grammar, and the target syntax tree includes syntax nodes. The target syntax tree can capture the bidirectional complex dependencies between adjacent words and non-adjacent words, and can effectively capture long-distance dependencies, thereby enhancing the translation effect of the model and reducing errors when generating long sentences. In addition, the process of PCFG generating sentences is parallel, which makes the inference time significantly lower than the autoregressive model, reduces the use of computing resources, greatly improves the speed and efficiency of translation, can quickly respond to the client's translation request, realize real-time translation, and accelerate the development and deployment of the target translation model.

[0129] See also Figure 5 In some embodiments, step S140 may include but is not limited to steps S510 to S520:

[0130] Step S510, generating a candidate syntax tree according to a probabilistic context-free grammar;

[0131] Step S520: Obtain the syntax tree scores of the candidate syntax trees, and screen the candidate syntax trees according to the syntax tree scores to obtain a target syntax tree.

[0132] In step S510 of some embodiments, the probabilistic context-free grammar can capture the complex syntactic structure in natural language to generate a reasonable syntactic tree to obtain a candidate syntactic tree. The candidate syntactic tree is a syntactic tree generated from a source language sentence that conforms to the rules of the target language.

[0133] In step S520 of some embodiments, in order to make the translated sentence structure conform to the grammatical rules of the target language and further improve the quality and readability of the translation, the candidate syntax trees are evaluated according to the scoring mechanism to obtain the syntax tree scores of the candidate syntax trees, and the better candidate syntax trees are selected according to the syntax tree scores to obtain the target syntax tree.

[0134] In the above steps S510 to S520, multiple candidate syntax trees can be generated by the probabilistic context-free grammar, ensuring the generation diversity. By screening the candidate syntax trees, the diversity of candidate results can be retained, which helps to generate richer and more diverse translations.

[0135] See also Figure 6 In some embodiments, step S510 may include but is not limited to steps S610 to S620:

[0136] Step S610, determining a target grammar from the syntactic type to the source word according to the context-free grammar of the syntactic type;

[0137] Step S620, obtaining a target grammar probability of the target grammar according to the original grammar probability;

[0138] Step S630: Select the target grammar with the highest grammar probability as the candidate syntax tree.

[0139] In step S610 of some embodiments, in order to make the grammar of the source language sentence conform to the target language, a set of context-free grammars covering the syntactic types of all source words in the source language sentence is obtained to get the target grammar.

[0140] The source language sentence consists of n source words. The source language sentence is represented as w = w1w2...wn, where wi represents the i-th source word. Create a table of size n×n, and the cell in the i-th row and j-th column of the table is denoted as T[i, j]. For the i-th source word of the source language sentence, obtain the set of context-free grammars that can generate the syntactic types required for the i-th source word, and fill this set into T[i, i]. For example, the context-free grammar of a prepositional phrase is PP (prepositional phrase) → P (preposition) + NP (noun phrase), and the context-free grammar of a preposition is P → and. For the source word "and", the set includes [PP → P + NP] and [P → and]. Update the cell T[i, j] by dynamic programming. Specifically, traverse all possible clause lengths. The clause length is the number of source words contained in the clause, and the clause length is greater than or equal to 1 and less than or equal to n. Starting from the i-th source word, determine the clause as w[i, i + l - 1] according to the clause length l, where 1 ≤ i ≤ n - l + 1. This clause includes multiple source words in the source language sentence starting from the i-th source word and ending at the (i + l - 1)-th source word. For the clause w[i, i + l - 1], traverse all possible split points k, where i ≤ k < i + l - 1. According to the split point k, divide the clause w[i, i + l - 1] into a first clause w[i, k] and a second clause w[k + 1, i + l - 1]. Obtain the set of context-free grammars that can generate the syntactic types of the first clause w[i, k] to get the first set, and obtain the set of context-free grammars that can generate the syntactic types of the second clause w[k + 1, i + l - 1] to get the second set. Multiply the original grammar probability of the context-free grammar in the first set by the original grammar probability of the context-free grammar in the second set to get the candidate probability. When the candidate probability reaches the maximum, fill the first set and the second set into the cell T[i, i + l - 1]. Take the grammar in T[1, n] as the target grammar.

[0141] In step S620 of some embodiments, multiply the original grammar probabilities of each context-free grammar in the target grammar to get the target grammar probability.

[0142] In step S630 of some embodiments, the target grammar with the highest target grammar probability is selected, and the syntax type corresponding to the context-free grammar in the target grammar is converted into a syntax node. The root node is screened out from the syntax node according to the syntax type, and other syntax nodes except the root node are recursively added according to the target grammar, and the target grammar is converted into a tree structure to obtain a candidate syntax tree. It should be noted that the same source language sentence can be generated by different target grammars, so there are also multiple different target grammars with equal target grammar probabilities, and multiple candidate syntax trees can be generated.

[0143] If the source language sentence is "The cat sits on the mat" and the target language is English, since adjectives in English are on the left side of nouns and verb phrases are on the right side of verbs, the candidate syntactic tree can be a right-heavy syntactic dependency tree. The candidate syntactic tree is as follows: Figure 7 As shown. The right-heavy syntactic dependency tree is a special syntactic tree that gives priority to the development of the right subtree and limits the height of the left subtree. The generation process of the right-heavy syntactic dependency tree is as follows: Create the root node of the syntactic tree to initialize the root node, and the syntactic node S represents the starting point of the entire sentence. For each syntactic node, recursively add syntactic nodes according to the target grammar. The newly added syntactic nodes are mainly placed on the right to form a right-heavy structure. The target grammar includes S→NP (noun phrase) + VP (verb phrase), NP→Det (determiner) + N (noun), VP→V (verb) + PP (prepositional phrase), PP→P (preposition) + NP (noun phrase).

[0144] In the above steps S610 to S630, a syntax tree is generated by using a probabilistic context-free grammar, and a complex dependency relationship between different words can be established according to the syntax tree to obtain a syntax tree that conforms to the rules of the target language, thereby improving the quality of machine translation.

[0145] See also Figure 8 In some embodiments, step S520 may include but is not limited to step S810 or step S820:

[0146] Step S810, performing attention transformation on the candidate syntax tree to obtain a syntax tree score;

[0147] Step S820: Select the candidate syntax tree with the largest syntax tree score as the target syntax tree.

[0148] In step S810 of some embodiments, an attention calculation is performed on the candidate syntax tree through an attention mechanism to obtain a syntax tree score. The candidate syntax tree is vectorized to obtain a syntax vector, and the syntax vector is multiplied with the query weight matrix, the key weight matrix, and the value weight matrix respectively to obtain a query vector, a key vector, and a value vector. The query vector, the key vector, and the value vector have the same feature dimension. Attention calculation is performed on the query vector, the key vector, and the value vector to obtain a syntax tree score. The method of attention calculation is as follows:

[0149]

[0150] Where A represents the syntactic tree score; Q represents the query vector; K represents the key vector; V represents the value vector; d k represents the feature dimension; T represents the transposition operation.

[0151] In step S820 of some embodiments, in order to enhance sentence fluency and enable the translation model to generate more fluent and natural sentences, a candidate syntax tree with the largest syntax tree score is selected as a target syntax tree.

[0152] Through the above steps S810 to S820, a more optimal syntax tree can be generated, ensuring that the generated target language sentence conforms to both the context semantics and the syntax norms of the target language.

[0153] In step S150 of some embodiments, a syntactic node of the target syntactic tree is traversed by using a depth optimization search or a breadth optimization search. If the syntactic node is a leaf node, the syntactic type of the syntactic node is obtained, and a word generation model is called according to the syntactic type and the target syntactic structure to generate words, and a target word of the syntactic type is obtained. The word generation model is a generation model based on deep learning, such as GPT, LSTM, etc. For example, if the syntactic type is a noun and the context provided by the target syntactic structure is a cat, then the target word is cat.

[0154] In step S160 of some embodiments, according to the structure of the target syntax tree, starting from the leaf nodes of the target syntax tree, the target words are gradually merged, and the sentence is constructed from the bottom up until the root node is reached to complete the sentence construction and obtain the target language sentence. For example, the target syntax tree is represented as [NP[Det][N]], the target word corresponding to the syntax type NN is cat, and the target word corresponding to the syntax type DT is the, and the noun phrase NP obtained by merging the target words is "the cat". The target translation model also includes an output layer, and in order to ensure the fluency of the target language sentence, the target language sentence is optimized through the output layer.

[0155] See also Fig. 9 The present application also provides a machine translation device that can implement the above-mentioned machine translation method. The device includes:

[0156] A sentence acquisition module 910, used to acquire source language sentences;

[0157] The syntax analysis module 920 is used to perform syntax analysis on the source language sentence to obtain a target syntax structure;

[0158] A grammar acquisition module 930, used to acquire a probabilistic context-free grammar of the target syntactic structure;

[0159] A generating module 940, configured to generate a target syntax tree according to the probabilistic context-free grammar; wherein the target syntax tree includes syntax nodes;

[0160] The syntactic type acquisition module 950 is used to acquire the syntactic type of the syntactic node, and call the word generation model to generate words according to the syntactic type to obtain the target word of the syntactic type;

[0161] The assembling module 960 is used to assemble the target words according to the target syntax tree to obtain a target language sentence.

[0162] The specific implementation of the machine translation device is substantially the same as the specific implementation of the above-mentioned machine translation method, and will not be described in detail here.

[0163] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned machine translation method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0164] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0165] The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0166] The memory 1020 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1020 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1020, and the processor 1010 calls and executes the machine translation method of the embodiment of this application;

[0167] Input / output interface 1030, used to implement information input and output;

[0168] The communication interface 1040 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0169] Bus 1050 , which transmits information between various components of the device (e.g., processor 1010 , memory 1020 , input / output interface 1030 , and communication interface 1040 );

[0170] The processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 are connected to each other in communication within the device via a bus 1050 .

[0171] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned machine translation method is implemented.

[0172] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0173] The machine translation method, machine translation device, electronic device and computer storage medium provided in the embodiments of the present application can capture the complex grammatical structure and semantic structure between natural language output words through probabilistic context-free syntax, so that the machine translation model does not rely on the syntactic structure of the source language sentence during the translation process. While improving the translation quality, it avoids the computational burden brought by the complex syntactic structure and improves the translation speed.

[0174] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0175] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0176] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0177] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0178] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0179] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0180] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0181] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0182] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0183] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0184] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A machine translation method, characterized in that: The method comprises: Obtain source language sentences; Performing syntactic analysis on the source language sentence to obtain a target syntactic structure; Obtaining a probabilistic context-free grammar of the target syntactic structure; Generate a target syntax tree according to the probabilistic context-free grammar; wherein the target syntax tree includes syntax nodes; Acquire the syntactic type of the syntactic node, call the word generation model to generate words according to the syntactic type, and obtain the target word of the syntactic type; The target words are assembled according to the target syntax tree to obtain a target language sentence.

2. The method according to claim 1, characterized in that The generating a target syntax tree according to the probabilistic context-free grammar comprises: generating a candidate syntax tree according to the probabilistic context-free grammar; The syntax tree scores of the candidate syntax trees are obtained, and the candidate syntax trees are screened according to the syntax tree scores to obtain the target syntax tree.

3. The method according to claim 2, characterized in that The target syntactic structure includes the syntactic type, the source language sentence includes source words, the probabilistic context-free grammar includes the context-free grammar of the syntactic type and the original grammar probability of the context-free grammar, and the generating a candidate syntactic tree according to the probabilistic context-free grammar includes: Determining a target grammar from the syntactic type to the source word according to the context-free grammar of the syntactic type; Acquire a target grammar probability of the target grammar according to the original grammar probability; The target grammar with the highest target grammar probability is selected as the candidate syntax tree.

4. The method according to claim 2, characterized in that: The obtaining the syntax tree score of the candidate syntax tree, screening the candidate syntax tree according to the syntax tree score, and obtaining the target syntax tree includes: Performing an attention transformation on the candidate syntax tree to obtain a score of the syntax tree; The candidate syntax tree with the largest syntax tree score is selected as the target syntax tree.

5. The method according to any one of claims 1 to 4, characterized in that: The step of performing syntactic analysis on the source language sentence to obtain a target syntactic structure includes: Acquire a target translation model, wherein the target translation model includes an encoder and a decoder; Segmenting the source language sentence to obtain source words; Encoding the source word by the encoder to obtain a source word vector of the source word; The source word vector is decoded by the decoder to obtain the target syntactic structure.

6. The method according to claim 5, characterized in that The target translation model is trained according to the following steps: Obtaining a sample source language sentence and a sample target language sentence; Translating the sample source language sentence using a preset translation model to obtain a first predicted target language sentence; Translating the first predicted target language sentence using the preset translation model to obtain a predicted source language sentence; Translating the predicted source language sentence using the preset translation model to obtain a second predicted target language sentence; Perform loss calculation based on the sample source language sentence, the sample target language sentence, the first predicted target language sentence, and the second predicted target language sentence to obtain target loss data; The model parameters of the preset translation model are adjusted according to the target loss data to obtain the target translation model.

7. The method according to claim 6, characterized in that The performing loss calculation according to the sample source language sentence, the sample target language sentence, the first predicted target language sentence and the second predicted target language sentence to obtain target loss data includes: Performing loss calculation on the sample target language sentence and the first predicted target language sentence to obtain a first loss; Performing loss calculation on the first predicted target language sentence and the second predicted target language sentence to obtain a second loss; Segmenting the sample source language sentence to obtain sample source segments; Segmenting the sample target language sentence to obtain sample target segments; Randomly mixing the sample source segment and the sample target segment to obtain a reference source language sentence; Translating the reference source language sentence using the preset translation model to obtain a third predicted target language sentence; Performing loss calculation on the sample target language sentence and the third predicted target language sentence to obtain a third loss; The first loss, the second loss and the third loss are summed to obtain the target loss data.

8. A machine translation device, characterized in that: The device comprises: A sentence acquisition module, used to acquire source language sentences; A syntactic analysis module, used for performing syntactic analysis on the source language sentence to obtain a target syntactic structure; A grammar acquisition module, used to acquire a probabilistic context-free grammar of the target syntactic structure; A generating module, configured to generate a target syntax tree according to the probabilistic context-free grammar; wherein the target syntax tree comprises syntax nodes; A syntactic type acquisition module, used to acquire the syntactic type of the syntactic node, call the word generation model to generate words according to the syntactic type, and obtain the target word of the syntactic type; An assembling module is used to assemble the target words according to the target syntax tree to obtain a target language sentence.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images