A text translation method and system based on a language translation model

Abstract

The present application relates to the field of translation, more particularly, it relates to a text translation method and system based on a language translation model. A text translation system based on a language translation model comprises a text to be translated acquisition module, a language translation model management module, a language translation module and a target translated text construction module. The present application translates the text to be translated through the language translation model based on the improved Transformer model, can combine the semantic information in the text to be translated, that is, the syntactic dependency relationship between the words in the text to be translated, so as to realize the fusion of the grammar structure of the corresponding language of the text to be translated, and further improve the accuracy of text translation.

Description

Technical Field

[0001] This invention relates to the field of translation, and more specifically, to a text translation method and system based on a language translation model. Background Technology

[0002] Using machine translation to assist human translation can save significant human resources costs. Currently, the Transformer model is generally used for machine translation, which involves building an end-to-end encoder-decoder neural network to encode the language to be translated into semantic vector representations in a continuous space, and then the decoder generates the target language based on these semantic vector representations. However, when using the traditional Transformer model for machine translation, word embedding is performed by adding positional encoding to word vectors. This method cannot incorporate syntactic structure information from the text, leading to mistranslations and omissions. Summary of the Invention

[0003] This invention uses a language translation model based on an improved Transformer model to translate the text to be translated. It can combine the semantic information in the text to be translated, that is, the syntactic dependency relationship between words in the text to be translated, thereby achieving the fusion of the grammatical structure of the language corresponding to the text to be translated, and thus improving the accuracy of text translation.

[0004] A text translation method based on a language translation model includes:

[0005] Obtain the text to be translated and perform word segmentation on it to obtain N words to be translated, where N is the total number of words in the text to be translated;

[0006] Input N words to be translated into a trained language translation model for processing, and output a sequence of target words. The sequence of target words stores M target words, which are the words in the target language corresponding to the text to be translated.

[0007] The target words in the target word sequence are concatenated to create the target translation text;

[0008] The language translation model is based on an improved Transformer model, comprising an encoder and a decoder. The encoder includes an encoder word embedding layer, a first encoding block, a second encoding block, a third encoding block, a fourth encoding block, a fifth encoding block, and a sixth encoding block. The decoder includes a decoder word embedding layer, a first decoding block, a second decoding block, a third decoding block, a fourth decoding block, a fifth decoding block, a sixth decoding block, and a word prediction layer. The encoder word embedding layer embeds words into the target language corresponding to the text to be translated. The first, second, third, fourth, fifth, and sixth encoding blocks extract features from the input matrix using a self-attention mechanism to obtain a hidden encoding matrix. The decoder word embedding layer embeds words into the target language corresponding to the translated text to obtain a target word vector matrix. The first, second, third, fourth, fifth, and sixth decoding blocks decode the input matrix using a self-attention mechanism to update the target word vector matrix. The encoder and decoder word embedding layers are improved based on semantic information. The word prediction layer predicts target words based on the target word vector matrix.

[0009] As a preferred aspect of the present invention, the word embedding operation through the encoder word embedding layer specifically includes the following steps:

[0010] N words to be translated are sequentially fed into the source word vector database for matching, and the source word vector sequence F is output, where F is {f1, f2, ..., fn}. n …f N}, where f n Let n be the source word vector corresponding to the nth word to be translated, and n = 1, 2, 3...N; there is a one-to-one correspondence between the words in the source word vector library and the source word vectors in the source word vector library are generated by the improved CBOW model;

[0011] The source word vector sequence F is fed into the trained syntactic dependency extraction model for processing, and the output source word vector matrix G to be translated is generated. The size of the source word vector matrix G is N×D, where D is the feature dimension corresponding to each word to be translated in the source word vector matrix G.

[0012] As a preferred aspect of the present invention, the syntactic dependency extraction model comprises a word encoding layer and L attention- and edge-label-based guided graph convolutional networks, wherein the word encoding layer is based on a Bi-LSTM model, and the attention- and edge-label-based guided graph convolutional networks comprise an edge-aware node joint update layer, an edge-aware node update layer, and M attention-guided graph convolutional networks, wherein the attention-guided graph convolutional networks comprise an attention-guided layer, a dense connection layer, and a linear combination layer;

[0013] The source word vector sequence F is processed using a trained syntactic dependency extraction model, specifically including the following steps:

[0014] S1: Feed the source word vector sequence F into the word encoding layer for processing, and output the word vector encoding sequence H, where H = {h1, h2, ..., h...} n …h N}, where h n For the source word vector f n The word vector encoding output by the word encoding layer has a dimension of D.

[0015] S2: The word vector encoding matrix U0 is fed into the joint update layer of edge-aware nodes for processing. The processing formula is as follows:

[0016]

[0017] Among them U l Let E be the word vector encoding matrix output by the joint update layer of the l-th edge-aware node. l-1 (:,:,i) represents the syntactic dependency embedding adjacency matrix E output by the update layer of the (l-1)th edge-aware node. l-1 The slice corresponding to the i-th channel, i = 1, 2, 3…I, where I is the syntactic dependency embedding adjacency matrix E. l-1 The total number of middle channels, W1 and W2 are the first and second weight matrices respectively, U l (i) is E l-1 (:,:,i) is a slice of the i-th word vector encoding matrix obtained by processing W2, where pool is the mean operation; σ is the ReLU activation function; A is the syntactic dependency matrix, and the size of the syntactic dependency matrix is ​​N×N;

[0018] S3: The adjacency matrix E embedded in the syntactic dependency relations is updated through the edge-aware node update layer. l Update the syntactic dependency relation embedding into the adjacency matrix E. l The syntactic dependency embedding adjacency matrix output by the update layer for the l-th edge-aware node is updated using the following formula:

[0019]

[0020] Among them, u l (k) is the word vector encoding matrix U l The word vector encoding corresponding to the k-th row in the code, u l (j) is the word vector encoding matrix U l The word vector encoding corresponding to the j-th row in the matrix, and W3 is the third weight matrix;

[0021] Embed all updated syntactic dependency relations into vector E l Concatenate (k, j, :) to obtain the syntactic dependency relation embedding adjacency matrix E. l ;

[0022] S4: The word vector encoding matrix U after processing through the joint update layer of edge-aware nodes. l The data is fed into M attention-guided graph convolutional networks for processing.

[0023] S5: Convert the word vector encoding matrix U output from step S4 l Feed the input into the next attention- and edge-label guided graph convolutional network, iteratively executing steps S2-S4; until all L attention- and edge-label guided graph convolutional networks have been executed, output the word vector encoding matrix U output by the last attention- and edge-label guided graph convolutional network. L G is the vector matrix of the source words to be translated.

[0024] In a preferred aspect of the present invention, the syntactic dependency matrix A is obtained by performing syntactic dependency analysis on the source word vector sequence F to obtain a syntactic dependency parsing tree, and then constructing the syntactic dependency matrix A based on the syntactic dependency parsing tree. The data A in the syntactic dependency matrix A... kj The value of A is either 1 or 0. kj This represents the data corresponding to the k-th row and j-th column in the syntactic dependency matrix A, where k = 1, 2, 3…N, j = 1, 2, 3…N. When A… kj =1 indicates that the k-th source word vector in the source word vector sequence F and the j-th source word vector in the source word vector sequence F have a syntactic dependency relationship on the syntactic dependency parsing tree. When A kj =0 indicates that the k-th source word vector in the source word vector sequence F and the j-th source word vector in the source word vector sequence F do not have a syntactic dependency relationship on the syntactic dependency parsing tree. In particular, when k=j, A kj =1;

[0025] When l = 0, the embedding adjacency matrix E l The size is N×N×I, where E l (k, j, :) is the embedding adjacency matrix E l The syntactic dependency embedding vector corresponding to the k-th row and j-th column is given by E. l The length of (k, j, :) is I, and the syntactic dependency embedding vector is E. l(k, j, :) is obtained through the syntactic dependency tree and the syntactic dependency relation embedding vector library. When the syntactic dependency tree indicates that the k-th source word vector in the source word vector sequence F has a syntactic dependency relation with the j-th source word vector in the source word vector sequence F, this syntactic dependency relation is matched with the syntactic dependency relation embedding vector library, and the syntactic dependency relation embedding vector corresponding to the syntactic dependency relation between the k-th source word vector in the source word vector sequence F and the j-th source word vector in the source word vector sequence F is output. The syntactic dependency relation embedding vector library stores the syntactic dependency relation and its corresponding syntactic dependency relation embedding vector, and the syntactic dependency relation embedding vector is obtained by training the source language corresponding to the text to be translated.

[0026] As a preferred aspect of the present invention, the word vector encoding matrix U after being processed by the edge-aware node joint update layer is... l The input is fed into M attention-guided graph convolutional networks for processing, and the output is the word vector encoding matrix U output by the last attention-guided graph convolutional network. l Specifically, it includes the following steps:

[0027] S4.1: Encode the word vector matrix U l The data is fed into the attention guidance layer for processing to obtain attention weights. Let be the attention weight output of the r-th attention-guided sublayer in the m-th attention-guided graph convolutional network of the l-th attention-guided graph convolutional network based on attention and edge labels; where m = 1, 2, 3…M, r = 1, 2, 3…R, and R is the total number of attention-guided sublayers in the attention-guided layer; the calculation formula is as follows:

[0028]

[0029] in Let r be the query weight matrix in the attention-guided sublayer. Let r be the value weight matrix in the attention-guided sublayer;

[0030] S4.2: Attention weights are transferred through dense connection layers. Joint matrix X with word vector encoding l Perform fusion and connection on the word vector encoding matrix U l The update is performed, including the joint matrix of word vector encodings. The updated formula is as follows:

[0031]

[0032] Among them U lThe updated word vector encoding matrix is ​​defined as follows: W4(l) is the fourth weight matrix among all densely connected layers in the l-th layer of the attention- and edge-label guided graph convolutional network, and b1 is the first bias term within the densely connected layers; the updated word vector encoding matrix is ​​defined as follows: U l This refers to the word vector encoding matrix U l After the update, the word vector encoding matrix U is updated. l For the original word vector encoding matrix U l Direct replacement; ρ is the sigmoid activation function;

[0033] S4.3: Word vector encoding matrix U is processed through a linear combination layer. l The formula has been updated again and is as follows:

[0034] U′ l =U l W5+b2

[0035] Where U′ l The updated word vector encoding matrix is ​​shown below. W5 is the fifth weight matrix in the linear combination layer, and b2 is the second bias term in the linear combination layer. During the update process, the word vector encoding matrix U′ will be updated. l Word vector encoding matrix U l Perform the replacement and change the word vector encoding matrix U l Output;

[0036] S4.4: Convert the word vector encoding matrix U output in step S4.3... l The input is fed into the next attention-guided graph convolutional network, and steps S4.1-S4.3 are executed iteratively until all M attention-guided graph convolutional networks have been executed. Finally, the word vector encoding matrix U output by the last attention-guided graph convolutional network is output. l .

[0037] As a preferred aspect of the present invention, the improved CBOW model includes an input layer, a hidden layer, and an output layer;

[0038] The input layer receives the one-hot encodings of the context words located in the sliding window of the center word, and the size of the sliding window is C.

[0039] The hidden layer is used to multiply the one-hot codes received from the input layer by the sixth weight matrix W6 and perform a weighted summation and averaging operation. The size of the sixth weight matrix W6 is V×D, where V is the length of the one-hot code.

[0040] The formula for applying the weighted average is as follows:

[0041]

[0042] Where Y is the hidden vector output by the hidden layer, y e+α β is the one-hot encoding of the word at a distance of α from the middle word. e+α β is the attention weight value corresponding to the word at a distance of α from the middle word. In the initial state, β e+α =1, and will be updated subsequently by training the syntactic dependency extraction model;

[0043] The output layer is used to multiply the hidden vector by the weight matrix between the hidden layer and the output layer to obtain an output vector of size 1×V. The output vector is then normalized by softmax to obtain a probability vector of size 1×V. The word corresponding to the highest probability value in the probability vector is taken as the prediction result.

[0044] The process of generating a source word vector library by improving the CBOW model includes the following steps:

[0045] Obtain several training texts;

[0046] For each training text, word vectors are obtained, each containing a one-hot encoding of each word in the text. All word vectors from the training text are combined to form a first training text set, which is then fed into an improved CBOW model for training, constructing a candidate source word vector library. Next, words in the training text are matched against the candidate source word vector library to obtain training source word vector sequences, each containing a candidate source word vector corresponding to each word in the training text. All training source word vector sequences from the training text are combined to form a second training text set, which is then fed into a syntactic dependency extraction model for training, yielding the attention weight β. e+α ; Adjust the attention weight β e+α The improved CBOW model is fed into the training text set for adjustment, and the training of the improved CBOW model is repeated. The candidate source word vector library trained by the improved CBOW model is then used to adjust the second training text set, and the training of the syntactic dependency extraction model is repeated to achieve joint training of the improved CBOW model and the syntactic dependency extraction model. This process continues until the improved CBOW model and the syntactic dependency extraction model respectively meet the first and second training conditions. The training of the improved CBOW model and the syntactic dependency extraction model using the current training text is then completed.

[0047] Once all training texts have been used to train the improved CBOW model and the syntactic dependency extraction model, the trained syntactic dependency extraction model is output, and the candidate source word vector library is output as the source word vector library.

[0048] As a preferred aspect of the present invention, the first training text set is fed into the improved CBOW model for training to construct a candidate source word vector library, specifically including the following steps:

[0049] The first training text set is fed into the improved CBOW model with initial parameters for training. During the training, the one-hot encoding of the context words in the sliding window of the center word is used as input, and the center word in the word vector of the training text is used as the target condition. The first loss value is calculated. If the first training condition is met, the first training condition is that the first loss value is within the first preset range. The trained improved CBOW model is output. Otherwise, the training continues iteratively.

[0050] Obtain the sixth weight matrix W6 in the trained improved CBOW model, and multiply the one-hot encoding of the word in the source language with the sixth weight matrix W6 to obtain the candidate source word vectors corresponding to the word in the source language. Map any word in the source language to its corresponding candidate source word vectors to form a candidate source word vector library.

[0051] The second training text set was fed into the syntactic dependency extraction model for training, and the attention weight value β was obtained. e+α Specifically, it includes the following steps:

[0052] Adding a named entity recognition layer to the end of the syntactic dependency extraction model creates an extended syntactic dependency extraction model. The named entity recognition layer encodes the word vector matrix U output by the last attention- and edge-label guided graph convolutional network. L Perform entity recognition and output a predicted entity label sequence;

[0053] The second training text set is fed into the extended syntactic dependency extraction model with initialization parameters for training. During the training, the actual entity label sequence corresponding to each training text in the second training text set is used as the target condition to calculate the second loss value. If the second training condition is met, the second training condition is that the second loss value is within the second preset range. The trained syntactic dependency extraction model is then output. Otherwise, the training continues iteratively.

[0054] Obtain the word vector encoding matrix U from the last attention- and edge-label guided graph convolutional network output in the trained syntactic dependency extraction model. L Let fe be the vector of the candidate source word corresponding to the center word in the training sample, and calculate the attention weight vector ζ. e :

[0055] ζ e =f e ·(U L ) T

[0056] The (e+α)th value in the attention weight vector ζe is taken as the attention weight β. e+α .

[0057] As a preferred aspect of the present invention, N words to be translated are input into a trained language translation model for processing, specifically including the following steps:

[0058] N words to be translated are input into the word embedding layer of the encoder to obtain the source word vector matrix G. The source word vector matrix G is then processed through the first, second, third, fourth, fifth, and sixth coding blocks. The specific processing methods of the first, second, third, fourth, fifth, and sixth coding blocks refer to the traditional Transformer model to obtain the hidden coding matrix.

[0059] The hidden encoding matrix is ​​right-shifted and then fed into the decoder word embedding layer for processing to obtain the target word vector matrix. This target word vector matrix is ​​then updated through the first, second, third, fourth, fifth, and sixth decoding blocks to obtain the target word vector matrix. This target word vector matrix is ​​then input into the word prediction layer, which normalizes the target word vector matrix using the softmax function to obtain the word probability matrix. The word corresponding to the highest probability value in the t-th row of the word probability matrix is ​​the target word, where t is the number of times the decoder is executed. Simultaneously, the target word vector matrix is ​​reconstructed as follows: the target word vector corresponding to the target word output by the t-th decoder is used to replace every t rows in the current target word vector matrix. The target word vectors are obtained from the target word vector library in the decoder word embedding layer. Each decoder execution outputs one target word until the output target word is a stop character, at which point the decoder execution stops. All target words are then arranged into a target word sequence according to the decoder execution order.

[0060] The operation and structure of the decoder word embedding layer are the same as those of the encoder word embedding layer. The difference is that the encoder word embedding layer is trained with training text of the source language, while the decoder word embedding layer is trained with training text of the target language. The decoder word embedding layer includes a target word vector library, and the words in the target word vector library correspond one-to-one with the target word vectors.

[0061] During the first execution of the decoder, the hidden encoding matrix is ​​embedded into words through the decoder word embedding layer to obtain the target word vector matrix. In the subsequent decoder operations, excluding the first execution, the decoder word embedding layer is not executed. Instead, the target word vector matrix output by the previous decoder is directly fed into the first decoding block to execute the decoder operations.

[0062] As a preferred aspect of the present invention, the training of the language translation model specifically includes the following steps:

[0063] Obtain several training texts to be translated and their corresponding training target texts; combine all training texts to be translated into a third training text set; then feed the third training text set into the language translation model with initialized parameters for training. During the training, use the training target texts corresponding to the training texts to be translated as the target conditions, calculate the third loss value. If the third training condition is met, the third training condition is that the third loss value is within the third preset range, and output the trained syntactic dependency extraction model; otherwise, continue iterative training.

[0064] A text translation system based on a language translation model includes:

[0065] The module for obtaining the text to be translated is used to acquire the text to be translated and perform word segmentation on the text to be translated to obtain the words to be translated.

[0066] The language translation model management module is used to store and train language translation models;

[0067] The language translation module is used to input the words to be translated into the trained language translation model for processing and output a sequence of target words. The sequence of target words stores the target words, which are the words in the target language that correspond to the text to be translated.

[0068] The target translation text construction module is used to concatenate target words in the target word sequence to create target translation text.

[0069] The present invention has the following advantages:

[0070] 1. This invention uses a language translation model based on an improved Transformer model to translate the text to be translated. It can combine the semantic information in the text to be translated, that is, the syntactic dependency relationship between words in the text to be translated, thereby achieving the fusion of the grammatical structure of the language corresponding to the text to be translated, and thus improving the accuracy of text translation.

[0071] 2. This invention performs word embedding operations on the source word vector sequence F through a syntactic dependency relation extraction model, which can take into account both long-distance information and long-term syntactic structure dependency information in the text to be translated. It solves the problems of lack of contextual information and lack of directionality in the word encoding plus position encoding operation in the traditional Transformer model, and can significantly improve the accuracy of text translation.

[0072] 3. This invention uses training samples to jointly train the improved CBOW model and the syntactic dependency extraction model. While obtaining the trained syntactic dependency extraction model, the improved CBOW model is trained, so that the source word vectors in the source word vector library can depend on semantic dependency relations and long-term dependency relations of sentences. Attached Figure Description

[0073] Figure 1 This is a schematic diagram of the structure of a text translation system based on a language translation model used in an embodiment of the present invention. Detailed Implementation

[0074] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this invention.

[0075] Example 1: A text translation method based on a language translation model, comprising:

[0076] Obtain the text to be translated and perform word segmentation on the text to be translated to obtain N words to be translated, where N is the total number of words in the text to be translated. It should be noted that the text to be translated can be any language. The translation of different languages ​​will only affect the training of the subsequent language translation model. In this application, the language translation model for translating English into Chinese is described. The word segmentation operation can be performed using the word segmentation plugin of the spaCY library.

[0077] Input N words to be translated into a trained language translation model for processing, and output a sequence of target words. The sequence of target words stores M target words, which are the words in the target language corresponding to the text to be translated.

[0078] The target words in the target word sequence are concatenated to create the target translation text;

[0079] The language translation model is based on an improved Transformer model, comprising an encoder and a decoder. The encoder includes an encoder word embedding layer, a first encoding block, a second encoding block, a third encoding block, a fourth encoding block, a fifth encoding block, and a sixth encoding block. The decoder includes a decoder word embedding layer, a first decoding block, a second decoding block, a third decoding block, a fourth decoding block, a fifth decoding block, a sixth decoding block, and a word prediction layer. The encoder word embedding layer embeds words into the target language corresponding to the text to be translated. The first, second, third, fourth, fifth, and sixth encoding blocks extract features from the input matrix using a self-attention mechanism to obtain a hidden encoding matrix. The decoder word embedding layer embeds words into the target language corresponding to the translated text to obtain a target word vector matrix. The first, second, third, fourth, fifth, and sixth decoding blocks decode the input matrix using a self-attention mechanism to update the target word vector matrix. The encoder and decoder word embedding layers are improved based on semantic information. The word prediction layer predicts target words based on the target word vector matrix.

[0080] This application uses a language translation model based on an improved Transformer model to translate the text to be translated. It can combine the semantic information in the text to be translated, that is, the syntactic dependency relationship between words in the text to be translated, thereby achieving the fusion of the grammatical structure of the language corresponding to the text to be translated, and thus improving the accuracy of text translation.

[0081] The word embedding operation through the encoder word embedding layer specifically includes the following steps:

[0082] N words to be translated are sequentially fed into the source word vector database for matching, and the source word vector sequence F is output, where F is {f1, f2, ..., fn}. n …f N}, where f n Let n be the source word vector corresponding to the nth word to be translated, where n = 1, 2, 3…N; there is a one-to-one correspondence between words in the source word vector library and source word vectors, and the source word vectors in the source word vector library are generated by improving the CBOW model based on semantic information; it should be noted that the traditional CBOW model is only trained based on contextual information, and is limited by the size of the sliding window, so it cannot learn the semantic dependency relationship between words well, nor can it learn the long-term dependency relationship well. Therefore, this application trains the traditional CBOW model through semantic information, so that the generated source word vectors can rely on semantic dependency relationship and long-term dependency relationship of sentences, thereby improving the accuracy of subsequent text translation;

[0083] The source word vector sequence F is fed into the trained syntactic dependency extraction model for processing, and the output source word vector matrix G to be translated is generated. The size of the source word vector matrix G to be translated is N×D, where D is the feature dimension of each word to be translated in the source word vector matrix G.

[0084] The syntactic dependency extraction model consists of a word encoding layer and L attention and edge-label guided graph convolutional networks (AELGCN). The word encoding layer is based on a Bi-LSTM model, and L is typically set to 3. The attention and edge-label guided graph convolutional networks include an edge-aware node joint update layer (EANJU), an edge-aware node update layer (EANU), and M attention-guided graph convolutional networks (AGGCN); M is typically set to 6. The attention-guided graph convolutional networks include attention guidance layers, dense connection layers, and linear combination layers.

[0085] The source word vector sequence F is processed using a trained syntactic dependency extraction model, specifically including the following steps:

[0086] S1: Feed the source word vector sequence F into the word encoding layer for processing, and output the word vector encoding sequence H, where H = {h1, h2, ..., h...} n …h N}, where h n For the source word vector f n The word vector encoding output by the word encoding layer has a dimension of D and satisfies:

[0087]

[0088] in For the source word vector f n The output vector of the forward LSTM model in the word encoding layer For the source word vector f n The output vector of the backward LSTM model in the word encoding layer For splicing operations, Forward LSTM model, It is a backward LSTM model. This is the set of training parameters for the feedforward LSTM model. This is the training parameter set for the backward LSTM model. It should be noted that the Bi-LSTM model is a conventional model, and its training parameter set is set with reference to the existing Bi-LSTM model, which will not be elaborated here. A word vector encoding matrix U0 is established based on the word vector encoding sequence H, and the size of the word vector encoding matrix U0 is N×D.

[0089] S2: The word vector encoding matrix U0 is fed into the joint update layer of edge-aware nodes for processing. The processing formula is as follows:

[0090]

[0091] Among them U l Let E be the word vector encoding matrix output by the joint update layer of the l-th edge-aware node. l-1 (:,:,i) is the syntactic dependency embedding adjacency matrix E output by the update layer of the (l-1)th edge-aware node. l-1 The slice corresponding to the i-th channel, i = 1, 2, 3…I, where I is the syntactic dependency embedding adjacency matrix E. l-1 The total number of channels, W1 and W2 are the first and second weight matrices respectively, and both W1 and W2 are of size D×D, U l (i) is E l-1 (:,:i) is the i-th word vector encoding matrix slice obtained by processing W2. pool is the mean operation, which is to sum all word vector encoding matrix slices and then calculate the arithmetic mean; σ is the ReLU activation function; A is the syntactic dependency matrix, and the size of the syntactic dependency matrix is ​​N×N; it should be noted that the updated word vector encoding matrix U l The size is also N×D;

[0092] The syntactic dependency matrix A is obtained as follows: the source word vector sequence F is subjected to syntactic dependency analysis using a syntactic dependency analysis tool in an NLP library to obtain a syntactic dependency parsing tree. Then, the syntactic dependency matrix A is constructed based on the syntactic dependency parsing tree. The data A in the syntactic dependency matrix A... kj The value of A is either 1 or 0. kj This represents the data corresponding to the k-th row and j-th column in the syntactic dependency matrix A, where k = 1, 2, 3…N, j = 1, 2, 3…N. When A… kj =1 indicates that the k-th source word vector in the source word vector sequence F and the j-th source word vector in the source word vector sequence F have a syntactic dependency relationship on the syntactic dependency parsing tree. When A kj =0 indicates that the k-th source word vector in the source word vector sequence F and the j-th source word vector in the source word vector sequence F do not have a syntactic dependency relationship on the syntactic dependency parsing tree. In particular, when k=j, A kj =1;

[0093] When l = 0, i.e., in the initial state, the embedding adjacency matrix E l The size is N×N×I, where E l (k, j, :) is the embedding adjacency matrix E l The syntactic dependency embedding vector corresponding to the k-th row and j-th column is given by E. l The length of (k, j, :) is I, and the syntactic dependency embedding vector is E. l (k, j, :) is obtained through a syntactic dependency tree and a syntactic dependency relation embedding vector library. When the syntactic dependency tree indicates that the k-th source word vector in the source word vector sequence F and the j-th source word vector in the source word vector sequence F have a syntactic dependency relation on the syntactic dependency parsing tree, such as a subject-verb relation or a direct object relation, this syntactic dependency relation is matched with the syntactic dependency relation embedding vector library, and the syntactic dependency relation embedding vector corresponding to the syntactic dependency relation between the k-th source word vector in the source word vector sequence F and the j-th source word vector in the source word vector sequence F is output. The syntactic dependency relation embedding vector library stores syntactic dependency relations and their corresponding syntactic dependency relation embedding vectors, and the syntactic dependency relation embedding vectors are obtained by training on the source language corresponding to the text to be translated. For details, please refer to word2vec.

[0094] S3: The adjacency matrix E embedded in the syntactic dependency relations is updated through the edge-aware node update layer. l Update the syntactic dependency relation embedding into the adjacency matrix E. l The syntactic dependency embedding adjacency matrix output by the update layer for the l-th edge-aware node is updated using the following formula:

[0095]

[0096] Among them, u l (k) is the word vector encoding matrix U l The word vector encoding corresponding to the k-th row in the code, u l (j) is the word vector encoding matrix U l The word vector encoding corresponding to the j-th row is given by W3, which is the third weight matrix with a size of (I+2D)×I. It should be noted that the word vector encoding u... l (k), word vector encoding u l (j) and the syntactic dependency embedding vector E l-1 After concatenating (k, j, :), the size is 1×(I+2D). Multiplying this by the third weight matrix W3 reverts to the size of 1×I. The syntactic dependency embedding adjacency matrix E is then updated through the edge-aware node update layer. l Updating the adjacency matrix E strengthens the embedding of syntactic dependency relations. l Dependence on the text context;

[0097] Embed all updated syntactic dependency relations into vector E l Concatenate (k, j, :) to obtain the syntactic dependency relation embedding adjacency matrix E. l ;

[0098] S4: The word vector encoding matrix U after processing through the joint update layer of edge-aware nodes. l The input is fed into M attention-guided graph convolutional networks for processing, and the output is the word vector encoding matrix U output by the last attention-guided graph convolutional network. l The handling method is as follows:

[0099] S4.1: Encode the word vector matrix U l The data is fed into the attention guidance layer for processing to obtain attention weights. Let be the attention weights output by the r-th attention-guided sublayer in the m-th attention-guided graph convolutional network of the l-th attention-guided graph convolutional network based on attention and edge labels; where m = 1, 2, 3…M, r = 1, 2, 3…R, and R is the total number of attention-guided sublayers in the attention-guided layer; the attention-guided layer can refer to the Multi-Head Attention layer in the traditional Transformer model, i.e., the multi-head attention mechanism, which can obtain information from different subspaces; the calculation formula is as follows:

[0100]

[0101] in Let r be the query weight matrix in the attention-guided sublayer. Let r be the value weight matrix in the attention-guided sublayer. and The size of each is D×D, and the attention weights are... The size is N×N;

[0102] S4.2: Attention weights are transferred through dense connection layers. Joint matrix X with word vector encoding l Perform fusion and connection on the word vector encoding matrix U l The update is performed, including the joint matrix of word vector encodings. The updated formula is as follows:

[0103]

[0104] Among them U lThe updated word vector encoding matrix is ​​defined as follows: W4(l) is the fourth weight matrix in all densely connected layers of the l-th layer of the attention and edge label guided graph convolutional network, with a size of 1D×D, and b1 is the first bias term in the densely connected layer. It should be noted that the updated word vector encoding matrix U... l This refers to the word vector encoding matrix U l After the update, the word vector encoding matrix U is updated. l For the original word vector encoding matrix U l Direct replacement; ρ is the sigmoid activation function;

[0105] S4.3: Word vector encoding matrix U is processed through a linear combination layer. l The formula has been updated again and is as follows:

[0106] U′ l =U l W5+b2

[0107] Where U′ l The updated word vector encoding matrix is ​​shown below. W5 is the fifth weight matrix in the linear combination layer, and b2 is the second bias term in the linear combination layer. During the update process, the word vector encoding matrix U′ will be updated. l Word vector encoding matrix U l Perform the replacement and change the word vector encoding matrix U l Output;

[0108] S4.4: Convert the word vector encoding matrix U output in step S4.3... l The input is fed into the next attention-guided graph convolutional network, and steps S4.1-S4.3 are executed iteratively until all M attention-guided graph convolutional networks have been executed. Finally, the word vector encoding matrix U output by the last attention-guided graph convolutional network is output. l ;

[0109] S5: Convert the word vector encoding matrix U output from step S4 l Feed the input into the next attention- and edge-label guided graph convolutional network, iteratively executing steps S2-S4; until all L attention- and edge-label guided graph convolutional networks have been executed, output the word vector encoding matrix U output by the last attention- and edge-label guided graph convolutional network. L G is the vector matrix of the source words to be translated;

[0110] This application performs word embedding operations on the source word vector sequence F through a syntactic dependency relation extraction model, which can take into account both long-distance information and long-term syntactic structural dependency information in the text to be translated. It solves the problems of lack of contextual information and lack of directionality in the word encoding plus position encoding operation in the traditional Transformer model, and can significantly improve the accuracy of text translation.

[0111] The improved CBOW model includes an input layer, hidden layers, and an output layer;

[0112] The input layer receives the one-hot encodings of the context words located in the sliding window of the center word, and the size of the sliding window is C.

[0113] The hidden layer is used to multiply the one-hot codes received from the input layer by the sixth weight matrix W6 and perform a weighted summation and averaging operation. The size of the sixth weight matrix W6 is V×D, where V is the length of the one-hot code.

[0114] The formula for applying the weighted average is as follows:

[0115]

[0116] Where Y is the hidden vector output by the hidden layer, y e+α β is the one-hot encoding of the word at a distance of α from the middle word. e+α β is the attention weight value corresponding to the word at a distance of α from the middle word. In the initial state, β e+α =1, and will be updated subsequently by training the syntactic dependency extraction model;

[0117] The output layer is used to multiply the hidden vector by the weight matrix between the hidden layer and the output layer to obtain an output vector of size 1×V. The output vector is then normalized by softmax to obtain a probability vector of size 1×V. The word corresponding to the highest probability value in the probability vector is taken as the prediction result.

[0118] It should be noted that the traditional CBOW model calculates the hidden vector output by directly summing and averaging, without taking into account the weights of syntactic dependencies. Under different syntactic dependency relations, the same word may have different meanings. The improved CBOW model in this application takes into account the weights under different syntactic dependency relations. As for the other parts, it is consistent with the traditional CBOW model.

[0119] The generation of a source word vector library by improving the CBOW model specifically includes the following:

[0120] To obtain several training texts, it should be noted that training texts refer to any ordered and meaningful sentence text from the source language.

[0121] For each training text, word vectors are obtained, each containing a one-hot encoding of each word in the text. All word vectors from the training text are combined to form a first training text set, which is then fed into an improved CBOW model for training, constructing a candidate source word vector library. Next, words in the training text are matched against the candidate source word vector library to obtain training source word vector sequences, each containing a candidate source word vector corresponding to each word in the training text. All training source word vector sequences from the training text are combined to form a second training text set, which is then fed into a syntactic dependency extraction model for training, yielding the attention weight β. e+α ; Adjust the attention weight β e+α The improved CBOW model is fed into the training text set for further adjustments, and the training of the improved CBOW model is repeated. The candidate source word vector library trained with the improved CBOW model is then used to adjust the second training text set, and the training of the syntactic dependency extraction model is repeated. This achieves joint training of the improved CBOW model and the syntactic dependency extraction model. Specifically, after each training iteration of the improved CBOW model, the second training text set is adjusted using the candidate source word vector library, and the syntactic dependency extraction model is trained again. After each training iteration of the syntactic dependency extraction model, the attention weight β is used to adjust the training text set. e+α Adjust the improved CBOW model, then train the improved CBOW model again, and repeat this process until the improved CBOW model and the syntactic dependency extraction model satisfy the first training condition and the second training condition respectively; complete the training of the improved CBOW model and the syntactic dependency extraction model with the current training text.

[0122] Once all training texts have been used to train the improved CBOW model and the syntactic dependency extraction model, the trained syntactic dependency extraction model is output, and the candidate source word vector library is output as the source word vector library.

[0123] This application uses training samples to jointly train the improved CBOW model and the syntactic dependency extraction model. While obtaining the trained syntactic dependency extraction model, the improved CBOW model is trained, so that the source word vectors in the source word vector library can depend on semantic dependency relations and long-term dependency relations of sentences.

[0124] The first training text set is fed into the improved CBOW model for training, and a candidate source word vector library is constructed, which includes the following steps:

[0125] The first training text set is fed into the improved CBOW model with initialized parameters for training. The first training of the improved CBOW model is regarded as the improved CBOW model being in the initial state. During this period, the one-hot encoding of the context words in the center word sliding window is used as input, and the center word in the word vector of the training text is used as the target condition. The first loss value is calculated. If the first training condition is met, the first training condition is that the first loss value is within the first preset range. The first preset range is set manually. The trained improved CBOW model is output. Otherwise, the iterative training continues.

[0126] Obtain the sixth weight matrix W6 in the trained improved CBOW model, and multiply the one-hot encoding of the word in the source language with the sixth weight matrix W6 to obtain the candidate source word vectors corresponding to the word in the source language. Map any word in the source language to its corresponding candidate source word vectors to form a candidate source word vector library.

[0127] The second training text set was fed into the syntactic dependency extraction model for training, and the attention weight value β was obtained. e+α Specifically, it includes the following steps:

[0128] An extended syntactic dependency extraction model is constructed by adding a named entity recognition layer to the end of the model. The named entity recognition layer is a Conditional Random Field (CRF). The named entity recognition layer encodes the word vector encoding matrix U output by the last attention- and edge-label guided graph convolutional network. L Perform entity recognition and output predicted entity label sequence. It should be noted that by adding a named entity recognition layer to the end of the syntactic dependency extraction model, the syntactic dependency extraction model can be trained with the accuracy of the predicted entity label sequence. During the training process, the target condition can be the actual entity label sequence that has been manually labeled.

[0129] The second training text set is fed into the extended syntactic dependency extraction model with initialization parameters for training. During the training, the actual entity label sequence corresponding to each training text in the second training text set is used as the target condition to calculate the second loss value. If the second training condition is met, the second training condition is that the second loss value is within the second preset range. The second preset range is set manually. The trained syntactic dependency extraction model is output. Otherwise, the training continues iteratively.

[0130] Obtain the word vector encoding matrix U from the last attention- and edge-label guided graph convolutional network output in the trained syntactic dependency extraction model. L Let fe be the vector of the candidate source word corresponding to the center word in the training sample, and calculate the attention weight vector ζ. e :

[0131] ζ e=f e ·(U L ) T

[0132] The attention weight vector ζ e The (e+α)th value is used as the attention weight β. e+α ;

[0133] The N words to be translated are input into the trained language translation model for processing, which includes the following steps:

[0134] N words to be translated are input into the word embedding layer of the encoder to obtain the source word vector matrix G. The source word vector matrix G is then processed through the first, second, third, fourth, fifth, and sixth coding blocks. The specific processing methods of the first, second, third, fourth, fifth, and sixth coding blocks refer to the traditional Transformer model to obtain the hidden coding matrix.

[0135] The hidden encoding matrix is ​​right-shifted and then fed into the decoder word embedding layer for processing to obtain the target word vector matrix. It should be noted that the right-shift operation involves deleting the last row of the hidden encoding matrix and adding the source word vector corresponding to the start symbol before the first row. The target word vector matrix is ​​updated through the first, second, third, fourth, fifth, and sixth decoding blocks. This target word vector matrix is ​​then input into the word prediction layer, which normalizes it using the softmax function to obtain a word probability matrix. The word with the highest probability value in the t-th row of the word probability matrix is ​​the target word, where t is the number of times the decoder is executed. Simultaneously, the target word vector matrix is ​​reconstructed as follows: the target word vector corresponding to the target word output by the t-th decoder is used to replace every t rows in the current target word vector. The target word vectors are obtained from the target word vector library in the decoder word embedding layer. Each decoder execution outputs one target word until the output target word is a stop symbol, at which point the decoder execution stops. All target words are then arranged into a target word sequence according to the decoder execution order.

[0136] The operation and structure of the decoder word embedding layer are the same as those of the encoder word embedding layer. The difference is that the encoder word embedding layer is trained with training text of the source language, while the decoder word embedding layer is trained with training text of the target language. The decoder word embedding layer includes a target word vector library, and the words in the target word vector library correspond one-to-one with the target word vectors.

[0137] During the first execution of the decoder, the hidden encoding matrix is ​​embedded into words through the decoder word embedding layer to obtain the target word vector matrix. In the subsequent decoder operations, excluding the first execution, the decoder word embedding layer is not executed. Instead, the target word vector matrix output by the previous decoder is directly fed into the first decoding block to execute the decoder operations.

[0138] The training of a language translation model specifically includes the following steps:

[0139] Several training texts to be translated and their corresponding training target texts are obtained. It should be noted that the training target texts refer to the training target texts corresponding to the target language obtained by manually translating the training texts to be translated. All training texts to be translated are combined into a third training text set. The third training text set is then fed into the language translation model with initialized parameters for training. During the training, the training target texts corresponding to the training texts to be translated are used as the target conditions to calculate the third loss value. If the third training condition is met, the third training condition is that the third loss value is within a third preset range. The third preset range is set manually. The trained syntactic dependency extraction model is then output. Otherwise, the training continues iteratively.

[0140] It should be noted that in the description of this embodiment, the number of words in the text to be translated is assumed to be N, which is consistent with the scale of the input syntactic dependency extraction model. However, in actual operation, the number of words in the text to be translated is not always N. Therefore, when the source word vector sequence corresponding to the text to be translated is input into the syntactic dependency extraction model, the source word vector sequence is usually filled to N source word vectors by the source word vectors corresponding to the padding characters.

[0141] Example 2, a text translation system based on a language translation model, see [link to example]. Figure 1 ,include:

[0142] The module for obtaining the text to be translated is used to acquire the text to be translated and perform word segmentation on the text to be translated to obtain the words to be translated.

[0143] The language translation model management module is used to store and train language translation models;

[0144] The language translation module is used to input the words to be translated into the trained language translation model for processing and output a sequence of target words. The sequence of target words stores the target words, which are the words in the target language that correspond to the text to be translated.

[0145] The target translation text construction module is used to concatenate target words in the target word sequence to create target translation text.

[0146] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims. Parts not described in detail in this specification are prior art known to those skilled in the art.

Claims

1. A text translation method based on a language translation model, characterized by, The method comprises the following steps: obtaining a text to be translated, and performing a word segmentation operation on the text to be translated to obtain N words to be translated, N being the total number of words in the text to be translated; inputting the N words to be translated into a trained language translation model for processing, and outputting a target word sequence, the target word sequence storing M target words, the target words being words in a target language corresponding to the text to be translated after text translation; splicing the target words in the target word sequence to establish a target translation text; the language translation model is established based on an improved Transformer model, and comprises an encoder and a decoder, wherein the encoder comprises an encoder word embedding layer, a first encoding block, a second encoding block, a third encoding block, a fourth encoding block, a fifth encoding block and a sixth encoding block; the decoder comprises a decoder word embedding layer, a first decoding block, a second decoding block, a third decoding block, a fourth decoding block, a fifth decoding block, a sixth decoding block and a word prediction layer; the encoder word embedding layer is used for performing word embedding operation on the words to be translated through a source language corresponding to the text to be translated; the first encoding block, the second encoding block, the third encoding block, the fourth encoding block, the fifth encoding block and the sixth encoding block are used for performing feature extraction on an input matrix through a self-attention mechanism to obtain a hidden encoding matrix; the decoder word embedding layer is used for performing word embedding operation on the target language corresponding to the text to be translated after text translation to obtain a target word vector matrix; the first decoding block, the second decoding block, the third decoding block, the fourth decoding block, the fifth decoding block and the sixth decoding block are used for performing decoding operation on an input matrix through a self-attention mechanism to update the target word vector matrix; and the encoder word embedding layer and the decoder word embedding layer are improved based on semantic information; and the word prediction layer is used for predicting target words according to the target word vector matrix; the word embedding operation through the encoder word embedding layer specifically comprises the following steps: The N words to be translated are sequentially sent into the source word vector library for matching, and a source word vector sequence F is output, where F is {f1, f2…fN}, f is a source word vector corresponding to the nth word to be translated, and n=1, 2, 3…N. n … N} n ​ the words in the source word vector library correspond one-to-one to the source word vectors, and the source word vectors in the source word vector library are generated through an improved CBOW model; the source word vector sequence F is sent into a trained syntax dependency relation extraction model for processing, and a translated source word vector matrix G is output, the size of the translated source word vector matrix G being N×D, wherein D is the feature dimension corresponding to each word to be translated in the translated source word vector matrix G; the syntax dependency relation extraction model comprises a word encoding layer and L attention-guided graph convolutional network groups based on attention and edge labels, wherein the word encoding layer is established based on a Bi-LSTM model, and the attention-guided graph convolutional network based on attention and edge labels comprises an edge perception node joint updating layer, an edge perception node updating layer and M attention-guided graph convolutional networks, wherein the attention-guided graph convolutional network comprises an attention-guided layer, a densely connected layer and a linear combination layer.

2. The text translation method based on a language translation model according to claim 1, characterized in that, processing the source word vector sequence F through the trained syntax dependency relation extraction model specifically comprises the following steps: S1: send the source word vector sequence F into the word encoding layer for processing, output the word vector encoding sequence H, the word vector encoding sequence H={h1, h2…h n … N}, wherein h n is the source word vector f n output by the word encoding layer, and the dimension size of the word vector encoding is D; S2: sending the word vector encoding matrix U0 into the edge perception node joint updating layer for processing, and the processing formula is as follows: U l Let E be the word vector encoding matrix output by the joint update layer of the l-th edge-aware node, where l = 1, 2, 3…L. l-1 (:,:,i) represents the syntactic dependency embedding adjacency matrix E output by the update layer of the (l-1)th edge-aware node. l-1 The slice corresponding to the i-th channel, i = 1, 2, 3…I, where I is the syntactic dependency embedding adjacency matrix E. l-1 The total number of middle channels, W1 and W2 are the first and second weight matrices respectively, U l (i) is E l-1 (:,:,i) is a slice of the i-th word vector encoding matrix obtained by processing with W2, where pool is the mean operation; σ is the ReLU activation function; A is the syntactic dependency matrix, and the size of the syntactic dependency matrix is ​​N×N; S3: Update the syntax dependency embedding adjacency matrix E by the edge-aware node update layer l Update the syntax dependency embedding adjacency matrix E l The syntax dependency embedding adjacency matrix output by the lth edge-aware node update layer is updated, and the update formula is as follows: wherein u l (k) is a word vector encoding matrix U l (k) is a word vector encoding matrix U l (j) is a word vector encoding matrix U l (j) is a word vector encoding matrix U concatenate all updated syntactic dependency relation embedding vectors E l (k, j, : ) to obtain syntactic dependency relation embedding adjacency matrix E l ; S4: the word vector encoding matrix U processed by the edge perception node joint update layer l is sent into M attention guide graph convolutional networks for processing; S5: encode the word vector matrix U outputted by step S4 l into the next attention-based and edge label guided graph convolution network, and iteratively perform steps S2-S4 until all L attention-based and edge label guided graph convolution networks are performed, and output the word vector matrix U outputted by the last attention-based and edge label guided graph convolution network L as the source word vector matrix G to be translated.

3. The method of claim 2, wherein, wherein the syntax dependency matrix A is obtained by performing syntax dependency analysis on the source word vector sequence F to obtain a syntax dependency analysis tree, and then constructing the syntax dependency matrix A based on the syntax dependency analysis tree, the data A kj in the syntax dependency matrix A has a value of 1 or 0, A kj represents data corresponding to the kth row and jth column in the syntax dependency matrix A, k = 1, 2, 3, …, N, j = 1, 2, 3, …, N, when A kj = 1, it indicates that the kth source word vector in the source word vector sequence F and the jth source word vector in the source word vector sequence F have a syntax dependency relationship on the syntax dependency analysis tree, and when A kj = 0, it indicates that the kth source word vector in the source word vector sequence F and the jth source word vector in the source word vector sequence F do not have a syntax dependency relationship on the syntax dependency analysis tree, and in particular, when k = j, A kj = 1. When l = 0, the embedding adjacency matrix E l has a size of N x N x I, where E l (k, j, :) is a syntax dependency relation embedding vector corresponding to the kth row and jth column in the embedding adjacency matrix E l , and the length of the syntax dependency relation embedding vector E l (k, j, :) is I, the syntax dependency relation embedding vector E l (k, j, :) is obtained through the syntax dependency tree and the syntax dependency relation embedding vector library, when the kth source word vector in the source word vector sequence F and the jth source word vector in the source word vector sequence F have a syntax dependency relation on the syntax dependency analysis tree, the syntax dependency relation is matched with the syntax dependency relation embedding vector library, and the syntax dependency relation embedding vector corresponding to the syntax dependency relation between the kth source word vector in the word vector sequence F and the jth source word vector in the source word vector sequence F on the syntax dependency analysis tree is output. The syntax dependency relation embedding vector library stores syntax dependency relations and corresponding syntax dependency relation embedding vectors, and the syntax dependency relation embedding vectors are obtained by training a source language corresponding to the text to be translated.

4. The text translation method based on a language translation model according to claim 3, characterized in that, The word vector encoding matrix U processed by the edge awareness node joint update layer l The word vector encoding matrix U output by the last attention guide map convolutional network is input into M attention guide map convolutional networks for processing l , and specifically includes the following steps: S4.1: encode the word vector matrix U l The input is processed by the attention guide layer to obtain attention weights is the attention weight output by the rth attention guide sub-layer in the mth attention guide graph convolution network in the lth attention and edge label guide graph convolution network. Wherein m=1, 2, 3…M, r=1, 2, 3…R, R is the total number of attention guide sub-layers in the attention guide layer; the calculation formula is as follows: wherein is a query weight matrix in the rth attention guide sub-layer, is a value weight matrix in the rth attention guide sub-layer. S4.2: Attention weights are computed by dense connection layers The joint matrix X of word vector encodings l Fusion connection is performed on the word vector encoding matrix U l Update, where the joint matrix X of word vector encodings The update formula is as follows: wherein U l is the updated word vector encoding matrix, W4(l) is the fourth weight matrix in all dense connection layers in the l-th layer of the attention and edge label guided graph convolutional network, and b1 is the first bias term in the dense connection layer. updated word vector encoding matrix U l is updated to an updated word vector encoding matrix U l is updated to an updated word vector encoding matrix U l is updated to an updated word vector encoding matrix U l is directly replaced by the updated word vector encoding matrix U ; and ρ is a sigmoid activation function. S4.3: Encode the word vector matrix U by linear combination layer l Again, update the formula as follows: U' l = U l W5 + b2 Where U′ l b1 is the updated word vector encoding matrix, W5 is the fifth weight matrix in the linear combination layer, and b2 is the second bias term in the linear combination layer. updating the word vector encoding matrix U' in the updating process l updating the word vector encoding matrix U l replacing the word vector encoding matrix U l outputting; S4.4: encode the word vector matrix U output by step S4.3 l The next attention-guided graph convolutional network is fed in, and steps S4.1-S4.3 are iteratively executed until the M attention-guided graph convolutional networks are all executed, and the word vector matrix U output by the last attention-guided graph convolutional network is output l .

5. The method of claim 4, wherein, The improved CBOW model includes an input layer, a hidden layer and an output layer. The input layer receives one-hot encoding corresponding to context words in the center word sliding window, and the size of the sliding window is C; the hidden layer is used to multiply the one-hot encoding received by the input layer by the sixth weight matrix W6 and perform weighted sum average operation, The size of the sixth weight matrix W6 is VxD, where V is the length of the one-hot encoding. The formula for performing weighted sum average is as follows: where Y is the hidden vector of the hidden layer output, y e+α is the one-hot encoding of the word pair corresponding to the intermediate word a, β e+α is the attention weight value corresponding to the word pair of the intermediate word a, initially β e+α = 1, and is subsequently updated through training of the syntactic dependency relation extraction model. The output layer is used to multiply the hidden vector by the weight matrix between the hidden layer and the output layer to obtain an output vector with a size of 1xV, and then the output vector is normalized by softmax to obtain a probability vector with a size of 1xV. The word corresponding to the maximum probability value in the probability vector is taken as the prediction result; The source word vector library is generated by improving the CBOW model, which specifically includes the following steps: Obtain a plurality of training texts; For each training text, a training text word vector is obtained based on the training text, the training text word vector including a one-hot encoding corresponding to each word in the training text, all training text word vectors corresponding to the training text are grouped to form a first training text set, the first training text set is sent to the improved CBOW model for training, and a candidate source word vector library is constructed; words in the training text are matched with the candidate source word vector library to obtain a training source word vector sequence, the training source word vector sequence including a candidate source word vector corresponding to each word in the training text, all training source word vector sequences corresponding to the training text are grouped to form a second training text set, the second training text set is sent to the syntactic dependency relation extraction model for training, and an attention weight value β is obtained e+α ; the attention weight value β e+α is sent to the improved CBOW model to adjust the improved CBOW model, and the training of the improved CBOW model is repeated; the candidate source word vector library trained by the improved CBOW model is adjusted to the second training text set, the training of the syntactic dependency relation extraction model is repeated, and the joint training of the improved CBOW model and the syntactic dependency relation extraction model is realized; until the improved CBOW model and the syntactic dependency relation extraction model respectively meet the first training condition and the second training condition; Complete the training of the improved CBOW model and the syntax dependency relation extraction model by the current training text; After the training of the improved CBOW model and the syntax dependency relation extraction model by all training texts is completed, output the trained syntax dependency relation extraction model; and output the candidate source word vector library as the source word vector library.

6. The method of claim 5, wherein, The first training text set is input into the improved CBOW model for training to construct the candidate source word vector library, which specifically includes the following steps: The first training text set is input into the improved CBOW model with initialized parameters for training, during which the one-hot encoding corresponding to the context words in the center word sliding window is taken as the input, and the center word in the word vector of the training text is taken as the target condition to calculate the first loss value. If the first training condition is met, the first training condition is that the first loss value is within the first preset range, the trained improved CBOW model is output, otherwise, the iterative training is continued; Obtain the sixth weight matrix W6 in the trained improved CBOW model, and multiply the one-hot encoding corresponding to the words in the source language by the sixth weight matrix W6 to obtain the candidate source word vector corresponding to the words in the source language. Any word in the source language and its corresponding candidate source word vector are mapped one by one to form a candidate source word vector library; The second training text set is input into a syntactic dependency relation extraction model for training to obtain an attention weight value β e+α , and specifically includes the following steps: An entity recognition layer is added at the end of the syntax dependency relation extraction model to construct an extended syntax dependency relation extraction model, and the entity recognition layer encodes the word vector matrix U output by the last attention-based and edge label guided graph convolutional network L entity recognition is performed, and a predicted entity label sequence is output The second training text set is input into the extended syntax dependency relation extraction model with initialized parameters for training, during which the actual entity label sequence corresponding to each training text in the second training text set is taken as the target condition to calculate the second loss value. If the second training condition is met, the second training condition is that the second loss value is within the second preset range, the trained syntax dependency relation extraction model is output, otherwise, the iterative training is continued; Obtaining the last word vector encoding matrix U output by the syntax dependency relation extraction model trained based on attention and edge label guided graph convolution network L , the candidate source word vector corresponding to the center word in the training sample is denoted as f e , the attention weight value vector ζ e is calculated z e = f e • (U L ) T a vector of attention weight values ζ e the e+αth value in the vector of attention weight values β e+α .

7. The method of claim 6, wherein, The N words to be translated are input into the trained language translation model for processing, which specifically includes the following steps: N input to the encoder word embedding layer, get the source word vector matrix G to be translated, and then the source word vector matrix G is processed through the first encoding block, the second encoding block, the third encoding block, the fourth encoding block, the fifth encoding block and the sixth encoding block, the specific processing mode of the first encoding block, the second encoding block, the third encoding block, the fourth encoding block, the fifth encoding block and the sixth encoding block refers to the traditional Transformer model, and the hidden encoding matrix is obtained; The hidden encoding matrix is right shifted and sent to the decoder word embedding layer for processing to obtain the target word vector matrix; the target word vector matrix is updated through the first decoding block, the second decoding block, the third decoding block, the fourth decoding block, the fifth decoding block and the sixth decoding block to obtain the target word vector matrix; the target word vector matrix is input into the word prediction layer, and the word prediction layer normalizes the target word vector matrix through the softmax function to obtain the word probability matrix, and the word corresponding to the maximum probability value in the tth row of the word probability matrix is the target word, t is the number of times of executing the decoder, and the target word vector matrix is reconstructed, and the reconstruction method is as follows: the target word vector corresponding to the target word output by the tth decoder replaces every t row in the current target word vector, and the target word vector is obtained through the target word vector library in the decoder word embedding layer; and a target word is output every time the decoder is executed, and the decoder is stopped when the output target word is a stop symbol; All target words are combined into a target word sequence according to the execution order of the decoder; The operation and structure of the decoder word embedding layer are consistent with those of the encoder word embedding layer, except that the encoder word embedding layer is trained through the training text of the source language, the decoder word embedding layer is trained through the training text of the target language, the decoder word embedding layer includes a target word vector library, and the words in the target word vector library correspond one by one to the target word vectors; When the decoder is executed for the first time, the hidden encoding matrix is word embedded through the decoder word embedding layer to obtain the target word vector matrix; in the operation of the decoder other than the first time, the decoder word embedding layer is not executed, and the target word vector matrix output by the previous decoder is directly sent to the first decoding block to execute the operation of the decoder.

8. The text translation method based on a language translation model according to claim 7, characterized in that, The training of the language translation model specifically includes the following steps: Obtain a plurality of training texts to be translated and corresponding training target texts; all training texts to be translated are combined into a third training text set; the third training text set is input into the language translation model with initialized parameters for training, during which the training target text corresponding to the training text to be translated is taken as a target condition to calculate a third loss value, and if the third training condition is met, the third training condition is that the third loss value is within a third preset range, the trained syntax dependency relationship extraction model is output, otherwise, the training is continued iteratively.

9. A text translation system based on a language translation model, characterized by, The system is applied to the text translation method based on the language translation model in any one of claims 1-8, comprising: A text to be translated acquisition module is configured to acquire a text to be translated and perform a word segmentation operation on the text to be translated to obtain words to be translated. The language translation model management module is configured to store and train the language translation model. The language translation module is configured to input the word to be translated into the trained language translation model for processing, and output a target word sequence, wherein the target word sequence stores target words, and the target words are words of a target language corresponding to the text to be translated after text translation. The target translation text construction module is configured to splice the target words in the target word sequence to establish a target translation text.

Images