Neural language translation method and device based on joint alignment and medium

Abstract

The invention discloses a joint alignment neural language translation method and device and a medium, and relates to the field of machine translations, the method comprises the following steps: obtaining to-be-translated text data, determining a source language and a target language corresponding to the text data, and selecting a corresponding deep learning model according to the source language and the target language; and preprocessing the text data, and inputting the preprocessed text data into the deep learning model so as to output a target language corresponding to the text data through the deep learning model. Through the deep learning model, the alignment information between the source language and the target language and the translation task of the neural network are jointly trained, so that the translation accuracy is optimized, and the understanding of the model on the language structure and semantics is enhanced through the alignment information.

Description

Technical Field

[0001] The present application relates to the field of machine translation, and in particular to a joint aligned neural language translation method, device and medium. Background Art

[0002] With the development of technology, deep learning models are gradually applied in various fields.

[0003] Neural Machine Translation (NMT) is a method of machine translation using deep learning models (e.g., recurrent neural networks (RNNs), long short-term memory networks (LSTMs), transformers, etc.). The advantages of NMT include: it can automatically learn the mapping relationship from the source language to the target language without explicit rules or manually designed features.

[0004] However, in the existing NMT process, when faced with the translation of long sentences or long articles, the focus is often on the translation between words, while ignoring the alignment relationship between words, resulting in the final translation result being incoherent and inaccurate. Summary of the invention

[0005] In order to solve the above problems, this application proposes a joint alignment neural language translation method, including: Acquire text data to be translated, determine a source language and a target language corresponding to the text data, and select a corresponding deep learning model according to the source language and the target language; Preprocessing the text data, and inputting the preprocessed text data into the deep learning model, so as to output a target language corresponding to the text data through the deep learning model; The deep learning model includes: an input layer, an encoder, a decoder, an alignment module, and an output layer; The input of the encoder is the source language of the text data, and the output is the first context representation of the source language; The input of the decoder is the first context representation and the target language of the text data at the previous moment that has been generated, and the output is the target language of the text data; The alignment module is an auxiliary module during training, whose input is the first context representation output by the encoder and the cross-attention map generated by the decoder based on the cross-attention mechanism, and whose output is the alignment matrix between the source language and the target language; The deep learning model is trained by a joint loss function, where the joint loss function includes a translation loss and an alignment loss.

[0006] On the other hand, the present application also proposes a jointly aligned neural language translation device, comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the joint-aligned neural language translation method described in the above example.

[0007] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as the joint-aligned neural language translation method described in the above example.

[0008] The joint alignment neural language translation method proposed in this application can bring the following beneficial effects: Through the deep learning model, the alignment information between the source language and the target language is jointly trained with the translation task of the neural network, which not only optimizes the accuracy of the translation, but also involves enhancing the model's understanding of language structure and semantics through alignment information. By introducing technologies such as alignment loss, multi-task learning, attention mechanism, and joint training, the model can learn the complex relationship between the source language and the target language while translating, thereby improving the quality and fluency of the translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of the process of the joint alignment neural language translation method in an embodiment of the present application; Figure 2 This is a schematic diagram of the architecture of a deep learning model during training in one scenario in an embodiment of the present application; Figure 3 This is a schematic diagram of the architecture when a deep learning model is used in one scenario in an embodiment of the present application; Figure 4 Schematic diagram of a jointly aligned neural language translation device in an embodiment of the present application. DETAILED DESCRIPTION

[0010] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0011] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0012] like Figure 1 As shown, the embodiment of the present application provides a joint aligned neural language translation method, including: S101: Acquire text data to be translated, determine a source language and a target language corresponding to the text data, and select a corresponding deep learning model according to the source language and the target language.

[0013] The source language corresponding to the text data (hereinafter referred to as the source language) and the target language corresponding to the text data (hereinafter referred to as the target language) are selected by the user. Of course, the source language can be automatically identified, and if the target language is not specified by the user, it can be one of the default common languages.

[0014] For different combinations of source and target languages, a corresponding deep learning model can be pre-set and trained. For example, when the source language and target language are English and Chinese (or Chinese and English), a corresponding deep learning model is set; when the source language and target language are English and Japanese (or other languages ​​such as Korean), at least one language in the combination is changed, and a new deep learning model can be set and trained. Of course, the architecture of each deep learning model is consistent, but due to different languages, the specific model parameters may be different.

[0015] S102: Preprocess the text data, and input the preprocessed text data into the deep learning model, so as to output a target language corresponding to the text data through the deep learning model.

[0016] The preprocessing process may include: data cleaning, word segmentation, stop word removal, word vector extraction, etc. When training a deep learning model, the preprocessing of text data may also include: text data enhancement, low-frequency word processing, etc.

[0017] Through the deep learning model, the alignment information between the source language and the target language is jointly trained with the translation task of the neural network, which not only optimizes the accuracy of the translation, but also involves enhancing the model's understanding of language structure and semantics through alignment information. By introducing technologies such as alignment loss, multi-task learning, attention mechanism, and joint training, the model can learn the complex relationship between the source language and the target language while translating, thereby improving the quality and fluency of the translation.

[0018] Specifically, if Figure 2 As shown, the deep learning model includes: input layer, encoder, decoder, alignment module (which is an auxiliary module during training and only exists during the training process), and output layer.

[0019] The input layer and the output layer are used to receive the source language and output the target language respectively. The source language is input to the input layer, processed by the encoder and decoder, and the target language is output through the output layer. The output layer can be a fully connected layer to achieve the output of the target language.

[0020] The encoder is mainly responsible for processing source language sentences and generating context representations. Its input is the source language of the text data, and its output is the first context representation of the source language. In actual processing, each sentence of the source language can be processed separately, and the first context representation corresponding to each sentence can be output.

[0021] Specifically, the source language input is passed through the encoder and processed by the Masked Self-Attention mechanism. The self-attention mechanism calculates the relationship between the words in the source language and outputs the source language context representation of each position based on this (for the convenience of description, it is called the first context representation here).

[0022] The contextual representation of the source language here refers to the encoder's understanding of the context of the word at each position, which contains information about all words in the source language (that is, it depends on the relationship with other words).

[0023] The decoder is mainly responsible for generating target language sentences and translating based on the output of the encoder. Its input consists of two parts, namely the first context representation and the target language of the text data at the previous moment that has been generated (that is, the words at each position have been generated. For example, assuming that the target language at moment t is currently being produced, the target language at the previous moment includes all target languages ​​before moment t-1. Of course, the count at this moment is usually counted separately for each sentence), and the output is the target language of the text data.

[0024] The decoder uses the self-attention mechanism to output the second context representation of the target language at the current moment based on the target language at the previous moment. This process is based on the contextual relationship within the target language, that is, the self-attention calculation within the decoder. The self-attention mechanism of the decoder calculates the dependency relationship between the generated partial words in the target language and outputs the second context representation of the target language at the current moment.

[0025] Through the cross-attention mechanism, a corresponding cross-attention map is generated according to the first context representation and the second context representation.

[0026] At each layer of the decoder, the decoder performs a cross-attention mechanism for the second context representation of the target language and the first context representation of the source language at time t. The cross-attention mechanism generates a cross-attention graph by calculating the attention weights between the current word in the target language and all words in the source language.

[0027] The cross attention map is essentially a weighted relationship between the target language words and the source language words, showing the degree to which the target language depends on each word in the source language when generating. This weighted relationship determines the contextual representation of the target language generation and reflects the alignment information between the two languages.

[0028] After the cross-attention map is generated, it does not directly affect the generation of the target language, but affects the target language generation through the weighted first context representation of the source language. In other words, the role of the cross-attention map is to determine the weighting method of the source language context, and the final target language generation depends on the combination of the weighted source language representation and the target language context.

[0029] To expand on this, when generating a Cross-Attention Map, we first determine the first context representation as ; Among them, the matrix size of the first context representation is ,in, is the number of words in the source language, and d is the representation dimension of each word.

[0030] Determine the cross attention mechanism as ; where Q is the query matrix, K is the key matrix, and V is the value matrix. The matrix sizes of Q, K, and V are Among them, Q comes from the second context representation of the target language, usually the hidden state of the decoder at the current time step, K comes from the first context representation of the source language of the encoder, and V also comes from the first context representation of the source language of the encoder, which is consistent with K.

[0031] Through the cross attention mechanism, the first context representation and the second context representation are weighted and summed according to the weights corresponding to the source language and the target language respectively to obtain a cross attention map. The matrix size is , is the number of words in the target language, cross attention map Each parameter in It represents the attention weight of the target language word i to the source language word j. Each row represents the dependency of a target word on a source language word, and each column represents the contribution of a source language word to the target language word.

[0032] Through the cross attention map, we can achieve: 1. Alignment between source and target languages: The cross-attention map is a direct reflection of the alignment information between the source and target languages. In machine translation, the cross-attention map can show which words in the source language contribute the most to the generation of the target word. By looking at the cross-attention map, you can see the relationship between the target language words and the source language words, which helps you understand how the model translates.

[0033] 2. Translation quality optimization: The cross-attention graph helps the model learn how to effectively map the first context information of the source language to the target language. By optimizing the calculation of cross-attention, the model can better capture the grammatical and semantic relationship between the source language and the target language, thereby improving the quality of translation.

[0034] 3. Visual analysis: The cross-attention map can be used to visualize and analyze how the translation model works. By observing the cross-attention map, it can help R&D personnel check whether the model correctly aligns the vocabulary between the source language and the target language. Usually, the strong association between the source language words and the target language words that the model notices can be indicated by marking the highlighted areas in the cross-attention map.

[0035] 4. Alignment module support: The output of the cross-attention map can also be used to generate an alignment matrix between the source language and the target language. The alignment matrix can calculate a more accurate alignment relationship by analyzing the pattern of the cross-attention map, further improving the translation accuracy.

[0036] After obtaining the cross-attention map, the attention weights between each target language word and all words in the source language are obtained. At this time, the cross-attention map is passed through the first context representation to generate the corresponding weighted context representation (for example, the cross-attention map is multiplied by the first context representation to obtain the weighted first context representation, that is, the weighted context representation). The weighted context representation reflects the dependence of the target language on the source language when generating.

[0037] The target language representation at the current moment is output by combining the weighted context representation with the second context representation. For example, the weighted context representation is combined with the second context representation by concatenation, addition, etc., so as to obtain the target language representation, which represents the output probability of each word in the target language. Thus, the word with the highest probability is selected through the target language representation, and the target language of the text data at the current moment is decoded and generated.

[0038] The encoder and decoder have been explained above, and the alignment module will be explained next.

[0039] like Figure 2 As shown in the figure, during the model training process, the alignment module is connected to both the encoder and the decoder. The alignment module is an auxiliary module during training. During the model training process, it is mainly used to generate an alignment matrix between the source language and the target language, which represents the alignment probability of each target language word with all source language words. Its input is the first context representation output by the encoder and the cross-attention map generated by the decoder based on the cross-attention mechanism. The output is the alignment matrix between the source language and the target language, and then the corresponding alignment loss is calculated through the alignment matrix.

[0040] like Figure 3 As shown, the alignment module is an auxiliary module during training. It exists explicitly during the model training process. After the model training is completed, the alignment module does not exist explicitly or participate in the actual translation generation process of the target language. At this time, the translation generation process of the target language can be completed through the architecture of input layer-encoder-decoder-output layer.

[0041] The alignment matrix is ​​a two-dimensional matrix that represents the alignment relationship between the source language and the target language. Each element in the matrix represents the degree of alignment between the source language word and the target language word (that is, the attention weight). Each row of the alignment matrix usually corresponds to a target language word, each column corresponds to a source language word, and the element value of the matrix represents the attention score between the target language word and the source language word.

[0042] The alignment module first extracts the attention distribution information between source and target language words from the cross-attention map of the decoder.

[0043] The alignment module then normalizes the cross-attention map so that the sum of the attention weights of each row of parameters in the cross-attention map is 1, indicating the degree of dependence of the target language words on the individual words of the source language words.

[0044] Next, we construct an alignment matrix between the source language and the target language by thresholding the attention weights of each row (which means removing parameters below the preset threshold in the normalized cross-attention map, so that the correspondence between words with too low probability is no longer considered, thereby reducing the computational complexity). The alignment matrix can be understood as a probability distribution, which represents the alignment relationship between each word in the target language and each word in the source language. The alignment matrix is ​​input into the alignment loss, and the corresponding loss is calculated to continue the model parameter optimization.

[0045] Of course, in some applications, the alignment matrix can also be visualized as a heat map to intuitively show the correspondence between the target language words and the source language words. This visualization can often help analyze the translation process and optimize the model's alignment strategy.

[0046] After the modules in the architecture of the deep learning model are set up, the deep learning model can be trained so that the trained deep learning model can perform corresponding translation tasks.

[0047] The deep learning model is trained through a joint loss function, which includes translation loss and alignment loss. Joint training is used in this process. In the same training process, the model optimizes multiple goals or tasks at the same time. In joint aligned neural translation, in addition to optimizing translation accuracy, it is also possible to optimize the alignment information between the source and target languages ​​at the same time, so that the neural network can take into account more context and alignment relationships when generating translations.

[0048] During the model training process, data preparation is first required. The training data (including the translated source and target languages) is obtained and preprocessed accordingly. Then, the model’s word embedding layer is initialized using pretrained word embeddings (e.g., Word2Vec or BERT), and the encoder and decoder are initialized.

[0049] The training data is input into the model, and the translation task is performed in the encoder and decoder through forward propagation, and the alignment task is performed using the alignment module.

[0050] After calculating the loss through the joint loss function, the model parameters are optimized through back propagation, and the training process is iterated until the deep learning model converges and the model training is completed.

[0051] In one embodiment, considering that in some translation tasks, a word in the target language may be related not only to words in the current source language sentence, but also to words in other sentences. This situation usually occurs in long text or paragraph-level translation tasks, especially in literary translation, technical document translation or dialogue systems, where the alignment of the source language and the target language is not limited to a single sentence pair.

[0052] Based on this, for this situation, the encoder is also used to: For the first sentence currently being translated, if it is determined that there is a first word that is a preset fixed word based on the source language corresponding to the first sentence and the target language generated in the first sentence at a previous moment, then in the sentence before the first sentence, find a second word that can be used as a specific instance description of the first word, and determine the second sentence where the second word is located.

[0053] For example, the first word is usually preset, for example, it can be a personal pronoun such as "he", "she" or other forms of pronouns, which are usually related to the words of the previous sentence, so the second word corresponding to it is found, for example, the second word can be a specific name, which can be used as a specific example description of the first word. Of course, the interval between the second sentence and the first sentence is less than the preset length (for example, the interval is less than 3-5 sentences), so as to prevent too many other sentences from being added.

[0054] At this time, starting from the second sentence and ending at the first sentence, for all the sentences contained therein, the encoder outputs the first context representation of the source language of all the sentences according to the self-attention mechanism, which is used as the first context representation of the source language of the first sentence for the generation of the cross-attention graph. In other words, the first context representation generated at this time includes not only the current first sentence that is currently performing the translation task, but also some other adjacent sentences before it.

[0055] At this point, the decoder is also used to: Through the self-attention mechanism, based on the target language at the previous moment generated in all the sentences corresponding to the second sentence to the first sentence, the second context representation of the target language at the current moment is output for the generation of the cross-attention graph and the target language at the current moment. In other words, when generating the second context representation, it is also generated based on all the sentences in this part, not just the current first sentence.

[0056] In this way, cross-sentence alignment is achieved, thereby expanding the scope of the alignment matrix so that it not only considers the alignment between a single source language sentence and a target language sentence, but also considers the alignment relationship across multiple sentences. The alignment matrix is ​​adjusted by the contextual information between sentences, so that the semantic associations across sentences can be fully captured.

[0057] In one embodiment, as described above, the joint loss function includes a translation loss and an alignment loss.

[0058] Among them, the translation loss is: ; The length of the target language is determined based on the number of words in the target language sentence. is the tth word in the target language, To generate a target word given the first t-1 target words and the source language sentence x probability.

[0059] The alignment loss is: ; The length of the source language is determined based on the number of words in the source language sentence. is the alignment probability or attention weight between the i-th word in the source language and the j-th word in the target language.

[0060] Joint loss function , , is a hyperparameter, at this time, the corresponding training task can be directly established based on . However, this situation is not only likely to lead to a long training time, but also to poor training results.

[0061] Based on this, a translation task is established for the deep learning model through translation loss, and the first stage model training process is performed. In the first stage model training process, after each preset number of training cycles (that is, after each corresponding batch of training), the first performance indicator of the deep learning model is calculated, and when the first performance indicator reaches the preset standard, the first stage model training process is stopped.

[0062] Among them, the first performance indicator is used to describe the translation ability of the deep learning model. For example, the first performance indicator can be BLEU score, TER, METEOR and other indicators, so that when it reaches the preset standard, early stopping is performed. At this time, the deep learning model often has not converged yet, which can save computing power and enable it to have preliminary translation ability in the first stage model training process, and prevent overfitting, which is conducive to the simultaneous training of translation ability and alignment ability in the second stage model training process.

[0063] Next, through the joint loss function, the translation task and alignment task are established for the deep learning model, and the second stage model training process is performed until the deep learning model converges, thus completing the training process of the deep learning model.

[0064] Furthermore, in the second stage of model training, the hyperparameters , It can be set to a fixed value or dynamically set during training to further increase the convergence speed during training and improve the final model capability.

[0065] At this time, the objective function is defined as ;in, is a hyperparameter , Configuration combination, , They are the first performance indicator and the second performance indicator of the deep learning model respectively; the second performance indicator is used to describe the alignment ability of the deep learning model, for example, the second performance indicator can be alignment accuracy, alignment recall rate, alignment F1 score, etc. The objective function refers to the joint loss function.

[0066] The surrogate model is constructed through a multi-objective Gaussian process. The Gaussian process model not only provides the predicted value of the objective function, but also provides the uncertainty (variance) of the prediction, while the multi-objective Gaussian process model can provide the predicted value and uncertainty of the objective function under multiple objectives.

[0067] Based on the first performance indicator, determine the joint loss function The initial value of and determines the joint loss function The default value corresponding to the initial value of , thus obtaining the hyperparameter , The initial configuration combination of The initial value of is negatively correlated. The higher the first performance index is, the stronger the translation ability of the deep learning model is during the first stage model training. At this time, in the second stage model training, it can be The value of , thus focusing more on the alignment ability of the model. There is also a corresponding lower limit value.

[0068] The deep learning model is trained through the initial configuration combination. , After the initial value of is determined, the joint loss function is determined. At this time, the joint loss function is used to train the deep learning model.

[0069] After a preset number of training cycles, the parameters of the model itself have changed due to the back-propagation optimization process. At this time, the first performance indicator and the second performance indicator corresponding to the initial configuration combination are determined, that is, after the preset number of training cycles are trained using the initial configuration combination, the current first performance indicator and the second performance indicator of the deep learning model are determined. Then, the proxy model is updated according to the first performance indicator and the second performance indicator, and the hyperparameters of the proxy model are optimized to better predict the objective function.

[0070] The initial configuration combination is updated through the Poisson acquisition function to obtain the updated configuration combination. Among them, the Poisson acquisition function (Pareto acquisition function) finds the next experimental point by selecting the point closest to the Pareto front in the current solution space.

[0071] Based on the update configuration combination, the deep learning model is repeatedly trained and the proxy model is updated until a preset condition is met, wherein the preset condition includes: the number of updates of the proxy model reaches an upper limit, or the deep learning model converges.

[0072] After obtaining the updated configuration combination, the objective function (that is, the joint loss function) is updated through it, and the updated joint loss function is used to continue training the deep learning model. After a preset number of training cycles, the first performance indicator, the second performance indicator, and the proxy model are repeatedly calculated until the preset conditions are met.

[0073] If the preset condition is that the deep learning model converges, the deep learning model can be directly obtained to complete the model training.

[0074] If the preset condition is that the number of updates of the proxy model reaches the upper limit, the update configuration combination of the last update is used as the setting value of the hyperparameter of the joint loss function in the deep learning model, and the deep learning model is continued to be trained according to the setting value until the deep learning model converges and the model training is completed.

[0075] In addition, during the training process, in order to reduce the reliance on manually annotated data, self-supervised learning can be used to generate alignment information. The model generates partial translation pairs (for example, words or sentence fragments) and uses the self-attention mechanism to perform self-alignment learning. In this way, training can be performed from a large amount of unannotated parallel corpus and implicit alignment information can be obtained. Of course, preliminary alignment data can also be generated through traditional statistical machine translation (SMT) methods (such as GIZA++), and then used to help train the neural network model. In this way, the reliance on explicit manual alignment data can be reduced.

[0076] like Figure 4 As shown, the embodiment of the present application also provides a joint-aligned neural language translation device, including: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the joint-aligned neural language translation method described in any of the above embodiments.

[0077] An embodiment of the present application further provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as the joint-aligned neural language translation method described in any of the above embodiments.

[0078] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0079] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0080] The above is only the embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A joint alignment neural language translation method, characterized in that: include: Acquire text data to be translated, determine a source language and a target language corresponding to the text data, and select a corresponding deep learning model according to the source language and the target language; Preprocessing the text data, and inputting the preprocessed text data into the deep learning model, so as to output a target language corresponding to the text data through the deep learning model; The deep learning model includes: an input layer, an encoder, a decoder, an alignment module, and an output layer; The input of the encoder is the source language of the text data, and the output is the first context representation of the source language; The input of the decoder is the first context representation and the target language of the text data at the previous moment that has been generated, and the output is the target language of the text data; The alignment module is an auxiliary module during training, whose input is the first context representation output by the encoder and the cross-attention map generated by the decoder based on the cross-attention mechanism, and whose output is the alignment matrix between the source language and the target language; The deep learning model is trained by a joint loss function, where the joint loss function includes a translation loss and an alignment loss.

2. The method according to claim 1, characterized in that The encoder inputs the source language of the text data, and outputs a first context representation of the source language through a self-attention mechanism; The input of the decoder is the first context representation and the target language of the generated text data at the previous moment; The decoder outputs a second context representation of the target language at the current moment according to the target language at the previous moment through a self-attention mechanism; Generate a corresponding cross-attention map according to the first context representation and the second context representation through a cross-attention mechanism, so that the cross-attention map generates a corresponding weighted context representation through the first context representation; Outputting the target language representation at the current moment by combining the weighted context representation with the second context representation; The target language of the text data at the current time is decoded and generated using the target language representation.

3. The method according to claim 2, characterized in that Generate a corresponding cross-attention graph according to the first context representation and the second context representation through the cross-attention mechanism, specifically including: Determine the first context representation as ; Wherein, the matrix size of the first context representation is ,in, is the number of words in the source language, and d is the representation dimension of each word; Determine the cross attention mechanism as ; where Q is the query matrix, K is the key matrix, and V is the value matrix. The matrix sizes of Q, K, and V are ; Through the cross attention mechanism, the first context representation and the second context representation are weighted and summed according to the weights corresponding to the source language and the target language respectively, to obtain a cross attention map; wherein the cross attention map The matrix size is , is the number of words in the target language, cross attention map Each parameter in It represents the attention weight of the i-th word in the target language to the j-th word in the source language.

4. The method according to claim 3, characterized in that The alignment module normalizes the cross attention map so that the sum of the attention weights of each row parameter of the cross attention map is 1; The cross-attention map is thresholded to remove parameters below a preset threshold in the normalized cross-attention map to obtain an alignment matrix.

5. The method according to claim 2, characterized in that: The encoder is further used for: For a first sentence currently being translated, if it is determined that a first word of a preset fixed word exists according to the source language corresponding to the first sentence and the target language generated in the first sentence at a previous moment, then a second word that can be described as a specific instance of the first word is searched in a sentence before the first sentence, and a second sentence in which the second word is located is determined; wherein the interval between the second sentence and the first sentence is less than a preset length; Taking the second sentence as the starting point and the first sentence as the end point, for all the sentences contained therein, outputting the first context representation of the source language of all the sentences through the encoder according to the self-attention mechanism as the first context representation of the source language of the first sentence for generating a cross-attention map; The decoder is further used for: Through the self-attention mechanism, based on the target language at the previous moment that has been generated in all sentences corresponding to the second sentence to the first sentence, a second context representation of the target language at the current moment is output for the generation of the cross-attention map and the target language at the current moment.

6. The method according to claim 1, characterized in that The training process of the deep learning model includes: Determine the joint loss function including translation loss and alignment loss; The translation loss is: ; The length of the target language is determined based on the number of words in the target language sentence. is the tth word in the target language, To generate a target word given the first t-1 target words and the source language sentence x probability; The alignment loss is: ; The length of the source language is determined based on the number of words in the source language sentence. is the alignment probability or attention weight between the i-th word in the source language and the j-th word in the target language; By using the translation loss, a translation task is established for the deep learning model, and a first-stage model training process is performed, so that during the first-stage model training process, a first performance indicator of the deep learning model is calculated after each preset number of training cycles, and the first-stage model training process is stopped when the first performance indicator reaches a preset standard; wherein the first performance indicator is used to describe the translation capability of the deep learning model; By combining the loss function, a translation task and an alignment task are established for the deep learning model, and a second-stage model training process is performed until the deep learning model converges; Among them, the joint loss function ; , is a hyperparameter.

7. The method according to claim 6, characterized in that The method further comprises: In the second stage of model training, the objective function is defined as ;in, is a hyperparameter , Configuration combination, , are respectively the first performance indicator and the second performance indicator of the deep learning model; wherein the second performance indicator is used to describe the alignment ability of the deep learning model; and the objective function is the joint loss function; Constructing surrogate models through multi-objective Gaussian processes; Based on the first performance indicator, determine the joint loss function The initial value of and determine the joint loss function The default value corresponding to the initial value of , thus obtaining the hyperparameter , An initial configuration combination; wherein the first performance indicator and the The initial value of is negatively correlated; The deep learning model is trained by the initial configuration combination, and after a preset number of training cycles, a first performance indicator and a second performance indicator corresponding to the initial configuration combination are determined, and the proxy model is updated according to the first performance indicator and the second performance indicator; The initial configuration combination is updated through a Poisson acquisition function to obtain an updated configuration combination, and based on the updated configuration combination, the deep learning model is repeatedly trained and the proxy model is updated until a preset condition is met; wherein the preset condition includes: the number of updates of the proxy model reaches an upper limit, or the deep learning model converges; If the preset condition is that the number of updates of the proxy model reaches an upper limit, the update configuration combination of the last update is used as the setting value of the hyperparameter of the joint loss function in the deep learning model, and the deep learning model continues to be trained according to the setting value until the deep learning model converges.

8. The method according to claim 1, characterized in that Before selecting a corresponding deep learning model according to the source language and the target language, the method further includes: For different combinations of source and target languages, corresponding deep learning models are pre-set and trained.

9. A jointly aligned neural language translation device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the joint-aligned neural language translation method described in any one of claims 1 to 8.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured as the joint aligned neural language translation method as described in any one of claims 1 to 8.

Images