Customized maintenance intelligent translation system based on Transformer model

Abstract

The invention discloses a customized locomotive intelligent translation system based on a Transformer model, which belongs to the technical field of locomotive machine translation systems, and includes an input module, a translation body, and an output module. The translation body is also connected to a locomotive customized translation vocabulary library, and the locomotive customized translation vocabulary library includes a vocabulary input module, a locomotive relevance evaluation module, and a sorting module. The invention can filter out words with poor relevance between the meaning of words and the daily meaning in locomotive data from a web page environment, construct a locomotive translation vocabulary library, and combine a self-attention network to perform customized translation of words with poor relevance between the meaning of words and the daily meaning in locomotive data, thereby improving the accuracy and professionalism of locomotive translation, and solving the problem that the poor accuracy of locomotive professional word translation in existing locomotive translation leads to insufficient understanding of locomotive-related technical information by locomotive personnel.

Description

Technical Field

[0001] The present invention relates to a translation system, in particular to a customized aircraft maintenance intelligent translation system based on the Transformer model, belonging to the technical field of aircraft maintenance machine translation systems. Background Art

[0002] Aircraft maintenance work is an industry engaged in aircraft maintenance and repair. When performing aircraft maintenance and repair, a lot of equipment data, aviation manuals, maintenance manuals, or repair instructions are in foreign languages, and aircraft maintenance translators are required to translate these materials with high quality and efficiency. The commonly used method in aircraft maintenance translation is machine translation. Machine translation uses a computer to convert one natural language into another natural language, which has important practical value and is an important reliance for bridging the language communication gap among aircraft maintenance workers. Machine translation has evolved from early dictionary matching to translation based on rules that combine dictionaries with the knowledge of linguistics experts, and then to corpus-based statistical machine translation. In recent years, the leapfrog development of neural network technology in the field of machine translation has significantly improved the quality and efficiency of machine translation. The dependency structure of neural network translators is the encoder-decoder structure. The encoder "understands" the input language and forms floating-point numbers in a specific dimension. Subsequently, the decoder generates the translation result of the target language word by word based on this vector. In the initial stage, recurrent neural networks (RNNs, Recurrent Neural Networks) were used as the network structures of the encoder and decoder in neural network translators. RNNs are good at modeling natural languages. RNN networks represented by long short-term memory networks (LSTMs, Long Short-Term Memory networks) and gated recurrent unit networks (GRUs, Gated Recurrent Unit networks) "remember" the more important words in a sentence through gate control, allowing the memory to be retained for a longer time. Later, convolutional neural networks (CNNs, Convolutional Neural Networks) and self-attention networks (Transformers) were proposed as the encoder and decoder structures. The translation effect far exceeds that of machine translation based on RNN networks, and it can also perform parallel multi-tasks during training, significantly improving the training efficiency. Among them, machine translation based on Transformers has become the mainstream framework in the industry. However, there are difficulties in aircraft maintenance translation, such as involving multiple specialties and having highly specialized terms. Traditional machine translation based on Transformers cannot meet the translation needs of aircraft maintenance workers and cannot accurately "understand" the specific meanings of the words to be translated in aircraft maintenance work. Take "bleed" as an example. In everyday English, this word means "to bleed", but in aircraft maintenance English, it means "bleed air". Similarly, "governor" in aircraft maintenance English refers to "regulator", but in everyday translation, it refers to "governor" or "ruler". Failures in aircraft structural components and aircraft systems can cause serious flight accidents. The analysis and elimination of faults by aircraft maintenance personnel are closely related to the translation quality of the technical materials related to aircraft maintenance that they read.Therefore, to address the above issues, the present invention proposes a customized aircraft maintenance intelligent translation system based on the Transformer model. This system can screen out words in aircraft maintenance materials with a poor relevance between their meanings and daily meanings from the web environment, construct an aircraft maintenance translation vocabulary database, and combine self-attention networks to perform customized translations on words in aircraft maintenance materials with a poor relevance between their meanings and daily meanings, improving the accuracy and professionalism of aircraft maintenance translations and solving the deficiency in existing aircraft maintenance translations where the poor accuracy of translating aircraft maintenance professional terms leads to insufficient understanding of aircraft maintenance-related technical materials by aircraft maintenance personnel. Summary of the Invention

[0003] The main objective of the present invention is to provide a customized aircraft maintenance intelligent translation system based on the Transformer model to solve the deficiency in existing aircraft maintenance translations where the poor accuracy of translating aircraft maintenance professional terms leads to insufficient understanding of aircraft maintenance-related technical materials by aircraft maintenance personnel.

[0004] The objective of the present invention can be achieved by adopting the following technical solutions:

[0005] A customized aircraft maintenance intelligent translation system based on the Transformer model includes an input module, a translation main body connected to the input module, and an output module connected to the translation main body. The translation main body is also connected to an aircraft maintenance customized translation vocabulary database. The aircraft maintenance customized translation vocabulary database includes a vocabulary input module, which is connected to an aircraft maintenance relevance evaluation module, and the aircraft maintenance relevance evaluation module is connected to a sorting module. The aircraft maintenance relevance evaluation module first determines whether the frequency ranking of the input word in aircraft maintenance materials is within the top three-quarters of the total ranking. If the frequency of the input word appears within the top three-quarters of the total ranking, the relevance between the input word and aircraft maintenance is positively correlated with the ratio of the frequency ranking of the word to the total ranking, positively correlated with the ratio of the number of times the most frequently appearing definition of the input word in aircraft maintenance materials to the total number of times all definitions appear in aircraft maintenance materials, and also positively correlated with the difference degree of the definitions of the input word. If the frequency ranking of the input word in aircraft maintenance materials is not within the top three-quarters of the total ranking, the relevance between the input word and aircraft maintenance work is 0. The formula for the relevance between the input word and aircraft maintenance is:

[0006] When the formula for the relevance between the input word and aircraft maintenance work is:

[0007] D R = γ*(α1*I PD + α2I PM );

[0008] Where: N A is denoted as the total number of words in aircraft maintenance materials, N C is denoted as the number of times the word appears in aircraft maintenance materials. Denoted as the pair The ranking number after ranking, D R Denoted as the relevance between the input word and the aircraft maintenance work, I PD Denoted as the frequency index of the input word, α1 is denoted as the weight of the input word frequency index, and the value of α1 is set to 0.6, I PM Denoted as the ranking index of the input word, α2 is denoted as the weight of the input word ranking index, and the value of α2 is set to 0.4. The expert scores the input word according to the difference degree of the input word, and the score of the semantic difference degree of the input word is denoted as γ, where the value range of γ is [0, 1]. When the difference degree of each interpretation of the input word is larger, the value of γ is larger;

[0009] Among them, the formula for the frequency index is:

[0010]

[0011] In the formula: u is denoted as the set of the frequency quantities of each interpretation of the input word appearing in the aircraft maintenance materials, p i Denoted as the frequency number of the i-th interpretation of the input word appearing in the aircraft maintenance materials, Denoted as the number of times the interpretation with the most total occurrences in the aircraft maintenance materials appears in the translated interpretations of the input word, and m is denoted as the total number of interpretations of the input word;

[0012] The formula for the ranking index is;

[0013]

[0014] When The relevance formula between the input word and the aircraft maintenance work is:

[0015] D R = 0.

[0016] As a further solution of the present invention, the translation main body includes an encoding component formed by connecting a plurality of encoders and a decoding component connected to the encoding component. The decoding component includes a plurality of decoders connected together. The encoder includes a first self-attention layer and a first feed-forward neural network connected to the first self-attention layer. The decoder includes a second self-attention layer, an encoding-decoding attention layer connected to the second self-attention layer, and a second feed-forward neural network connected to the encoding-decoding attention layer

[0017] As a further solution of the present invention, the sorting module sorts the interpretations of the input word according to the relevance between each interpretation of the input word output by the aircraft maintenance relevance, and uses the interpretation with high relevance to the aircraft maintenance work as the interpretation preferentially queried by the query vector in the translation main body.

[0018] The usage process of a customized aircraft maintenance intelligent translation system based on the Transformer model is characterized by including the following steps:

[0019] S1. Word vector and position information encoding: Convert the vocabulary input by the input module into word vectors, where the word vectors include word embeddings and position information encoding;

[0020] S2. Encoder calculation: Send the word vectors into the encoder in the translation main body. The first self-attention layer in the encoder receives the word vectors, and after residual connection and normalization operations, it is sent to the first feed-forward neural network. Then, after residual connection and normalization operations, the output result is input into the decoder;

[0021] S3. Decoder calculation: The second self-attention layer receives the output result. After residual connection and normalization operations, the output result of the first feed-forward network and the output result of the second self-attention layer are input into the encoder-decoder attention layer. During decoding, in combination with the sorting module, the relevance between the input vocabulary interpretation and aircraft maintenance work is used to select the interpretation of the vocabulary. Perform a linear transformation and softmax operation on the output of the decoder to obtain the probability of each target word in the vocabulary table until the end symbol is predicted and the calculation stops;

[0022] S4. Output the translation result: The output module outputs the translation result.

[0023] The beneficial technical effects of the present invention: According to the customized aircraft maintenance intelligent translation system based on the Transformer model of the present invention, through the setting of the aircraft maintenance customized translation vocabulary library, it is possible to screen out the vocabulary in aircraft maintenance materials with a poor relevance between the word meaning and the daily word meaning from the web environment, construct an aircraft maintenance translation vocabulary library, and combine the self-attention network to perform customized translation on the vocabulary in aircraft maintenance materials with a poor relevance between the word meaning and the daily word meaning, optimize the order of selecting the vocabulary interpretation during aircraft maintenance translation, select the interpretation with a high relevance in aircraft maintenance work materials, improve the accuracy and professionalism of aircraft maintenance translation, and solve the deficiency in the existing aircraft maintenance translation that the accuracy of translating aircraft maintenance professional terms is poor, resulting in insufficient understanding of aircraft maintenance-related technical materials by aircraft maintenance personnel. Description of the Drawings

[0024] Figure 1 It is the structural block diagram of the customized aircraft maintenance intelligent translation system based on the Transformer model of the present invention;

[0025] Figure 2 It is the structural block diagram of the encoder and decoder of the customized aircraft maintenance intelligent translation system based on the Transformer model of the present invention. Detailed Embodiment

[0026] To make the technical solution of the present invention clearer and more explicit to those skilled in the art, the present invention will be further described in detail below in conjunction with embodiments and the accompanying drawings. However, the implementation manners of the present invention are not limited thereto.

[0027] As Figure 1-2 shown, the customized maintenance intelligent translation system based on the Transformer model provided in this embodiment includes an input module, a translation main body connected to the input module, and an output module connected to the translation main body. The translation main body is also connected to a maintenance customized translation vocabulary library. The maintenance customized translation vocabulary library includes a vocabulary input module. The vocabulary input module is connected to a maintenance relevance evaluation module. The maintenance relevance evaluation module is connected to a sorting module. The maintenance relevance evaluation module first determines whether the frequency ranking of the input vocabulary in the maintenance materials is in the top three-quarters of the total ranking number. If the frequency of the input vocabulary appears in the top three-quarters of the total ranking number, the relevance of the input vocabulary to maintenance is positively correlated with the ratio of the frequency ranking of the vocabulary to the total ranking, positively correlated with the ratio of the number of times the most frequently occurring interpretation in the maintenance materials appears in the translation interpretation of the input vocabulary to the total number of times all interpretations appear in the maintenance materials, and also positively correlated with the difference degree of the interpretations of the input vocabulary. If the frequency ranking of the input vocabulary in the maintenance materials is not in the top three-quarters of the total ranking number, the relevance of the input vocabulary to maintenance work is 0. The formula for the relevance of the input vocabulary to maintenance is:

[0028] When the relevance formula of the input vocabulary to maintenance work is:

[0029] D R = γ * (α1 * I PD + α2I PM );

[0030] In the formula: N A is recorded as the total number of words in the maintenance materials, N C is recorded as the number of times the vocabulary appears in the maintenance materials, is recorded as after ranking R is recorded as the relevance of the input vocabulary to maintenance work, D PD is recorded as the frequency index of the input vocabulary, α1 is recorded as the weight of the frequency index of the input vocabulary, and the value of α1 is set to 0.6. I PM is recorded as the ranking index of the input vocabulary, α2 is recorded as the weight of the ranking index of the input vocabulary, and the value of α2 is set to 0.4. Experts score the input vocabulary according to the difference degree of the input vocabulary. The score of the semantic difference degree of the input vocabulary is recorded as γ, where the value range of γ is [0, 1]. When the difference degree of each interpretation of the input vocabulary is greater, the value of γ is greater;

[0031] Among them, the formula for the frequency index is:

[0032]

[0033] Where: u is denoted as the set of the frequency quantities of each interpretation of the input vocabulary appearing in the maintenance materials, and p i is denoted as the frequency number of the i-th interpretation of the input vocabulary appearing in the maintenance materials, is denoted as the number of times the interpretation with the largest total number of appearances in the maintenance materials appears in the translated interpretations of the input vocabulary, and m is denoted as the total number of interpretations of the input vocabulary;

[0034] The formula for the ranking index is;

[0035]

[0036] When the relevance formula between the input vocabulary and the maintenance work is:

[0037] D R = 0.

[0038] When the relevance between the input vocabulary and the maintenance work is set to 0, which is convenient for excluding the vocabulary with relatively low frequencies of appearance in the maintenance materials.

[0039] The customized maintenance intelligent translation system based on the Transformer model proposed by the present invention can screen out the vocabulary with relatively poor relevance between the meanings in the maintenance materials and the daily meanings from the web environment through the setting of the maintenance customized translation vocabulary library, construct the maintenance translation vocabulary library, combine the self-attention network to perform customized translation on the vocabulary with relatively poor relevance between the meanings in the maintenance materials and the daily meanings, optimize the selection order of the vocabulary interpretations during maintenance translation, select the interpretations with high relevance in the maintenance work materials, improve the accuracy and professionalism of maintenance translation, and solve the deficiency in the existing maintenance translation that the translation accuracy of maintenance professional terms is poor, resulting in insufficient understanding of maintenance-related technical materials by maintenance personnel.

[0040] The translation main body includes an encoding component composed of multiple encoders connected in series and a decoding component connected to the encoding component. The decoding component includes multiple decoders connected in series. The encoder includes a first self-attention layer and a first feed-forward neural network connected to the first self-attention layer. The decoder includes a second self-attention layer, an encoder-decoder attention layer connected to the second self-attention layer, and a second feed-forward neural network connected to the encoder-decoder attention layer

[0041] Through the setting of the translation main body structure, it is convenient to utilize the multi-head attention mechanism of the Transformer model, improve the understanding of its own sentence meaning during translation, and improve the accuracy and efficiency of translation.

[0042] The sorting module sorts the interpretations of each input word according to the relevance of each interpretation of the input word output according to the relevance to aircraft maintenance work, and selects the interpretation with a high relevance to aircraft maintenance work as the interpretation preferentially queried by the query vector in the translation subject.

[0043] Through the setting of the sorting module, the order of vocabulary interpretation selection can be optimized, so that the translated interpretation approaches the common interpretation of aircraft maintenance work, and the inaccurate aircraft maintenance translation caused by mechanical translation of daily interpretations can be avoided.

[0044] A usage process of a customized aircraft maintenance intelligent translation system based on the Transformer model, characterized by including the following steps:

[0045] S1. Word vector and position information encoding: Convert the vocabulary input by the input module into word vectors, and the word vectors include word embeddings and position information encoding;

[0046] S2. Encoder calculation: Send the word vectors into the encoder in the translation subject. The first self-attention layer in the encoder receives the word vectors, and after residual connection and normalization operations, it is sent to the first feed-forward neural network, and then after residual connection and normalization operations, the output result is input into the decoder;

[0047] S3. Decoder calculation: The second self-attention layer receives the output result, and after residual connection and normalization operations, the output result of the first feed-forward network and the output result of the second self-attention layer are input into the encoder-decoder attention layer. During decoding, the relevance of the input word interpretation to aircraft maintenance work is combined by the sorting module to select the interpretation of the word, and linear transformation and softmax operations are performed on the output of the decoder to obtain the probability of each target word in the word table until the end symbol is predicted and the calculation stops;

[0048] S4. Output translation result: The output module outputs the translation result.

[0049] In summary, in this embodiment, the customized aircraft maintenance intelligent translation system based on the Transformer model according to this embodiment can, through the relevance formula of the input vocabulary and aircraft maintenance, combine the frequency of the meaning of the input vocabulary appearing in aircraft maintenance work materials and the ranking number of the frequency to obtain an exact actual value for the relevance of the vocabulary and aircraft maintenance work, which is convenient for providing a data basis for optimizing the selection of vocabulary interpretation relevance. At the same time, it combines the difference degree of each vocabulary interpretation by experts to make a positive correlation correction to the relevance, which is convenient for making a positive correlation correction to the relevance on the basis of the word meaning difference degree, that is, to distinguish the difference between the interpretation of aircraft maintenance relevance and the daily interpretation. The greater the difference, the higher the relevance, so that the interpretation with a larger difference degree is selected into the customized vocabulary library of aircraft maintenance work during the gradual training process. Through the setting of the frequency index, it is convenient to integrate the frequency number of the i-th interpretation of the input vocabulary appearing in aircraft maintenance materials and the number of times the interpretation with the most occurrences in the translated interpretation of the input vocabulary appears in aircraft maintenance materials into the internal frequency index, which is convenient for realizing the expression of two factors. Through the setting of the ranking index, the formula can be used to balance the order of magnitude of the frequency index and the ranking index, calculate the two indexes with the same order of magnitude, balance the influence degree of the two factors, and avoid the adverse expression of the influence on the index caused by the imbalance of the data order of magnitude. When it is the case, set the relevance of the input vocabulary and aircraft maintenance work to 0, which is convenient for excluding the vocabulary with a low frequency of occurrence in aircraft maintenance materials. Through the setting of the translation main structure, it is convenient to utilize the multi-head attention mechanism of the Transformer model to improve the understanding of its own sentence meaning during translation, and improve the accuracy and efficiency of translation. Through the setting of the sorting module, the order of vocabulary interpretation selection can be optimized, so that the translated interpretation approaches the common interpretation of aircraft maintenance work, and avoid the inaccuracy of aircraft maintenance translation caused by mechanical translation of daily interpretation.

[0050] The above is only a further embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the scope disclosed by the present invention, according to the technical solution and its concept of the present invention, makes equivalent replacement or change, all belong to the protection scope of the present invention.

Claims

1. A customized aircraft maintenance intelligent translation system based on the Transformer model, comprising an input module, a translation main body connected to the input module, and an output module connected to the translation main body, characterized in that, The translation main body is also connected to a customized maintenance translation vocabulary library. The customized maintenance translation vocabulary library includes a vocabulary input module. The vocabulary input module is connected to a maintenance relevance evaluation module. The maintenance relevance evaluation module is connected to a sorting module. The maintenance relevance evaluation module first determines whether the frequency ranking of the input vocabulary in maintenance materials is in the top three-quarters of the total ranking. If the frequency of the input vocabulary appears in the top three-quarters of the total ranking, the relevance between the input vocabulary and maintenance is positively correlated with the ratio of the frequency ranking of the vocabulary to the total ranking, positively correlated with the ratio of the number of times the most frequently appearing interpretation in the maintenance materials in the translation interpretation of the input vocabulary to the total number of times all interpretations appear in the maintenance materials, and also positively correlated with the difference degree of the interpretations of the input vocabulary. If the frequency ranking of the input vocabulary in the maintenance materials is not in the top three-quarters of the total ranking, the relevance between the input vocabulary and maintenance work is 0. The formula for the relevance between the input vocabulary and maintenance is: When The relevance formula between the input vocabulary and aircraft maintenance work is as follows: D R = γ * (α1 * I PD + α2 * I PM ); Where: N A Denoted as the total number of words in the maintenance data, N C Denoted as the number of times the word appears in the maintenance data, Denoted as for The ranking number after ranking, D R Denoted as the relevance of the input word to the maintenance work, I PD Denoted as the frequency index of the input word, α1 is denoted as the weight of the frequency index of the input word, and the value of α1 is set to 0.6, I PM Denoted as the ranking index of the input word, α2 is denoted as the weight of the ranking index of the input word, and the value of α2 is set to 0.

4. The expert scores the input word according to the difference degree of the input word, and the score of the semantic difference degree of the input word is denoted as γ, where the value range of γ is [0, 1]. When the difference degree of each interpretation of the input word is greater, the value of γ is greater; Among them, the formula for the frequency index is: Where: u is denoted as the set of the frequency numbers of each interpretation of the input vocabulary appearing in the maintenance materials, p i is denoted as the frequency number of the i-th interpretation of the input vocabulary appearing in the maintenance materials, is denoted as the number of times the interpretation with the largest total number of occurrences in the maintenance materials appears in the translated interpretations of the input vocabulary, and m is denoted as the total number of interpretations of the input vocabulary; The formula for the ranking index is; When The relevance formula between the input vocabulary and aircraft maintenance work is as follows: D R ＝0。 2. The customized aircraft maintenance intelligent translation system based on the Transformer model according to claim 1, wherein The translation main body includes an encoding component composed of multiple encoders connected to each other and a decoding component connected to the encoding component. The decoding component includes multiple decoders connected to each other. The encoder includes a first self-attention layer and a first feed-forward neural network connected to the first self-attention layer. The decoder includes a second self-attention layer, an encoder-decoder attention layer connected to the second self-attention layer, and a second feed-forward neural network connected to the encoder-decoder attention layer.

3. The customized aircraft maintenance intelligent translation system based on the Transformer model according to claim 1, characterized in that, The sorting module sorts the interpretations of the input vocabulary according to the relevance between each interpretation of the input vocabulary output by the maintenance relevance and maintenance work, and uses the interpretation with a high relevance to maintenance work as the interpretation that the query vector in the translation main body preferentially queries.

4. A method for using a customized aircraft maintenance intelligent translation system based on a Transformer model, characterized in that, Adopt a customized maintenance intelligent translation system based on the Transformer model according to any one of claims 1-3, including the following steps: S1. Word vector and position information encoding: Convert the vocabulary input by the input module into word vectors, and the word vectors include word embeddings and position information encoding; S2. Encoder calculation: Send the word vectors into the encoder in the translation main body. The first self-attention layer in the encoder receives the word vectors, passes through residual connection and normalization operations, and then sends them into the first feed-forward neural network. After passing through residual connection and normalization operations again, the output result is input into the decoder; S3. Decoder calculation: The second self-attention layer receives the output result. After performing residual connection and normalization operations, it inputs the output result of the first feed-forward network and the output result of the second self-attention layer into the encoder-decoder attention layer. During decoding, the sorting module is combined to select the interpretation of the vocabulary according to the relevance between the interpretation of the input vocabulary and maintenance work, and perform linear transformation and softmax operations on the output of the decoder to obtain the probability of each target word in the vocabulary table until the end symbol is predicted and the calculation stops; S4. Output the translation result: The output module outputs the translation result.

Images