Translation method and system based on knowledge base and improved Transform model

Abstract

The invention provides a translation method and system based on a knowledge base and an improved Transform model, an input module firstly obtains and cleans literature texts to ensure the quality and consistency of input data, then an analysis and judgment module carries out sentiment analysis and style judgment on the cleaned texts and outputs results in a vector form, and the analysis and judgment module carries out analysis and style judgment on the cleaned texts. And rich context information is provided for subsequent modules. The knowledge base module calculates the complexity of the text according to the analysis results, retrieves related knowledge information from a knowledge base and fuses the related knowledge information with the analysis results to generate an information enhanced text, and the translation module receives the information enhanced text, captures an inter-word relationship by using the self-adaptive differential linear attention module, generates a context sensing representation, and translates the context sensing representation into the text. And a translation result is generated through an encoder architecture, and finally, a feedback module dynamically adjusts parameters of each module according to the translation effect so as to continuously optimize the translation quality and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to a translation method and system based on a knowledge base and an improved Transformer model. Background Art

[0002] With the acceleration of globalization, the cross-border exchange of literary works has become increasingly frequent, promoting the interaction of different languages and cultures. However, traditional manual translation is inefficient and difficult to meet the large-scale translation needs, especially when dealing with the complex cultural connotations, emotional expressions, and rhetorical devices in literary works. To address this challenge, deep learning-based technologies have provided new approaches for literary translation.

[0003] Modern translation systems integrate technologies such as natural language processing, machine learning, and deep learning, which not only significantly improve translation efficiency but also can more accurately capture the semantics, emotions, and language styles of the original text. In the face of complex language phenomena such as metaphors and puns, deep learning technologies can effectively retain the unique charm of the original work and generate translations that fit the cultural background of the target language.

[0004] To further improve translation quality, the introduction of a knowledge base provides strong support for the translation system. By constructing a knowledge base covering rich cultural backgrounds, domain knowledge, and language rules, the translation system can more deeply understand the meaning of the source language and accurately convey it to the target language. When dealing with specific cultural phenomena, historical backgrounds, or professional terms, the system can use the knowledge base for accurate interpretation to avoid misunderstandings or translation deviations.

[0005] For example, in the patent document with the patent application number 202310825794.4 and the publication date of October 3, 2023, a personalized dialogue method and system combined with sentiment analysis are disclosed. The method includes: inputting a multimodal statement, processing the multimodal statement, and classifying the multimodal statement through a sentiment classification model to output a high-fineness sentiment classification result; modifying the high-fineness sentiment classification result through a Prompt template to guide a pre-trained large model to output a sentiment-rich statement; wherein, the classification head of the sentiment classification model is designed based on a secondary sentiment category system, and the sentiment classification model uses a Transformer structure as a feature extraction unit. The ability of the dialogue system to express personality can be enhanced by improving the fineness of sentence sentiment discrimination.

[0006] The above literature proposed a secondary emotion category system, which further subdivided emotion classification from the traditional positive, negative, and neutral. However, this subdivision may still not fully meet the complex and ever-changing human emotion expressions. Human emotions are extremely rich and delicate. Many times, one emotion may contain multiple mixed emotions, such as a complex mood that contains both joy and worry. The secondary emotion category system may be difficult to accurately classify and express it. And for the sake of accurate translation, if a more complex translation method is adopted for relatively simple texts, it will lead to a longer overall translation time. For complex texts, if a simple translation method is used, the accuracy of the translation cannot be ensured. Summary of the Invention

[0007] The present invention provides a translation method and system based on a knowledge base and an improved Transformer model, which enhances the recognition accuracy, avoids misunderstandings or translation deviations, and has a simple method.

[0008] To achieve the above object, one aspect of the technical solution provided by the present invention is: a translation method based on a knowledge base and an improved Transformer model, including: S1 Obtain the input literary text, clean the literary text to obtain a standardized text; S2 Perform text emotion analysis and text style judgment on the standardized text. The emotion analysis result and the text style analysis result are output in the form of vectors and form a formatted result; S3 Calculate the emotion and style complexity according to the output formatted result. Integrate the emotion complexity and the style complexity, weight the emotion complexity and the style complexity to determine the total complexity, search for relevant knowledge information from the knowledge base according to the total complexity, and fuse it with the structured result to generate an information-enhanced text; S4 Input the information-enhanced text, and determine the position encoding through the Transformer model. Then translate the encoded text through an adaptive differential linear attention module. The adaptive differential linear attention module includes a simple linear attention mechanism and a multi-head attention mechanism. The translation text obtained by the simple linear attention mechanism and the translation text obtained by the multi-head attention mechanism are weighted with the total complexity to obtain the translated text for output; S5 Adjust the parameters in steps S1-S3 according to the translation effect.

[0009] The above settings first perform standardized processing on the input text to ensure the reliability of the input text. Then, perform emotion analysis and writing style analysis on the standardized text, which can identify the emotional tendency contained in the text. The writing style judgment can determine the literary style genre to which the text belongs, which helps to better convey the emotion and style of the original text during the translation process and avoid translation distortion caused by misunderstandings; By means of the emotional complexity and the style complexity, and comprehensively obtaining the total complexity, an information-enhanced text is obtained. Relevant knowledge information is searched in the knowledge base according to the total complexity. For texts with higher complexity, relevant information in the knowledge base can be more deeply mined, and then after determining the position under the Transformer model, the multi-head attention mechanism is adopted for translation, which can translate more precisely; for simple texts, they can be simplified through a simple linear attention mechanism to improve the translation efficiency; during the translation process, the adaptive differential linear attention module can dynamically adjust the allocation of computing resources according to the complexity of the text, ensuring both the high-quality translation of complex texts and the translation speed of simple texts, achieving the dual optimization of translation efficiency and effect. At the same time, when fusing complex texts and simple texts, the total complexity is used to determine the weighting coefficient, so that the translation result can be closely related to the complexity of the text writing style and the emotional style, further ensuring that the translation effect is closely related to the text writing and emotion, ensuring the reliability of the translation. In addition, the parameters of the above steps can be adjusted according to the translation effect to further ensure the reliability of the translation.

[0010] Further, in step S1, the redundant spaces, irrelevant symbols, error information and possible special characters in the text are identified and deleted; then the punctuation marks are standardized, including unifying different forms of punctuation; the spelling mistakes and grammar anomalies in the text are corrected to obtain a standardized text.

[0011] The above settings are to eliminate the interference factors that may affect the translation quality, and ultimately can provide cleaner, more consistent and reliable input data for the subsequent translation process.

[0012] Further, step S2 specifically includes: S21 inputs the standardized text into GPT, and GPT analyzes the emotions and writing styles contained in the content of the standardized text; S22: The emotional analysis result returned by GPT needs to include the text object, the thing subject, the behavior of the thing subject, the emotion corresponding to the behavior and the confidence corresponding to the emotion; S23: The writing style analysis result returned by GPT needs to include the text object, the characteristics of the text object, the corresponding style type of the text and the confidence corresponding to the style type; Step S24 combines all the emotional analysis results and writing style analysis results in the form of vectors to form a formatted result.

[0013] With the above settings, the emotion classification includes: happy, sad, angry, fearful, calm, surprised, and undetermined, and the writing style classification includes: classicism, romanticism, modernism, realism, gothic, postmodernism, satire, poetry, drama, minimalism, casual style, and undetermined. By determining the corresponding emotion and the confidence level of the emotion in the emotion analysis result, and at the same time determining the corresponding style type and the confidence level of the style type in the writing style analysis result, the emotion analysis and writing style analysis become more reliable, and it is also convenient to determine the calculation method of complexity according to the emotion analysis result later.

[0014] Further, step S3 specifically includes step S31: dividing the emotion analysis result into complex emotion content and simple emotion content, and presetting the weight coefficients of complex emotion content and simple emotion content; S32: calculating the emotion complexity , where is the complexity of the th emotion analysis result. The complexity of each emotion analysis result is determined by the sum of the category it belongs to and its corresponding weight. S33: dividing the writing style into clear style content and complex style content, and presetting the weight coefficients of clear style content and complex style content; S34: calculating the style complexity , where is the complexity of the th writing style analysis result. The complexity of each emotion analysis result is determined by the sum of the category it belongs to and its corresponding weight. If a result contains multiple categories, its weight is the sum of the weights of each category is the complexity of the th writing style analysis result; the complexity of each writing style result is determined by the sum of the category it belongs to and its corresponding weight; S35: The calculation process of calculating the complexity C of the formatted result is as follows: , where is the normalized score of the total emotion complexity, is the normalized score of the total writing style complexity, is the complexity of the structured result, and are the weight coefficients for emotion analysis and writing style complexity, are the minimum and maximum values of emotion analysis complexity respectively, are the maximum and minimum values of writing style complexity respectively.

[0015] With the above settings, by calculating the emotional and style complexity, the system can better understand the emotional and style characteristics of the text, and thus more accurately convey these characteristics during the translation process. For example, for texts with complex emotions, the system can handle emotional expressions more meticulously to ensure that the translated text can convey the emotional color of the original text; for texts with complex styles, the system can handle the writing style more flexibly to ensure that the translated text can retain the style features of the original text.

[0016] Furthermore, S32 also includes: if a result contains multiple categories, its weight is the sum of the weights of each category. The calculation formula is as follows: Among them, is the weight of category in the th sentiment analysis result. If a certain category is not included in the sentiment analysis result, then

[0017] S33 also includes: if a result contains multiple categories, its weight is the sum of the weights of each category. The calculation formula is as follows: , among which, is the weight of category in the th writing style analysis result. If a certain category is not included in the writing analysis result, then .

[0018] With the above settings, during the sentiment analysis result and writing style analysis process, for the case where a result includes multiple categories, calculations are made to ensure that the situation where a result contains multiple categories can be analyzed and translated more reliably.

[0019] Furthermore, after step S35, when performing step S36, when the sentiment analysis confidence is greater than the preset confidence, the search information is associated with the sentiment analysis result. If the sentiment analysis result is marked as complex emotional content, knowledge information related to the sentiment analysis result will be further searched and these knowledge information will be associated with the sentiment analysis result; In S37, when the text style analysis confidence is greater than the preset style analysis confidence value, all translation strategies and translation precautions corresponding to the Transformer model are determined, and the searched information is associated with the style type analysis result. If the style type analysis result is marked as complex style content, the system will further search for knowledge information related to the result and associate the result-related knowledge information with the style type analysis result; S38 associates the fused knowledge information with the results of sentiment analysis and style type analysis, and outputs the fused information-enhanced text.

[0020] With the above settings, for the sentiment analysis results with a relatively high confidence level, they are marked as complex sentiment content. Information will be retrieved from the knowledge base to associate with the translated text. Then, for the writing style text with a relatively high confidence level, translation metrics and translation precautions will be determined and then associated with relevant knowledge information. Finally, a fused information-enhanced text is formed to ensure the accuracy and reliability of the translation.

[0021] Further, step S4 includes: After receiving the information-enhanced text and the complexity of the structured result, the information-enhanced text is first converted into word embeddings. During the process of converting the input text into word embeddings, positional encoding is applied to the word embeddings. The input after positional encoding is fed into the encoder, and the encoder captures the relationships between words through an adaptive differential linear attention module to generate context-aware representations. After the feature vectors are input into the adaptive differential linear attention module, the input feature vectors are processed through two parallel branches. The first branch adopts the multi-head differential attention mechanism, and the second branch adopts the self-attention mechanism. After the feature vectors pass through these two branches respectively, the outputs of the two branches are combined, and a weighted sum of the outputs of the two branches and the total complexity is calculated to generate an output feature representation.

[0022] With the above settings, by combining the multi-head differential attention mechanism and the simple linear attention mechanism, the system can handle different types of texts more flexibly. The multi-head differential attention mechanism can capture more subtle relationships between words and is suitable for processing complex literary texts; the simple linear attention mechanism can improve the efficiency of processing simple texts and reduce the waste of computing resources.

[0023] Further, the calculation process of the first branch in step S42 is as follows: S421 For the input feature vector , there is , where is the number of texts, is the dimension of the input feature; project the input onto the and matrices, that is: ; Among them, is the projection matrix used to map the input features to different spaces, are respectively vectors, V is the value matrix, S422 Calculate two softmax attention maps. For each pair Matrix sum Calculate attention scores: ; where are two softmax attention maps, representing the attention distributions based on two sets of query matrices and key matrices respectively, and the softmax function is a normalization function.

[0024] S423 calculates the differential attention scores, weights the second softmax attention map using a learnable scalar λ, and subtracts the weighted second map from the first map: ; where is a learnable scalar, and the learnable scalar is initialized by reparameterizing λ as: ; where is the learnable vector used for reparameterizing to increase the flexibility and expressive power of , is 's initialization constant; S424 uses the multi - head mechanism to enhance the model's representation ability; let be the number of attention heads, and different projection matrices are used for each head. The output of each head is: ; where is the output of the -th attention head, representing the calculation result of the -th attention head, is the differential attention mechanism.

[0025] S425 applies layer normalization processing and scaling processing ; where the scaling factor is ; S426 concatenates the outputs of all heads and projects them through a linear layer to the final output dimension to obtain the final output d of the first branch: ; where is the final output of the first branch, is the projection matrix of the final output.

[0026] With the above settings, the first analysis uses the multi-head self-attention mechanism for translation. First, the text is projected, then the attention map and differential attention scores are calculated. Then, based on the differential attention scores, multiple attention heads are obtained, and the multiple attention heads are concatenated through the concatenation method to obtain the final output matrix.

[0027] Further, the calculation process for the second branch in step S42 is as follows: S4211 For the input feature vector , there is where is the number of input texts, is the dimension of the input features; project the input onto the , and matrices, that is: ; where, is the projection matrix; S4212 Use the ReLU function as the kernel function to calculate the similarity between the query matrix and the key matrix: ; where, are the i-th row and the j-th row of the query matrix and the key matrix respectively; S4213 Perform weighted summation according to the similarity to obtain the preliminary output: ; where, is the j-th column of the value matrix , S4214 Add the preliminary output result to the result enhanced by depth convolution to obtain the final output of the second branch: where, is the preliminary output matrix, is the result after applying depth convolution to the value matrix , is the final output of the second branch; S4215 The outputs of the two branches are fused through a gating mechanism, and the fusion process is expressed as: where, represent the outputs of the first branch and the second branch respectively, is the fusion weight output by the gating mechanism, is the value of the structural result complexity calculated in step S31.

[0028] With the above settings, by calculating the similarity of the input feature vectors and weighting them, and performing depth convolution independently on each input channel without mixing channel information to enhance local features, and then performing a weighted sum of the depth convolution values and the preliminary output results, enhanced local features can be obtained to ensure the accuracy of translation.

[0029] On the other hand, the present invention also provides a translation system based on a knowledge base and an improved Transformer model, including an input module, an analysis and judgment module, a knowledge base module, a translation module, and a feedback module. The input module is used to obtain the input literary text, clean the literary text to obtain a standardized text. The analysis and judgment module is used to perform text sentiment analysis and text style judgment on the standardized text. The sentiment analysis result and the text style analysis result are output in the form of vectors and form a formatted result. The knowledge base module is used to calculate the sentiment complexity and style complexity respectively according to the output formatted result, determine the total complexity by weighting the sentiment complexity and style complexity, search for relevant knowledge information from the knowledge base according to the total complexity, and fuse it with the formatted result to generate an information-enhanced text. The translation module is used to input the information-enhanced text and determine the positional encoding through the Transformer model, and then translate the encoded text through an adaptive differential linear attention module. The adaptive differential linear attention module includes a self-attention mechanism and a multi-head attention mechanism. The translation text obtained by the self-attention mechanism and the translation text obtained by the multi-head attention mechanism are weighted with the total complexity to obtain the translated text for output. The feedback module is used to adjust the parameters of the input module, the analysis and judgment module, the knowledge base module, and the translation module according to the translation effect.

[0030] In the above system, first, the input text is standardized to ensure the reliability of the input text, and then sentiment analysis and writing style analysis are performed on the standardized text, which can identify the emotional tendency contained in the text. The writing style judgment can determine the literary style genre to which the text belongs, which helps to better convey the emotion and style of the original text during the translation process and avoid translation distortion caused by misunderstanding. By means of the emotional complexity and the style complexity, and comprehensively obtaining the total complexity, an information-enhanced text is obtained. Relevant knowledge information is searched in the knowledge base according to the total complexity. For texts with higher complexity, relevant information in the knowledge base can be more deeply mined. Then, after determining the position under the Transformer model, the multi-head attention mechanism is adopted for translation, which can translate more precisely; for simple texts, the simple linear attention mechanism can be used for simplification to improve the translation efficiency; during the translation process, the adaptive differential linear attention module can dynamically adjust the allocation of computing resources according to the complexity of the text, which not only ensures the high-quality translation of complex texts but also improves the translation speed of simple texts, realizing the dual optimization of translation efficiency and effect. At the same time, when fusing complex texts and simple texts, the total complexity is used to determine the weighting coefficient, so that the translation result can be closely related to the complexity of the text writing style and the emotional style, further ensuring that the translation effect is closely related to the text writing and emotion, ensuring the reliability of the translation. In addition, the parameters of the above steps can be adjusted according to the translation effect to further ensure the reliability of the translation. Description of the Drawings

[0031] Figure 1 It is a block diagram of the translation system in the present invention.

[0032] Figure 2 It is a network structure diagram of the improved Transformer model in the present invention. Detailed Embodiments

[0033] Embodiment 1

[0034] As Figure 1-2 shown, a translation method based on a knowledge base and an improved Transformer model is implemented through a translation system. The translation system includes an input module, an analysis and judgment module, a knowledge base module, a translation module, and a feedback module. The specific steps include: S1 The input module obtains the input literary text and cleans the literary text to obtain a standardized text; S2 The standardized text is input into the analysis and judgment module for text emotional analysis and text style judgment. The emotional analysis result and the text style analysis result are output in the form of vectors and form a formatted result; S3 The formatted result output from the analysis and judgment module is input into the knowledge base module. The emotional and style complexities are calculated respectively, the emotional complexity and the style complexity are comprehensively calculated, the total complexity is calculated, relevant knowledge information is searched from the knowledge base according to the calculated total complexity, and is fused with the structured result to generate an information-enhanced text; S4 inputs the text with enhanced information into the translation module for text translation, determines the positional encoding through the Transformer model, and then translates the encoded text through the adaptive differential linear attention module. The adaptive differential linear attention module includes simple linear attention and multi-head attention mechanism. The translation text obtained by the simple linear attention and the translation text obtained by the multi-head attention mechanism are weighted with the total complexity to obtain the translated text for output; S5 According to the translation effect, the parameters in the input module, analysis and judgment module, knowledge base module, and translation module are adjusted through the feedback module to optimize the translation quality and effect. In this embodiment, the preset confidence in the confidence of the translation module and the learnable vector, etc. can be adjusted according to the translation effect. This kind of adjustment can be manual adjustment. For example, when a sad emotion appears in the translation result and the translation result is not enough to express the emotion, that is, the vocabulary used is not enough to express sadness, the preset confidence corresponding to the sad emotion can be lowered a little. When it is higher than this preset confidence, relevant vocabulary will be searched in the knowledge base to expand the vocabulary selection of sad emotions.

[0035] In step S1, the redundant spaces, irrelevant symbols, error information, and possible special characters in the text are identified and deleted; then the punctuation marks are standardized, including unifying different forms of punctuation marks; for example, the same punctuation marks in different modes can be unified. Then, the spelling mistakes and grammar anomalies in the text are corrected to eliminate the interference factors that may affect the translation quality. Finally, the standardized text obtained through these steps can provide cleaner, more consistent, and reliable input data for the subsequent translation process.

[0036] Step S2 specifically includes: S21 inputs the standardized text into the analysis and judgment module such as GPT, and GPT analyzes the emotions and writing styles contained in the content of the standardized text; S22: The emotional analysis result returned by GPT needs to include the text object, the subject of the thing, the behavior of the subject of the thing, the emotion corresponding to the behavior, and the confidence corresponding to the emotion; S23: The writing style analysis result returned by GPT needs to include the text object, the characteristics of the text object, the corresponding style type of the text, and the confidence corresponding to the style type; S24 combines all the emotional analysis results and writing style analysis results in the form of vectors to form a formatted result. The emotion classification includes: happy, sad, angry, fearful, calm, surprised, and undetermined. The writing style classification includes: classicism, romanticism, modernism, realism, gothic, postmodernism, satire, poetry, drama, minimalism, casual style, and undetermined; One of the reference formats for the returned sentiment analysis results is: <(text object), (subject of the thing), (action), (sentiment), (confidence)>. A more specific example is: <(Li Fang had tears in her eyes and softly told the sorrow in her heart.), (Li Fang), (telling sorrow), (sadness), (0.75)>.

[0037] The writing style analysis results returned by GPT need to include the text object, the characteristics of the text object, the corresponding style type of the text, and the confidence corresponding to this style type. One of the reference formats for the returned writing style analysis results is: <(She walked on the desolate street, the lights were dim, and her shadow was stretched, as if forgotten by time.), (symbolic, sense of loneliness), (modernism), (0.9)>.

[0038] In another embodiment, for the same piece of text, there can be more than one sentiment analysis result and one writing style analysis result. Combining all the sentiment analysis results and writing style analysis results in the form of a vector can form a list or array containing multiple vectors. This list or array is the so-called "formatted result". One of the formats that can refer to the "formatted result" is [(sentiment analysis result 1, sentiment analysis result 2, sentiment analysis result 3..., sentiment analysis result n), (writing style analysis result 1, writing style analysis result 2, writing style analysis result 3..., writing style analysis result n)]. The specific formula is as follows: where represents the formatted result, represents all the sentiment analysis results, represents all the writing style results, , where, represents the th sentiment analysis result, represents the th text object, represents the th subject of the thing, represents the th action of the subject of the thing represents the th sentiment corresponding to the action of the subject of the thing represents the th confidence corresponding to the sentiment, where, represents the th writing style analysis result, represents the th text object, represents the th characteristic of the text object, represents the The style type corresponding to the text indicating the confidence level corresponding to the

[0039] More specifically, the formatted output is: [<(Li Fang's eyes are filled with tears as she softly tells of the sorrow in her heart.), (Li Fang), (telling of sorrow), (sadness), (0.75)>; <(She walks on the desolate street with dim yellow lights and stretched shadows, as if forgotten by time.), (symbolism, sense of loneliness), (modernism), (0.9)>].

[0040] Step S3 specifically includes: S31 classifies the sentiment analysis results into complex sentiment content and simple sentiment content. The determination of the sentiment analysis results can be made through the confidence level value. For example, when the confidence level value of the sentiment analysis result is higher than the preset sentiment analysis result confidence level value, it is determined as simple sentiment content. For example, for the preset confidence level of happiness in the sentiment analysis result being 70, when it is higher than the preset confidence level value, it can be determined as a clear happy style; preset the weight coefficients for complex sentiment content and simple sentiment content; among them, the weight coefficient for complex sentiment content is 3, and the weight coefficient for simple sentiment content is 1; S32 calculates the sentiment complexity wherein, is the complexity of the th sentiment analysis result. The complexity of each sentiment analysis result is determined by the sum of the categories it belongs to and their corresponding weights. If a result contains multiple categories, its weight is the sum of the weights of each category, and its calculation formula is as follows: is the weight of category in the th sentiment analysis result. If the sentiment analysis result does not have a certain category , then ; S33 classifies the writing style into clear style content and complex style content. The determination of the writing style can be made through the confidence level value. For example, when the writing style confidence level is higher than the preset writing style confidence level value, it is determined as clear style content. For example, for the preset confidence level of romanticism in the writing style being 70, when it is higher than the preset confidence level value, it can be determined as a clear romanticism style; preset the weight coefficients for clear style content and complex style content; among them, the weight coefficient for clear style content is 2, and the weight coefficient for complex style content is 4; S34 calculates the style complexity , wherein, is the complexity of the th sentiment analysis result. The complexity of each sentiment analysis result is determined by the sum of the categories it belongs to and their corresponding weights. If a result contains multiple categories, its weight is the sum of the weights of each category Among them is the complexity of the th writing style analysis result; the complexity of each writing style result is determined by the sum of the category it belongs to and its corresponding weight; if a result contains multiple categories, its weight is the sum of the weights of each category, and the calculation formula is as follows: Among them, is the th weight of category in the th writing style analysis result; if a certain category is not included in the writing analysis result, then

[0041] The calculation process of calculating the complexity C of the formatted result in S35 is as follows: , among which, is the score after standardizing the total sentiment complexity, is the score after standardizing the total writing style complexity, are respectively the minimum and maximum values of the sentiment analysis complexity, the maximum and minimum values of the writing style complexity. is the weight coefficient for sentiment analysis and writing style complexity. Assuming their importance is equal, then ranges from 0 to 1. The closer the value is to 1, the higher the complexity of the text to be translated; the closer it is to 0, the simpler the text to be translated. By calculating the sentiment and style complexity, the system can better understand the sentiment and style characteristics of the text, and thus more accurately convey these characteristics during the translation process. For example, for a text with complex sentiment, the system can handle the sentiment expression more meticulously to ensure that the translated text can convey the sentiment color of the original text; for a text with complex style, the system can handle the writing style more flexibly to ensure that the translated text can retain the style features of the original text.

[0042] After step S35, it also includes: step S36 when the sentiment analysis confidence is high, such as when the confidence of the happy sentiment is 0.95, corresponding all English expressions such as ecstasy, overjoyed, and relate these expressions to the sentiment analysis result. If the sentiment analysis result is marked as "complex sentiment content", the system will further search for knowledge information related to this result and establish an association between this knowledge information and the sentiment analysis result; When the confidence level of text style analysis is relatively high, such as when the confidence level of text style analysis is higher than the preset confidence value, for example, when the confidence level of the realistic style is 0.7, all corresponding translation strategies, translation precautions, etc. are retrieved, and the information retrieved is associated with the result of style type analysis. If the result of style type analysis is marked as "complex style content", the system will further search for knowledge information related to this result and establish an association between this knowledge information and the result of style type analysis; S38 Integrate the association between knowledge information, sentiment analysis results, and style type analysis results. For example, translate the English literary text to be translated "The mayor’s speech dripped with honeyed words about ‘equality,’ while his polished boots crushed the petitions of the hungry. Ah, the glorious symphony of democracy!" into Chinese. If the literal translation of "dripped with" in the text to be translated is "filled with" or "overflowing with", these Chinese translations usually do not have a restricted context of use. However, the sentiment analysis results show that this text mainly contains two emotions: anger with a confidence level of 0.85 and surprise (irony) with a confidence level of 0.7.

[0043] Obviously, directly translating 'dripped with' as 'filled with' or 'overflowing with' is not the best choice for the context of this text.

[0044] After the knowledge base retrieves according to the results of sentiment analysis, it is considered that in the context of anger and irony, it may be more appropriate to translate "dripped with" as "talking nonsense" or "being glib-tongued", etc. Therefore, establish a connection: in the current context of anger and irony, give priority to using words such as "talking nonsense" or "being glib-tongued" to translate "dripped with". The result of style type analysis shows that the text to be translated belongs to satirical literature, and its confidence level is 0.75.

[0045] Confidence levels will also be assigned to this part of the knowledge, associating the knowledge related to emotions with confidence levels. The range of confidence levels is any value between 0 and 1. The greater the confidence level, the stronger the emotion; conversely, the smaller the confidence level, the milder the emotion. Specifically, in the database, each different phrase or word expressing a certain emotion will have one or more confidence levels. For example, for the emotion of "happiness", in English, it can be expressed as "pleasure", "happiness", "joy", "delight", and "ecstasy", etc. However, the degrees of happiness expressed by these words are different. That is, "pleasure" is more suitable for describing mild happiness, and "ecstasy" is more suitable for expressing extreme happiness. Therefore, in the database, the confidence level of the word "pleasure" corresponding to "happiness" is 0.6, and the confidence level of the word "ecstasy" corresponding to "happiness" is 0.95. It should be noted that some expressions about emotions can contain multiple different emotions. For example, "mixed feelings of grief and joy" describes an emotional state, where "grief" represents sadness and "joy" represents happiness. These two emotions coexist and are intertwined in the word, forming a complex psychological state. Therefore, "mixed feelings of grief and joy" contains both "happiness" and "sadness" and their respective confidence levels corresponding to "happiness" and "sadness".

[0046] First, in the high confidence interval (0.9 - 1), the stylistic features of the text are highly prominent and can be clearly attributed. The core strategy at this time is style-enhanced translation. At the operational level, at the lexical level, it is required to compulsorily use the signature term bank of this style (for example, for the realistic style, "vulgar slang + list of concrete description words" should be selected). At the syntactic level, the typical structural features should be strictly replicated, such as maintaining dense narration in long paragraphs and avoiding lyrical rewriting.

[0047] Second, in the medium confidence interval (0.6 - 0.89), although the main style of the text has been clarified, there may be mixed features. At this time, first judge whether there is only one style type. If there is indeed only one style, adopt the same strategy as in the corresponding high confidence interval. If there are multiple style types, the high-frequency words should be preferentially matched with the main style dictionary, and the rhetorical devices can allow appropriate penetration of the secondary style. At the same time, for the parts with ambiguous styles, academic annotations can be made by adding translator's notes.

[0048] Then, in the low confidence interval (0.3 - 0.59), the stylistic features are relatively mixed or not obvious. The core strategy at this time is conservative and neutral translation.

[0049] Finally, when the confidence level is lower than 0.3, the system will automatically trigger a request for manual review. In terms of operation, the principle of minimal style intervention should be followed, retaining the structural features of the original text and using a cross-style general vocabulary bank.

[0050] The setting and division of confidence levels are also accomplished by referring to expert opinions and consulting professional dictionaries to determine the confidence levels, ensuring the accuracy and professionalism of the data.

[0051] Step S4 also includes: After the translation module receives the information-enhanced text and the complexity of the structured result, the information-enhanced text is first converted into word embeddings. During the process of converting the input text into word embeddings, in order to retain the position information of words in the sentence, positional encoding is applied to the word embeddings. The input after positional encoding is fed into the encoder, and the encoder is the positional encoder in the improved Transformer model. The encoder captures the relationships between words through the adaptive differential linear attention module and generates context-aware representations. After the feature vectors are input into the adaptive differential linear attention module, the input feature vectors are processed through two parallel branches. The first branch adopts the multi-head differential attention mechanism, and the second branch adopts the simple linear attention mechanism. After the feature vectors pass through these two branches respectively, the outputs of the two branches are fused through a gating mechanism to generate a comprehensive feature representation.

[0052] After the feature vectors enter the first branch, the following process will occur: First, the input feature vectors will be projected into three different spaces: the query matrix the key matrix and the value matrix . This is accomplished through three independent linear transformations (i.e., matrix multiplications), and each transformation uses a learnable projection matrix. This process maps the original features into a new space that is more suitable for attention mechanism calculations.

[0053] Next, the mechanism will calculate two softmax attention maps. This is obtained by respectively taking the dot product of two groups of query matrices and key matrices and normalizing them through the softmax function. These two attention maps reflect different correlations or importances between the feature vectors.

[0054] Then, the core operation of the MHD (multi-head Attention) mechanism is to calculate the differential attention scores. It weights the second attention map using a learnable scalar and subtracts this weighted second attention map from the first attention map. This operation aims to highlight the differences between the two attention maps, thereby capturing more subtle relationships between the feature vectors.

[0055] To enhance the representation ability and flexibility of the model, this learnable scalar is usually obtained through reparameterization, so that its value can be dynamically adjusted during the training process.

[0056] Finally, the result obtained through differential attention calculation will be weighted and summed with the value vector to obtain the output of each attention head. In the multi-head mechanism, multiple such attention heads will be calculated in parallel, and their outputs will be concatenated together. Finally, the concatenated result will be projected to the final output dimension through a linear layer to obtain the output of the MHD (multi-head Attention) mechanism. This output is the enhanced representation of the original feature vector after being processed by the differential attention mechanism, and it contains richer and more detailed relationship information between the feature vectors.

[0057] The complete calculation process after the first branch of the feature vector is as follows: S421 For the input feature vector where is the number of tokens (i.e., the number of texts), is the dimension of the input feature.

[0058] Input matrix, that is: ; where, is the projection matrix used to map the input features to different spaces, are respectively vectors, is the value matrix. The projection of the input X to the Q matrix, K matrix, and V matrix is implemented through existing projection methods, which are respectively used to calculate the attention scores of two different attention maps.

[0059] S422 Calculate two softmax attention maps. For each pair of calculate the attention score: ; where, are two softmax attention maps, respectively representing the attention distributions based on two sets of query matrices and key matrices. The softmax function is a normalization function.

[0060] S423 Calculate the differential attention score. Use a learnable scalar λ to weight the second softmax attention map and subtract the weighted second map from the first map: ; where, is a learnable scalar used to balance the contributions of the two attention maps and implement the differential attention mechanism.

[0061] Learnable scalar Initialization. To synchronize the learning dynamics, λ is reparameterized as: ; Among them, is a learnable vector for reparameterization The learnable vector can be preset or adjusted by input, and is used to increase flexibility and expressive power is the initialization constant for initializing the value to ensure the stability of the training process.

[0062] S424 uses a multi-head mechanism to enhance the model's representation ability. Let be the number of attention heads, and different projection matrices are used for each head. The output of each head is: ; Among them, is the output of attention heads, representing the calculation result of the th attention head,

[0063] S425 applies layer normalization and a scaling factor to the output of each head: Concatenate the outputs of all heads and project them through a linear layer to the final output dimension to obtain the final output d of the first branch: ; Among them, is the final output of the first branch, is the projection matrix of the final output.

[0064] After the feature vector enters the second branch, the following process will occur: First, the input feature vector will be projected into three different spaces: the query matrix , the key matrix and the value matrix . This is done through three independent linear transformations (i.e., matrix multiplications), and each transformation uses a learnable projection matrix. This process maps the original features into a new space that is more suitable for attention mechanism calculations.

[0065] Next, the SLAttention (Simple Linear Attention) mechanism uses the ReLU function as the kernel function to calculate the similarity between the query matrix and the key matrix. The element-wise application of the ReLU function ensures that the similarity values are non-negative, which helps with subsequent calculation stability. By calculating the dot product of the query matrix and the key matrix (after ReLU activation), a similarity matrix is obtained, which reflects the degree of correlation between each query matrix and all key matrices.

[0066] After obtaining the similarity matrix, the SLAttention mechanism normalizes it and performs a weighted sum on the value matrix according to the normalized similarity. This process is similar to the traditional attention mechanism, but the SLAttention mechanism reduces the computational complexity in advance by calculating the product of the transpose of the key matrix and the value matrix, achieving linear complexity. The result of the weighted sum is a re-weighted representation of the input features, emphasizing the features most relevant to the current query.

[0067] To further enhance this re-weighted feature representation, the SLAttention mechanism also applies a depth convolution operation to the original value matrix. Depth convolution is a lightweight convolution method that independently convolves each channel without mixing channel information. This operation helps capture local features and provides additional context information, thereby further enhancing the feature representation.

[0068] Finally, the SLAttention mechanism adds the result of the weighted sum to the result enhanced by depth convolution to obtain the final output. This output not only contains the re-weighted feature information but also incorporates the enhanced information of local features, enabling subsequent processes to handle richer and more accurate feature representations. The entire process maintains computational efficiency while ensuring that the model can capture important information in the input data.

[0069] The complete calculation process of the feature vector after passing through the second branch is as follows: S4211 For the input feature vector where is the number of tokens (input text), is the dimension of the input features.

[0070] Take the input matrix, i.e.: ; where, is the projection matrix used to map the input features to a different space.

[0071] S4212 Use the ReLU function as the kernel function to calculate the similarity between the query matrix and the key matrix: ; where, are the i-th and j-th rows of the query matrix and the key matrix respectively, and the ReLU function is applied element-wise to ensure non-negativity.

[0072] S4213 Perform a weighted sum according to the similarity to obtain the preliminary output: ; Among them, for the column, the denominator is used to normalize the similarity weight for the value matrix Apply depth convolution. Depth convolution is a special convolution operation where each input channel is convolved independently without mixing channel information to enhance local features.

[0073] S4214 adds the result of weighted summation to the result enhanced by depth convolution to obtain the final output of the second branch: Among them, is the preliminary output matrix obtained after weighted summation, is the result after applying depth convolution to the value matrix , is the final output of the second branch. The depth convolution model is an existing depth convolution model and will not be elaborated here.

[0074] S4215 The outputs of the two branches are fused through a gating mechanism. The specific fusion process can be expressed as: respectively represent the outputs of the multi-head differential attention branch and the simple linear attention branch, is the fusion weight output by the gating mechanism, The value is the complexity of the structured result calculated in step S31.

[0075] The working principle of the present invention: First, the input text is standardized to ensure the reliability of the input text. Then, sentiment analysis and writing style analysis are performed on the standardized text, which can identify the sentiment tendency contained in the text. The writing style judgment can determine the literary style genre to which the text belongs, which helps to better convey the emotion and style of the original text during the translation process and avoid translation distortion caused by misunderstanding; By combining emotional complexity and stylistic complexity to obtain the overall complexity, an information-enhanced text is generated. Relevant knowledge information is searched in the knowledge base based on the overall complexity. For texts with higher complexity, more in-depth relevant information in the knowledge base can be mined. Then, after determining the position under the Transformer model, the multi-head attention mechanism is used for translation, enabling more refined translation. For simple texts, a simple linear attention mechanism can be used for simplification to improve translation efficiency. During the translation process, the adaptive differential linear attention module can dynamically adjust the allocation of computing resources according to the complexity of the text, ensuring both high-quality translation of complex texts and increased translation speed of simple texts, achieving a dual optimization of translation efficiency and effect. At the same time, when fusing complex and simple texts, the overall complexity is used to determine the weighting coefficient, making the translation result closely related to the complexity of the text writing style and emotional style, further ensuring that the translation effect is closely related to the text writing and emotion, and ensuring the reliability of the translation. Additionally, the parameters of the above steps can be adjusted according to the translation effect to further ensure the reliability of the translation.

[0076] Embodiment 2.

[0077] As Figure 1 shown, a translation system for a knowledge base and an improved Transformer model, the translation system includes an input module, an analysis and judgment module, a knowledge base module, a translation module, and a feedback module.

[0078] The input module is used to obtain the input literary text, clean the literary text to obtain a standardized text; The analysis and judgment module is used to perform text sentiment analysis and text style judgment on the standardized text. The sentiment analysis result and the text style analysis result are output in the form of vectors and form a formatted result; The knowledge base module is used to calculate the emotional complexity and stylistic complexity respectively according to the output formatted result, weight the emotional complexity and stylistic complexity to determine the overall complexity, search relevant knowledge information from the knowledge base according to the overall complexity, and fuse it with the formatted result to generate an information-enhanced text; The translation module is used to input the information-enhanced text and determine the position encoding through the Transformer model, and then translate the encoded text through the adaptive differential linear attention module. The adaptive differential linear attention module includes a self-attention mechanism and a multi-head attention mechanism. The translation text obtained by the self-attention mechanism and the translation text obtained by the multi-head attention mechanism are weighted with the overall complexity to obtain the translated text for output; The feedback module is used to adjust the parameters of the input module, the analysis and judgment module, the knowledge base module, and the translation module according to the translation effect.

Claims

1. A translation method based on a knowledge base and an improved Transformer model, characterized in that: include: S1 obtains the input literary text, cleans the literary text, and obtains a standardized text; S2 performs text sentiment analysis and text style judgment on the standardized text. The sentiment analysis results and text style analysis results are output in the form of vectors and formatted. S3 calculates the emotional complexity and style complexity according to the output formatting results, weights the emotional complexity and style complexity to determine the total complexity, searches for relevant knowledge information from the knowledge base according to the total complexity, and fuses it with the formatting results to generate information-enhanced text; S4 inputs the information-enhanced text and determines the position encoding through the Transformer model. Then, the encoded text is translated through the adaptive differential linear attention module. The adaptive differential linear attention module includes a simple linear attention and a multi-head attention mechanism. The translated text obtained by the simple linear attention and the translated text obtained by the multi-head attention mechanism are weighted with the total complexity to obtain the translated text for output. S5 adjusts the parameters in steps S1-S3 according to the translation effect.

2. The translation method based on the knowledge base and the improved Transformer model according to claim 1, characterized in that: In step S1, redundant spaces, irrelevant symbols, erroneous information, and possible special characters in the text are identified and deleted; punctuation marks are then standardized, including unifying different forms of punctuation marks; spelling errors and grammatical anomalies in the text are corrected to obtain standardized text.

3. The translation method based on the knowledge base and the improved Transformer model according to claim 1, characterized in that: Step S2 specifically includes: S21 inputting the standardized text into GPT, and GPT analyzing the emotions and writing style contained in the standardized text; S22: The sentiment analysis results returned by GPT need to include the text object, the subject of the object, the behavior of the subject of the object, the sentiment corresponding to the behavior, and the confidence level corresponding to the sentiment; S23: The writing style analysis results returned by GPT need to include the text object, the features of the text object, the style type corresponding to the text, and the confidence level corresponding to the style type; Step S24 combines all sentiment analysis results and writing style analysis results in the form of vectors to form a formatted result.

4. The translation method based on the knowledge base and the improved Transformer model according to claim 1, characterized in that: Step S3 specifically includes: S31 divides the sentiment analysis result into complex sentiment content and simple sentiment content, and presets a weight coefficient of the complex sentiment content and a weight coefficient of the simple sentiment content; S32 Calculate emotional complexity ,in, It is The complexity of each sentiment analysis result The complexity of is determined by the sum of the categories it belongs to and their corresponding weights. S33 divides the writing style analysis results into clear style content and complex style content, and presets the weight coefficient of clear style content and the weight coefficient of complex style content; S34 Computational style complexity ,in, It is The complexity of each writing style analysis result; The complexity of is determined by the sum of the categories it belongs to and their corresponding weights; S35 calculates the formatting result complexity C: is the standardized score of total emotional complexity, is the standardized score of the total writing style complexity, is the structured result complexity, is the weight coefficient for sentiment analysis and writing style complexity, They are the minimum and maximum complexity of sentiment analysis, are the maximum and minimum values ​​of writing style complexity respectively.

5. The translation method based on the knowledge base and the improved Transformer model according to claim 4, characterized in that: S32 also includes: if a result contains multiple categories, its weight is the sum of the weights of each category, The calculation formula is as follows: Categories in sentiment analysis results If the sentiment analysis result does not have a certain category ; S34 also includes: if a result contains multiple categories, its weight is the sum of the weights of each category, It is Writing style analysis results If the writing analysis result does not have a certain category ,but .

6. The translation method based on the knowledge base and the improved Transformer model according to claim 4, characterized in that: After step S35, step S36 is performed. When the sentiment analysis confidence is greater than the preset confidence, the search information is associated with the sentiment analysis result. If the sentiment analysis result is marked as complex sentiment content, knowledge information related to the sentiment analysis result is further searched, and the knowledge information is associated with the sentiment analysis result. S37 When the text style analysis confidence is greater than the preset style analysis confidence value, all translation strategies and translation considerations corresponding to the Transformer model are determined, and the searched information is associated with the style type analysis result. If the style type analysis result is marked as complex style content, the system will further search for knowledge information related to the result, and associate the knowledge information related to the result with the style type analysis result; S38 fuses the knowledge information with the association between the sentiment analysis results and the style type analysis results, and outputs the fused information enhanced text.

7. The translation method based on the knowledge base and the improved Transformer model according to claim 1, characterized in that: Step S4 includes: Step S41: after receiving the information-enhanced text and the structured result complexity, the information-enhanced text is first converted into a word embedding. In the process of converting the input text into the word embedding, the position encoding is applied to the word embedding. The position-encoded input is passed to the encoder. The encoder captures the relationship between words through an adaptive differential linear attention module to generate a context-aware representation. After the S42 feature vector is input into the adaptive differential linear attention module, the input feature vector is processed by two parallel branches. The first branch adopts the multi-head differential attention mechanism, and the second branch adopts the self-attention mechanism. After the feature vector passes through these two branches respectively, the outputs of the two branches are weighted and summed with the total complexity to generate an output feature representation.

8. The translation method based on the knowledge base and the improved Transformer model according to claim 1, characterized in that: The calculation process of the first branch in step S42 is as follows: S421 For the input feature vector ,have ,in is the number of texts, is the dimension of the input feature; Project to and The matrix is: ； in, is the projection matrix, which is used to map the input features into a different space. They are vector, V is the value matrix, S422 calculates two softmax attention maps, for each pair Matrix and Calculate the attention score: ； in, There are two softmax attention graphs, which represent the attention distribution based on two sets of query matrices and key matrices respectively. The softmax function is a normalization function. S423 computes the differential attention score by weighting the second softmax attention map using a learnable scalar λ and subtracting the weighted second map from the first map: ； in, is a learnable scalar, a learnable scalar Initialization, reparameterize λ as: ； in, is used to reparameterize The learnable vectors are used to increase flexibility and expressiveness, yes Initialization constants; S424 uses a multi-head mechanism to enhance the representation capability of the model; is the number of attention heads, using a different projection matrix for each head The output for each header is: ； in, It is The output of the attention head represents The calculation result of the attention head is: It is the differential attention mechanism; S425 applies layer normalization to the output of each head and scaling ; The scaling factor is ; S426 concatenates the outputs of all heads and projects them to the final output dimension through a linear layer to obtain the final output d of the first branch: ； in, is the final output of the first branch, To string characters or words together, A function that concatenates strings or words together. The final output projection matrix.

9. The translation method based on the knowledge base and the improved Transformer model according to claim 8, characterized in that: The calculation process of the second branch in step S42 is as follows: S4211 For the input feature vector ,have in is the number of input texts, is the dimension of the input feature; Project to , and The matrix is: ； in, is the projection matrix; S4212 uses the ReLU function as the kernel function to calculate the similarity between the query matrix and the key matrix: ； in, are the i-th and j-th rows of the query matrix and key matrix respectively; S4213 performs weighted summation based on similarity and obtains preliminary output: ； in, is the value matrix The jth column of S4214 combines the preliminary output results with the results of deep convolution enhancement Add together to get the final output of the second branch: in, is the initial output matrix, is a pair matrix The result after applying depthwise convolution, is the final output of the second branch; The outputs of the two branches of S4215 are fused through a gating mechanism. The fusion process is expressed as: in, denote the outputs of the first branch and the second branch respectively, The fusion weights output by the gating mechanism, The value of is the complexity of the structured result calculated in step S31.

10. The translation system based on the knowledge base and the improved Transformer model according to claim 1, characterized in that: It includes input module, analysis and interpretation module, knowledge base module, translation module and feedback module. The input module is used to obtain the input literary text, clean the literary text, and obtain a standardized text; The analysis and interpretation module is used to perform sentiment analysis and text style judgment on the standardized text. The sentiment analysis results and text style analysis results are output in the form of vectors and formatted. The knowledge base module is used to calculate the emotional complexity and style complexity according to the output formatting results, weight the emotional complexity and style complexity to determine the total complexity, search for relevant knowledge information from the knowledge base according to the total complexity, and fuse it with the formatting results to generate information-enhanced text; The translation module is used to input the information-enhanced text and determine the position encoding through the Transformer model, and then translate the encoded text through the adaptive differential linear attention module. The adaptive differential linear attention module includes a self-attention mechanism and a multi-head attention mechanism. The translated text obtained by the self-attention mechanism and the translated text obtained by the multi-head attention mechanism are weighted with the total complexity to obtain the translated text for output; The feedback module is used to adjust the parameters of the input module, analysis and interpretation module, knowledge base module and translation module according to the translation effect.

Images