Translation method and system based on knowledge base and improved transformer model

Abstract

The application provides a translation method and system based on a knowledge base and an improved Transformer model. The input module first acquires literary texts and performs cleaning to ensure the quality and consistency of the input data. Then, the analysis and judgment module performs sentiment analysis and style judgment on the cleaned texts and outputs the results in vector form, providing rich context information for subsequent modules. The knowledge base module calculates the complexity of the texts based on these analysis results and retrieves relevant knowledge information from the knowledge base. The analysis results are combined with the retrieved information to generate information-enhanced texts. The translation module receives the information-enhanced texts, uses an adaptive differential linear attention module to capture the relationship between words, generates context-aware representations, and generates translation results through an encoder architecture. Finally, the feedback module dynamically adjusts the parameters of each module based on the translation effect to continuously optimize the translation quality and efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a translation method and system based on a knowledge base and an improved Transformer model. BACKGROUND

[0002] With the acceleration of globalization, cross-border exchange of literary works is increasingly frequent, promoting interaction between different languages and cultures. However, traditional manual translation is inefficient and difficult to meet the large-scale translation needs, especially in dealing with complex cultural connotations, emotional expression and rhetorical devices in literary works. To address this challenge, deep learning-based technology provides a new way for literary translation.

[0003] Modern translation systems integrate natural language processing, machine learning and deep learning technologies, not only significantly improving translation efficiency, but also more accurately capturing the semantics, emotions and language style of the original text. In the face of complex language phenomena such as metaphors and puns, deep learning technology can effectively preserve the unique charm of the original work and generate translated text that fits the cultural background of the target language.

[0004] To further improve translation quality, the introduction of a knowledge base provides strong support for the translation system. By building a knowledge base that covers rich cultural background, domain knowledge and language rules, the translation system can better understand the meaning of the source language and accurately convey it to the target language. When dealing with specific cultural phenomena, historical background or professional terms, the system can use the knowledge base for accurate interpretation to avoid misinterpretation or translation bias.

[0005] As disclosed in the patent document with patent application number 202310825794.4 and publication date October 3, 2023, a personalized conversation method and system combined with sentiment analysis, the method includes: inputting a multi-modal sentence, processing the multi-modal sentence, and classifying the multi-modal sentence through a sentiment classification model, outputting a high-fineness sentiment classification result; modify the high-fineness sentiment classification result through the Prompt template, guide the pre-training large model to output the sentence rich in emotion; wherein the classification head of the sentiment classification model is designed based on a two-level sentiment category system, and the sentiment classification model takes a Transformer structure as a feature extraction unit. The fineness of sentence sentiment discrimination can be improved, and the ability of the dialogue system to express individualization can be enhanced.

[0006] The above documents propose a two-level emotion category system, which further subdivides the traditional positive, negative and neutral emotions, but this subdivision may still not fully meet the complex and varied human emotional expressions. Human emotions are extremely rich and delicate, and many times an emotion may contain a mixture of multiple emotions, such as a complex mood containing both joy and worry, and the two-level emotion category system may not be able to accurately classify and express it. In order to translate accurately, if a relatively complex translation method is used for relatively simple text, it will result in a longer translation time, and if a simple translation method is used for complex text, it will not be able to ensure the accuracy of the translation. SUMMARY

[0007] The present application provides a translation method and system based on a knowledge base and an improved Transformer model, which enhances recognition accuracy, avoids misinterpretation or translation bias, and is simple in method.

[0008] To achieve the above purpose, the technical scheme provided by the present application is as follows: a translation method based on a knowledge base and an improved Transformer model, comprising:

[0009] S1 obtains the input literary text, cleanses the literary text, and obtains a standardized text;

[0010] S2 performs text sentiment analysis and text style judgment on the standardized text, and outputs the sentiment analysis result and the text style analysis result in the form of a vector, and forms a formatted result;

[0011] S3 calculates the sentiment and style complexity according to the output formatted result, determines the total complexity by weighting the sentiment complexity and the style complexity, searches for related knowledge information from the knowledge base according to the total complexity, and fuses the structured result to generate an information enhanced text;

[0012] S4 inputs the information enhanced text, determines the position encoding through the Transformer model, then translates the encoded text through an adaptive differential linear attention module, the adaptive differential linear attention module includes a simple linear attention mechanism and a multi-head attention mechanism, the simple linear attention mechanism obtains a translated text, and the multi-head attention mechanism obtains a translated text weighted with the total complexity, and the translated text is outputted;

[0013] S5 adjusts the parameters in steps S1-S3 according to the translation effect.

[0014] The above settings first standardize the input text to ensure the reliability of the input text, and then perform sentiment analysis and writing style analysis on the standardized text. The sentiment analysis can identify the emotional tendencies contained in the text, and the writing style analysis can determine the literary style of the text, which helps to better convey the emotions and style of the original text during translation and avoid translation distortion due to misunderstanding;

[0015] By combining the emotional complexity and the style complexity, the total complexity is obtained, and the information-enhanced text is obtained. According to the total complexity, relevant knowledge information is searched in the knowledge base. For complex texts, more in-depth mining of relevant information in the knowledge base can be performed, and then translation is performed through multi-head attention mechanism after determining the position in the Transformer model, which can translate more finely. For simple texts, simple linear attention mechanism can be used for simplification processing to improve translation efficiency. In the translation process, the self-adaptive differential linear attention module can dynamically adjust the allocation of computing resources according to the complexity of the text, ensuring high-quality translation of complex texts and improving the translation speed of simple texts, achieving dual optimization of translation efficiency and effect. At the same time, in the fusion process of complex texts and simple texts, the total complexity is used to determine the weighting coefficient, so that the translation result is closely related to the complexity of the writing style and emotional style of the text, further ensuring the translation effect and the relevance of the text writing and emotional complexity, ensuring the reliability of translation. In addition, the parameters of the above steps can be adjusted according to the translation effect, further ensuring the reliability of translation.

[0016] Further, in step S1, redundant spaces, irrelevant symbols, errors, and possible special characters in the text are identified and deleted; then the punctuation is standardized, including unifying different forms of punctuation; spelling errors and syntax abnormalities in the text are corrected to obtain the standardized text.

[0017] The above settings eliminate possible interference factors that may affect the translation quality, and ultimately provide cleaner, consistent, and reliable input data for the subsequent translation process.

[0018] Further, in step S2, specifically including: S21, input the standardized text into GPT, and GPT analyzes the sentiment and writing style contained in the standardized text;

[0019] S22: The sentiment analysis result returned by GPT needs to include the text object, the thing subject, the behavior of the thing subject, the emotion corresponding to the behavior, and the confidence corresponding to the emotion;

[0020] S23: The writing style analysis result returned by GPT needs to include the text object, the characteristics of the text object, the style type corresponding to the text, and the confidence corresponding to the style type;

[0021] Step S24 combines all sentiment analysis results and writing style analysis results into a vector to form a formatted result.

[0022] The above settings include sentiment categories such as happiness, sadness, anger, fear, calm, surprise, and uncertainty; and writing style categories such as classicism, romanticism, modernism, realism, gothic, postmodernism, satire, poetry, drama, minimalism, casual style, and uncertainty. By determining the corresponding sentiment and confidence level of the behavior in the sentiment analysis results, and the corresponding style type and confidence level of the style type in the writing style analysis results, the sentiment analysis and writing style analysis become more reliable, and it also facilitates the subsequent determination of the complexity calculation method based on the sentiment analysis results.

[0023] Furthermore, step S3 specifically includes step S31 dividing the sentiment analysis results into complex sentiment content and simple sentiment content, and setting weight coefficients for complex sentiment content and simple sentiment content.

[0024] S32 calculates the emotional complexity. ,in, It is the first The complexity of each sentiment analysis result, each sentiment analysis result The complexity is determined by the sum of its category and its corresponding weights.

[0025] S33 divides writing style into explicit style content and complex style content, and pre-sets weight coefficients for explicit style content and complex style content.

[0026] S34 computational style complexity ,in, It is the first The complexity of each writing style analysis result is determined by the sum of its category and its corresponding weights. If a result contains multiple categories, its weight is the sum of the weights of each category.

[0027] It is the first The complexity of each writing style analysis result; the complexity of each writing style result The complexity is determined by the sum of its category and its corresponding weights;

[0028] The calculation process for the complexity C of the formatted result of S35 is as follows: ,in, It is the standardized score of the total emotional complexity. is the total writing style complexity score normalized, is the structured result complexity, and is the weight coefficient for sentiment analysis and writing style complexity, is the minimum and maximum value of sentiment analysis complexity, respectively, is the maximum and minimum value of writing style complexity, respectively.

[0029] With the above settings, by calculating sentiment and style complexity, the system can better understand the emotional and stylistic characteristics of the text, and thus more accurately convey these characteristics in the translation process. For example, for emotionally complex texts, the system can handle emotional expression more carefully, ensuring that the translation conveys the emotional color of the original text; for stylistically complex texts, the system can handle writing style more flexibly, ensuring that the translation preserves the stylistic features of the original text.

[0030] Further, S32 also includes: if a result contains multiple categories, its weight is the sum of the weights of each category, The calculation formula is as follows: wherein, is the weight of category in the th sentiment analysis result, if the sentiment analysis result does not have a certain category ,

[0031] S33 also includes: if a result contains multiple categories, its weight is the sum of the weights of each category, The calculation formula is as follows: wherein, is the weight of category in the th writing style analysis result, if the writing analysis result does not have a certain category , .

[0032] The above settings calculate and determine the case where a result includes multiple categories during sentiment analysis and writing style analysis, ensuring that multiple categories in a result can be analyzed and translated more reliably.

[0033] Further, after step S35, step S36 is performed when the sentiment analysis confidence is pre-set. The search information is associated with the sentiment analysis result, and if the sentiment analysis result is marked as complex emotional content, further search for knowledge information related to the sentiment analysis result, and establish an association between these knowledge information and the sentiment analysis result;

[0034] S37When the text style analysis confidence is greater than the preset style analysis confidence value, determine all translation strategies and translation precautions corresponding to the Transformer model, and associate these searched information with the style type analysis result. If the style type analysis result is marked as complex style content, the system will further search for knowledge information related to the result, and associate the result-related knowledge information with the style type analysis result;

[0035] S38Fuse the association between knowledge information and emotional analysis results and style type analysis results, and output the fused information-enhanced text.

[0036] The above settings, for the emotional analysis result with high confidence, are marked as complex emotional content, and the knowledge base is searched for information to associate with the translated text. Then, for the writing style text with high confidence, the translation measurement and translation precautions are determined, and then associated with the relevant knowledge information. Finally, the fused information-enhanced text is formed to ensure the accuracy and reliability of translation.

[0037] Further, step S4 includes: after receiving the information-enhanced text and the structured result complexity, the information-enhanced text is first converted into word embedding. During the conversion of the input text into word embedding, position coding is applied to the word embedding. The input after position coding is transmitted to the encoder. The encoder captures the relationship between words through the adaptive differential linear attention module to generate context-aware representation;

[0038] S42After the feature vector is input into the adaptive differential linear attention module, the input feature vector is processed through two parallel branches. The first branch uses a multi-head differential attention mechanism, and the second branch uses a self-attention mechanism. After the feature vector passes through the two branches respectively, the outputs of the two branches are weighted and summed with the total complexity to generate an output feature representation.

[0039] The above settings, by combining the multi-head differential attention mechanism and the simple linear attention mechanism, the system can more flexibly process different types of text. The multi-head differential attention mechanism can capture more subtle inter-word relationships and is suitable for processing complex literary texts. The simple linear attention mechanism can improve the efficiency of processing simple texts and reduce the waste of computing resources.

[0040] Further, the calculation process of the first branch in step S42 is as follows:

[0041] S421For the input feature vector , where is the number of texts, is the dimension of the input feature; the input Projected to and Matrix, that is:

[0042] ;

[0043] in, It is a projection matrix, used to map input features to different spaces. They are Vector, V is a value matrix,

[0044] S422 computes two softmax attention maps, for each pair Matrix and Calculate attention score:

[0045] ;

[0046] in, These are two softmax attention maps, representing the attention distribution based on two sets of query matrices and key matrices, respectively. The softmax function is a normalization function.

[0047] S423 calculates the differential attention score by weighting the second softmax attention map with a learnable scalar λ, and subtracting the weighted second map from the first map:

[0048] ;

[0049] in, It is a learnable scalar. The initialization re-parameterizes λ as follows:

[0050] ;

[0051] in, It is used for reparameterization Learnable vectors are used to increase flexibility and expressive ability yes Initialization constants;

[0052] S424 uses a multi-head mechanism to enhance the representational power of the model; let... To determine the number of attention heads, use a different projection matrix for each head. The output of each head is:

[0053] ;

[0054] in, It is the first The output of the attention head represents the output of the first attention head. The calculation results of each attention head, This is a differential attention mechanism.

[0055] S425 applies layer normalization processing to the output of each head. and scaling ;

[0056] Where the scaling factor is ;

[0057] S426 concatenates the outputs of all heads and projects them onto the final output dimension through a linear layer to obtain the final output d of the first branch:

[0058] ;

[0059] in, This is the final output of the first branch. The final output projection matrix.

[0060] The above settings, the first analysis uses a multi-head self-attention mechanism for translation. First, the text is projected, then the attention map and differential attention score are calculated. Then, based on the differential attention score, multiple attention heads are obtained, and the multiple attention heads are concatenated to obtain the final output matrix.

[0061] Furthermore, the calculation process for the second branch in step S42 is as follows:

[0062] S4211 For the input feature vector ,have in It is the number of input texts. It is the dimension of the input features; the input Projected to , and Matrix, that is:

[0063] ;

[0064] in, It is the projection matrix;

[0065] S4212 uses the ReLU function as the kernel function to calculate the similarity between the query matrix and the key matrix:

[0066] ;

[0067] in, These are the i-th and j-th rows of the query matrix and the key matrix, respectively;

[0068] S4213 weighted sum according to the similarity, get the preliminary output:

[0069] ;

[0070] wherein, is the jth column of the value matrix ,

[0071] S4214 the preliminary output results and depth convolution enhanced results add, get the final output of the second branch: wherein, is the preliminary output matrix, is the result of applying depth convolution to the value matrix , is the final output of the second branch;

[0072] S4215 the output of the two branches is fused through a gating mechanism, and the fusion process is represented as: wherein, respectively represent the output of the first branch and the second branch, the fusion weight of the gating mechanism output, the value of the structural result complexity calculated in step S31.

[0073] The above settings, by calculating the similarity of the input feature vector and weighting, and through the depth convolution, each input channel is independently convolved without mixing channel information, to enhance local features, and then the depth convolution value is weighted with the preliminary output result, so that enhanced local features can be obtained, ensuring the accuracy of translation.

[0074] Another aspect of the present application also provides a translation system based on knowledge base and improved Transformer model, comprising an input module, an analysis and interpretation module, a knowledge base module, a translation module and a feedback module,

[0075] The input module is used for obtaining the input literary text, cleaning the literary text to obtain a standardized text;

[0076] The analysis and interpretation module is used for text sentiment analysis and text style judgment on the standardized text, and the sentiment analysis result and the text style analysis result are output in the form of a vector and form a formatted result;

[0077] The knowledge base module is used for calculating the sentiment complexity and the style complexity according to the output formatted result respectively, determining the total complexity by weighting the sentiment complexity and the style complexity, searching relevant knowledge information from the knowledge base according to the total complexity, and fusing the formatted result to generate information enhanced text;

[0078] The translation module is used for inputting the information-enhanced text and determining position coding through the Transformer model, and then translating the coded text through the adaptive differential linear attention module, the adaptive differential linear attention module comprising a self-attention mechanism and a multi-head attention mechanism, the self-attention mechanism obtaining the translated text and the multi-head attention mechanism obtaining the translated text and the total complexity, so as to obtain the translated text for output;

[0079] The feedback module is used for adjusting the parameters of the input module, the analysis and interpretation module, the knowledge base module and the translation module according to the translation effect.

[0080] The above system firstly standardizes the input text to ensure the reliability of the input text, and then performs sentiment analysis and writing style analysis on the standardized text, which can identify the sentiment tendency contained in the text, and the writing style judgment can determine the literary style of the text, which helps to better convey the sentiment and style of the original text in the translation process and avoid translation distortion caused by misunderstanding.

[0081] The total complexity is obtained by the sentiment complexity and the style complexity, and the information-enhanced text is obtained, the relevant knowledge information in the knowledge base is searched according to the total complexity, for the text with high complexity, the relevant information in the knowledge base can be further mined, and then the multi-head attention mechanism is used for translation after determining the position in the Transformer model, which can translate more finely, for simple text, simple linear attention mechanism can be used for simplification processing to improve the translation efficiency, in the translation process, the adaptive differential linear attention module can dynamically adjust the allocation of computing resources according to the complexity of the text, which not only ensures the high-quality translation of complex text, but also improves the translation speed of simple text, realizes the double optimization of translation efficiency and effect, at the same time, in the fusion process of complex text and simple text, the weighting coefficient is determined by using the total complexity, so that the translation result is closely related to the complexity of the writing style and the sentiment style of the text, further ensuring the reliability of the translation, in addition, the parameters of the above steps can be adjusted according to the translation effect, further ensuring the reliability of the translation. BRIEF DESCRIPTION OF DRAWINGS

[0082] Figure 1 The above system firstly standardizes the input text to ensure the reliability of the input text, and then performs sentiment analysis and writing style analysis on the standardized text, which can identify the sentiment tendency contained in the text, and the writing style judgment can determine the literary style of the text, which helps to better convey the sentiment and style of the original text in the translation process and avoid translation distortion caused by misunderstanding.

[0083] Figure 2 The above system firstly standardizes the input text to ensure the reliability of the input text, and then performs sentiment analysis and writing style analysis on the standardized text, which can identify the sentiment tendency contained in the text, and the writing style judgment can determine the literary style of the text, which helps to better convey the sentiment and style of the original text in the translation process and avoid translation distortion caused by misunderstanding. DETAILED DESCRIPTION

[0084] Embodiment one.

[0085] AsFigures 1-2 As shown, a translation method based on a knowledge base and an improved Transformer model is implemented through a translation system, which includes an input module, an analysis and judgment module, a knowledge base module, a translation module, and a feedback module. The specific steps include:

[0086] S1 The input module obtains input literary text, cleanses the literary text, and obtains standardized text;

[0087] S2 The standardized text is input into the analysis and judgment module for text sentiment analysis and text style judgment. The sentiment analysis result and the text style analysis result are output in the form of a vector and form a formatted result;

[0088] S3 The formatted result output from the analysis and judgment module is input into the knowledge base module, the sentiment and style complexity are calculated respectively, the total complexity is calculated by integrating the sentiment complexity and the style complexity, the relevant knowledge information is searched from the knowledge base according to the calculated total complexity, and is fused with the structured result to generate an information enhanced text;

[0089] S4 The information enhanced text is input into the translation module for text translation, and the position encoding is determined through the Transformer model. Then the encoded text is translated through the self-adaptive differential linear attention module, which includes simple linear attention and multi-head attention mechanism. The simple linear attention obtains the translated text, and the multi-head attention mechanism obtains the translated text and the total complexity for weighting, and the translated text is output;

[0090] S5 According to the translation effect, the parameters in the input module, the analysis and judgment module, the knowledge base module, and the translation module are adjusted through the feedback module to optimize the translation quality and effect. In this embodiment, the pre-set confidence and the learnable vector in the confidence in the translation module can be adjusted according to the translation effect. This adjustment can be artificial adjustment. For example, when a sad emotion appears in the translation result, the translation result is not enough to express the emotion, that is, the vocabulary used is not enough to express sadness, the pre-set confidence corresponding to the sad emotion can be adjusted to be lower. When it is higher than this pre-set confidence, the relevant vocabulary is searched from the knowledge base, so as to expand the vocabulary selection of the sad emotion.

[0091] In step S1, the superfluous spaces, irrelevant symbols, errors and possible special characters in the text are identified and deleted; then the punctuation marks are standardized, including unifying different forms of punctuation; for example, different modes of the same punctuation mark can be uniformly processed, then the spelling errors and syntax abnormalities in the text are corrected to eliminate possible interference factors affecting the translation quality. Finally, the standardized text obtained through these steps can provide cleaner, consistent and reliable input data for the subsequent translation process.

[0092] Step S2 specifically includes:

[0093] S21: input the standardized text into the analysis and judgment module such as GPT, GPT analyzes the content of the standardized text including emotion and writing style;

[0094] S22: the emotion analysis result returned by GPT needs to include the text object, the subject of the matter, the behavior of the subject of the matter, the emotion corresponding to the behavior and the confidence degree corresponding to the emotion;

[0095] S23: the writing style analysis result returned by GPT needs to include the text object, the characteristics of the text object, the style type corresponding to the text and the confidence degree corresponding to the style type;

[0096] S24: combine all emotion analysis results and writing style analysis results in the form of vectors to form a formatted result, the emotion classification includes: happiness, sadness, anger, fear, calmness, surprise and uncertainty, and the writing style classification includes: classicism, romanticism, modernism, realism, gothic, postmodernism, satirical literature, poetry, drama, minimalism, casual style and uncertainty;

[0097] One of the reference formats of the returned emotion analysis result is: <(text object), (subject of matter), (behavior), (emotion), (confidence degree)> and more specific examples are: <(Li Fang's eyes are filled with tears, and she speaks softly about her sadness in her heart.), (Li Fang), (speak sadness), (sadness), (0.75)>.

[0098] The writing style analysis result returned by GPT needs to include the text object, the characteristics of the text object, the style type corresponding to the text and the confidence degree corresponding to the style type, one of the reference formats of the returned writing style analysis result is: <(She walks on the desolate street, the light is dim, the shadow is long, as if forgotten by time.), (symbolic, sense of loneliness), (modernism), (0.9)>.

[0099] In another embodiment, for the same text, there can be more than one sentiment result analysis result and one writing style analysis result, all the sentiment analysis results and writing style analysis results are combined in the form of a vector, which can form a list or array containing multiple vectors, this list or array is called the so-called "formatted result", a format that can refer to the "formatted result" is as follows: [ (sentiment analysis result 1, sentiment analysis result 2, sentiment analysis result 3 …, sentiment analysis result n), (writing style analysis result 1, writing style analysis result 2, writing style analysis result 3 …, writing style analysis result n)], the specific formula is as follows: wherein represents the formatted result, represents all sentiment analysis results, represents all writing style results, ,

[0100] wherein, represents the result of the th sentiment analysis, represents the th text object, represents the th subject, represents the th subject's behavior represents the th subject's behavior corresponding to the sentiment represents the th sentiment corresponding confidence, wherein, represents the result of the th writing style analysis, represents the th text object, represents the th text object's characteristics, represents the th text corresponding style type, represents the th style type corresponding confidence.

[0101] More specifically, the formatted output is: [< (Li Fang's eyes are filled with tears, and she speaks softly about her sadness in her heart.), (Li Fang), (speak sadness), (sadness), (0.75)>; < (She walks on the desolate street, the light is dim, the shadow is long, as if forgotten by time.), (symbolic, sense of loneliness), (modernism), (0.9)>].

[0102] Step S3 specifically comprises: S31 dividing the sentiment analysis result into complex sentiment content and simple sentiment content, the determination of the sentiment analysis result can be determined by the confidence value, such as when the sentiment analysis result confidence value is higher than the preset sentiment analysis result confidence value, it is determined as simple sentiment content, such as for the preset confidence of happiness in the sentiment analysis result is 70, when it is higher than the preset confidence value, it is determined as explicit happy style; the preset complex sentiment content weight coefficient and the simple sentiment content weight coefficient; wherein the complex sentiment content weight coefficient is 3, and the simple sentiment content weight coefficient is 1;

[0103] S32 calculates the sentiment complexity Wherein, is the complexity of the first sentiment analysis result, and the complexity of each sentiment analysis result is determined by the sum of the categories to which it belongs and the corresponding weights, if a result contains multiple categories, the weight is the sum of the weights of the categories, The calculation formula of is as follows: is the weight of the category in the first sentiment analysis result, if the sentiment analysis result does not have a category , ;

[0104] S33 divides the writing style into explicit style content and complex style content, the determination of the writing style can be determined by the confidence value, such as when the writing style confidence is higher than the preset writing style confidence value, it is determined as explicit style content, such as for the preset confidence of romanticism in the writing style is 70, when it is higher than the preset confidence value, it is determined as explicit romanticism style; the preset explicit style content weight coefficient and the complex style content weight coefficient; wherein the explicit style content weight coefficient is 2, and the complex style content weight coefficient is 4;

[0105] S34 calculates the style complexity , wherein, is the complexity of the first sentiment analysis result. The complexity of each sentiment analysis result is determined by the sum of the categories to which it belongs and the corresponding weights. If a result contains multiple categories, the weight is the sum of the weights of the categories

[0106] Wherein is the complexity of the first writing style analysis result; the complexity of each writing style result is determined by the sum of the categories to which it belongs and the corresponding weights; if a result contains multiple categories, the weight is the sum of the weights of the categories, The calculation formula of is as follows: Wherein, is the first category in the writing style analysis result .

[0107] The calculation process of the formatting result complexity C is as follows: wherein, is the normalized score of the total sentiment complexity, is the normalized score of the total writing style complexity, are the minimum and maximum values of the sentiment analysis complexity, respectively, are the minimum and maximum values of the writing style complexity, respectively. is the weight coefficient for the sentiment analysis and the writing style complexity, assuming that both are equally important, The value of C ranges from 0 to 1, with a value closer to 1 indicating a higher complexity of the text to be translated, and a value closer to 0 indicating a relatively simple text to be translated. By calculating the sentiment and style complexity, the system can better understand the emotional and stylistic characteristics of the text, and thus more accurately convey these characteristics during translation. For example, for a text with complex emotions, the system can handle emotional expression more carefully to ensure that the translation conveys the emotional color of the original text; for a text with complex style, the system can handle writing style more flexibly to ensure that the translation preserves the stylistic characteristics of the original text.

[0108] After step S35, there is also step S36: when the sentiment analysis confidence is high, such as when the happy sentiment confidence is 0.95, all corresponding English expressions such as ecstasy, overjoyed, and the like are searched, and these expressions are associated with the sentiment analysis result. If the sentiment analysis result is marked as "complex emotional content", the system will further search for knowledge information related to the result, and establish an association between the knowledge information and the sentiment analysis result;

[0109] S37 When the text style analysis confidence is high, such as when the text style analysis confidence is higher than the pre-set confidence value, such as when the realism style confidence is 0.7, all corresponding translation strategies, translation considerations, and the like are searched, and these searched information is associated with the style type analysis result. If the style type analysis result is marked as "complex style content", the system will further search for knowledge information related to the result, and establish an association between the knowledge information and the style type analysis result;

[0110] ​​S38 fuses the association between the knowledge information and the results of sentiment analysis and style type analysis, such as translating the English literary text "The mayor's speech dripped with honeyed words about 'equality,' while his polished boots crushed the petitions of the hungry. Ah, the glorious symphony of democracy!" into Chinese. "Dripped with" in the text to be translated is directly translated as "full of" or "overflowing with", which usually does not limit the context of use. However, the results of sentiment analysis show that this text mainly contains two kinds of emotions: anger with a confidence of 0.85 and surprise (irony) with a confidence of 0.7.

[0111] Obviously, directly translating 'dripped with' as 'full of' or 'overflowing with' is not the best choice for the context of this text.

[0112] After the knowledge base retrieves according to the results of sentiment analysis, it considers that in the context of anger and irony, it is more appropriate to translate "dripped with" as "full of empty words" or "big talk". Therefore, the association is established: in the current context of anger and irony, prefer to use "full of empty words" or "big talk" to translate "dripped with". The results of style type analysis show that the text to be translated belongs to satirical literature, with a confidence of 0.75.

[0113] The part of knowledge will also be divided by confidence, and the knowledge related to emotion will be associated with confidence. The range of confidence is any value between 0 and 1. The greater the confidence, the stronger the emotion. Conversely, the smaller the confidence, the weaker the emotion. Specifically, different phrases or words expressing a certain emotion in the database will have one or more confidence values. For example, for the emotion of "happiness", English can use "pleasure", "happiness", "joy", "delight" and "ecstasy" to express it. However, the degree of happiness expressed by these words is not the same. Pleasure is more suitable for describing mild happiness, and ecstasy is more suitable for expressing extreme happiness. Therefore, in the database, the confidence of the word "pleasure" corresponding to "happiness" is 0.6, and the confidence of the word "ecstasy" corresponding to "happiness" is 0.95. It should be noted that some expressions about emotion can contain multiple different emotions, such as "mixed feelings of joy and sorrow". It describes a state of emotion, in which "sadness" represents sadness and unhappiness, and "joy" represents joy and happiness. These two emotions coexist and interweave in the word, forming a complex psychological state. Therefore, "mixed feelings of joy and sorrow" contains "happiness" and "sadness" and their respective confidence values.

[0114] First, in the high-confidence interval (0.9-1), the style characteristics of the text are highly significant and can be clearly attributed. The core strategy at this time is style-enhanced translation. At the operational level, the lexical level requires the forced use of a symbolic term library of this style (for example, the realistic style requires the use of "street slang + concrete description word table"), and the syntactic level strictly replicates typical structural features, such as maintaining dense narrative long paragraphs and avoiding lyrical rewriting.

[0115] Second, in the medium-confidence interval (0.6-0.89), although the main style of the text is clear, there may be mixed characteristics. At this time, first determine whether there is only one style type. If there is only one style, the same strategy as in the high-confidence interval is adopted. If there are multiple style types, high-frequency words should be matched to the main style dictionary first, and rhetorical devices can allow secondary styles to penetrate appropriately. At the same time, for the part of the style that is ambiguous, academic annotation can be made by adding translator's notes.

[0116] Then, in the low-confidence interval (0.3-0.59), the style characteristics are mixed or not obvious. The core strategy at this time is conservative and neutral translation.

[0117] Finally, when the confidence is less than 0.3, the system will automatically trigger a request for manual review. In operation, the principle of minimal style intervention should be followed, the structure of the original text should be preserved, and a universal vocabulary library across styles should be used.

[0118] The setting and division of confidence are also determined by referring to expert opinions and consulting professional dictionaries to ensure the accuracy and professionalism of the data.

[0119] Step S4 further includes: after the information enhancement text and the structured result complexity are received by the step S41 translation module, the information enhancement text is first converted into word embedding. In the process of converting the input text into word embedding, in order to preserve the position information of the words in the sentence, position encoding is applied to the word embedding. The input after position encoding is transmitted to the encoder. The encoder is a position encoder in the improved Transformer model. The encoder captures the relationship between words through the adaptive differential linear attention module and generates context-aware representation.

[0120] After the feature vector is input into the adaptive differential linear attention module, the input feature vector is processed through two parallel branches. The first branch adopts a multi-head differential attention mechanism, and the second branch adopts a simple linear attention mechanism. After the feature vector passes through these two branches respectively, the outputs of the two branches are fused through a gating mechanism to generate a comprehensive feature representation.

[0121] After the feature vector enters the first branch, it will undergo the following process: first, the input feature vector will be projected into three different spaces: query matrix key matrix and value matrix This is done through three independent linear transformations (i.e., matrix multiplication), each of which uses a learnable projection matrix. This process maps the original features to a new space that is more suitable for attention mechanism calculation.

[0122] Next, the mechanism calculates two softmax attention maps. This is done by performing dot product on the two sets of query matrix and key matrix respectively, and then normalizing them through the softmax function. These two attention maps reflect the different correlations or importance between feature vectors.

[0123] Then, the core operation of the MHD (multi-head Attention) mechanism is to calculate the differential attention score. It uses a learnable scalar to weight the second attention map and subtracts this weighted second attention map from the first attention map. This operation aims to highlight the differences between the two attention maps, thereby capturing more subtle relationships between feature vectors.

[0124] In order to enhance the representation ability and flexibility of the model, this learnable scalar is usually obtained through reparameterization, so that its value can be dynamically adjusted during the training process.

[0125] Finally, the results from the differential attention calculation are weighted and summed with the value vectors to obtain the output of each attention head. In the multi-head mechanism, multiple such attention heads are calculated in parallel, and their outputs are concatenated. Finally, the concatenated result is projected to the final output dimension by a linear layer to obtain the output of the MHD (multi-head Attention) mechanism. This output is the enhanced representation of the original feature vectors after the differential attention mechanism, which contains more abundant and detailed relationship information between the feature vectors.

[0126] The complete calculation process after the first branch of the feature vector is as follows:

[0127] S421 For the input feature vector Wherein is the number of tokens (i.e. the number of texts), is the dimension of the input feature.

[0128] The input is a matrix, that is:

[0129] ;

[0130] Wherein, is the projection matrix, which is used to map the input feature to different spaces, are vectors, is the value matrix, and the input X is projected to the Q matrix, the K matrix and the V matrix through the existing projection method, which are used to calculate the attention scores of two different attention graphs.

[0131] S422 Calculate two softmax attention graphs, and calculate the attention score for each pair

[0132] ; wherein, are two softmax attention graphs, which represent the attention distribution based on two sets of query matrices and key matrices, respectively, and the softmax function is a normalization function.

[0133] S423 Calculate the differential attention score, use a learnable scalar λ to weight the second softmax attention graph, and subtract the weighted second graph from the first graph:

[0134] ;

[0135] Wherein, is a learnable scalar, which is used to balance the contribution of the two attention graphs and realize the differential attention mechanism. ​

[0136] Learnable scalar Initialization, to synchronize learning dynamics, re-parameterize λ as:

[0137] ;

[0138] where, is a learnable vector for re-parameterizing , which can be preset or adjusted through input, to increase the flexibility and expressive power of , and is an initialization constant for initializing the value of , to ensure the stability of the training process.

[0139] S424 uses multi-head mechanism to enhance the representation ability of the model. Let be the number of attention heads, use different projection matrices for each head. The output of each head is:

[0140] ;

[0141] where, is the output of the i-th attention head, representing the calculation result of the i-th attention head, is the difference attention mechanism. S425 applies layer normalization

[0142] and scaling factor to the output of each head:

[0143] Concatenate the outputs of all heads and project them through a linear layer to the final output dimension d to get the final output d of the first branch:

[0144] ;

[0145] where, is the final output of the first branch, is the projection matrix of the final output.

[0146] After the feature vector enters the second branch, it will undergo the following process:

[0147] First, the input feature vector is projected into three different spaces: the query matrix , the key matrix and the value matrix . This is done through three independent linear transformations (i.e. matrix multiplication), each using a learnable projection matrix. This process maps the original features into a new space that is more suitable for attention mechanism calculation.​

[0148] Next, the SLAttention mechanism uses a ReLU function as the kernel function to compute the similarity between the query matrix and the key matrix. The element-wise application of the ReLU function ensures that the similarity values are non-negative, which helps with the stability of subsequent calculations. By computing the dot product (after ReLU activation) of the query matrix and the key matrix, a similarity matrix is obtained, which reflects the degree of correlation between each query matrix and all key matrices.

[0149] With the similarity matrix, the SLAttention mechanism normalizes it and performs a weighted sum of the value matrix based on the normalized similarity values. This process is similar to traditional attention mechanisms, but the SLAttention mechanism reduces computational complexity by computing the product of the transpose of the key matrix and the value matrix in advance, achieving linear complexity. The result of the weighted sum is a reweighted representation of the input features, emphasizing the features most relevant to the current query.

[0150] To further enhance this reweighted feature representation, the SLAttention mechanism also applies a deep convolution operation to the original value matrix. Deep convolution is a lightweight convolution method that independently convolves each channel without mixing channel information. This operation helps capture local features and provides additional contextual information, further enhancing the feature representation.

[0151] Finally, the SLAttention mechanism adds the result of the weighted sum to the result of the deep convolution enhancement, obtaining the final output. This output not only contains the reweighted feature information but also incorporates the enhanced information of local features, allowing subsequent processes to handle richer and more accurate feature representations. The entire process maintains computational efficiency while ensuring that the model can capture important information from the input data.

[0152] The complete calculation process of the feature vector after the second branch is as follows:

[0153] S4211 For the input feature vector where is the number of tokens (input text), is the dimension of the input feature.

[0154] The input matrix, i.e.,

[0155] ;

[0156] where is the projection matrix, used to map the input features to different spaces.

[0157] S4212 uses the ReLU function as the kernel function to calculate the similarity between the query matrix and the key matrix:

[0158] ;

[0159] wherein, are the i-th row and j-th row of the query matrix and the key matrix respectively, the ReLU function is applied element-wise to ensure non-negativity.

[0160] S4213 performs weighted summation according to the similarity to obtain a preliminary output:

[0161] ;

[0162] wherein, columns, the denominator is used to normalize the similarity weight, and the value matrix apply deep convolution, which is a special convolution operation where each input channel is convolved independently without mixing channel information to enhance local features.

[0163] S4214 adds the weighted summation result and the deep convolution enhanced result to obtain the final output of the second branch: wherein, is the preliminary output matrix obtained after weighted summation, is the result of the value matrix after applying deep convolution, is the final output of the second branch, and the deep convolution model is an existing deep convolution model which will not be repeated here.

[0164] S4215 the outputs of the two branches are fused through a gating mechanism, and the specific fusion process can be represented as: respectively represent the output of the multi-head differential attention branch and the simple linear attention branch, the fusion weight of the gating mechanism output, the value is the complexity of the structured result calculated in step S31.

[0165] The working principle of the present application: first, the input text is standardized to ensure the reliability of the input text, then the standardized text is analyzed for sentiment and writing style, which can identify the emotional tendencies contained in the text, and the writing style judgment can determine the literary style of the text, which helps to better convey the emotions and style of the original text in the translation process, and avoids translation distortion caused by misunderstanding;

[0166] The information enhanced text is obtained by the emotional complexity and the style complexity, and the total complexity is synthesized, the related knowledge information is searched in the knowledge base according to the total complexity, for the text with high complexity, the related information in the knowledge base can be further mined, then the translation is carried out through the multi-head attention mechanism after the position is determined under the Transformer model, the translation can be more fine; for simple text, simple linear attention mechanism can be used for simplified processing to improve the translation efficiency; in the translation process, the adaptive differential linear attention module can dynamically adjust the allocation of computing resources according to the complexity of the text, which not only ensures the high-quality translation of complex text, but also improves the translation speed of simple text, realizes the double optimization of translation efficiency and effect, and in the fusion process of complex text and simple text, the weighting coefficient is determined by using the total complexity, so that the translation result is closely related to the complexity of the writing style and emotional style, further ensuring the translation effect and the text writing and emotional style, ensuring the reliability of the translation, in addition, the parameters of the above steps can be adjusted according to the translation effect, further ensuring the reliability of the translation.

[0167] Embodiment two.

[0168] As Figure 1 shown, a knowledge base and improved Transformer model translation system, the translation system includes an input module, an analysis and interpretation module, a knowledge base module, a translation module and a feedback module.

[0169] The input module is used for obtaining the input literary text, cleaning the literary text to obtain the standardized text;

[0170] The analysis and interpretation module is used for text sentiment analysis and text style judgment on the standardized text, the sentiment analysis result and the text style analysis result are output in the form of vector, and the formatted result is formed;

[0171] The knowledge base module is used for calculating the emotional complexity and the style complexity according to the output formatted result respectively, determining the total complexity by weighting the emotional complexity and the style complexity, searching the related knowledge information from the knowledge base according to the total complexity, and fusing to generate the information enhanced text;

[0172] The translation module is used for inputting the information enhanced text and determining the position coding through the Transformer model, then translating the coded text through the adaptive differential linear attention module, the adaptive differential linear attention module includes self-attention mechanism and multi-head attention mechanism, the self-attention mechanism obtains the translated text, and the multi-head attention mechanism obtains the translated text and the total complexity, and the translated text is output after weighting;

[0173] The feedback module is used to adjust the parameters of the input module, the analysis and interpretation module, the knowledge base module and the translation module according to the effect of the translation.

Claims

1. A translation method based on a knowledge base and an improved Transformer model, characterized by: include: S1 acquires the input literary text, cleans the literary text, and obtains standardized text; S2 performs sentiment analysis and style judgment on standardized text. The sentiment analysis results and style analysis results are output in vector form and formatted. S3 calculates the sentiment complexity and style complexity based on the output formatting results, weights the sentiment complexity and style complexity to determine the total complexity, searches for relevant knowledge information from the knowledge base based on the total complexity, and merges it with the formatting results to generate information-enhanced text. S4 takes the augmented text as input and uses the Transformer model to determine the positional encoding. Then, the encoded text is translated using an adaptive differential linear attention module, which includes simple linear attention and multi-head attention mechanisms. The translated text obtained through simple linear attention and the translated text obtained through multi-head attention are weighted with the total complexity to obtain the translated text for output. The calculation process of the first branch in step S42 is as follows: S421 For the input feature vector ,have ,in It is the number of texts. It is the dimension of the input features; the input Projected to and Matrix, that is: ； in, It is a projection matrix, used to map input features to different spaces. They are Vector, V is a value matrix; S422 computes two softmax attention maps, for each pair Matrix and Calculate attention score: ； in, These are two softmax attention maps, representing the attention distribution based on two sets of query matrices and key matrices, respectively. The softmax function is a normalization function. S423 calculates the differential attention score by weighting the second softmax attention map with a learnable scalar λ, and subtracting the weighted second map from the first map: ； in, It is a learnable scalar. The initialization re-parameterizes λ as follows: ； in, It is used for reparameterization Learnable vectors are used to increase flexibility and expressive ability yes Initialization constants; S424 uses a multi-head mechanism to enhance the representational power of the model; let... To determine the number of attention heads, use a different projection matrix for each head. The output of each head is: ； in, It is the first The output of the attention head represents the output of the first attention head. The calculation results of each attention head, This is a differential attention mechanism; S425 applies layer normalization processing to the output of each head. and scaling ; Where the scaling factor is ; S426 concatenates the outputs of all heads and projects them onto the final output dimension through a linear layer to obtain the final output d of the first branch: ； in, This is the final output of the first branch. To string characters or words together, A function that concatenates strings or words together. The final output projection matrix; S5. Adjust the parameters in steps S1-S3 based on the translation effect.

2. The translation method based on a knowledge base and an improved Transformer model according to claim 1, characterized in that: In step S1, redundant spaces, irrelevant symbols, error messages, and special characters in the text are identified and deleted; then, punctuation marks are standardized, including unifying different forms of punctuation; and spelling errors and grammatical anomalies in the text are corrected to obtain standardized text.

3. The translation method based on a knowledge base and an improved Transformer model according to claim 1, characterized in that: Step S2 specifically includes: S21 Inputting standardized text into GPT, and GPT analyzing the sentiment and writing style contained in the standardized text; S22: The sentiment analysis results returned by GPT should include the text object, the subject, the subject's behavior, the sentiment corresponding to the behavior, and the confidence level corresponding to the sentiment. S23: The writing style analysis results returned by GPT should include the text object, the features of the text object, the style type corresponding to the text, and the confidence level corresponding to the style type; Step S24 combines all sentiment analysis results and writing style analysis results into a vector to form a formatted result.

4. The translation method based on a knowledge base and an improved Transformer model according to claim 1, characterized in that: Step S3 specifically includes: S31 divides the sentiment analysis results into complex sentiment content and simple sentiment content, and presets the weighting coefficients for complex sentiment content and simple sentiment content. S32 calculates the emotional complexity. ,in, It is the first The complexity of each sentiment analysis result, each sentiment analysis result The complexity is determined by the sum of its category and its corresponding weights; S33 divides the writing style analysis results into explicit style content and complex style content, and pre-sets the weight coefficients for explicit style content and complex style content. S34 computational style complexity ,in, It is the first The complexity of each writing style analysis result; the complexity of each writing style result The complexity is determined by the sum of its category and its corresponding weights; The complexity C of calculating the formatted result of S35: It is the standardized score of the total emotional complexity. It is the standardized score of the overall writing style complexity. It is the complexity of the structured result. These are weighting coefficients for sentiment analysis and writing style complexity. These are the minimum and maximum complexity values ​​for sentiment analysis, respectively. These are the maximum and minimum values ​​of writing style complexity, respectively.

5. The translation method based on a knowledge base and an improved Transformer model according to claim 4, characterized in that: S32 also includes: if a result contains multiple categories, its weight is the sum of the weights of each category. The calculation formula is as follows: Category of sentiment analysis results The weighting of a sentiment analysis result if no category is specified. ; S34 also includes: if a result contains multiple categories, its weight is the sum of the weights of each category. It is the first The categories in the writing style analysis results The weighting, if the writing analysis results do not have a certain category ,but .

6. The translation method based on a knowledge base and an improved Transformer model according to claim 4, characterized in that: After step S35, proceed to step S36. When the confidence level of the sentiment analysis is greater than the preset confidence level, the search information is associated with the sentiment analysis results. If the sentiment analysis results are marked as complex sentiment content, further search will be conducted for knowledge information related to the sentiment analysis results, and this knowledge information will be associated with the sentiment analysis results. S37 When the confidence level of text style analysis is greater than the preset confidence level value of style analysis, determine all translation strategies and translation considerations corresponding to the Transformer model, and associate the searched information with the style type analysis results. If the style type analysis results are marked as complex style content, the system will further search for knowledge information related to the results and establish an association between the knowledge information related to the results and the style type analysis results. S38 integrates the correlation between knowledge information, sentiment analysis results, and style type analysis results, and outputs the integrated information-enhanced text.

7. The translation method based on a knowledge base and an improved Transformer model according to claim 1, characterized in that: Step S4 includes: After receiving the information-enhanced text and the structured result complexity in step S41, the information-enhanced text is first converted into word embeddings. During the process of converting the input text into word embeddings, position encoding is applied to the word embeddings. The input after position encoding is passed to the encoder. The encoder captures the relationship between words through the adaptive differential linear attention module and generates a context-aware representation. After the S42 feature vector is input into the adaptive differential linear attention module, the input feature vector is processed through two parallel branches. The first branch adopts a multi-head differential attention mechanism, and the second branch adopts a self-attention mechanism. After the feature vector passes through these two branches, the outputs of the two branches are weighted and summed with the total complexity to generate an output feature representation.

8. The translation method based on a knowledge base and an improved Transformer model according to claim 1, characterized in that: The calculation process for the second branch in step S42 is as follows: S4211 For the input feature vector ,have in It is the number of input texts. It is the dimension of the input features; the input Projected to , and Matrix, that is: ； in, It is the projection matrix; S4212 uses the ReLU function as the kernel function to calculate the similarity between the query matrix and the key matrix: ； in, These are the i-th and j-th rows of the query matrix and the key matrix, respectively; S4213 performs a weighted summation based on similarity to obtain the initial output: ； in, Value matrix The j-th column; S4214 combines the initial output with the results of depthwise convolution enhancement. Adding them together, we get the final output of the second branch: in, This is the initial output matrix. log-value matrix The result after applying depthwise convolution, This is the final output of the second branch; The outputs of the two branches of S4215 are merged through a gating mechanism. The fusion process is represented as follows: in, These represent the outputs of the first branch and the second branch, respectively. The fusion weights output by the gating mechanism The value is the complexity of the structured result calculated in step S31.

9. The translation system based on a knowledge base and an improved Transformer model according to claim 1, characterized in that: It includes an input module, an analysis and interpretation module, a knowledge base module, a translation module, and a feedback module. The input module is used to acquire the input literary text, clean the literary text, and obtain standardized text; The analysis and interpretation module is used to perform sentiment analysis and style judgment on standardized text. The sentiment analysis results and style analysis results are output in vector form and formatted. The knowledge base module is used to calculate the sentiment complexity and style complexity based on the output formatted results, weight the sentiment complexity and style complexity to determine the total complexity, search for relevant knowledge information from the knowledge base based on the total complexity, and fuse it with the formatted results to generate information-enhanced text. The translation module takes the augmented text as input and uses the Transformer model to determine the position encoding. Then, the encoded text is translated by the adaptive differential linear attention module, which includes a self-attention mechanism and a multi-head attention mechanism. The translated text obtained by the self-attention mechanism and the translated text obtained by the multi-head attention mechanism are weighted with the total complexity to obtain the translated text for output. The feedback module is used to adjust the parameters of the input module, analysis and interpretation module, knowledge base module, and translation module based on the translation effect.

Images