Translation method and translation system

Abstract

The invention provides a translation method and a translation system. According to the translation method, firstly, a predefined text knowledge base is adopted to preprocess text segments needing to be precisely processed in a to-be-translated text, so that a prompt of an AI large language model contains a consistent text comparison table customized for the to-be-translated text; wherein fixed translation of specific text segments (such as proper nouns, phrases or sentences) in a to-be-translated text in a target language is defined; and translating the preprocessed text by adopting the AI large language model. The long text such as the novel can be segmented into chapters and sections by adopting an automatic text segmentation technology before translation, and the translated texts of all the chapters and sections are recombined into a target file. According to the technical scheme, the automatic text segmentation technology and a cooperation mechanism of text consistency preprocessing and comparison table guiding type Prompt guidance are combined, the problems of key expression inconsistency, semantic drift and the like occurring in long text translation are effectively avoided, and the continuity and readability of translated texts are improved.

Description

Technical Field

[0001] This invention relates to the field of machine translation technology or the translation application of large AI models. Specifically, it relates to a translation method and system that can improve named entity consistency, stylistic consistency, and semantic transfer accuracy in the translation of long texts. Background Technology

[0002] Currently, mainstream automatic translation systems are primarily based on the Neural Machine Translation (NMT) framework. These systems employ an encoder-decoder structure and superimpose an attention mechanism to achieve sentence-level or paragraph-level translation. The core components of these translation systems mainly include an input preprocessing module, an encoder, a decoder, and a context management module.

[0003] The input preprocessing module is used to segment and vectorize the input text. When a user inputs raw text, the system first segments the text. Using sub-word algorithms such as WordPiece or SentencePiece, long words are broken down into smaller linguistic units (e.g., "unhappiness" is broken down into "un" + "happiness"), thereby improving the system's ability to handle new or rare words. After segmentation, each sub-word is mapped to a fixed-dimensional vector for subsequent model processing.

[0004] After the input preprocessing module processes the input, these sub-word vectors are fed into a multi-layered Transformer encoder. Each encoder layer has a self-attention mechanism that automatically determines the relationships between words in the sentence. After attention calculation, the model transforms the vector at each position using a feedforward neural network, while using residual connections and normalization operations to ensure the stability and efficiency of the deep network during training. The contextual semantic information output by the encoder is passed to the decoder. The decoder gradually generates translations of the target language. In this process, it relies on the already generated partial translations (through self-attention) and combines the contextual information provided by the encoder (through cross-attention) to determine which word to generate next. Finally, the model predicts the next most likely word through a softmax layer and continues iterating until the sentence ends.

[0005] The context management module retains a recently generated context (e.g., the last 512 tokens) during decoding and calibrates the cached context information to improve translation coherence. This short-term caching mechanism helps the model maintain a certain level of coherence. However, it also introduces a problem: when the text is too long, content exceeding the window range is discarded, potentially leading to inconsistencies in terminology or context drift.

[0006] Despite the progress made by current neural machine translation (NMT) technology and large language models (LLM) in semantic understanding and sentence translation, the following key issues still exist in practical applications: 1. Insufficient Contextual Consistency: Most existing mainstream translation models process texts at the sentence or paragraph level, lacking the ability to model the overall context of the entire text. This "partial translation" approach often leads to inconsistencies in the translation of names, place names, and terminology when dealing with long texts (such as novels and screenplays), and may even result in character confusion and semantic drift, affecting the overall reading experience and logical coherence.

[0007] 2. Weak Context Adaptability: Current translation systems struggle to accurately identify and track consistent entity information within the context, such as character identities, titles, and proper nouns. This leads to inconsistencies in character titles and mistranslations of proper nouns in multi-paragraph or cross-chapter texts (such as dialogue scripts and novels), thereby reducing the comprehensibility of the content and the professionalism of the translation.

[0008] 3. Lack of custom knowledge intervention mechanisms: Existing large language models struggle to incorporate user-provided "external knowledge" during the translation process, such as proprietary name and place name lookup tables, industry terminology glossaries, and cultural localization rules. This limits the model's controllability and adaptability in translating professional texts (such as legal and medical texts) or documents with regional cultural characteristics, making it difficult to meet users' demands for personalization and accuracy.

[0009] 4. Difficulty in maintaining consistency in language style and genre: Especially in literary translation, existing methods often fail to maintain the rhythm, sentence structure, and stylistic features of the original text, resulting in a lack of literary quality and consistency in the translation. This problem is particularly prominent for texts such as novels, which have high requirements for maintaining style, often causing reading difficulties such as inconsistent tone and narrative breaks. Summary of the Invention

[0010] To address the aforementioned shortcomings of current automatic translation systems when translating long texts, this application comprehensively adopts a processing method combining "terminology consistency preprocessing + text consistency comparison table" with automated text segmentation technology. This improves the effectiveness of automatic translation systems in avoiding common problems such as terminology drift and name confusion in the context transfer translation of long texts, thereby enhancing the coherence, consistency, and readability of the translation.

[0011] A first aspect of the present invention provides a translation method. The translation method includes: preprocessing a text to be translated using a predefined text knowledge base, the preprocessing including traversing the text to be translated and replacing text segments requiring precise translation with target text segments in the target language. Then, using an AI large language model to perform a translation task, the preprocessed text to be translated is translated to obtain the target text. The final execution prompt generated by the AI ​​large language model for the translation task integrates a consistent text lookup table as a translation guide for the text to be translated. The consistent text lookup table defines the type of a specific text segment in the text to be translated, the specific text segment, and its target translation in the target language. The types of specific text segments defined in the consistent text lookup table include, but are not limited to: titles of persons, place names, organizations, items / props, skills / moves, and phrases or sentences that require a fixed translation.

[0012] Furthermore, the final execution prompt is generated by selecting a translation template from a preset prompt library or prompt template library and filling the consistency text lookup table and the text content to be translated into the template.

[0013] Furthermore, the translation method also includes: dynamically expanding the predefined text knowledge base, which is used to store text fragments that need to be accurately translated and their corresponding translations in the target language. The text fragments that need to be accurately translated include, but are not limited to, slang, culturally specific words, proper abbreviations, or fixed expressions.

[0014] Furthermore, the translation task of using an AI large language model to translate the preprocessed text to be translated to obtain the target text includes: using a text segmentation algorithm to segment the preprocessed text to be translated into chapters, then using the AI ​​large language model to translate the segmented chapters to obtain the corresponding target chapters, and reorganizing the obtained target chapters into the target file according to the order of their original text in the text to be translated.

[0015] Preferably, the translation method further includes: performing forced calibration (i.e., post-processing) on ​​the target text using the consistency text lookup table.

[0016] In contrast to the above translation method, a second aspect of the present invention also provides a translation system.

[0017] The translation system includes a text replacement module, a text segmentation module, an AI large-scale language model, and a reorganization module. The text replacement module uses a predefined text knowledge base to traverse the text to be translated, replacing text segments requiring precise translation with target text in the target language. The text segmentation module uses a text segmentation algorithm to divide the text to be translated into chapters, obtaining the individual chapter texts. The AI ​​large-scale language model performs the translation task, translating each chapter text to obtain the target chapter. The final execution prompt generated for this translation task integrates a consistency text lookup table as a guide for translating the text to be translated. The consistency text lookup table defines the type of a specific text segment in the text to be translated, the specific text segment, and its target translation in the target language. The reorganization module reorganizes the target chapters translated by the AI ​​large-scale language model into a target file according to their original order in the text to be translated. The types of specific text segments defined in the consistency text lookup table include, but are not limited to: names of people, place names, organizations, items, skills, and phrases or sentences that require a fixed translation.

[0018] Furthermore, the predefined text knowledge base is used to store text fragments that need to be accurately translated and their corresponding translations in the target language. The text fragments that need to be accurately translated include, but are not limited to, slang, culturally specific words, proper abbreviations, or fixed expressions. The text replacement module also includes an interface for replacing or expanding the predefined text knowledge base.

[0019] Preferably, the translation system further includes a post-processing module, which uses the consistency text lookup table to perform forced calibration on the target file.

[0020] Corresponding to the translation method described above, a third aspect of this application also provides a computer-readable storage medium. The computer-readable storage medium stores code that, when executed by a processor / computer, implements the translation method described above.

[0021] The translation method and system provided in this application utilize an AI-powered large-scale language AI prompt word integration mechanism to create a text consistency checklist for the long text to be translated. Combined with a pre-set, general system-level noun knowledge base, a unified terminology input context is constructed, effectively guiding the AI ​​to maintain consistency in proper nouns such as names of people, places, and technical terms during the translation of long texts. Simultaneously, the prompt word injection method avoids the ambiguity and semantic confusion that may result from direct text replacement, improving the contextual coherence and contextual fidelity in long text translation. While ensuring translation quality, the solution achieves adaptation to personalized corpora and accurate restoration of professional terminology, possessing good versatility and scalability; it is particularly suitable for translating long texts such as novels and professional documents. Attached Figure Description

[0022] Figure 1 A flowchart of one embodiment of the translation method provided by the present invention.

[0023] Figure 2 A schematic diagram of one embodiment of the translation system provided by the present invention. Detailed Implementation

[0024] To make the technical problems solved by the present invention, the technical solutions, and the beneficial effects clearer, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0025] like Figure 1 As shown, the translation method provided by this invention includes the following steps: S1. Construct a predefined text knowledge base. This predefined text knowledge base stores text fragments in a general domain that require precise translation, along with their corresponding translations in the target language. These text fragments cover a variety of forms, from single words to fixed phrases, such as slang, culturally specific terms (e.g., internet slang), proper abbreviations, or expressions requiring fixed translations. This predefined text knowledge base can be pre-built by experts and supports dynamic expansion during subsequent use.

[0026] S2. Traverse the text to be translated using the predefined text knowledge base, and replace the text fragments that need to be accurately translated with the target text fragments in the target language of translation. The processing program / computer will traverse the entire text and, in comparison with the predefined text knowledge base, replace the text fragments that need to be accurately translated in the text to be translated with the target text fragments in the target language of translation. The predefined text knowledge base is used to store the text fragments that need to be accurately translated and their corresponding translations in the target language. The text fragments that need to be accurately translated include, but are not limited to: slang, culture-specific vocabulary, proprietary abbreviations, or fixed expressions. Traversing the text to be translated using the predefined text knowledge base and replacing the matching text fragments can avoid or reduce the possibility of mistranslation or non-translation by the AI large language model due to lack of specific cultural background.

[0027] S3. Construct a text consistency comparison table for the text to be translated; the consistency text comparison table defines the types of specific text fragments in the text to be translated, the specific text fragments, and their target translations in the target language of translation. The types of the specific text fragments include, but are not limited to: personal appellations, place names, organizations, items and props, skill moves, and phrases or sentences that need to maintain fixed translations.

[0028] Preferably, the user can upload a custom consistency text comparison table for a specific work or project to be translated, following the writing rules of "text fragment type", "original text fragment", and "target translation result". This table defines the mapping relationship between the original text and the target language of a series of proper nouns. For example, in the case of Chinese-to-English translation, "person", "Zhang San", and "San Zhang" can be used. Deal with the content that the AI model may mistranslate or not translate due to lack of specific cultural background.

[0029] S4. Integrate the consistency text comparison table into the final execution prompt generated by the AI large language model for performing the translation task of the text to be translated, as the translation guidance for this text to be translated. When calling the AI large language model to translate a text unit, the system will append the content of the consistency text comparison table uploaded by the user as a prompt word to the prompt of the AI large language model to construct a prompt word context. This prompt word mechanism enables the AI large language model to clearly define the corresponding method of named entities in each translation, ensuring that even if each text unit (for example, dividing a long text into a series of chapters) is processed independently, a translation result with consistent naming and smooth semantics can be obtained. In the original text, a person may be called informal appellations such as "he", "Old Zhang", "That person Zhang", etc. If the risk of pre-replacement is relatively high, the method of injecting prompt words is more flexible and accurate.

[0030] Furthermore, the final execution prompt is generated by selecting a translation template from a preset prompt library or prompt template library and filling the consistency text lookup table and the text content to be translated into the template.

[0031] S5. The AI ​​large language model is used to translate the text to be translated after step S2 to obtain the target text. For long texts such as novels and professional documents, this step can also be implemented as follows: the text to be translated is segmented into chapters after word replacement using a text segmentation algorithm, and then the segmented chapters are translated using the AI ​​large language model to obtain the corresponding target chapters. The obtained target chapters are then reorganized into the target file according to the original text order in the text to be translated.

[0032] Preferably, the translation method further includes performing forced calibration on the target file using the consistency text lookup table.

[0033] Corresponding to the translation method described above, this invention provides a computer-readable storage medium. The computer-readable storage medium stores code that, when executed by a processor / computer, implements the aforementioned translation method. The computer-readable storage medium includes SDRAM, RAM, flash memory, NOR flash memory, hard disk, magnetic disk, USB storage, and network storage.

[0034] Corresponding to the above translation method, the present invention also provides a translation system. For example... Figure 2 As shown, the translation system 200 includes: a text replacement module 201, a text segmentation module 202, an AI large language model 203, and a chapter module 204.

[0035] The text replacement module 201 uses a predefined text knowledge base to traverse the text to be translated, replacing text segments requiring precise translation with target text in the target language. The predefined text knowledge base stores text segments requiring precise translation in a general domain and their corresponding translations in the target language. These text segments cover a variety of forms, from single words to fixed phrases, such as slang, culturally specific terms (e.g., internet slang), proper abbreviations, or expressions requiring fixed translations. This predefined text knowledge base can be pre-built by experts and supports dynamic expansion during subsequent use.

[0036] The text segmentation module 202 employs a text segmentation algorithm to divide the text to be translated into chapters, thereby obtaining the text of each chapter. The system automatically divides a long document into multiple independent text units using a preset text segmentation algorithm. This algorithm can be based on one or more rules, such as recognizing specific structured identifiers (e.g., subheadings, specific separators), monitoring logical breaks in the text (e.g., consecutive blank lines), or segmenting according to a preset length (e.g., number of characters or paragraphs). When the system identifies a segmentation point according to the preset rules, it treats the subsequent content as a new text unit until the end of the document. In this way, the present invention can flexibly handle text documents of different formats and achieve automated text segmentation.

[0037] The AI ​​large language model 203 performs a translation task, translating the text of each chapter to obtain the target chapter. The final execution prompt generated for this translation task integrates a consistent text lookup table as a guide for translating the text to be translated. The consistent text lookup table defines the type of a specific text segment in the text to be translated, the specific text segment, and its target translation in the target language. The reorganization module reorganizes the target chapter translated by the AI ​​large language model into a target file according to the original order in the text to be translated. The types of specific text segments defined in the consistent text lookup table include, but are not limited to: names of people, place names, organizations, items, skills, and phrases or sentences that require a fixed translation.

[0038] The chapter reorganization module 204 reorganizes the target chapters translated by the AI ​​large language model 203 into a target file according to the original text order in the text to be translated.

[0039] Furthermore, the translation system 200 also includes a post-processing module, which uses the text consistency checklist to perform forced calibration on the target file.

[0040] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A translation method, characterized in that, The translation method includes: preprocessing the text to be translated using a predefined text knowledge base, wherein the preprocessing includes traversing the text to be translated and replacing the text segments that need to be accurately translated with target text segments in the target language; and performing the translation task using an AI large language model to translate the preprocessed text to obtain the target text, wherein the final execution prompt generated by the AI ​​large language model for the translation task integrates a consistent text lookup table as a translation guide for the text to be translated; the consistent text lookup table defines the type of a specific text segment in the text to be translated, the specific text segment and its target translation in the target language.

2. The translation method as described in claim 1, characterized in that, The final execution prompt is generated by selecting a translation template from a preset prompt library or prompt template library, and filling the consistency text lookup table and the text content to be translated into the template.

3. The translation method as described in claim 1, characterized in that, The translation method further includes: dynamically expanding the predefined text knowledge base, which is used to store text fragments that need to be accurately translated and their corresponding translations in the target language. The text fragments that need to be accurately translated include, but are not limited to, slang, culturally specific words, proper abbreviations, or fixed expressions.

4. The translation method as described in claim 3, characterized in that, The types of specific text fragments defined in the consistency text lookup table include, but are not limited to: character titles, place names, organizations, items and props, skill moves, and phrases or sentences that need to maintain a fixed translation.

5. The translation method as described in claim 4, characterized in that, The translation task using an AI large language model to translate preprocessed text to obtain target text includes: using a text segmentation algorithm to segment the preprocessed text to be translated into chapters, then using the AI ​​large language model to translate the segmented chapters to obtain corresponding target chapters, and reorganizing the obtained target chapters into the target file according to the original text order in the text to be translated.

6. The translation method according to any one of claims 1-5, characterized in that, The translation method further includes: performing forced calibration on the target text using the consistency text lookup table.

7. A translation system, characterized in that, The translation system includes: a text replacement module, a text segmentation module, an AI large language model, and a reconstruction module; The text replacement module uses a predefined text knowledge base to traverse the text to be translated, and replaces the text segments that need to be accurately translated with the target text in the target language. The text segmentation module uses a text segmentation algorithm to segment the text to be translated into chapters, so as to obtain the text of each chapter of the text to be translated; The AI ​​large language model translation task translates the text of each chapter to obtain the target chapter. The final execution prompt generated for the translation task integrates a consistent text lookup table as a translation guide for the text to be translated. The consistent text lookup table defines the type of a specific text segment in the text to be translated, the specific text segment and its target translation in the target language. The reorganization module reorganizes the target chapters translated by the AI ​​large language model into a target file according to the original text order in the text to be translated.

8. The translation system as described in claim 7, characterized in that, The predefined text knowledge base is used to store text fragments that need to be accurately translated and their corresponding translations in the target language. The text fragments include, but are not limited to, slang, culturally specific words, proper abbreviations, or fixed expressions. The text replacement module also includes an interface for replacing or expanding the predefined text knowledge base.

9. The translation system as described in claim 8, characterized in that, The types of specific text fragments defined in the consistency text lookup table include, but are not limited to: character titles, place names, organizations, items and props, skill moves, and phrases or sentences that need to maintain a fixed translation.

10. The translation system according to any one of claims 7-9, characterized in that, The translation system also includes a post-processing module; the post-processing module uses the consistency text lookup table to perform forced calibration on the target file.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores code that, when executed by a processor / computer, implements the translation method as described in any one of claims 1-6.

Images