A large model machine translation method based on RAG
Through the combination of RAG technology and external knowledge base, synonymous language pairs and KL divergence calculations are used to solve the problem of poor translation tasks in different fields, achieving rapid adaptability and efficient translation, and reducing computing resource consumption .
Patent Information
- Application Number
- CN202410091557.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-23
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-01-23
AI Technical Summary
Large language models are not effective when performing translation tasks other than training data, and fine-tuning of the model consumes a large amount of computing resources, which is not conducive to the rapid update of information.
Using RAG (retrieval enhancement generation) method, the large language model is used to adapt to various fields and translation habits through external knowledge base, and embedding model training, word segmentation processing and embedding representation transformation using synonymous language pairs, combined with KL divergence calculation and Weaviate vector database, generates translations that fit the needs.
The quality and efficiency of translations are significantly improved in specific fields and translation stylization, the consumption of computing resources is reduced, and the rapid adaptation of large language models and the satisfaction of translation needs is achieved.
Smart Images

Figure CN117993396B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of large language model translation, and is a large model machine translation method based on RAG. Background Art
[0002] With the continuous development of neural network models, large language models have emerged, but they have made a qualitative breakthrough in functionality compared to previous neural network models. With the continuous improvement of language model parameters and the increasing amount of training data, the amount of knowledge contained in the model has increased by leaps and bounds. Large language models obtain extensive general knowledge by training a large amount of data, and this knowledge is stored in the weights of the neural network.
[0003] Although the amount of knowledge contained in the large language model can already meet the basic requirements of machine translation tasks, that is, it performs well in high-resource languages in general fields, the large language model is not very effective when performing translation tasks that require more than training data. In this case, we can usually fine-tune the model to adapt it to different fields, but this method consumes a lot of computing resources and is not conducive to the rapid updating of information.
[0004] The main idea of the present invention is to help the model quickly adapt to various fields and even personal translation habits by injecting external knowledge. Specifically, the RAG technology (retrieval enhanced generation) is used to use external knowledge bases to help the large language model adapt to various fields and translation habits, and complete the stylization and domainization of the large language model translation. Summary of the invention
[0005] In order to solve the above problems, the present invention proposes a large model machine translation method based on RAG.
[0006] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0007] The present invention provides a large model machine translation method based on RAG, and the present invention provides the following technical solutions:
[0008] A large model machine translation method based on RAG, the method comprising the following steps:
[0009] Step 1: Segment the original language sequence, transform the language sequence embedding representation, and establish a translation database based on the same language;
[0010] Step 2: Based on the translation requirements, the original language is split and the original text embedding representation is obtained;
[0011] Step 3: Calculate similarity based on the original text embedding representation and the translation database;
[0012] Step 4: Based on the similarity calculation, obtain the original text translation examples that meet the needs, and combine them with the big model to generate the translation that meets the needs.
[0013] Preferably, the step 1 is specifically:
[0014] The language is trained using synonymous language pairs in the field to generate an embedding model suitable for the field;
[0015] The original language sequence is segmented. The BPE segmentation strategy of the Sentence Piece tool is used to train all the language sequences in the corpus to obtain the vocabulary and tokenizer for the corpus for segmentation.
[0016] The obtained vocabulary is used to segment the language sequences of all translation databases and convert them into token sequence representations.
[0017] Preferably, the m-Bert model is used to train the embedding layer:
[0018] Train m-Bert on the corpus of the translation database. When training m-Bert, fix other layers and only train the embedding layer, further reducing the computing resources required for training.
[0019] The trained m-Bert model is used to generate embedded representations. The token sequence of the translation database is Bert-embedded to obtain the embedded sequence representation.
[0020] When storing the above-generated embedding sequence and language sequence, the Weaviate vector database is used to store the language sequence and its corresponding embedding sequence in pairs.
[0021] Preferably, the step 2 is specifically:
[0022] A user's translation requirement is divided into three pieces of information: translation direction, original text, and translation style requirement. The translation style requirement determines the choice of translation database. For example, if an ancient style or a humorous style is required, the corresponding translation database can be used to retrieve the final translation example.
[0023] First, the original text to be translated is segmented through the previously trained tokenizer to obtain the sequence represented by its tokens;
[0024] The above Token sequence is then embedded in the embedding model to obtain its vector form.
[0025] Preferably, the step 4 is specifically:
[0026] The KL divergence calculation is performed on the embedding representation of the token sequence and the embedding sequence in the translation data vector database. The obtained score is used as the similarity to obtain all the similarity scores.
[0027] The lower the KL divergence, the higher the similarity. Under the two constraints, we can find the vector representation of the most similar k language sequences (src1, src2…src k );
[0028] Constraint 1 (r1) is that the language is consistent with the original text, and constraint 2 (r2) is that the synonymous language pairs of the language sequence contain the required translation direction. The search interval is obtained by the two constraints, and then the sequence with the highest similarity is searched:
[0029] The formula for constraint 1 is expressed as:
[0030]
[0031] Where p is the original text, q is the current text in the translation database, and lang(x) is the language of the x sequence;
[0032] The formula of constraint 2 (r2) is expressed as:
[0033]
[0034] Where t is the target language sequence, lang(t) is the target language, and q 1…n Indicates a language sequence group synonymous with q, that is, the translation direction exists in the language sequence group synonymous with the current language sequence;
[0035] Obtain the search interval using constraints 1 (r1) and 2 (r2):
[0036] Q'(q)=R(Q(q),r1,r2)
[0037] Where Q'(q) is the search interval obtained from the initial translation database Q(q) through two constraints.
[0038] Preferably, for similarity calculation, KL divergence relative entropy is used as the similarity score, and the specific calculation formula is as follows:
[0039]
[0040] Among them, P(x) is the embedded representation of the original text, Q(x) is the embedded representation of a language sequence in the retrieval space, the smaller the KL divergence, the closer the two representations are. The similarity retrieval process is to search for the k language sequences in the retrieval space with the lowest KL divergence between the language sequence and the original text.
[0041] Preferably, k demos are obtained after retrieval, arranged at the beginning of a new prompt, and combined with the original translation requirement to obtain a new prompt. At this time, the prompt large language model generates a translation that meets the requirement.
[0042] A RAG-based large-model machine translation library is formed by a RAG-based large-model machine translation method, and generates translations that meet requirements in specific field translation and translation stylization.
[0043] A computer-readable storage medium stores a computer program, which is executed by a processor to implement a large-model machine translation method based on RAG.
[0044] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements a large model machine translation method based on RAG when executing the computer program.
[0045] The present invention has the following beneficial effects:
[0046] Compared with the prior art, the present invention has the following advantages:
[0047] The present invention performs word segmentation processing on the original language sequence, converts the language sequence embedding representation, establishes a translation database based on the same language; splits the original language based on the translation requirements, obtains the original text embedding representation; performs similarity calculation based on the original text embedding representation and the translation database; obtains translation examples close to the requirements based on the similarity calculation, combines the translation examples and the large model, and generates a translation that meets the requirements. Compared with ordinary prompts and prompts that use examples at random, the prompt construction method adopted by the invention can better help the large language model generate translations that meet the requirements, and has a good effect on translation in specific fields and translation stylization.
[0048] Compared with ordinary prompts and prompts that use random examples, the prompt construction method adopted by the present invention can better help large language models generate translations that meet the requirements, and has a good effect on translation in specific fields and translation stylization. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0050] Figure 1 It is the overall architecture diagram;
[0051] Figure 2 It is a synonymous language comparison chart of the translation data vector database;
[0052] Figure 3 It is a flow chart for constructing a translation data vector database;
[0053] Figure 4 It is a graph of language sequence embedding representations of the translation data vector database;
[0054] Figure 5 It is a flowchart of similar language sequence retrieval;
[0055] Figure 6 It is a translation stylized picture. DETAILED DESCRIPTION
[0056] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0057] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.
[0058] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0059] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0060] The present invention is described in detail below in conjunction with specific embodiments. Specific embodiment one:
[0062] according to Figures 1 to 6 As shown, the specific optimization technical solution adopted by the present invention to solve the above technical problems is: the present invention relates to a large model machine translation method based on RAG.
[0063] A large model machine translation method based on RAG, the method comprising the following steps:
[0064] Step 1: Segment the original language sequence, transform the language sequence embedding representation, and establish a translation database based on the same language;
[0065] The step 1 is specifically as follows:
[0066] The language is trained using synonymous language pairs in the field to generate an embedding model suitable for the field;
[0067] The original language sequence is segmented. The BPE segmentation strategy of the Sentence Piece tool is used to train all the language sequences in the corpus to obtain the vocabulary and tokenizer for the corpus for segmentation.
[0068] The obtained vocabulary is used to segment the language sequences of all translation databases and convert them into token sequence representations.
[0069] Use the m-Bert model to train the embedding layer:
[0070] Train m-Bert on the corpus of the translation database. When training m-Bert, fix other layers and only train the embedding layer, further reducing the computing resources required for training.
[0071] The trained m-Bert model is used to generate embedded representations. The token sequence of the translation database is Bert-embedded to obtain the embedded sequence representation.
[0072] When storing the above-generated embedding sequence and language sequence, the Weaviate vector database is used to store the language sequence and its corresponding embedding sequence in pairs.
[0073] Step 2: Based on the translation requirements, the original language is split and the original text embedding representation is obtained;
[0074] The step 2 is specifically as follows:
[0075] A user's translation requirement is divided into three pieces of information: translation direction, original text, and translation style requirement. The translation style requirement determines the choice of translation database. For example, if an ancient style or a humorous style is required, the corresponding translation database can be used to retrieve the final translation example.
[0076] First, the original text to be translated is segmented through the previously trained tokenizer to obtain the sequence represented by its tokens;
[0077] The above Token sequence is then embedded in the embedding model to obtain its vector form.
[0078] Step 3: Calculate similarity based on the original text embedding representation and the translation database;
[0079] The step 4 is specifically as follows:
[0080] The KL divergence calculation is performed on the embedding representation of the token sequence and the embedding sequence in the translation data vector database. The obtained score is used as the similarity to obtain all the similarity scores.
[0081] The lower the KL divergence, the higher the similarity. Under the two constraints, we can find the vector representation of the most similar k language sequences (src1, src2…src k );
[0082] Constraint 1 (r1) is that the language is consistent with the original text, and constraint 2 (r2) is that the synonymous language pairs of the language sequence contain the required translation direction. The search interval is obtained by the two constraints, and then the sequence with the highest similarity is searched:
[0083] The formula for constraint 1 is expressed as:
[0084]
[0085] Where p is the original text, q is the current text in the translation database, and lang(x) is the language of the x sequence;
[0086] The formula of constraint 2 (r2) is expressed as:
[0087]
[0088] Where t is the target language sequence, lang(t) is the target language, and q 1…n Indicates a language sequence group synonymous with q, that is, the translation direction exists in the language sequence group synonymous with the current language sequence;
[0089] Obtain the search interval using constraints 1 (r1) and 2 (r2):
[0090] Q'(q)=R(Q(q),r1,r2)
[0091] Where Q'(q) is the search interval obtained from the initial translation database Q(q) through two constraints.
[0092] For similarity calculation, KL divergence relative entropy is used as the similarity score. The specific calculation formula is as follows:
[0093]
[0094] Among them, P(x) is the embedded representation of the original text, Q(x) is the embedded representation of a language sequence in the retrieval space, the smaller the KL divergence, the closer the two representations are. The similarity retrieval process is to search for the k language sequences in the retrieval space with the lowest KL divergence between the language sequence and the original text.
[0095] Step 4: Obtain translation examples that meet the needs based on similarity calculation, and generate translations that meet the needs by combining with the big model.
[0096] After the retrieval, k demos are obtained, arranged at the beginning of the new prompt, and combined with the original translation requirements to obtain a new prompt. At this time, the prompt large language model generates a translation that meets the requirements.
[0097] The present invention provides a RAG-based large-model machine translation library, wherein the translation library is formed based on a RAG-based large-model machine translation method, and generates translations that meet requirements in specific field translation and translation stylization.
[0098] The present invention performs word segmentation processing on the original language sequence, converts the language sequence embedding representation, establishes a translation database based on the same language; splits the original language based on the translation requirements, obtains the original text embedding representation; performs similarity calculation based on the original text embedding representation and the translation database; obtains translation examples close to the requirements based on the similarity calculation, combines the translation examples and the large model, and generates a translation that meets the requirements. Compared with ordinary prompts and prompts that use examples at random, the prompt construction method adopted by the invention can better help the large language model generate translations that meet the requirements, and has a good effect on translation in specific fields and translation stylization. Specific embodiment 2:
[0100] The difference between the second embodiment of the present application and the first embodiment is that:
[0101] Translation data vector database construction:
[0102] Synonymous language pair construction:
[0103] The same language sequence needs to be translated into different styles for different fields. In different fields, individual translators and field-recognized translation materials can, after long-term accumulation, construct translation language pairs with translation habits belonging to the field.
[0104] Specifically, the original text and the translations in different languages are combined into synonymous language pairs, as shown in the following figure
[0105] According to the needs of users, translation databases of different fields or styles can be constructed for subsequent retrieval needs.
[0106] Language sequence segmentation representation:
[0107] The representation of language pairs is very difficult to use directly for information retrieval and cannot be directly calculated. In order to achieve semantic search of the original text, we need to generate vector embeddings for each language sequence and then store them together with the embeddings.
[0108] When generating vector embeddings, this solution uses synonymous language pairs in the field for training to generate an embedding model suitable for the field.
[0109] In order to reduce the length of the language sequence and better obtain its semantics, the original language sequence needs to be segmented.
[0110] The present invention firstly trains all language sequences in a corpus using the BPE word segmentation strategy of a Sentence Piece tool to obtain a vocabulary and a tokenizer for the corpus for word segmentation.
[0111] The present invention uses the obtained vocabulary to segment the language sequences of all translation databases and convert them into token sequence representations.
[0112] Language sequence embedding representation transformation:
[0113] In order to better adapt to the translation characteristics of the translation database, the m-Bert model performs well in semantic understanding and requires very little computing resources for base-level model training. This solution uses the m-Bert model to train the embedding layer.
[0114] The present invention trains m-Bert on the corpus of the translation database. When training m-Bert, other layers are fixed and only the embedding layer is trained, thereby further reducing the computing resources required for training.
[0115] The present invention uses the trained m-Bert model to generate embedded representations, and the Token sequence of the translation database is subjected to Bert-embedding to obtain an embedded sequence representation.
[0116] When storing the above-generated embedding sequence and language sequence, the present invention uses the Weaviate vector database to store the language sequence and its corresponding embedding sequence in pairs.
[0117] Through the above steps, the present invention can construct a translation data vector database suitable for a specific field at a low cost.
[0118] Language pair search and prompt construction use:
[0119] The original text embedding representation is obtained:
[0120] A user's translation requirement can be divided into three pieces of information: translation direction, original text, and translation style requirements. Specifically, the translation style requirements determine the choice of translation database. For example, if you need an ancient translation style or a humorous translation style, you can use the corresponding translation database to retrieve the final translation example.
[0121] The present invention firstly segments the original text to be translated through the previously trained tokenizer to obtain a sequence represented by its tokens.
[0122] The above Token sequence is then embedded in the embedding model to obtain its vector form.
[0123] Similar embedding representation retrieval:
[0124] The present invention calculates the KL divergence between the embedding representation of the Token sequence and the embedding sequence in the translation data vector database, and uses the obtained score as the similarity to obtain all similarity scores.
[0125] The lower the KL divergence, the higher the similarity. Under the two constraints, we find the vector representations of the k most similar language sequences (src1, src2…srck). These language sequences are considered to be the most relevant sentences.
[0126] Constraint 1 (r1) is that the language is consistent with the original text, and constraint 2 (r2) is that the synonymous language pairs of the language sequence contain the required translation direction. The two constraints can be used to obtain the search interval, and then the sequence with the highest similarity can be retrieved.
[0127] The formula for constraint 1 is expressed as:
[0128]
[0129] Where p is the original text, q is the current text in the translation database, and lang(x) is the language of the x sequence.
[0130] The formula of constraint 2 (r2) is expressed as:
[0131]
[0132] Where t is the target language sequence, lang(t) is the target language, and q1…n represents a language sequence group synonymous with q, that is, the translation direction exists in the language sequence group synonymous with the current language sequence.
[0133] The present invention obtains the search interval through constraint 1 (r1) and constraint 2 (r2):
[0134] Q'(q)=R(Q(q),r1,r2)
[0135] Where Q'(q) is the search interval obtained from the initial translation database Q(q) through two constraints.
[0136] For similarity calculation, this scheme uses KL divergence (relative entropy) as the similarity score. The specific calculation formula is as follows:
[0137]
[0138] Where P(x) is the embedded representation of the original text, and Q(x) is the embedded representation of a language sequence in the retrieval space. The smaller the KL divergence, the closer the two representations are. The similarity retrieval process is to search for the k language sequences in the retrieval space with the lowest KL divergence between the language sequence and the original text.
[0139] Prompt combination and use:
[0140] After the search, k demos are obtained, arranged at the beginning of the new prompt, and combined with the original translation requirement query to obtain a new prompt. At this time, the prompt can help the large language model generate a translation that better meets the requirements.
[0141] The combined new prompt is input into the large language model to obtain a translation that better meets the user's needs. Specific embodiment three:
[0143] The difference between the third embodiment of the present application and the second embodiment is that:
[0144] The present invention provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to implement a large model machine translation method based on RAG. Specific embodiment four:
[0146] The difference between the fourth embodiment of the present application and the fifth embodiment is that:
[0147] The present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements a large model machine translation method based on RAG when executing the computer program.
[0148] In the description of this specification, the description of reference terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction. In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. Any process or method description in the flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code including one or more executable instructions for implementing the steps of a custom logic function or process, and the scope of the preferred embodiment of the present invention includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the art to which the embodiments of the present invention belong. The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as a sequenced list of executable instructions for implementing the logic function, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or N wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM).In addition, the computer readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, then editing, interpreting or processing in other suitable ways as necessary, and then storing it in a computer memory. It should be understood that the various parts of the present invention can be implemented with hardware, software, firmware, or a combination thereof. In the above-mentioned embodiment, N steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented with hardware, as in another embodiment, any one of the following technologies known in the art or their combination can be used to implement: a discrete logic circuit with a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit with a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0149] The above is only a preferred implementation of a large model machine translation method based on RAG. The protection scope of a large model machine translation method based on RAG is not limited to the above embodiment. All technical solutions under this idea belong to the protection scope of the present invention. It should be pointed out that for those skilled in the art, several improvements and changes without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
Claims
1. A large model machine translation method based on RAG, characterized by: The method comprises the following steps: Step 1: Segment the original language sequence, transform the language sequence embedding representation, and establish a translation database based on the same language; The step 1 is specifically as follows: Use synonymous language pairs in the field for training to generate an embedding model suitable for the field; The original language sequence is segmented. The BPE segmentation strategy of the Sentence Piece tool is used to train all the language sequences in the corpus to obtain the vocabulary and tokenizer for the corpus for segmentation. Use the obtained vocabulary to segment the language sequences of all translation databases and convert them into token sequence representations; Use the m-Bert model to train the embedding layer: Train m-Bert on the corpus of the translation database. When training m-Bert, fix other layers and only train the embedding layer, further reducing the computing resources required for training. The trained m-Bert model is used to generate embedded representations. The token sequence of the translation database is Bert-embedded to obtain the embedded sequence representation. When storing the above generated embedding sequence and language sequence, the Weaviate vector database is used to store the language sequence and its corresponding embedding sequence in pairs; Step 2: Based on the translation requirements, the original language is split and the original text embedding representation is obtained; The step 2 is specifically as follows: A user's translation requirement is divided into three pieces of information: translation direction, original text, and translation style requirement. The translation style requirement determines the choice of translation database. For example, if an ancient style or a humorous style is required, the corresponding translation database can be used to retrieve the final translation example. First, the original text to be translated is segmented through the previously trained tokenizer to obtain the sequence represented by its tokens; The above token sequence is then embedded through the embedding model to obtain its embedding representation, that is, in vector form; Step 3: Calculate similarity based on the original text embedding representation and the translation database; Step 4: Obtain translation examples that meet the needs based on similarity calculation, and generate translations that meet the needs by combining with the big model.
2. The method according to claim 1, characterized in that: The step 4 is specifically as follows: The KL divergence calculation is performed on the embedding representation of the token sequence and the embedding sequence in the translation data vector database. The obtained score is used as the similarity to obtain all the similarity scores. The lower the KL divergence, the higher the similarity. Under the two constraints, we can find the vector representation of the most similar k language sequences (src1, src2…src k ); Constraint 1 (r1) is that the language is consistent with the original text, and constraint 2 (r2) is that the synonymous language pairs of the language sequence contain the required translation direction. The search interval is obtained by the two constraints, and then the sequence with the highest similarity is searched: The formula for constraint 1 is expressed as: Where p is the original text, q is the current text in the translation database, and lang(x) is the language of the x sequence; The formula of constraint 2 (r2) is expressed as: Where t is the target language sequence, lang(t) is the target language, and q 1…n Indicates a language sequence group synonymous with q, that is, the translation direction exists in the language sequence group synonymous with the current language sequence; Obtain the search interval using constraints 1 (r1) and 2 (r2): Q'(q)=R(Q(q),r1,r2) Where Q'(q) is the search interval obtained from the initial translation database Q(q) through two constraints.
3. The method according to claim 2, characterized in that: For similarity calculation, KL divergence relative entropy is used as the similarity score. The specific calculation formula is as follows: Among them, P(x) is the embedded representation of the original text, Q(x) is the embedded representation of a language sequence in the retrieval space, the smaller the KL divergence, the closer the two representations are. The similarity retrieval process is to search for the k language sequences in the retrieval space with the lowest KL divergence between the language sequence and the original text.
4. The method according to claim 3, characterized in that: After the retrieval, k demos are obtained, arranged at the beginning of the new prompt, and combined with the original translation requirements to obtain a new prompt. At this time, the prompt large language model generates a translation that meets the requirements.
5. A large model machine translation library based on RAG, characterized by: The translation library is formed based on the RAG-based large-model machine translation method described in claim 1, and generates translations that meet the requirements in specific field translation and translation stylization.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 4.
7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.
Citation Information
Patent Citations
Reading understanding intelligent question and answer method based on RPR fusion model
CN114064871A
Corpus classification method and vertical industry machine translation method and device
CN114461799A
Low-resource machine translation method for fusing domain terms by using semi-autoregression
CN114492468A
Low-resource neural machine translation method based on multi-strategy prototype generation
CN114676708A
Text reduction and analysis interface to a text generation modeling system
US11861320B1
