Ancient Chinese understanding method based on large model

Abstract

The invention provides a big language model-based ancient Chinese understanding method, which comprises the following steps of: constructing a historical information knowledge base by utilizing a retrieval enhancement generation technology; in combination with a historical information knowledge base, a linked chain thinking prompt mechanism is designed to guide the big language model to carry out a gradual and explainable reasoning process, and related historical background and word meaning analysis and translation results are obtained; a method of combining comparative learning with knowledge distillation is adopted, the small-parameter model is finely adjusted, and translation and knowledge reasoning ability is transferred to the small-parameter model from the large-parameter model; and introducing a generative evaluation index based on iteration word alignment, and outputting an index score of model translation. Through experiments on a real data set, high-quality historical background analysis and accurate semantic understanding can be generated, and the effectiveness of the method in ancient Chinese education scenes is proved.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing and machine learning. Background Art

[0002] With the acceleration of globalization and the deepening of cultural exchanges, the importance of ancient Chinese as a treasure of Chinese culture has become increasingly prominent. Ancient Chinese is not only an important tool for studying Chinese history, philosophy and literature, but also an important carrier for inheriting Chinese civilization and enhancing cultural confidence. However, there are many challenges in understanding ancient Chinese, such as the scarcity of corpus, complex language structure, changeable word meanings, and profound historical and cultural background, which increase the difficulty of understanding. Therefore, accurately understanding and translating ancient Chinese is of great significance to promoting cultural exchanges and deepening historical research.

[0003] Currently, key tasks for understanding ancient Chinese sentences include word sense disambiguation and text translation. Word sense disambiguation aims to select the most appropriate meaning of a polysemous word based on its context. For text translation, traditional methods use an encoder-decoder approach to achieve literal translation of sentences. However, due to insufficient historical context, traditional models often suffer from semantic inaccuracies. In addition, these models usually require computational coding skills, which limits their accessibility and reduces their applicability to educational scenarios.

[0004] For word sense disambiguation and text translation, the existing technical solutions include the following methods:

[0005] a) In the word sense disambiguation part, it is mainly regarded as a classification problem, using a word embedding method that includes context and word annotations, and using K nearest neighbors to infer the meaning of the word ji;

[0006] b) In the text translation part, it mainly uses a specific pre-trained model and an encoder-decoder framework to complete the text translation task.

[0007] With the emergence of large models, they can improve the interpretability of generated content by providing rich historical context, interacting with users, and enhancing language understanding. However, large models sometimes generate false content, which may mislead users in educational scenarios. In addition, the large model reasoning process requires high computational costs, and its training and deployment also consume a lot of resources.

[0008] On the other hand, evaluating the generative ability of a large model is crucial to measuring its effectiveness. In general, the indicators currently used to evaluate the quality of generated content are usually divided into the following categories:

[0009] a) Word overlap-based evaluation indicators measure the quality by evaluating the word overlap between the translated text and the standard translation;

[0010] b) Based on the evaluation index of word vectors, the text is converted into vector representation and compared in a low-dimensional semantic space;

[0011] c) Language model-based evaluation indicators use the pre-trained model to evaluate the semantic and grammatical consistency between the translated text and the standard translation.

[0012] However, evaluation indicators based on word overlap require translation accuracy, cannot identify synonymous fields, and are highly dependent on external databases; evaluation indicators based on word vectors do not adequately consider the ambiguity of vocabulary and also ignore word order and syntactic structure; although evaluation indicators based on language models alleviate the limitations of the above two methods, they still cannot solve the problem of containing useful but additional information and overly detailed explanations of the original words. Summary of the invention

[0013] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0014] To this end, the purpose of the present invention is to propose an ancient Chinese understanding method based on a large model for ancient Chinese understanding.

[0015] To achieve the above-mentioned purpose, the first embodiment of the present invention proposes a method for understanding ancient Chinese based on a large model, comprising:

[0016] Using search enhancement generation technology, a historical information knowledge base is built, which contains highly reliable historical records and authoritative ancient Chinese vocabulary explanations;

[0017] Combined with the historical information knowledge base, a chain of interlocking thinking prompts is designed to guide the large model to conduct a step-by-step, explainable reasoning process, and obtain relevant historical background and word meaning analysis and translation results;

[0018] The method of contrastive learning combined with knowledge distillation is used to fine-tune the small parameter model and transfer the translation and knowledge reasoning capabilities from the large parameter model to the small parameter model.

[0019] A generative evaluation metric based on iterative word alignment is introduced to output the metric score of the model translation.

[0020] In addition, the ancient Chinese understanding method based on a large model according to the above embodiment of the present invention may also have the following additional technical features:

[0021] Furthermore, in one embodiment of the present invention, the use of search enhancement generation technology to build a historical information knowledge base includes:

[0022] Build a knowledge base of historical information based on ancient historical materials and authoritative Chinese interpretations of key vocabulary.

[0023] Furthermore, in one embodiment of the present invention, the design of an interlocking chain thinking prompt mechanism by combining the historical information knowledge base includes:

[0024] Analyze the relevant historical background of the sentence to be translated, select the best interpretation of each keyword from the provided annotations, and use the historical background information and the best interpretation of the keyword to obtain the literal translation result of the sentence to be translated.

[0025] Furthermore, in one embodiment of the present invention, the method of combining contrastive learning with knowledge distillation is used to fine-tune the small parameter model, and transfer the translation and knowledge reasoning capabilities from the large parameter model to the small parameter model:

[0026] Use the standard translation as the output label for training, and optimize the negative log-likelihood of the conditional language modeling objective to make the model generation close to the standard translation;

[0027] The original translation generated by the large model is used as a negative sample, and the standard translation is used as a positive sample. The language model is adjusted through the contrast loss function to learn the translation style of ancient Chinese language texts.

[0028] Furthermore, in one embodiment of the present invention, the introduction of a generative evaluation index based on iterative word alignment and the output of an index score of model translation include:

[0029] Preprocess the model translation and standard translation, use the word alignment self-labeling method, and iterate multiple times to obtain high-precision self-labeled alignment labels;

[0030] According to the self-labeled alignment labels, an optimized BLEU index and an optimized ROUGE index are calculated.

[0031] Further, in one embodiment of the present invention, the optimized BLEU index is expressed as follows:

[0032]

[0033]

[0034]

[0035] in, Represents the penalty factor for sentence length, which penalizes sentences that are too short (incomplete translation) and too long (too many translations). Represents the alignment label Statistical participle exist The number of times it appears in .

[0036] Furthermore, in one embodiment of the present invention, the optimized ROUGE index is expressed as follows:

[0037]

[0038] in, Represents the penalty factor for word segmentation length. Represents the alignment label Statistical participle and participle The number of co-occurrences of n-grams (n-tuples).

[0039] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes an ancient Chinese comprehension device based on a large model, comprising the following modules:

[0040] A historical information base module, which builds a historical information knowledge base, which contains highly reliable historical records and authoritative ancient Chinese vocabulary annotations;

[0041] Chain thinking prompt module, which designs an interlocking chain thinking prompt mechanism to guide the large model to conduct a step-by-step and explainable reasoning process to obtain relevant historical background and word meaning analysis and translation results;

[0042] The contrastive knowledge distillation module uses contrastive learning combined with knowledge distillation to fine-tune the small-parameter model and transfer the translation and knowledge reasoning capabilities from the large-parameter model to the small-parameter model.

[0043] A generative evaluation module that outputs metric scores for model translation.

[0044] The ancient Chinese understanding method based on a large model proposed in the embodiment of the present invention uses retrieval enhancement generation technology to build a historical information knowledge base, which contains high-reliability historical corpus records and authoritative ancient Chinese vocabulary annotations; combined with the historical information knowledge base, a chain-like thinking prompt mechanism is designed to guide the large model to perform a step-by-step, explainable reasoning process to obtain relevant historical background and word meaning analysis and translation results; a method of contrastive learning combined with knowledge distillation is used to fine-tune the small parameter model to transfer the translation and knowledge reasoning capabilities from the large parameter model to the small parameter model; a generative evaluation index based on iterative word alignment is introduced to output the index score of the model translation. This application uses relevant knowledge and technologies in the fields of natural language processing and machine learning to process ancient Chinese texts, deeply mine the historical knowledge and semantic information contained in ancient Chinese texts, effectively improve the accuracy and explainability of ancient Chinese understanding, and finally form an ancient Chinese learning system that can be used for education, providing certain help for understanding and inheriting Chinese culture. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0046] Figure 1 A flowchart of a method for understanding ancient Chinese based on a large model provided in an embodiment of the present invention.

[0047] Figure 2 A flow chart of the large model-assisted ancient Chinese comprehension provided by an embodiment of the present invention.

[0048] Figure 3 A schematic diagram of the structure of an ancient Chinese comprehension device based on a large model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0050] The following describes an ancient Chinese understanding method based on a large model according to an embodiment of the present invention with reference to the accompanying drawings.

[0051] Figure 1 A schematic flow chart of a method for understanding ancient Chinese based on a large model provided in an embodiment of the present invention.

[0052] like Figure 1 As shown, the ancient Chinese understanding method based on the large model includes the following steps:

[0053] S101: Using retrieval enhancement generation technology, building a historical information knowledge base, the historical information knowledge base contains high-reliability historical records and authoritative ancient Chinese vocabulary annotations;

[0054] Furthermore, in one embodiment of the present invention, a historical information knowledge base is constructed using retrieval enhancement generation technology, including:

[0055] Build a knowledge base of historical information based on ancient historical materials and authoritative Chinese interpretations of key vocabulary.

[0056] like Figure 2 As shown in the figure, the construction of the historical information knowledge base mainly refers to the existing external professional knowledge. Its content includes the complete ancient version of the "Twenty-Four Histories" and related novels, poems and other text materials, supplemented by a total of 169,742 words with a frequency greater than 5 in the ancient Chinese corpus and their authoritative Chinese explanations, to build a database supported by Milvus.

[0057] S102: Combined with the historical information knowledge base, a chain of interlocking thinking prompts is designed to guide the large model to conduct a step-by-step, explainable reasoning process to obtain relevant historical background and word meaning analysis and translation results;

[0058] Furthermore, in one embodiment of the present invention, a chain-type thinking prompt mechanism is designed in conjunction with the historical information knowledge base, including:

[0059] Analyze the relevant historical background of the sentence to be translated, select the best interpretation of each keyword from the provided annotations, and use the historical background information and the best interpretation of the keyword to obtain the literal translation result of the sentence to be translated.

[0060] S103: Use contrastive learning combined with knowledge distillation to fine-tune the small parameter model and transfer the translation and knowledge reasoning capabilities from the large parameter model to the small parameter model;

[0061] Furthermore, in one embodiment of the present invention, a method combining contrastive learning and knowledge distillation is used to fine-tune the small parameter model, transferring the translation and knowledge reasoning capabilities from the large parameter model to the small parameter model:

[0062] Use the standard translation as the output label for training, and optimize the negative log-likelihood of the conditional language modeling objective to make the model generation close to the standard translation;

[0063] The original translation generated by the large model is used as a negative sample, and the standard translation is used as a positive sample. The language model is adjusted through the contrast loss function to learn the translation style of ancient Chinese language texts.

[0064] like Figure 2 As shown, the purpose of fine-tuning the small parameter model is to make the translation output of the small parameter model closer to the standard translation. The LoRA technology is used. The fine-tuning process includes 10 epochs, the batch size is 64, the maximum input length is 1,024, the output length is 512, and the learning rate is 5e-5.

[0065] S104: Introduce a generative evaluation metric based on iterative word alignment to output the metric score of the model translation;

[0066] Furthermore, in one embodiment of the present invention, a generative evaluation index is introduced to output an index score of model translation, including:

[0067] Preprocess the model translation and standard translation, use the word alignment self-labeling method, and iterate multiple times to obtain high-precision self-labeled alignment labels;

[0068] According to the self-labeled alignment labels, an optimized BLEU index and an optimized ROUGE index are calculated.

[0069] Furthermore, in one embodiment of the present invention, the BLEU index is optimized, as shown below:

[0070]

[0071]

[0072]

[0073] in, Represents the penalty factor for sentence length, which penalizes sentences that are too short (incomplete translation) and too long (too many translations). Represents the alignment label Statistical participle exist The number of times it appears in .

[0074]

[0075] in, Represents the penalty factor for word segmentation length. Represents the alignment label Statistical participle and participle The number of co-occurrences of n-grams (n-tuples).

[0076] The ancient Chinese understanding method based on a large model proposed in the embodiment of the present invention uses retrieval enhancement generation technology to build a historical information knowledge base, which contains high-reliability historical records and authoritative ancient Chinese vocabulary annotations; combined with the historical information knowledge base, an interlocking chain thinking prompt mechanism is designed to guide the large model to perform a step-by-step, explainable reasoning process to obtain relevant historical background and word meaning analysis and translation results; a method of combining contrastive learning with knowledge distillation is used to fine-tune the small parameter model to transfer the translation and knowledge reasoning capabilities from the large parameter model to the small parameter model; a generative evaluation index based on iterative word alignment is introduced to output the index score of the model translation. This method makes full use of the knowledge of the large model, realizes in-depth Chinese understanding, and effectively improves the effect of ancient Chinese translation.

[0077] Compared with the prior art, the advantages of the present invention are:

[0078] 1) Compared with the traditional ancient Chinese text understanding method that handles word sense disambiguation and text translation tasks independently, the innovation of the present invention is that it fully considers the complexity of ancient Chinese and the importance of historical background, combines retrieval enhancement generation technology, builds a historical information knowledge base, and provides a unified and more comprehensive and in-depth ancient Chinese text understanding solution;

[0079] 2) Combining the historical information knowledge base and the knowledge adaptive distillation method, a step-by-step chain thinking prompt mechanism is designed to effectively transfer the knowledge and reasoning ability of the large language model to the small student model, significantly reducing the problems of hallucination generation and over-free translation;

[0080] 3) This invention greatly improves the translation quality of ancient Chinese texts, and outperforms the algorithms proposed by most current solutions on the largest ancient Chinese translation dataset Erya;

[0081] 4) The technical solution of the present invention can be run in various ancient Chinese education systems, providing more accurate historical background and semantic analysis for ancient Chinese learning, and has considerable educational application value;

[0082] 5) The historical and semantic information obtained by the present invention can also assist in fields such as historical research and cultural heritage, and help solve related historical understanding problems.

[0083] In order to implement the above-mentioned embodiment, the present invention also proposes an ancient Chinese understanding device based on a large model.

[0084] Figure 3 A schematic structural diagram of an ancient Chinese comprehension device based on a large model provided in an embodiment of the present invention.

[0085] like Figure 3 As shown, the ancient Chinese comprehension device based on the large model includes: a historical information library module 100, a chain thinking prompt module 200, a comparative knowledge distillation module 300, and a generative evaluation module 400, wherein:

[0086] A historical information base module, which builds a historical information knowledge base, which contains highly reliable historical records and authoritative ancient Chinese vocabulary annotations;

[0087] Chain thinking prompt module, which designs an interlocking chain thinking prompt mechanism to guide the large model to conduct a step-by-step and explainable reasoning process to obtain relevant historical background and word meaning analysis and translation results;

[0088] The contrastive knowledge distillation module uses contrastive learning combined with knowledge distillation to fine-tune the small-parameter model and transfer the translation and knowledge reasoning capabilities from the large-parameter model to the small-parameter model.

[0089] A generative evaluation module that outputs metric scores for model translation.

[0090] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0091] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0092] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for understanding ancient Chinese based on a large model, characterized in that: The following steps are involved: Using search enhancement generation technology, a historical information knowledge base is built, which contains highly reliable historical records and authoritative ancient Chinese vocabulary annotations; Combined with the historical information knowledge base, a chain of interlocking thinking prompts is designed to guide the large model to conduct a step-by-step, explainable reasoning process, and obtain relevant historical background and word meaning analysis and translation results; The method of contrastive learning combined with knowledge distillation is used to fine-tune the small parameter model and transfer the translation and knowledge reasoning capabilities from the large parameter model to the small parameter model. A generative evaluation metric based on iterative word alignment is introduced to output the metric score of the model translation.

2. The method according to claim 1, characterized in that The use of retrieval enhancement generation technology to build a historical information knowledge base includes: Build a knowledge base of historical information based on ancient historical materials and authoritative Chinese interpretations of key vocabulary.

3. The method according to claim 1, characterized in that The above-mentioned combination of historical information knowledge base and the design of an interlocking chain thinking prompt mechanism includes: Analyze the relevant historical background of the sentence to be translated, select the best interpretation of each keyword from the provided annotations, and use the historical background information and the best interpretation of the keyword to obtain the literal translation result of the sentence to be translated.

4. The method according to claim 1, characterized in that: The method of combining contrastive learning with knowledge distillation is used to fine-tune the small parameter model and transfer the translation and knowledge reasoning capabilities from the large parameter model to the small parameter model: Use the standard translation as the output label for training, and optimize the negative log-likelihood of the conditional language modeling objective to make the model generation close to the standard translation; The original translation generated by the large parameter model is used as a negative sample, and the standard translation is used as a positive sample. The language model is adjusted through the contrast loss function to learn the translation style of ancient Chinese language texts.

5. The method according to claim 1, characterized in that The generative evaluation indicators output the indicator scores of the model translation, including: Preprocess the model translation and standard translation, use the word alignment self-labeling method, and iterate multiple times to obtain high-precision self-labeled alignment labels; According to the self-labeled alignment labels, an optimized BLEU index and an optimized ROUGE index are calculated.

6. The method according to claim 5, characterized in that The optimized BLEU index is expressed as follows: in, Represents the penalty factor for sentence length, which penalizes sentences that are too short (incomplete translation) and too long (too many translations). Represents the alignment label Statistical participle exist The number of times it appears in .

7. The method according to claim 5, characterized in that The optimized ROUGE indicator is expressed as follows: in, Represents the penalty factor for word segmentation length. Represents the alignment label Statistical participle and participle The number of co-occurrences of n-grams (n-tuples).

8. A method for understanding ancient Chinese based on a large model, characterized in that: Includes the following modules: A historical information base module, which builds a historical information knowledge base, which contains highly reliable historical records and authoritative ancient Chinese vocabulary annotations; Chain thinking prompt module, which designs an interlocking chain thinking prompt mechanism to guide the large model to conduct a step-by-step and explainable reasoning process to obtain relevant historical background and word meaning analysis and translation results; The contrastive knowledge distillation module uses contrastive learning combined with knowledge distillation to fine-tune the small-parameter model and transfer the translation and knowledge reasoning capabilities from the large-parameter model to the small-parameter model. A generative evaluation module that outputs metric scores for model translation.

Images