A method for fine-tuning and evaluating large language models based on retrieval enhancement generation

Abstract

The present disclosure relates to the technical field of artificial intelligence, and provides a large language model fine-tuning and evaluation method based on retrieval enhancement generation, which comprises: cutting the input original text document content into multiple independent paragraphs according to a preset character level cutting rule; extracting entities and inter-entity relationships from the multiple independent paragraphs to construct a knowledge graph; based on the knowledge graph, generating noise data, complex data, rejection data and multi-hop data through a noise sampling module, a complex attribution module, a rejection analysis module and a thinking reasoning module respectively; performing format conversion on the noise data, the complex data, the rejection data, the multi-hop data and general data to obtain fine-tuning data; fine-tuning a large language model according to the fine-tuning data to obtain a fine-tuned retrieval enhancement generation model; and evaluating the fine-tuned retrieval enhancement generation model according to an answer correlation index and an answer similarity index, thereby improving the accuracy, reliability and reasoning ability of the retrieval enhancement generation model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method for fine-tuning and evaluating a large language model based on retrieval enhancement generation. Background Technology

[0002] Large language models, as a type of language model based on artificial neural networks, typically contain billions or even more parameters and are trained on massive amounts of unlabeled text data through self-supervised or semi-supervised learning methods. They have demonstrated powerful capabilities in multiple fields such as natural language processing. However, in practical applications, users can directly perceive three core problems with this type of model: first, bias, where the model output may implicitly contain a tendency to be unobjective and unfair; second, illusion, i.e., generating false information that does not conform to the facts; and third, obsolescence, because the training data has time limitations, making it difficult to cope with emerging knowledge or dynamically changing information needs.

[0003] To address the aforementioned issues, Retrieval-Augmented Generation (RAG) technology has emerged. Currently, among techniques combining RAG with large language models, there is no fine-tuning technique for large language models based on RAG. Some studies optimize the question-answering performance of large language models by controlling the number of recalled documents. While this can alleviate bias, illusion, and obsolescence issues in large language models to some extent, it cannot fundamentally solve the core problems mentioned above. Therefore, existing RAG-based question-answering optimization techniques for large language models still face significant bottlenecks. Summary of the Invention

[0004] This disclosure provides a method for fine-tuning and evaluating a large language model based on retrieval-enhanced generation. Through refined data processing, diverse fine-tuning data construction, efficient model fine-tuning, and a scientific evaluation system, it comprehensively improves the accuracy, robustness, reliability, and reasoning ability of the retrieval-enhanced generation model, enabling it to better cope with complex and diverse question-answering scenarios in practical applications.

[0005] According to a first aspect of this disclosure, a method for fine-tuning and evaluating a large language model based on retrieval enhancement generation is provided, including:

[0006] The input original text document content is divided into multiple independent paragraphs according to the preset character-level segmentation rules;

[0007] Entities and relationships between entities are extracted from the multiple independent paragraphs to construct a knowledge graph;

[0008] Based on the knowledge graph, noisy data, complex data, rejection data, and multi-hop data are generated through the noise sampling module, complex attribution module, rejection analysis module, and reasoning module, respectively.

[0009] The noisy data, complex data, rejection data, multi-hop data, and general data are converted into different formats to obtain fine-tuning data. The general data is the question-and-answer data used for pre-training the open-source model.

[0010] The large language model is fine-tuned based on the fine-tuning data to obtain the fine-tuned retrieval enhancement generative model;

[0011] The fine-tuned retrieval enhancement generation model was evaluated based on answer relevance and answer similarity metrics.

[0012] In a preferred embodiment, the step of dividing the input original text document content into multiple independent paragraphs according to a preset character-level segmentation rule includes:

[0013] According to the preset character-level segmentation rules, the original text is segmented based on the preset number of characters and commonly used punctuation marks;

[0014] Perform semantic integrity checks on the independent paragraphs formed after segmentation;

[0015] If a paragraph ends with an incomplete sentence, then that paragraph will be merged into the next paragraph and re-segmented.

[0016] If the last independent paragraph is an incomplete sentence, then it is merged into the previous independent paragraph.

[0017] In a preferred embodiment, the step of extracting entities and relationships between entities from the independent paragraphs and constructing a knowledge graph includes:

[0018] Using a large language model, the system parses entities and relationships between entities in a paragraph based on a preset prompt word template.

[0019] The extracted entities and relationships between entities are mapped to nodes and edges in a knowledge graph to construct the knowledge graph.

[0020] In a preferred embodiment, the preset prompt word template includes a target instruction and step-by-step parsing rules, wherein the step-by-step parsing rules include:

[0021] Identify all entities and extract entity information, including entity name, entity type, and entity description;

[0022] Identify related entity pairs from the identified entities and extract relationship information between entities, including source entity name, target entity name, relationship description, and relationship strength.

[0023] In a preferred implementation, the extracted entities and relationships between entities are mapped to nodes and edges in a knowledge graph to construct the knowledge graph, including:

[0024] The entity name is used as the unique identifier of the node, the entity type is used as the node category label, and the entity description is used as the node attribute.

[0025] The source entity name and target entity name are respectively assigned to the start node and end node of the edge, the relationship description is used as the edge type, and the relationship strength is used as the edge weight.

[0026] The mapped nodes and edges are stored in triple format, and the node attributes and edge weights are associated to form an extended triple structure, thus constructing a knowledge graph.

[0027] In a preferred embodiment, based on the knowledge graph, noisy data, complex data, rejection data, and multi-hop data are generated through a noise sampling module, a complex attribution module, a rejection analysis module, and a reasoning module, respectively, including:

[0028] A first simple question-and-answer pair is constructed based on entities and relationships between entities in the knowledge graph. Paragraph information related to the first simple question-and-answer pair is selected from the set of independent paragraphs and added to the first simple question-and-answer pair as noise information according to a preset ratio to form noise data.

[0029] M entities with similar attributes or relationships are selected from the knowledge graph to form an entity cluster. Complex question-answer pairs are constructed based on the entity cluster to form complex data.

[0030] A second simple question-and-answer pair is constructed based on entities in the knowledge graph and absolute paragraphs containing explicit answers. Paragraphs other than the absolute paragraphs are selected from the set of independent paragraphs as relevant texts for the second question-and-answer pair, forming rejection data.

[0031] In the knowledge graph, bridging entities are filtered out, and candidate paragraph pairs containing three entity relationships are generated based on the bridging entities and associated entities to construct multi-hop question-answer pairs, forming multi-hop data. The bridging entity is associated with at least two other entities at the same time, and there is no direct relationship between the associated entities.

[0032] In a preferred embodiment, the noise data, complex data, rejection data, multi-hop data, and general data are format-converted to obtain fine-tuned data, including:

[0033] Call the triplet data structure that contains a task description field, a context information field, and a target output field;

[0034] The noisy data, complex data, rejected data, multi-hop data, and general data are mapped to the triplet data structure according to the preset mapping structure;

[0035] The mapped triplet data is normalized to obtain fine-tuning data that meets the input requirements of the model fine-tuning framework.

[0036] In a preferred embodiment, mapping the noisy data, complex data, rejected data, multi-hop data, and general data to the triplet data structure according to a preset mapping structure includes:

[0037] The original questions of the noisy data, complex data, and multi-hop data are mapped to the task description field, the associated background text is mapped to the context information field, and the deduced standard answer is mapped to the target output field.

[0038] The original question of the rejected data is mapped to the task description field, the interference text without the correct answer is mapped to the context information field, and the preset knowledge rejection label is mapped to the target output field;

[0039] The original task description of the general data is mapped to the task description field, the original input information is mapped to the context information field, and the original output result is mapped to the target output field.

[0040] In a preferred embodiment, the evaluation of the fine-tuned retrieval enhancement generation model based on answer relevance and answer similarity indicators includes:

[0041] Based on the generated answer, at least one related question is derived in reverse, and the related question is combined with the original input question to form a question set;

[0042] Calculate the semantic relevance score between each question in the question set, as an indicator of answer relevance;

[0043] The generated answers are compared with the standard answers for semantic equivalence to obtain an answer similarity index.

[0044] As a preferred embodiment, the formula for calculating the answer relevance index is:

[0045] ;

[0046] in, It is the generated first Each question corresponds to a vector. It is the vector corresponding to the user's question. It represents the number of problems generated.

[0047] According to a second aspect of this disclosure, a device for fine-tuning and evaluating a large language model based on retrieval enhancement generation is provided, comprising:

[0048] The text segmentation module is used to segment the input raw text document content into multiple independent paragraphs according to preset character-level segmentation rules;

[0049] The knowledge graph construction module is used to extract entities and relationships between entities from the multiple independent paragraphs and construct a knowledge graph.

[0050] The feature data generation module is used to generate noisy data, complex data, rejection data, and multi-hop data based on the knowledge graph through the noise sampling module, complex attribution module, rejection analysis module, and reasoning module, respectively.

[0051] The data conversion module is used to convert the noisy data, complex data, rejection data, multi-hop data and general data into formats to obtain fine-tuning data, wherein the general data is the question-and-answer data used for pre-training of the open-source model;

[0052] The large language model fine-tuning module is used to fine-tune the large language model based on the fine-tuning data to obtain the fine-tuned retrieval enhancement generation model;

[0053] The large language model evaluation module is used to evaluate the fine-tuned retrieval enhancement generation model based on answer relevance and answer similarity metrics.

[0054] According to a third aspect of this disclosure, an electronic device is provided, comprising at least one processor and a memory communicatively connected to said at least one processor; wherein the memory stores a computer program executable by said at least one processor, said computer program being executed by said at least one processor to enable said at least one processor to perform the method as described in any of the preceding claims.

[0055] Compared with the prior art, this disclosure achieves the following beneficial effects:

[0056] (1) The original text is segmented by a preset character-level segmentation rule, and paragraphs are adjusted by combining semantic integrity detection to ensure the semantic integrity of the paragraphs, thus laying a high-quality data foundation for subsequent entity and relation extraction. This segmentation method avoids semantic loss caused by paragraph breakage, making the extraction of entities and relationships between entities from paragraphs more accurate. The knowledge graph constructed is also more in line with the real semantics of the text, providing reliable knowledge support for model training.

[0057] (2) Noisy data, complex data, rejected data, and multi-hop data were generated based on the knowledge graph, and combined with general data to form fine-tuning data. The diverse data types can specifically improve the model's capabilities in different scenarios: noisy data can enhance the model's ability to resist interference from interfering information, enabling it to accurately focus on the core question even when irrelevant information is present; complex data helps the model handle questions involving multiple similar entities or complex relationships, improving its ability to cope with complex tasks; the introduction of rejected data allows the model to learn to identify unanswerable questions, reducing the occurrence of fabricated answers and enhancing the reliability of the model's output; multi-hop data can train the model's logical reasoning ability, enabling it to handle questions that require multiple derivations. The integration of general data ensures that the model has the ability to handle specific scenarios while not losing its general question-answering capabilities, achieving a balance between specialized and general capabilities.

[0058] (3) By converting multiple types of data into triplet data structures and standardizing them through unified format conversion, data of different types and sources can be adapted to the model fine-tuning framework, which improves the efficiency and convenience of the fine-tuning process and enhances the fine-tuning effect.

[0059] (4) The model after fine-tuning is evaluated by combining the answer relevance index and the answer similarity index, which is more comprehensive and scientific than a single index. The answer relevance index ensures that the answer generated by the model is closely related to the question and avoids answering the wrong question; the answer similarity index measures the degree of consistency between the generated answer and the standard answer. The combination of the two can more accurately reflect the quality of the answer generated by the model and provide a reliable basis for evaluating the model performance.

[0060] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0061] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0062] Figure 1 A flowchart illustrating a method for fine-tuning and evaluating a large language model based on retrieval enhancement generation, according to an embodiment of this disclosure, is shown.

[0063] Figure 2 A flowchart illustrating the fine-tuning process of a large language model based on retrieval enhancement generation, according to an embodiment of this disclosure, is shown.

[0064] Figure 3A block diagram of a large language model fine-tuning and evaluation apparatus based on retrieval enhancement generation according to an embodiment of the present disclosure is shown;

[0065] Figure 4 A schematic diagram of an exemplary electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0067] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0068] Example 1

[0069] Combination Figure 1 and Figure 2 This disclosure provides an explanation of a method for fine-tuning and evaluating a large language model based on retrieval enhancement generation, according to an embodiment of the present disclosure.

[0070] like Figure 1 The diagram shows a flowchart of a method for fine-tuning and evaluating a large language model based on retrieval enhancement generation, according to an embodiment of this disclosure. The method 100 includes:

[0071] S110: Divide the input original text document content into multiple independent paragraphs according to the preset character-level segmentation rules.

[0072] In some embodiments, the original text is segmented according to a preset character-level segmentation rule, based on a preset number of characters and commonly used punctuation marks;

[0073] The preset character-level segmentation rules include a segmentation base and a dynamic adjustment mechanism: First, the original text is initially segmented based on a preset number of characters (such as 500-800 characters) and commonly used punctuation marks (period, question mark, exclamation mark, semicolon).

[0074] Secondly, in order to ensure the semantic integrity of the paragraphs and lay a high-quality data foundation for subsequent entity and relation extraction, the segmented paragraphs are further detected by a semantic integrity detection model (a pre-trained text understanding model can be used) to identify whether the end of the paragraph is a complete sentence (the judgment criteria are whether the punctuation at the end is a sentence-ending punctuation and whether the sentence components are complete).

[0075] If an independent paragraph ends with an incomplete sentence (such as ending with a comma, pause mark, or missing predicate), then that independent paragraph is merged into the next independent paragraph and re-segmented; if the last independent paragraph ends with an incomplete sentence, then it is merged into the previous independent paragraph.

[0076] This embodiment uses a process of "preliminary segmentation - semantic verification - dynamic merging" to ensure that all independent paragraphs have complete semantics, providing a coherent contextual basis for subsequent entity extraction.

[0077] S120: Extract entities and relationships between entities from the multiple independent paragraphs to construct a knowledge graph.

[0078] In some embodiments, entities and relationships between entities are extracted from multiple independent paragraphs to construct a knowledge graph using a two-level processing approach of "large language model parsing + structured mapping":

[0079] First, a large language model (such as GPT-4, LLaMA series) is used to parse the entities and relationships between entities in the paragraph based on a preset prompt word template. The preset prompt word template includes a clear task objective, a list of entity types (such as people, organizations, events, etc.), examples of relationship types (such as membership, causality, chronology, etc.), and output format constraints.

[0080] Among them, parsing entities and relationships between entities in a paragraph based on a preset prompt word template includes:

[0081] First, identify all entities and extract entity information (duplicates need to be removed, and entities with the same name but different characteristics are marked with a distinguisher). Entity information includes entity name, entity type, and entity description.

[0082] Secondly, related entity pairs are identified from the identified entities, and relationship information between entities is extracted. The relationship information includes the source entity name (associated unique entity identifier), the target entity name (associated unique entity identifier), relationship description, and relationship strength.

[0083] It is important to note that the identification of related entity pairs can be based on co-occurrence frequency and semantic relevance. Entity pairs with a relevance below a threshold are marked as "weakly related," while those with a relevance above the threshold are marked as "strongly related."

[0084] For example, the prompt word template is as follows:

[0085] Objective: Given a text document that may be related to the activity and a set of entity types, identify all of these entity types from the text and all relationships between the identified entities.

[0086] step:

[0087] (1) Identify all entities. For each identified entity, extract the following information:

[0088] entity_name: Entity name, first letter capitalized;

[0089] entity_type: One of the following types: [{entity_types}];

[0090] entity_description: A comprehensive description of the entity's attributes and activities, formatted as ("entity"{tuple_delimiter}).<entity_name> {tuple_delimiter}<entity_type> {tuple_delimiter}<entity_description> );

[0091] (2) From the entities identified in step (1), identify all (source entity, target entity) pairs that are clearly related to each other. For each pair of related entities, extract the following information:

[0092] source_entity: The name of the source entity, as identified in step (1);

[0093] target_entity: The name of the target entity;

[0094] relationship_description: Explains why the source entity and the target entity are considered to be related;

[0095] relationship_strength: A numerical score representing the strength of the relationship between the source and target entities. Each relationship is formatted as follows:

[0096] ("relationship"{tuple_delimiter}<source_entity> {tuple_delimiter}<target_entity> {tuple_delimiter}<relationship_description> {tuple_delimiter}<relationship_strength> ).

[0097] (3) The English function returns a single list containing all entities and relationships identified in steps (1) and (2), using **{record_delimiter}** as the list delimiter.

[0098] (4) After completion, output {completion_delimiter}.

[0099] Furthermore, the extracted entities and relationships between entities are mapped to nodes and edges in the knowledge graph to construct the knowledge graph, including:

[0100] (a) Node mapping rules:

[0101] The identified entity name is used as the unique identifier of the node. If there are duplicate entity names but different entity types or entity descriptions, a unique identifier (such as entity type abbreviation + serial number, for example, "Beijing_City_001" or "Beijing_Company_002") is added after the entity name to distinguish them.

[0102] Entity type serves as a node classification label. A pre-defined type hierarchy system is used to standardize entity types, ensuring that entities of the same category correspond to the same label.

[0103] Entity descriptions are stored as node attributes in key-value pairs (e.g., {"Date of Birth": "1955", "Nationality": "United States"}). The description content is deduplicated and noise-reduced, retaining key information and supporting the associative storage of multilingual descriptions.

[0104] (b) Edge mapping rules:

[0105] The identified source entity name and target entity name are respectively mapped to the start node and end node of the edge. If the entity name is duplicated, its unique identifier is associated to ensure the accuracy of the edge pointing.

[0106] The relation description is used as the edge type. A mapping table between relation descriptions and standard relation types is established, and non-standard relation descriptions are normalized.

[0107] The relationship strength is used as the weight of the edge, and is represented by a value range of 0-1. It is calculated by quantitative indicators such as entity co-occurrence frequency and relationship mention frequency. The weight value is retained to two decimal places. The higher the weight value, the closer the relationship.

[0108] (c) Knowledge graph storage and optimization:

[0109] The mapped nodes and edges are stored in the format of triples (head entity, relation type, tail entity), and the node attributes and edge weight information are associated to form an extended triple structure.

[0110] Redundancy detection is performed on the constructed knowledge graph, and duplicate nodes, edges, and triples are deleted.

[0111] Perform consistency checks to ensure that the mapping logic between entities and relationships is consistent. If conflicting relationships exist, mark the conflict information and retain the original data source for manual verification.

[0112] When a new entity or relation is added, it is expanded according to the above mapping rules, and the triplet storage content is updated.

[0113] S130: Based on the knowledge graph, noise data, complex data, rejection data, and multi-hop data are generated through the noise sampling module, complex attribution module, rejection analysis module, and reasoning module, respectively.

[0114] In some embodiments, based on the knowledge graph, noisy data, complex data, rejection data, and multi-hop data are generated through a data feature module, wherein the data feature module includes four major modules: noise sampling, complex attribution, rejection analysis, and reasoning.

[0115] Since entities and their relationships are already acquired during the knowledge graph construction, the noise sampling module can construct a first simple question-and-answer pair (e.g., "What is the relationship between entity A and entity B?") based on the entities and their relationships in the knowledge graph. Then, it selects paragraphs with low relevance from the set of independent paragraphs using semantic similarity calculation (e.g., cosine similarity greater than or equal to 0.3) as noise information. These paragraphs are then combined with the question-and-answer pair at a preset ratio (e.g., noise accounting for 30%-50%) to form noisy data containing interfering information. For example, for the question "What theory did Einstein propose?" + the answer "relativity", paragraphs with low relevance such as "Einstein studied unified field theory in his later years" and "Newton discovered universal gravitation" are added as noise.

[0116] The complex attribution module requires manually identifying M (e.g., 20-30) similar entities from the knowledge graph, covering at least 5 entity categories. For each entity cluster, it generates complex question-and-answer pairs (e.g., "What are the main differences between Apple and Microsoft in their product ecosystems?"), utilizing related paragraphs within the entity cluster to construct context and enhance the model's ability to analyze complex questions.

[0117] The rejection analysis module, similar to the noise sampling module, constructs a second simple question-and-answer pair using known entities and absolute paragraphs containing explicit answers. Apart from absolute paragraphs, other noise paragraphs have no positive effect on the actual question and answer. This module only uses noise paragraphs as relevant text, ensuring that the model only answers known information and refuses to answer unknown questions.

[0118] The reasoning module is used to associate entity c with both entities a and b in a knowledge graph. c can be considered a bridging entity between entities a and b. Filtering these bridging entities can generate candidate paragraph pairs. For example, to find the question "When was the singer of 'Orchid Grass' born?", you first need to determine who the singer of "Orchid Grass" is, and then find their birthday. This module often contains relationships between three entities, providing potential clues for reasoning.

[0119] In summary, the data feature module can generate noisy data, complex data, rejected data, and multi-hop data to obtain question-and-answer data that meets the requirements of knowledge retrieval enhancement.

[0120] For example, questions in knowledge retrieval-enhanced question-answering data should conform to the following format:

[0121] {

[0122] --- Role---

[0123] You are a helpful assistant, responsible for answering questions about the data provided in the information.

[0124] --- Target---

[0125] Generate a response that meets the target length and format to answer the user's question, summarizing all relevant information from the input data to fit the response's length and format.

[0126] You should use the data provided below as the primary context for generating your response.

[0127] If you don't know the answer, or if the data table doesn't provide enough information to provide an answer, please state it directly. Do not make anything up.

[0128] The key points supported by the data should be listed in the following manner, along with their data references:

[0129] {Reference Example}

[0130] ---Target Response Length and Format---

[0131] {Example Answer}

[0132] ---Reference Information---

[0133] {Reference Content}

[0134] }

[0135] S140: Convert the format of the noisy data, complex data, rejection data, multi-hop data and general data to obtain fine-tuning data, wherein the general data is the question-and-answer data used for pre-training the open-source model.

[0136] In some embodiments, to ensure the model's general question-answering capability, general data is still required in addition to noisy data, complex data, rejected data, and multi-hop data. The general data is the question-answering data used for pre-training the open-source model.

[0137] In addition, since the question-and-answer data format differs significantly from the format required for model fine-tuning, it is necessary to perform format adaptation and conversion through the data preprocessing module. The specific steps of format conversion include:

[0138] First, a triplet data structure containing "instruction" (task description field), "input" (context information field), and "output" (target output field) is invoked to adapt to the input format of mainstream fine-tuning frameworks (such as LLaMA Factory and Hugging Face Transformers); according to the preset mapping structure, the noisy data, complex data, rejected data, multi-hop data, and general data are mapped to the triplet data structure.

[0139] Specifically, the mapping rules are refined as follows:

[0140] For noisy, complex, and multi-hop data: the original question is mapped to the "instruction" field, the associated background text is mapped to the "input" field, and the derived standard answer is mapped to the "output" field; for rejected data, the original question is mapped to the "instruction" field, the interference text without a correct answer is mapped to the "input" field, and the preset knowledge rejection flag is mapped to the "output" field; for general data, the original task description is mapped to the "instruction" field, the original input information is mapped to the "input" field, and the original output result is mapped to the "output" field.

[0141] Finally, the mapped triplet data is normalized to obtain fine-tuning data that meets the input requirements of the model fine-tuning framework.

[0142] Example fine-tuning data:

[0143] { “instruction”: “Identify and explain two scientific theories from the given list: cell theory and heliocentrism.”,

[0144] "input": ",

[0145] "output": "The cell theory is a theory in biological science that posits that all living organisms are composed of tiny basic units—cells. It is a fundamental theory of biology, holding that the cell is the basic structural and functional unit of all living things, and that all organisms are composed of one or more cells, with cells only producing new cells through cell division. This theory was first proposed in 1839 by Schrödinger, Schwann, and Shelley. The heliocentric model states that the sun is the center of the solar system, meaning that the planets revolve around the sun. This theory broke with the traditional geocentric view, arguing that the Earth is not the center of the universe. The heliocentric model was proposed by Nicolaus Copernicus, who published his work *On the Revolutions of the Heavenly Spheres* in the early 16th century, outlining a model of the planets revolving around the sun, making a significant contribution to the development of astronomy."

[0146] S150: Fine-tune the large language model based on the fine-tuning data to obtain the fine-tuned retrieval enhancement generation model.

[0147] In some embodiments, the fine-tuning data obtained in step S140 above is used to fine-tune the open-source model. This embodiment has no special requirements for the open-source model. Both domestic and foreign open-source large language models can be used for fine-tuning, such as Qwen, Deepseek, Llama and other open-source large models.

[0148] For example, taking LoRA fine-tuning of the llama-factory framework as an example, the optimized fine-tuning parameter settings are as follows:

[0149] llamafactory-cli train \

[0150] --stage sft \

[0151] --do_train \

[0152] --model_name_or_path local_model\

[0153] --dataset local_data\

[0154] --dataset_dir . / data \

[0155] --template local_model_template \

[0156] --finetuning_type lora \

[0157] --output_dir . / saves / local_model \

[0158] --overwrite_cache \

[0159] --overwrite_output_dir \

[0160] --cutoff_len 1024 \

[0161] --preprocessing_num_workers 16 \

[0162] --per_device_train_batch_size 2 \

[0163] --per_device_eval_batch_size 1 \

[0164] --gradient_accumulation_steps 8 \

[0165] --lr_scheduler_type cosine \

[0166] --logging_steps 50 \

[0167] --warmup_steps 20 \

[0168] --save_steps 100 \

[0169] --eval_steps 50 \

[0170] --evaluation_strategy steps \

[0171] --load_best_model_at_end \

[0172] --learning_rate 5e-5 \

[0173] --num_train_epochs 5.0 \

[0174] --max_samples 1000 \

[0175] --val_size 0.1 \

[0176] --plot_loss \

[0177] --fp16

[0178] It is worth mentioning that a data balancing strategy was incorporated during the fine-tuning process to ensure that the ratio of noisy data, complex data, rejected data, multi-hop data, and general data was 2:2:1:2:3, thus avoiding model bias caused by an excessive proportion of one type of data.

[0179] S160: Evaluate the fine-tuned retrieval enhancement generation model based on answer relevance and answer similarity metrics.

[0180] In some embodiments, the evaluation metric of answer relevance focuses on the degree of close correlation between the generated answer and the input question. This embodiment innovatively adopts an indirect but highly effective method. By issuing specific prompts to the large language model, it is required to generate possible related questions based on the generated answer dialogue. The related questions are combined with the original input question to form a question set. Subsequently, text similarity calculation technology is used to measure the mean cosine similarity between these generated related questions and the original input question, thereby determining the answer relevance of the generated answer.

[0181] The specific prompt is: "Generate a related question for the given answer and determine whether the answer is explicit. If the answer is implicit, give noncommittal as 1; if the answer is explicit, give noncommittal as 0 (an implicit answer refers to an evasive, vague, or ambiguous response). For example: 'I don't know' or 'I'm not sure' are implicit answers."

[0182] Specifically, the formula for calculating answer relevance is:

[0183] ;

[0184] in, It is the generated first Each question corresponds to a vector. It is the vector corresponding to the user's question. This is the number of questions generated (the default value is usually 3).

[0185] The answer similarity metric primarily assesses the semantic similarity between the generated answer and the input question. In practice, this evaluation is strictly based on the ideal dialogue and the generated dialogue, with values ​​limited to 0 to 1. A higher score clearly indicates better semantic consistency between the generated answer and the standard answer.

[0186] According to the above embodiments of this disclosure, the following technical effects are achieved:

[0187] (1) The original text is segmented by a preset character-level segmentation rule, and paragraphs are adjusted by combining semantic integrity detection to ensure the semantic integrity of the paragraphs, thus laying a high-quality data foundation for subsequent entity and relation extraction. This segmentation method avoids semantic loss caused by paragraph breakage, making the extraction of entities and relationships between entities from paragraphs more accurate. The knowledge graph constructed is also more in line with the real semantics of the text, providing reliable knowledge support for model training.

[0188] (2) Noisy data, complex data, rejected data, and multi-hop data were generated based on the knowledge graph, and combined with general data to form fine-tuning data. The diverse data types can specifically improve the model's capabilities in different scenarios: noisy data can enhance the model's ability to resist interference from interfering information, enabling it to accurately focus on the core question even when irrelevant information is present; complex data helps the model handle questions involving multiple similar entities or complex relationships, improving its ability to cope with complex tasks; the introduction of rejected data allows the model to learn to identify unanswerable questions, reducing the occurrence of fabricated answers and enhancing the reliability of the model's output; multi-hop data can train the model's logical reasoning ability, enabling it to handle questions that require multiple derivations. The integration of general data ensures that the model has the ability to handle specific scenarios while not losing its general question-answering capabilities, achieving a balance between specialized and general capabilities.

[0189] (3) By converting multiple types of data into triplet data structures and standardizing them through unified format conversion, data of different types and sources can be adapted to the model fine-tuning framework, which improves the efficiency and convenience of the fine-tuning process and enhances the fine-tuning effect.

[0190] (4) The model after fine-tuning is evaluated by combining the answer relevance index and the answer similarity index, which is more comprehensive and scientific than a single index. The answer relevance index ensures that the answer generated by the model is closely related to the question and avoids answering the wrong question; the answer similarity index measures the degree of consistency between the generated answer and the standard answer. The combination of the two can more accurately reflect the quality of the answer generated by the model and provide a reliable basis for evaluating the model performance.

[0191] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0192] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0193] Example 2

[0194] Figure 3 A block diagram of a large language model fine-tuning and evaluation apparatus based on retrieval enhancement generation, according to an embodiment of the present disclosure, is shown. Figure 3 As shown, the device 300 includes:

[0195] The text segmentation module 310 is used to segment the input original text document content into multiple independent paragraphs according to preset character-level segmentation rules;

[0196] The knowledge graph construction module 320 is used to extract entities and relationships between entities from the multiple independent paragraphs to construct a knowledge graph.

[0197] The feature data generation module 330 is used to generate noisy data, complex data, rejection data and multi-hop data based on the knowledge graph through the noise sampling module, complex attribution module, rejection analysis module and reasoning module, respectively.

[0198] The data conversion module 340 is used to convert the noisy data, complex data, rejection data, multi-hop data and general data into formats to obtain fine-tuned data, wherein the general data is the question-and-answer data used for pre-training of the open-source model;

[0199] The large language model fine-tuning module 350 is used to fine-tune the large language model according to the fine-tuning data to obtain the fine-tuned retrieval enhancement generation model;

[0200] The large language model evaluation module 360 ​​is used to evaluate the fine-tuned retrieval enhancement generation model based on answer relevance and answer similarity indicators.

[0201] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0202] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0203] Figure 4A schematic block diagram of an electronic device 400 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0204] Electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in ROM 402 or a computer program loaded into RAM 403 from storage unit 408. RAM 403 may also store various programs and data required for the operation of electronic device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via bus 404. I / O interface 405 is also connected to bus 404.

[0205] Multiple components in electronic device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of displays, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0206] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as method 100. For example, in some embodiments, method 100 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of method 100 described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform method 100 by any other suitable means (e.g., by means of firmware).

[0207] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0208] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0209] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0210] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0211] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0212] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0213] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0214] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for large language model fine-tuning and evaluation based on retrieval-augmented generation, characterized in that, The method comprises the following steps: According to the preset character level segmentation rule, the input original text document content is segmented into multiple independent paragraphs; Extracting entities and relationships between entities from the multiple independent paragraphs to construct a knowledge graph; Based on the knowledge graph, noise data, complex data, rejection data and multi-hop data are generated through a noise sampling module, a complex attribution module, a rejection analysis module and a thinking reasoning module, specifically including: Based on the entities and relationships between entities in the knowledge graph, a first simple question and answer pair is constructed, and the relevant paragraph information of the first simple question and answer pair is selected from the independent paragraph set as noise information according to a preset proportion to form noise data. Select M entities with similar attributes or relationships from the knowledge graph to form an entity cluster, and construct a complex question and answer pair based on the entity cluster to form complex data. Based on the entities in the knowledge graph and the absolute paragraph containing the explicit answer, a second simple question and answer pair is constructed, and other paragraphs except the absolute paragraph are selected from the independent paragraph set as related text of the second question and answer pair to form rejection data. Filtering bridge entities in the knowledge graph, and generating candidate paragraph pairs containing three entity relationships based on the bridge entities and associated entities to construct multi-hop question and answer pairs to form multi-hop data, wherein the bridge entity is associated with at least two other entities, and there is no direct relationship between the associated entities. Format conversion is performed on the noise data, complex data, rejection data, multi-hop data and general data to obtain fine-tuning data, wherein the general data is the question and answer data used for pre-training of an open source model. According to the fine-tuning data, the large language model is fine-tuned to obtain a fine-tuned retrieval enhancement generation model. According to the answer relevance index and the answer similarity index, the fine-tuned retrieval enhancement generation model is evaluated.

2. The method of claim 1, wherein, According to the preset character level segmentation rule, the input original text document content is segmented into multiple independent paragraphs, including: According to the preset character level segmentation rule, the original text is segmented based on the preset word number and common punctuation; The semantic integrity of the segmented independent paragraphs is detected; If the end of an independent paragraph is an incomplete sentence, the independent paragraph is merged into the next independent paragraph for re-segmentation; If the last independent paragraph is an incomplete sentence, it is merged into the previous independent paragraph.

3. The method of claim 2, wherein, The method comprises the following steps: Using a large language model, the entities and relationships between entities in the paragraph are analyzed based on a preset prompt word template; The extracted entities and relationships between entities are mapped to nodes and edges in the knowledge graph to construct the knowledge graph.

4. The method of claim 3, wherein, The preset prompt word template includes target instructions and step-by-step analysis rules, wherein the step-by-step analysis rules include: Identify all entities and extract entity information, including entity name, entity type and entity description; Identify related entity pairs from the identified entities and extract relationship information between entities, including source entity name, target entity name, relationship description and relationship strength.

5. The method of claim 4, wherein, The extracted entities and relationships between entities are mapped to nodes and edges in the knowledge graph to construct the knowledge graph, including: The entity name is taken as a unique identification of a node, the entity type is taken as a classification label of the node, and the entity description is taken as a node attribute; The source entity name and the target entity name are taken as a starting node and a terminal node of an edge respectively, the relationship description is taken as a type of the edge, and the relationship strength is taken as a weight of the edge; The mapped nodes and edges are stored in a triple format, and the node attributes and the edge weights are associated, so that an extended triple structure is formed, and a knowledge graph is constructed.

6. The method of claim 1, wherein, The noise data, the complex data, the rejection data, the multi-hop data and the general data are format-converted to obtain fine-tuning data, including: a triple data structure including a task description field, a context information field and a target output field is called; the noise data, the complex data, the rejection data, the multi-hop data and the general data are mapped to the triple data structure according to a preset mapping structure; the mapped triple data is normalized to obtain fine-tuning data meeting input requirements of a model fine-tuning framework.

7. The method of claim 6, wherein, The noise data, the complex data, the rejection data, the multi-hop data and the general data are mapped to the triple data structure according to the preset mapping structure, including: the original question of the noise data, the complex data and the multi-hop data is mapped to the task description field, the associated background text is mapped to the context information field, and the derived standard answer is mapped to the target output field; the original question of the rejection data is mapped to the task description field, the interference text without a correct answer is mapped to the context information field, and the preset knowledge rejection identifier is mapped to the target output field; the original task description of the general data is mapped to the task description field, the original input information is mapped to the context information field, and the original output result is mapped to the target output field.

8. The method of claim 1, wherein, The fine-tuned retrieval enhancement generation model is evaluated according to an answer relevance index and an answer similarity index, including: at least one associated question is derived based on the generated answer, and the associated question and the input original question form a question set; a semantic association score between questions in the question set is calculated as the answer relevance index; the generated answer and the standard answer are compared in semantic equivalence to obtain the answer similarity index.

9. The method of claim 8, wherein, The answer relevance index calculation formula is: ; wherein, is the generated first question corresponding vector, is the user question corresponding vector, is the number of generated questions.

Images