Method for creating and correcting large model fine tuning data set

Abstract

The invention discloses a method for creating and correcting a large model fine tuning data set, and relates to the technical field of large models. The method comprises the following steps: converting received structured and unstructured texts into structured content units in a unified format; generating and distilling an initial question and answer pair by using a teacher model taking a pre-trained large language model as a core; screening out high-quality question and answer pairs according to a quantitative evaluation system containing correlation, accuracy and integrity dimensions; performing diversified optimization on high-quality question and answer pairs by applying a generalization strategy and multi-round distillation, and performing dynamic regulation and control based on language feature difference to avoid content homogenization; and finally, guiding the teacher model to generate ternary structure data containing a thinking chain, and executing a verification and self-correction algorithm so as to generate a final fine tuning data set containing a verified reasoning process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large model, and in particular to a method for creating and correcting large model fine-tuning dataset. BACKGROUND

[0002] In recent years, artificial intelligence technology represented by large language models has developed rapidly. In order to make general large models better adapt to the needs of specific industries or specific tasks, fine-tuning technology is usually used, that is, using high-quality datasets in specific fields to further train pre-trained models. The quality of the fine-tuning dataset directly determines the performance of the model on specific tasks.

[0003] Currently, the construction of datasets for large model fine-tuning, especially question and answer datasets, mainly relies on manual processing. This method usually requires domain experts to manually read, understand a large amount of raw materials, and extract and write paired knowledge or question and answer pairs from them. These raw materials have various forms, including structured text with regular formats (such as database tables) and unstructured text with free content (such as technical documents, reports, and web pages). However, the existing dataset construction methods have many technical defects: first, the purely manual processing method is inefficient, labor-intensive, and time-consuming, especially when dealing with massive amounts of raw text, which is costly and difficult to scale. Second, for unstructured text, it is difficult to accurately anchor the key information when manually extracting it due to the lack of fixed formats, and the complex logical relationships implied in the text require deep inference based on context, which can cause the generated questions to deviate from the topic or have low value. Third, for structured text, information is often scattered in different fields, and a single field may not be able to form a complete and meaningful answer, while the combination of cross-field information can easily produce logical gaps. In addition, in the process of converting raw text into question and answer pairs, the pursuit of neat sentence structure or uniform form may distort the original text semantics, leading to unbalanced question and answer semantics.

[0004] In summary, the existing large model fine-tuning dataset construction method has obvious shortcomings in processing efficiency, information extraction accuracy, logical relationship integrity, and the quality of the final generated data, which seriously restricts the application and development of large models in vertical fields. SUMMARY

[0005] The purpose of the present application is to solve the problems in the prior art and provide a method for creating and correcting large model fine-tuning dataset. To achieve the above purpose, the embodiments of the present application adopt the following technical solutions: The embodiments of the present application provide A method for creating and correcting large model fine-tuning dataset, the method comprising: The received structured text and unstructured text are identified and classified, and the structured text and unstructured text are converted into structured content units with uniform format fields according to a preset text conversion rule library and a data field mapping engine; The structured content units are input into a teacher model using a teacher model distillation framework based on a pre-trained large language model, and an initial question and answer pair is generated and distilled through the reasoning ability and built-in attention mechanism of the teacher model; According to a quantitative evaluation system including relevance, accuracy and completeness dimensions, the initial question and answer pair is scored to calculate a comprehensive quality score, and high-quality question and answer pairs are selected according to a preset quality threshold; At least one strategy in a generalization strategy library is applied to the high-quality question and answer pair to generate diversified question and answer pair variants, and a multi-round distillation mechanism is used to optimize the language expression logic and information density of the question and answer pair variants. At the same time, based on the quantitative analysis of the language feature difference degree of the question and answer pair variants, the combination of the generalization strategy is dynamically adjusted to avoid content homogenization; Based on the preset thinking chain structure generation rule, the teacher model generates ternary structure data including reasoning process based on the optimized question and answer pair, the ternary structure data includes question, thinking chain corresponding to the question and answer derived from the thinking chain, and performs thinking chain accuracy verification algorithm to decompose and verify the reasoning process. The self-correction ability of the teacher model is used to locate and correct the identified error reasoning nodes, thereby generating a final fine-tuning data set containing verified and corrected thinking chains.

[0006] Preferably, the step of converting the structured text and unstructured text into structured content units with uniform format fields comprises: Using natural language processing to extract keywords, syntax structures and semantic relationships of structured text and unstructured text as text features; Using machine learning classification algorithms to identify and classify the types of structured text and unstructured text based on text features; According to the identified and classified text categories, corresponding conversion rules are called from the text conversion rule library to extract key information elements from structured text and unstructured text and convert them into fields with uniform format; Through the data field mapping engine, the field names from different data sources but with the same semantics are mapped to standardized field names by string matching or semantic similarity calculation And the converted and mapped key information elements and standardized field names are combined to form structured content units.

[0007] Preferably, after generating and distilling the initial question and answer pair through the reasoning ability and built-in attention mechanism of the teacher model, it further comprises: The built-in attention mechanism of the teacher model is applied to focus on the core information elements in the input structured content unit; Based on the weight distribution of the attention mechanism, redundant or secondary information in the structured content unit is identified and filtered, thereby generating an initial question-answer pair with higher information density and content conciseness.

[0008] Preferably, the step of scoring the initial question-answer pair to calculate the comprehensive quality score and screening high-quality question-answer pairs according to the preset quality threshold includes: For each initial question-answer pair, its relevance score , accuracy score and completeness score are calculated respectively; According to the preset weight coefficients , and , the comprehensive quality score is calculated by a weighted sum formula , wherein the weight coefficients are in the range of (0, 1) and satisfy ; The calculated comprehensive quality score is compared with the preset quality threshold , and only the question-answer pairs that meet the condition are retained as high-quality question-answer pairs.

[0009] Preferably, the step of dynamically regulating the combination of generalization strategies to avoid content homogenization includes: Select at least one generalization method from the generalization strategy library, the generalization method including synonym replacement, sentence transformation, word order adjustment and / or context expansion, and apply the selected generalization method to the high-quality question-answer pairs to generate a set of question-answer pair variants; Through a language feature analysis model, the language feature difference degree of the question-answer pair variants in vocabulary selection, sentence structure and semantic style is quantitatively analyzed, and the quantitative formula of the difference degree is: , wherein and are two different question-answer pair variants, is the language feature set, is the value of the question-answer pair on the feature ; Based on the calculated difference degree , the combination of generalization strategies applied to the next batch of high-quality question-answer pairs and the related parameters are dynamically adjusted to ensure that the difference degree between the generated question-answer pair variants is not lower than the preset difference threshold.

[0010] Preferably, when the generalization method is synonym substitution, the original word With candidate words Replacement probability Determined by the following formula:

[0011] in, For the original word a collection of synonyms For the original word With candidate words Word vector similarity, Candidate words The frequency of occurrence in the corpus; a set of synonyms Any synonym in For the original word Synonyms Word vector similarity, Synonyms The frequency of occurrence of the corpus.

[0012] Preferably, the multi-round distillation mechanism includes: The question-answer pair variants generated after the generalization strategy are then input back into the teacher model for secondary distillation. During the secondary distillation process, the teacher model optimizes the expression of question-and-answer pairs based on its internal knowledge base’s understanding of the logic and information density of language expression. This improves the rationality and information compactness of the question-and-answer pairs in terms of language expression, while retaining the linguistic diversity features introduced by different generalization strategies.

[0013] Preferably, the thought chain accuracy verification algorithm includes: decomposing the thought chain generated by the teacher model into a series of discrete, logically related reasoning nodes, where each reasoning node contains a premise, a reasoning action, and a conclusion; for each reasoning node, independently verifying the authenticity of its premise, the validity of its reasoning action, and the correctness of its conclusion by comparing it with an external factual knowledge base or an internal logical rule set; generating a verification result for each reasoning node, and summarizing the verification results of all nodes to quantitatively evaluate the accuracy and reliability of the entire thought chain.

[0014] Preferably, the self-correcting capability of the teacher model includes: When the thought chain accuracy verification algorithm identifies an incorrect inference node, it feeds back the incorrect inference node and its context information as a correction instruction to the teacher model. After receiving the correction instruction, the teacher model performs internal reasoning to locate the root cause of the error; The teacher model autonomously regenerates or modifies the wrong reasoning node, and cascades to adjust all subsequent reasoning nodes affected by the node, thereby ensuring the logical consistency of the entire thinking chain and the correctness of the final answer.

[0015] Preferably, the method further comprises a data set optimization iteration process, which comprises: Training the target large model with the generated fine-tuning data set; Through the large model training effect evaluation system, the performance change of the target large model before and after training is evaluated on a set of standard test sets, and the performance change indicators include question and answer accuracy, thinking chain integrity and generalization ability; Based on the evaluation results fed back by the evaluation system, the combination and parameters of the quality threshold and the generalization strategy, and the distillation hyperparameters in the teacher model distillation framework are dynamically and adaptively adjusted, and a new round of data set creation and correction process is started.

[0016] Beneficial effects: The method for creating and correcting the large model fine-tuning data set provided by the application introduces an automatic distillation generation process centered on the teacher model, and combines multi-dimensional quality evaluation, diversity enhancement, and thinking chain injection and self-correction closed-loop mechanism, effectively overcoming the problems of low efficiency, high cost, uneven quality and lack of logical reasoning ability caused by relying on artificial construction of data set in the prior art. Compared with the prior art, the application automatically generates and selects high-quality question and answer pairs using the teacher model, greatly improving the construction efficiency and scalability of the data set. At the same time, through the dynamic regulation of the generalization strategy and multiple rounds of distillation, the richness and diversity of the data set in content and expression are ensured, and semantic imbalance is avoided. More importantly, the application creatively introduces the generation, verification and self-correction mechanism of the thinking chain, not only injects a deep logical reasoning process into the data set, but also ensures the accuracy of the reasoning chain through automatic checking and correction, significantly enhancing the complex reasoning and analysis ability of the final fine-tuned model. BRIEF DESCRIPTION OF DRAWINGS

[0017] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for purposes of illustration only and are not considered a limitation of the application. Moreover, like reference numerals are used to designate identical components throughout the specification and drawings. In the drawings: Figure 1 A flowchart of the method for creating and correcting the large model fine-tuning data set proposed in the embodiments of the application; Figure 2 A structural schematic diagram of the computer device provided by the embodiments of the application. DETAILED DESCRIPTION

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the application will be briefly introduced below in conjunction with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the drawings only constitutes some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor. It should be noted that the description of these embodiments is used to help understand the application, but does not constitute a limitation on the application.

[0019] In the first aspect, with reference to Figure 1 A method for creating and correcting a large model fine-tuning dataset is provided in the embodiments of the application, which can be executed on a computer device. The computer device can be a server, a personal computer (PC), or an embedded system with sufficient computing power for creating and correcting a large model fine-tuning dataset. The device generally includes, but is not limited to, a processor, a memory, a communication interface, and an input / output device. The memory stores computer program instructions, and the processor executes these instructions to implement the method for creating and correcting a large model fine-tuning dataset of the application. The method aims to solve the problems of low efficiency, high cost, and difficult quality guarantee in the prior art of manually constructing a fine-tuning dataset, and efficiently generates a high-quality dataset containing deep logical reasoning information through an automated process. The specific process of the embodiment method includes the following steps: S101: Identify and classify the received structured text and unstructured text, and convert the structured text and unstructured text into structured content units with uniform format fields according to the preset text conversion rule library and data field mapping engine.

[0020] This step is the starting point of the entire data processing process, and its core goal is to achieve standardized and normalized preprocessing of heterogeneous data sources, laying a solid data foundation for accurate processing of the teacher model. In a specific implementation, this step is completed by a text type identification module. The module first receives external input data from diverse sources, which can be divided into two categories: structured text, which refers to text with fixed format and explicit data fields, such as tables in databases, Excel or CSV format spreadsheets, JSON or XML files, etc. Unstructured text, which refers to free text without a predetermined format, such as Word documents, PDF reports, web content, emails, plain text documents, etc.

[0021] Preferably, the step of converting structured text and unstructured text into structured content units with uniform format fields includes: S1011, using natural language processing technology to extract keywords, syntax structures, and semantic relationships of structured text and unstructured text as text features.

[0022] To effectively identify and classify the received text, the system first employs a series of Natural Language Processing (NLP) techniques for deep text analysis. These techniques include: tokenization, which segments continuous text strings into words or symbolic units with independent meaning; Part-of-Speech Tagging (POS Tagging), which labels each word with its grammatical attributes, such as noun, verb, or adjective; and Named Entity Recognition (NER), which identifies and labels entities with specific meanings in the text, such as names of people, places, organizations, times, and proper nouns.

[0023] By comprehensively applying the aforementioned NLP techniques, the system can extract key text features from the original text, such as high-frequency keywords, core grammatical structures (e.g., subject-verb-object structures), and deep semantic relationships between words. These features collectively constitute a vectorized or structured representation of the text content, serving as the input basis for subsequent classification algorithms.

[0024] S1012. Employ machine learning classification algorithms to identify and classify structured and unstructured text based on text features.

[0025] After extracting text features, the system employs one or more machine learning classification algorithms to automatically identify the text's type. Optional classification algorithms include, but are not limited to: Naive Bayes, a simple and efficient classification algorithm based on Bayes' theorem; Support Vector Machine (SVM), a supervised learning model that performs well when handling high-dimensional data; and deep learning classification models, such as text classification models based on Recurrent Neural Networks (RNNs) or Transformer architectures, which can capture more complex text patterns.

[0026] Taking the Naive Bayes algorithm as an example, for a text to be classified... It belongs to the category The posterior probability can be calculated using the following formula: in: Indicates in the given text In this case, the text belongs to the category The probability of. Indicates category prior probability, i.e. the frequency of occurrence of the class in the entire dataset. represents the likelihood probability of the text appearing in the class . represents the marginal probability of the text . The system will calculate the probability of the text belonging to all predefined classes (e.g. contract class, technical report class, product description class, daily question and answer class, etc.), and select the class with the maximum posterior probability as the final classification result of the text.

[0027] S1013, according to the identified and classified text category, call the corresponding conversion rule from the text conversion rule library, extract the key information elements in the structured text and unstructured text and convert them into a unified format field.

[0028] The system pre-constructs and maintains a text conversion rule library. The rule library stores one or more sets of corresponding parsing and extraction rules for each predefined text category. When a text is classified, the system will call the corresponding rules from the rule library according to its category.

[0029] These rules specify which information is the key information element in this type of text, and how to locate and extract these elements from the text. For example: for "contract" text, the rules may define the need to extract "Party A", "Party B", "contract amount", "signing date", "effective provisions" and other key information. For "technical report" text, the rules may focus on extracting "experimental purpose", "experimental data", "conclusion", "technical indicators", etc. The system parses the text in depth according to these rules, and fills the extracted key information elements into the pre-set, standardized fields.

[0030] S1014, through the data field mapping engine, using string matching or semantic similarity calculation, mapping the field names from different data sources but with the same semantics into standardized field names.

[0031] Because texts from different sources may use different names for the same semantic field (e.g. "company name" and "customer name", "purchase amount" and "contract total price"), in order to ensure data consistency, the present embodiment also includes a data field mapping engine. The engine maintains a field mapping relationship table or a synonym dictionary, and uses string exact / fuzzy matching algorithm or semantic similarity calculation method based on word vector, to automatically map various heterogeneous field names to a unified standard field name.

[0032] ​After the processing of S1011 to S1014, the original, heterogeneous structured and unstructured text is successfully transformed into "structured content units" composed of a series of uniformly formatted fields. Each unit contains core information under a certain topic, preparing for the subsequent processing steps.

[0033] S102: Using a teacher model distillation framework centered on a pre-trained large language model, input the structured content units into the teacher model, and generate and distill the initial question-answer pairs through the reasoning ability and built-in attention mechanism of the teacher model.

[0034] The core of this step is to use the generation ability of a powerful pre-trained large language model (as a "teacher model") to automatically convert the structured content units output in the previous step into initial question-answer (QA) pairs.

[0035] This embodiment constructs a teacher model distillation framework, the core of which is a superior large language model, such as the GPT (Generative Pre-trained Transformer) series, LLaMA series or other models with strong natural language understanding and generation capabilities.

[0036] Preferably, after generating and distilling the initial question-answer pairs through the reasoning ability and built-in attention mechanism of the teacher model, it further includes: S1021, applying the built-in attention mechanism of the teacher model to focus on the core information elements in the input structured content units.

[0037] When the structured content units are input into the teacher model, the attention mechanism (Attention Mechanism) inside the model begins to play a role. This mechanism can automatically calculate the importance weight of different parts of the input sequence, so that the model can allocate more computing resources to the information most relevant to the current task when processing information. In this scenario, the attention mechanism will focus on the core information fields and key values in the structured content units, while giving lower weights to auxiliary or descriptive text.

[0038] S1022, based on the weight distribution of the attention mechanism, identify and filter redundant or secondary information in the structured content units, and generate initial question-answer pairs with higher information density and concise content.

[0039] The teacher model generates the question-answer pair based on the weight distribution calculated by the attention mechanism. Core information with high weights is more likely to be used as key material for generating questions or answers. At the same time, the model is guided to identify and filter out redundant or secondary information with low weights. This process is similar to "distillation", that is, to extract the most essential part from the original, possibly complex content.

[0040] For example, a content unit may contain a large number of legal provisions, but the core information is "the liquidated damages of the contract is 10% of the total amount". The teacher model focuses on the latter through the attention mechanism, and the generated question may be "What is the liquidated damages of the contract?", and the answer is "10% of the total amount of the contract", rather than generating questions related to complex legal provisions.

[0041] In this way, the initial question-answer pair generated has higher information density and better content conciseness, and the quality is much higher than that of the simple template matching method.

[0042] S103: According to the quantitative evaluation system including relevance, accuracy and completeness dimensions, score the initial question-answer pair to calculate the comprehensive quality score, and screen out high-quality question-answer pairs according to the preset quality threshold.

[0043] Although the initial question-answer pair generated by the teacher model has high quality, there may still be defects. Therefore, an automatic quality evaluation and screening mechanism is established to ensure that only the best question-answer pairs can enter the next step.

[0044] Preferably, the step of scoring the initial question-answer pair to calculate the comprehensive quality score comprises: S1031, for each initial question-answer pair, calculate its relevance score Rel, accuracy score Acc and completeness score Com, respectively.

[0045] The system quantitatively evaluates each question-answer pair from the following three core dimensions: relevance (Relevance, Rel), which evaluates the semantic association between the question and the answer. A high-relevance question-answer pair should have an answer that directly and explicitly answers the question. This score can be obtained by calculating the vector similarity of the question and answer texts. Accuracy (Accuracy, Acc), which evaluates the truthfulness of the answer content, that is, whether the answer is consistent with the factual information in the original structured content unit. This score can be calculated by comparing the answer content with the source data. Completeness (Completeness, Com), which evaluates whether the answer covers all the information required by the question. For example, if the question is "Who are the parties A and B in the contract?", the answer only gives the A party, and the completeness score is low. The score of each dimension is normalized to the interval.

[0046] S1032、According to the preset weight coefficient , and , the comprehensive quality score is calculated by a weighted sum formula , and the question and answer pairs that meet the conditions are screened out.

[0047] The comprehensive quality score is calculated by the following weighted sum formula:

[0048] , and are the weight coefficients of relevance, accuracy and completeness scores, respectively, and satisfy . These weights can be adjusted according to the needs of specific application scenarios. For example, when building a factual knowledge base, the weight of accuracy can be increased . After calculating the comprehensive quality score , the system compares it with a preset quality threshold . Only question and answer pairs that meet the conditions are considered "high-quality question and answer pairs" and are retained for subsequent processing.

[0049] S104: Apply at least one strategy in the generalization strategy library to generate diversified question and answer pair variants, and optimize the language expression logic and information density of the question and answer pair variants through multiple rounds of distillation mechanism, while dynamically adjusting the combination of generalization strategies based on the quantitative analysis of the language feature difference of the question and answer pair variants to avoid content homogenization.

[0050] In order to increase the richness of the fine-tuning data set and the generalization ability of the model, this step carries out diversity enhancement and depth optimization on the selected high-quality question and answer pairs.

[0051] Preferably, the step of dynamically adjusting the combination of generalization strategies to avoid content homogenization comprises: S1041, selecting at least one generalization method from the generalization strategy library to generate question and answer pair variants.

[0052] The system has a built-in question and answer pair generalization strategy library, which contains a variety of text transformation techniques: Synonym replacement, replacing some words in the question-answer pair with their synonyms without changing the core semantics of the sentence. Sentence transformation, changing the grammatical structure of the sentence, such as active to passive, declarative to interrogative, etc. Order adjustment, reasonably adjusting the position of adverbial, attributive, and other modifying components in the sentence. Context expansion, supplementing reasonable context information to the question or answer to make it more complete. The system will select one or more appropriate generalization strategies for each high-quality question-answer pair and generate a set of question-answer pair variants with different language expressions but consistent core semantics.

[0053] Preferably, when the generalization method is synonym replacement, the original word is replaced by the candidate word with a replacement probability which is determined by the following formula:

[0054] wherein: is the original word. is the candidate synonym. is the synonym set of the original word . is the word vector similarity between the original word and the candidate word , used to measure the closeness of semantics. is the frequency of the candidate word in the general corpus, used to ensure that the replaced word is commonly used. is any synonym in the synonym set . and are the word vector similarity of the original word and the synonym and the corpus frequency of the synonym . This formula calculates a scientific replacement probability by considering semantic similarity and word frequency, thereby ensuring semantic consistency while improving language diversity.

[0055] S1042, quantitatively analyze the language feature difference between question-answer pair variants through a language feature analysis model, and dynamically control the generalization strategy.

[0056] To avoid the generated question-answer pair variants being too similar in expression (i.e., content homogenization), the system also includes a language expression diversity enhancement module. This module uses a language feature analysis model to quantitatively analyze the language feature differences between different question-answer pair variants from multiple dimensions such as vocabulary selection, sentence structure complexity, average word length, and semantic style.

[0057] The quantitative formula for difference degree can be:

[0058] wherein and are two different question-answer pair variants, is a set of linguistic features, is a question-answer pair on the values of the features ; the system forms a feedback loop based on the calculated diversity . If it is found that the diversity between the generated variants is generally lower than a preset diversity threshold, the system will automatically adjust the combination of generalization strategies and their related parameters applied to the next batch of high-quality question-answer pairs (for example, increase the weight of sentence transformation, reduce the proportion of synonym replacement), so as to actively improve the language expression richness of the entire data set.

[0059] Preferably, the multi-round distillation mechanism comprises: S1043, input the question-answer pair variants generated after the generalization strategy processing into the teacher model again for secondary distillation.

[0060] Although the question-answer pair variants after generalization processing have enhanced diversity, the naturalness of language expression and the rigor of logic may be reduced. Therefore, the present embodiment introduces a multi-round distillation mechanism.

[0061] The system inputs the generated question-answer pair variants into the same teacher large model again for secondary distillation. In the secondary distillation process, the teacher model will use its powerful internal knowledge base and deep understanding of language expression logic to polish and optimize the expression of these variants, so that they maintain diversity features while having more reasonable, natural language expression and more compact information.

[0062] S105: Based on the preset thought chain structure generation rule, the teacher model generates ternary structure data containing the reasoning process based on the optimized question-answer pair, and performs a thought chain accuracy verification algorithm to decompose and check the reasoning process, and then uses the self-correction ability of the teacher model to locate and correct the identified incorrect reasoning nodes, thereby generating a final fine-tuning data set.

[0063] In order to let the fine-tuned model not only answer questions, but also understand and imitate complex reasoning processes, this step injects "Chain-of-Thought" (CoT) information into the data set.

[0064] Preferably, the thought chain accuracy verification algorithm comprises: S1051, based on the preset thought chain structure generation rule, guide the teacher model to generate ternary structure data.

[0065] The system pre-establishes the structure generation rules of the thought chain, defines the constituent elements of the thought chain, such as premise, reasoning step, intermediate conclusion, final conclusion, etc. Then, the system guides the teacher model to generate not only the final answer but also the detailed thinking process of obtaining the answer when processing the optimized question and answer pair.

[0066] In this way, each data point is expanded from a simple question and answer pair to a ternary structure data of "question-thought chain-answer". For example: Question: If A is 5 years older than B, and B is 3 years older than C, how many years older is A than C? Thought chain: Step 1: Given that A is 5 years older than B, it can be expressed as A = B + 5. Step 2: Given that B is 3 years older than C, it can be expressed as B = C + 3. Step 3: Substitute the expression of Step 2 into Step 1 to get A = (C + 3) + 5. Step 4: Simplify the expression to get A = C + 8. Conclusion: Therefore, A is 8 years older than C.

[0067] Answer: 8 years.

[0068] S1052、The thought chain generated by the teacher model is decomposed into a series of discrete, logically related reasoning nodes, and is independently verified.

[0069] In order to ensure the accuracy of the thought chain, the system constructs a thought chain accuracy verification algorithm. The algorithm first decomposes a complete thought chain into a series of discrete, logically related reasoning nodes. Each reasoning node usually contains a premise, a reasoning action and a conclusion.

[0070] Then, the algorithm verifies the truth of the premise, the validity of the reasoning action (such as whether the mathematical operation is correct) and the correctness of the conclusion of each reasoning node independently by comparing with external fact knowledge base (such as public online encyclopedia) or internal logical rule set.

[0071] S1053、For each reasoning node, generate a verification result and summarize the accuracy of the entire thought chain.

[0072] The system generates a verification result (e.g. correct, incorrect, uncertain) for each reasoning node and summarizes the verification results of all nodes to give a quantitative evaluation of the accuracy and reliability of the entire thought chain.

[0073] Preferably, the self-correction ability of the teacher model includes: S1054、When an incorrect reasoning node is identified, use the self-correction ability of the teacher model to correct it.

[0074] The present embodiment makes full use of the self-correction capability of advanced large language models. When the thought chain accuracy verification algorithm identifies an incorrect reasoning node, the system will package the incorrect reasoning node and its context information into a correction instruction and feed it back to the teacher model.

[0075] After receiving the instruction, the teacher model will perform an internal reasoning and reflection process to locate the root cause of the error. Then, the model will autonomously regenerate or modify the incorrect reasoning node and adjust all subsequent reasoning nodes affected by it in a cascading manner, ensuring the logical consistency of the entire thought chain and the correctness of the final answer.

[0076] After all the above steps, the system finally generates a high-quality fine-tuning dataset containing verified and corrected thought chains.

[0077] Preferably, the method of the present application further comprises a dataset optimization iteration process, which includes: S1061, training the target large model using the generated fine-tuning dataset and establishing an evaluation system.

[0078] To form a continuous optimization closed loop, the present embodiment also designs a dataset optimization iteration process. First, the final generated fine-tuning dataset is used to train a target large model. Then, through a pre-established large model training effect evaluation system, the performance change of the target large model before and after training is evaluated on a set of standard, independent test sets. The evaluation indicators can include: question and answer accuracy, thought chain integrity, generalization ability, etc.

[0079] S1062, based on the evaluation results, dynamically and adaptively adjust the hyperparameters in the process, and start a new round of process.

[0080] Based on the evaluation results fed back by the evaluation system, the system dynamically and adaptively adjusts various hyperparameters in the entire dataset creation and correction process, such as the quality threshold in S103 , the combination and parameters of the generalization strategy in S104, and the distillation hyperparameters (such as temperature coefficient, etc.) of the teacher model distillation framework in S102. After adjustment, the system can start a new round of dataset creation and correction process, thereby continuously iterating and optimizing to generate fine-tuning datasets with higher and higher quality.

[0081] In summary, the present embodiment realizes a method for quickly and efficiently creating and correcting large model fine-tuning datasets through a series of interlocking and deeply optimized automated steps, significantly improving the quality, diversity, and logical depth of the dataset, and providing strong support for training more performant domain-specific large models.

[0082] The second aspect of the embodiment of the present application provides a computer device for creating and correcting a large model fine-tuning data set, please refer to Figure 2 comprises a memory and a processor connected in sequence, wherein the memory is configured to store a computer program, and the processor is configured to read the computer program and execute a method for creating and correcting a large model fine-tuning data set according to the first aspect of the embodiment. Specifically, the memory can include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a flash memory, a first-in first-out memory (FIFO), a first-in last-out memory (FILO), and the like; the processor can be, but is not limited to, a microprocessor of STM32F105 series, an ARM (Advanced RISC Machines) processor, an X86 architecture processor, or a processor integrated with an NPU (neural-network processing units). The working process, working details and technical effects of the device provided in the second aspect of the embodiment can be referred to the first aspect of the embodiment, which will not be repeated here.

[0083] The third aspect of the embodiment provides a computer readable storage medium storing instructions of the method for creating and correcting a large model fine-tuning data set according to the first aspect of the embodiment, that is, the computer readable storage medium stores instructions, when the instructions run on a computer, the method for creating and correcting a large model fine-tuning data set according to the first aspect is executed. The computer readable storage medium refers to a carrier for storing data, which can include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash disk, a memory stick, and the like. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device systems. The working process, working details and technical effects of the computer readable storage medium provided in the third aspect of the embodiment can be referred to the first aspect of the embodiment, which will not be repeated here.

[0084] The fourth aspect of the embodiment provides a computer program product containing instructions, when the instructions run on a computer, the computer executes the method for creating and correcting a large model fine-tuning data set according to the first aspect of the embodiment, wherein the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0085] The above-described embodiments are merely illustrative for the present application, wherein the units illustrated as separated components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0086] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions to make a computer or programmable device execute the methods of the embodiments or some parts of the embodiments.

[0087] Finally, it should be noted that: although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for creating and correcting large model fine-tuning datasets, characterized in that, The method includes: The received structured and unstructured texts are identified and classified, and then converted into structured content units with uniform format fields according to a preset text conversion rule library and data field mapping engine. Using a teacher model distillation framework based on a pre-trained large language model, the structured content units are input into the teacher model, and the initial question-answer pairs are generated and distilled through the reasoning ability and built-in attention mechanism of the teacher model. The initial question-and-answer pairs are scored according to a quantitative evaluation system that includes relevance, accuracy and completeness dimensions to calculate a comprehensive quality score, and high-quality question-and-answer pairs are selected according to a preset quality threshold. At least one strategy from the generalization strategy library is applied to the high-quality question-answer pairs to generate diverse question-answer pair variants. The language expression logic and information density of the question-answer pair variants are optimized through a multi-round distillation mechanism. At the same time, based on the quantitative analysis of the language feature differences of the question-answer pair variants, the combination of the generalization strategies is dynamically adjusted to avoid content homogenization. Based on the preset thought chain structure generation rules, the teacher model generates ternary structure data containing the reasoning process based on the optimized question-answer pairs. The ternary structure data includes a question, the thought chain corresponding to the question, and the answer derived from the thought chain. The reasoning process is decomposed and verified by the thought chain accuracy verification algorithm. Then, the self-correction capability of the teacher model is used to locate and correct the identified erroneous reasoning nodes, thereby generating the final fine-tuned dataset containing the verified and corrected thought chains.

2. The method according to claim 1, characterized in that, The step of converting the structured and unstructured text into structured content units with uniformly formatted fields includes: Natural language processing is used to extract keywords, grammatical structures, and semantic relationships from the structured and unstructured texts as text features; A machine learning classification algorithm is used to identify and classify the structured and unstructured texts based on the text features. Based on the identified and classified text categories, the corresponding conversion rules are called from the text conversion rule base to extract key information elements from the structured and unstructured texts and convert them into fields in a unified format; The data field mapping engine uses string matching or semantic similarity calculation to map semantically identical field names from different data sources to standardized field names. Furthermore, the transformed and mapped key information elements are combined with the standardized field names to form the structured content unit.

3. The method according to claim 1 or 2, characterized in that, After generating and distilling the initial question-answer pairs using the reasoning ability and built-in attention mechanism of the teacher model, the process also includes: The built-in attention mechanism of the teacher model is applied to focus on the core information elements in the input structured content unit; Based on the weight distribution of the attention mechanism, redundant or secondary information in the structured content units is identified and filtered, thereby generating initial question-answer pairs with higher information density and content conciseness.

4. The method according to claim 1, characterized in that, The steps of scoring the initial question-and-answer pairs to calculate a comprehensive quality score, and selecting high-quality question-and-answer pairs based on a preset quality threshold, include: For each initial question-answer pair, calculate its relevance score. Accuracy score and integrity score ; According to the preset weighting coefficients , and Using the weighted summation formula Calculate the overall quality score The weighting coefficients range from (0,1) and satisfy the following condition: ; Calculate the overall quality score With the preset quality threshold Compare and retain only those that meet the conditions. The question-answer pairs are considered as high-quality question-answer pairs.

5. The method according to claim 1, characterized in that, The step of dynamically adjusting the combination of the generalization strategies to avoid content homogenization includes: Select at least one generalization method from the generalization strategy library, the generalization method including synonym substitution, sentence transformation, word order adjustment and / or context expansion, and apply the selected generalization method to the high-quality question-answer pair to generate a set of question-answer pair variants; Using a language feature analysis model, the differences in language features of the question-answer pair variants in terms of vocabulary selection, sentence structure, and semantic style are quantitatively analyzed. The quantitative formula for these differences is as follows: ， in and For two different question-answer pair variants, For a set of language features, For question and answer In features The value of ; Based on the calculated degree of difference The combination of generalization strategies and related parameters applied to the next batch of high-quality question-answer pairs are dynamically adjusted to ensure that the difference between the generated question-answer pair variants is not lower than the preset difference threshold.

6. The method according to claim 5, characterized in that, When the generalization method is synonym replacement, the original word With candidate words Replacement probability Determined by the following formula: in, For the original word a collection of synonyms For the original word With candidate words Word vector similarity, Candidate words The frequency of occurrence in the corpus; For the set of synonyms Any synonym in For the original word Synonyms Word vector similarity, Synonyms The frequency of occurrence of the corpus.

7. The method according to claim 1 or 5, characterized in that, The multi-round distillation mechanism includes: The question-answer pair variants generated after generalization are then input back into the teacher model for secondary distillation. During the secondary distillation process, the teacher model optimizes the expression of the question-and-answer pair variants based on its internal knowledge base's understanding of the logic and information density of language expression. This improves the rationality and information compactness of the question-and-answer pair variants in terms of language expression, while preserving the linguistic diversity features introduced by different generalization strategies.

8. The method according to claim 1, characterized in that, The algorithm for verifying the accuracy of the thought chain includes: decomposing the thought chain generated by the teacher model into a series of discrete, logically related reasoning nodes, where each reasoning node contains a premise, a reasoning action, and a conclusion; for each reasoning node, independently verifying the authenticity of its premise, the validity of its reasoning action, and the correctness of its conclusion by comparing it with an external factual knowledge base or an internal logical rule set; generating a verification result for each reasoning node, and summarizing the verification results of all nodes to quantitatively evaluate the accuracy and reliability of the entire thought chain.

9. The method according to claim 1 or 8, characterized in that, The self-correcting capability of the teacher model includes: When the thought chain accuracy verification algorithm identifies an incorrect inference node, it feeds back the incorrect inference node and its context information as a correction instruction to the teacher model. After receiving the correction instruction, the teacher model performs internal reasoning to locate the root cause of the error; The teacher model autonomously regenerates or modifies the erroneous reasoning node and cascades and adjusts all subsequent reasoning nodes affected by it, thereby ensuring the logical consistency of the entire thought chain and the correctness of the final answer.

10. The method according to claim 1, characterized in that, The method also includes a dataset optimization iterative process, which includes: The target large model is trained using the generated fine-tuning dataset; The performance changes of the target large model before and after training are evaluated on a set of standard test sets through a large model training effect evaluation system. The performance change indicators include question answering accuracy, mental chain integrity and generalization ability. Based on the evaluation results fed back by the evaluation system, the quality threshold, the combination and parameters of the generalization strategy, and the distillation hyperparameters in the teacher model distillation framework are dynamically and adaptively adjusted, and a new round of dataset creation and correction process is initiated.

Images