Model fine-tuning method, device, equipment, medium and product

Abstract

The present disclosure relates to the technical field of model fine-tuning, and in particular, to a model fine-tuning method, device, equipment, medium and product. The method comprises: obtaining input data, and constructing an initial training set carrying a plurality of initial question-answer pairs based on the input data. In the process of any round of iterative fine-tuning, the model of the previous round is fine-tuned based on the target training set of the current round, and the model of the current round is obtained after fine-tuning; the model of the previous round corresponding to the first round of iterative fine-tuning is a base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set. The target question-answer pair is generated based on the model of the current round. The target question-answer pair that passes the quality screening is added to the target training set, so as to perform the next round of iterative fine-tuning based on the added target training set. The target question-answer pair that passes the quality screening in any round in the present disclosure can be continuously used as training data in subsequent rounds, so that each round of iteration continues to optimize based on the model fine-tuned in the previous round, and the iteration efficiency of the model is improved.

Description

Technical Field

[0001] This disclosure relates to the technical field of model fine-tuning, and more specifically, to a model fine-tuning method, apparatus, device, medium, and product. Background Technology

[0002] In model fine-tuning, data construction and fine-tuning methods are two interrelated key steps that jointly determine the model's ability to respond to domain-specific problems. Therefore, it is necessary to fine-tune the base model with domain-specific data to enhance its professionalism. During model fine-tuning, the quality and scale of the training data are two core factors determining the model's final performance.

[0003] In related technologies, supervised fine-tuning is usually used to directly optimize parameters using labeled data. However, manual labeling is costly and prone to overfitting in the case of small data, making it difficult to support the training needs of large-scale domain models, resulting in insufficient model fine-tuning efficiency. Summary of the Invention

[0004] This disclosure is made in view of the above-mentioned problems. This disclosure provides a model fine-tuning method, apparatus, device, medium, and product.

[0005] In a first aspect, embodiments of this disclosure provide a model fine-tuning method, comprising: Obtain the input data and construct an initial training set carrying multiple initial question-answer pairs based on the input data.

[0006] During any round of iterative fine-tuning, the model from the previous round is fine-tuned based on the target training set of the current round, and the model for the current round is obtained after fine-tuning. The previous round model corresponding to the first round of iterative fine-tuning is the base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set.

[0007] The target question-answer pair is generated based on this round of model.

[0008] The target question-answer pairs that pass the quality screening are added to the target training set, and the next round of iterative fine-tuning is carried out based on the added target training set.

[0009] In one embodiment of the first aspect, constructing an initial training set carrying multiple initial question-answer pairs based on input data includes: The input data is segmented and processed to generate multiple text blocks.

[0010] For each text block, question-answer pairs are extracted, and initial question-answer pairs are generated.

[0011] Calculate the quality score of the initial question-and-answer pairs.

[0012] Use the initial question-answer pairs that meet the quality scoring criteria as the initial training set.

[0013] In one embodiment of the first aspect, the input data is segmented to generate multiple text blocks, including: The input data is initially segmented based on fixed-length segmentation and punctuation segmentation, resulting in multiple initial text blocks.

[0014] Calculate the semantic similarity between adjacent initial text blocks.

[0015] Multiple text blocks are generated based on the comparison results between semantic similarity and similarity threshold.

[0016] In one embodiment of the first aspect, multiple text blocks are generated based on a comparison result between semantic similarity and a similarity threshold, including: If the semantic similarity is less than the similarity threshold, the text is segmented between adjacent initial text blocks corresponding to the semantic similarity to obtain the segmentation result.

[0017] If the semantic similarity is greater than or equal to the similarity threshold, the adjacent preliminary text blocks corresponding to the semantic similarity are merged to obtain the merged result.

[0018] By integrating the segmentation and merging results, multiple text blocks are obtained.

[0019] In one embodiment of the first aspect, calculating the quality score of the initial question-answer pair includes: The initial question-and-answer pairs are scored in terms of quality from multiple dimensions, and the score results for each dimension are obtained. These dimensions include answer accuracy, question relevance, information completeness, language standardization, logical coherence, and security compliance.

[0020] The scores for each dimension are weighted and summed to obtain the quality score.

[0021] In one embodiment of the first aspect, the scoring criteria include: The quality score is greater than the first scoring threshold, and the score results for each dimension of the initial question-answer pair are greater than the second scoring threshold; wherein, the first scoring threshold is greater than the second scoring threshold.

[0022] In one embodiment of the first aspect, the iteration is terminated after each round of iterative fine-tuning is completed and before the next round of iterative fine-tuning begins, in response to the satisfaction of any of the following conditions: During two consecutive rounds of iterative fine-tuning, the performance improvement of the current model compared to the previous model is less than the preset performance improvement threshold; the current iteration round reaches the preset maximum iteration round; and the performance evaluation result of the current model on the validation set is less than the performance evaluation result of the previous model on the validation set.

[0023] In a second aspect, embodiments of this disclosure provide a model fine-tuning apparatus, comprising: The data acquisition unit is used to acquire input data and construct an initial training set carrying multiple initial question-answer pairs based on the input data. The model fine-tuning unit is used to fine-tune the previous model based on the target training set of the current round during any round of iterative fine-tuning, and obtain the current model after fine-tuning; wherein, the previous model corresponding to the first round of iterative fine-tuning is the base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set. The question-answer pair generation unit is used to generate target question-answer pairs based on the current round of the model. The training set update unit is used to add the target question-answer pairs that have passed the quality screening to the target training set, so as to perform the next round of iterative fine-tuning based on the added target training set.

[0024] In a third aspect, embodiments of this disclosure provide an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps in the above-described model fine-tuning method are performed.

[0025] In a fourth aspect, embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps in the model fine-tuning method described above.

[0026] In a fifth aspect, embodiments of this disclosure provide a computer program product stored in a storage medium, the program product being executed by at least one processor to implement the steps in the model fine-tuning method described above.

[0027] As will be described in detail below, a model fine-tuning method, apparatus, device, medium, and product according to embodiments of this disclosure are disclosed. In embodiments of this disclosure, input data is first acquired, and an initial training set carrying multiple initial question-answer pairs is constructed based on the input data. Then, during any round of iterative fine-tuning, the previous round model is fine-tuned based on the target training set of the current round, resulting in the current round model; wherein, the previous round model corresponding to the first round of iterative fine-tuning is the base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set. Target question-answer pairs are generated based on the current round model. Finally, the target question-answer pairs that pass quality screening are added to the target training set, and the next round of iterative fine-tuning is performed based on the added target training set. Through the above processing method, this disclosure constructs an initial training set through an automated question-answer pair extraction and quality screening mechanism, filtering out low-quality samples before the data enters the training stage, avoiding resource waste on invalid data in subsequent models, and ensuring the data quality of the initial training set. Moreover, the target question-answer pairs that pass the quality screening in any round are continuously used as training data in all subsequent rounds. The generation cost only occurs once, which significantly reduces the data acquisition cost. This allows each iteration to continue to optimize the model based on the fine-tuned model of the previous round, without having to rebuild the complete training set in each round, further improving the efficiency of model iteration.

[0028] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0029] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0030] Figure 1 This is a flowchart of a model fine-tuning method according to an embodiment of the present disclosure.

[0031] Figure 2 This is a flowchart of a model fine-tuning process according to an embodiment of the present disclosure.

[0032] Figure 3 This is a schematic diagram of a model fine-tuning device according to an embodiment of the present disclosure.

[0033] Figure 4 This is a schematic diagram of an electronic device according to an embodiment of the present disclosure.

[0034] Figure 5 This is a schematic diagram of a computer program product according to an embodiment of the present disclosure. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this disclosure more apparent, exemplary embodiments according to this disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments of this disclosure. It should be understood that this disclosure is not limited to the exemplary embodiments described herein.

[0036] To facilitate understanding of this embodiment, a model fine-tuning method disclosed in this disclosure will first be described in detail. (See also...) Figure 1 The diagram shows a flowchart of a model fine-tuning method provided in this embodiment of the present disclosure. This method can be applied to an intelligent development platform and includes steps S101 to S104, wherein: Step S101: Obtain input data and construct an initial training set carrying multiple initial question-answer pairs based on the input data.

[0037] Here, the input data consists of unstructured documents collected by users in vertical fields, including financial research reports, legal provisions, medical literature, corporate rules and regulations, product user manuals, Q&A documents, and historical customer service dialogue records.

[0038] In this embodiment of the disclosure, the user can provide input data to the system by uploading. After receiving the uploaded document, the system extracts text from the document content to obtain the original text content. The system can also obtain input data through a network interface or database import method; this disclosure does not require it to be done in this way.

[0039] To address this, multiple high-quality initial question-answer pairs can be constructed based on the input data, and an initial training set can be built using these initial question-answer pairs.

[0040] Step S102: During any round of iterative fine-tuning, the model of the previous round is fine-tuned based on the target training set of the current round, and the model of the current round is obtained after fine-tuning; wherein, the previous round model corresponding to the first round of iterative fine-tuning is the base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set.

[0041] Here, the previous model refers to the model to be fine-tuned when entering the current iteration, the current model refers to the model obtained after fine-tuning the previous model, and the target training set of the current iteration refers to the training dataset used when entering the current iteration.

[0042] Based on industry-specific documentation, we constructed a high-quality seed dataset and then introduced a self-evolutionary distillation method to perform multiple rounds of iterative fine-tuning on the pre-trained model. The initial training set can be understood as high-quality seed data generated by professional annotation or construction tools.

[0043] When performing the first round of iterative fine-tuning: the previous model is the base model, the target training set is the initial training set, and then the base model is fine-tuned under supervision based on the initial training set, and the first round model is obtained after fine-tuning.

[0044] The base model is a large language model that has been pre-trained on a large scale, including but not limited to pre-trained models with general text generation capabilities such as the Qwen series, LLaMA series, and ChatGLM series.

[0045] In the second round of iterative fine-tuning, the previous model is the first-round model, and the target training set for the current round is the union of the initial training set and the subsequently added question-answer pairs. Based on this union, the first-round model is fine-tuned under supervision to obtain the second-round model.

[0046] Similarly, during the j-th iteration of fine-tuning, the model from the previous round is the (j-1)-th round model, and the target training set for the current round is the union of the initial training set and the newly added question-answer pairs in the previous (j-1)-th iterations. Based on this union, the (j-1)-th round model is fine-tuned under supervision to obtain the j-th round model.

[0047] In each round of supervised fine-tuning, the training loss is calculated using the cross-entropy loss function, and the AdamW optimizer is used to update the model parameters, gradually reducing the model's loss value on the training set. As the number of iterations increases, the model continuously optimizes its parameters on the target training set, gradually learning and consolidating domain knowledge.

[0048] In the above embodiments, this disclosure performs one or more rounds of iterative fine-tuning on the base model. Each round of iterative fine-tuning is based on the target training set of the current round to supervise the fine-tuning of the previous round model. After fine-tuning, the current round model is obtained, thereby gradually adapting to the professional knowledge and expression habits of the target domain.

[0049] Step S103: Generate target question-answer pairs based on the current round model.

[0050] Step S104: Add the target question-answer pairs that have passed the quality screening to the target training set, so as to perform the next round of iterative fine-tuning based on the added target training set.

[0051] After each round of iterative fine-tuning, this disclosure, based on the model obtained after that round of fine-tuning, inputs the same or different input data as in step S101 into the current round model, and the current round model generates corresponding question-answer pairs. The structure of the generated target question-answer pairs is the same as the initial question-answer pairs, that is, each target question-answer pair contains a question field and an answer field.

[0052] Since the model in this round has absorbed more domain knowledge than the previous model, the generated question-answer pairs have improved in terms of content coverage and knowledge depth. Therefore, the generated target question-answer pairs are used to expand the target training set of the previous round, thereby providing new information increments to the target training set.

[0053] Furthermore, since the target question-answer pairs generated by this model may have inconsistent quality, they are not all directly added to the training set, but are screened for quality before being added.

[0054] In this embodiment of the disclosure, after generating the target question-answer pair, a quality screening process is performed on the target question-answer pair. Specifically, the target question-answer pair can be input into a quality assessment model, which scores the target question-answer pair from multiple dimensions and determines whether the target question-answer pair passes the quality screening based on the scoring results. Target question-answer pairs that pass the quality screening are considered high-quality question-answer pairs.

[0055] After the target question-answer pair passes the quality screening, the target question-answer pair is added to the target training set of the current round, so that the added target training set can be used for the next round of iterative fine-tuning.

[0056] In this disclosure, the target question-answer pairs selected through quality screening are appended to the existing training set without replacing or deleting the original question-answer pairs in the training set. Through the above incremental accumulation mechanism, the data size of the training set continues to expand with each iteration, and the amount of information in the model training increases with each iteration.

[0057] In the above embodiments, this disclosure constructs an initial training set through an automated question-answer pair extraction and quality screening mechanism. This filters out low-quality samples before the data enters the training stage, avoiding resource waste on invalid data in subsequent models and ensuring the data quality of the initial training set. Moreover, any target question-answer pair that passes the quality screening in any round is continuously used as training data in all subsequent rounds, with the generation cost occurring only once. This significantly reduces data acquisition costs and allows each iteration to continue optimizing the model based on the fine-tuned model from the previous round, eliminating the need to rebuild a complete training set each round and further improving model iteration efficiency.

[0058] In an optional implementation, the above steps construct an initial training set carrying multiple initial question-answer pairs based on the input data, specifically including the following steps: The input data is segmented and processed to generate multiple text blocks.

[0059] For each text block, question-answer pairs are extracted, and initial question-answer pairs are generated.

[0060] Calculate the quality score of the initial question-and-answer pairs.

[0061] Use the initial question-answer pairs that meet the quality scoring criteria as the initial training set.

[0062] After obtaining the input data, this disclosure first performs segmentation processing on the input data. Since the input data is usually a long and complex unstructured document, its overall length may exceed the context window limit of the language model, making it impossible to directly extract question-answer pairs. Therefore, it is necessary to first segment the input data into text blocks of appropriate length and semantic completeness.

[0063] The segmentation process employs a two-stage strategy of rule-first and semantic optimization: In the first stage—rule-based segmentation—the system executes two segmentation mechanisms in parallel to ensure processing efficiency and structural rationality; in the second stage—semantic optimization—the system further introduces the Sentence-BERT deep learning model to verify the semantic coherence of the text blocks after rule segmentation.

[0064] After obtaining multiple text blocks, question-answer pairs are extracted from each text block to generate initial question-answer pairs. This disclosure can extract question-answer pairs from the text blocks based on a first language model, which is a large language model with question-answer pair generation capability, such as the Qwen2.5-32B-Instruct model.

[0065] To address this, each text block can be input into the first language model, and a structured prompt template can be used to guide the first language model to generate one or more questions and answers related to the text content for each text block, forming an initial question-answer pair.

[0066] The prompt template includes: task role settings and instruction descriptions, output format constraints, Few-shot examples, text cleanup and formatting requirements.

[0067] For example, the task role setting and instruction description: the first language model is set as a "question-answer pair extraction expert", and it is required to generate fluent, relevant and factually accurate question-answer pairs based on the given text.

[0068] Output format constraints: The model must be output strictly in JSON format, each question-answer pair contains two fields, "question" and "answer", and the whole pair is organized in list form.

[0069] Few-shot examples: Provides several typical examples to show the question-and-answer pair formats corresponding to different types of text fragments (e.g., factual, explanatory, inferential). The examples show that the questions to be generated should be clear and the answers should fully cover the key information.

[0070] Text cleaning and formatting requirements: The model should avoid introducing external knowledge, not copy question-answer pairs from examples, and ensure that the language is concise and free of redundant expressions.

[0071] To effectively ensure the quality and reliability of the generated initial question-and-answer pairs, a multi-stage verification mechanism was constructed, including format verification, integrity verification, semantic relevance verification, and manual sampling verification.

[0072] The format validation process first checks whether the output is valid JSON through automated parsing and verifies whether it contains required fields ("question", "answer"). Resulting results with incorrect format are automatically intercepted and recorded.

[0073] Integrity check: Detects whether there are missing elements in the question-answer pair (e.g., incomplete question, empty answer, or obvious truncation), and excludes invalid or low-completeness results.

[0074] Semantic relevance verification: Based on the pre-trained model, the semantic similarity between the question and the answer is calculated, and combined with rule filtering (e.g., keyword overlap rate, question word matching, etc.), the initial question-answer pairs that do not match the content, are irrelevant to the question, or are logically inconsistent are eliminated.

[0075] Manual sampling verification: Samples are periodically taken from the generated results, the quality is evaluated by experts, and error analysis is performed to iteratively optimize the prompt templates and verification rules.

[0076] The above implementation fully combines the powerful generation capabilities of the large language model with a multi-level automatic verification mechanism, significantly improving the accuracy, usability, and system stability of question-answer pair extraction, making it suitable for high-quality structured extraction scenarios for large-scale text information.

[0077] The prompt template is as follows: # Roles and Missions You are a question-and-answer pair extraction expert. Your task is to generate one or more high-quality, factually accurate, and semantically relevant question-and-answer pairs based on text snippets provided by users. The question-and-answer pairs must be based solely on the provided text content and must not incorporate external knowledge.

[0078] # Output format requirements You must strictly adhere to the following JSON format when outputting the results, and output only the JSON object without any additional interpretation: { "qa_pairs": [ { "question": "text-based generation question", "answer": "The answer found in the text" } ] } # Few-shot Learning Example Please refer to the following three examples to understand how to generate question-answer pairs from a given text: Example 1: Factual Questions and Answers Text: "Tesla was founded in 2003 by Martin Eberhard and Marc Tarpenning. Elon Musk joined the company in 2004 and led the Series A funding round." Output: { "qa_pairs": [ { Question: In what year was Tesla Motors founded? Answer: 2003. } ] } Example 2: Explanatory Questions and Answers Text: "Photosynthesis is the process by which plants, algae, and certain bacteria use light energy to convert carbon dioxide and water into organic matter (such as glucose) and release oxygen." Output: { "qa_pairs": [ { Question: What is photosynthesis? Answer: Photosynthesis is the process by which plants, algae, and certain bacteria use light energy to convert carbon dioxide and water into organic matter and release oxygen. } ] } Example 3: Inference-based Question and Answer Text: "Although the weather forecast predicted rain, the entire afternoon was sunny. As a result, the school sports meet, which was originally scheduled to be held indoors, was moved to the playground." Output: { "qa_pairs": [ { Question: Why was the location of the school sports meet changed? Answer: "Because the actual weather was sunny, which contradicted the rain forecast, the event was moved from indoors to the playground." } ] } # Text fragment to be processed Now, please generate question-and-answer pairs for the following text based on the rules and examples above: {text} # Important Requirements 1. The generated questions must be clear and unambiguous, and the answers must be inferred directly or indirectly from the text.

[0079] 2. Ensure that the question-and-answer pair fully covers the core information of the text.

[0080] 3. Copying any specific content from the above examples is strictly prohibited.

[0081] After obtaining the initial question-and-answer pairs, a quality score is calculated for each pair. Because first-language models may suffer from illusion problems during generation (i.e., generating inaccurate or fabricated content), not all generated initial question-and-answer pairs possess sufficient quality for model fine-tuning. Therefore, a quality score is required for each initial question-and-answer pair to assess its quality level.

[0082] This disclosure enables quality scoring of initial question-and-answer pairs based on a second language model, which is a large language model with quality discrimination capabilities, such as the GLM4-9B-Chat model. This model demonstrates superior performance in Chinese semantic understanding and discrimination tasks. While its model architecture and training data composition differ from the Qwen2.5 generative model, introducing this model to score the generated question-and-answer pairs effectively reduces subjective bias in the evaluation process, ensuring the objectivity and reliability of the scoring results.

[0083] After obtaining the quality scores of each initial question-answer pair, the initial question-answer pairs that meet the scoring criteria are selected as the initial training set.

[0084] In one optional implementation, the scoring criteria include: The quality score is greater than the first scoring threshold, and the score results for each dimension of the initial question-answer pair are greater than the second scoring threshold; wherein, the first scoring threshold is greater than the second scoring threshold.

[0085] For example, the first scoring threshold can be set to 4.5 points, and the second scoring threshold can be set to 3 points. That is, only initial question-answer pairs with a quality score greater than 4.5 points and a score greater than 3 points for each dimension are included in the initial training set.

[0086] By applying the above dual conditions, we can ensure that the overall quality of the question-and-answer pairs entering the training set is high enough, while avoiding significant shortcomings in individual dimensions.

[0087] In an optional implementation, the above steps segment the input data to generate multiple text blocks, specifically including the following steps: The input data is initially segmented based on fixed-length segmentation and punctuation segmentation, resulting in multiple initial text blocks.

[0088] Calculate the semantic similarity between adjacent initial text blocks.

[0089] Multiple text blocks are generated based on the comparison results between semantic similarity and similarity threshold.

[0090] This disclosure provides an exemplary description of the first-phase strategy.

[0091] Specifically, this disclosure can perform preliminary segmentation of input data based on two methods: fixed-length segmentation and punctuation segmentation.

[0092] The fixed-length segmentation divides the data into equal parts based on a preset number of characters (e.g., 500 characters) and sets an overlap buffer of a preset number of characters (e.g., 50 characters) to avoid truncation of critical information due to improper segmentation.

[0093] Punctuation segmentation is based on natural language boundaries such as periods, question marks, exclamation marks, and paragraph marks. LangChain's RecursiveCharacterTextSplitter tool can be used to achieve intelligent segmentation that conforms to language habits, improving the readability and logical integrity of text blocks.

[0094] The two segmentation methods described above are executed in parallel. Fixed-length segmentation ensures that the length of the text block is moderate and does not exceed the input length limit of the subsequent language model. At the same time, punctuation segmentation ensures the rationality of the text block in terms of language structure, balancing processing efficiency and language structure integrity.

[0095] Furthermore, an exemplary description of the second-stage strategy is provided here.

[0096] Specifically, the semantic coherence of each text block after initial segmentation is verified based on a semantic similarity model, which means calculating the semantic similarity between adjacent initial text blocks. The semantic similarity model used is the Sentence-BERT model.

[0097] This disclosure can first encode adjacent initial text blocks into high-dimensional semantic vectors, calculate the semantic similarity between adjacent initial text blocks, and finally segment or merge the initial text blocks based on the comparison results between semantic similarity and similarity threshold to generate multiple text blocks.

[0098] Semantic similarity measures the degree of semantic coherence between adjacent text blocks. Higher semantic similarity indicates stronger semantic coherence between adjacent text blocks; lower semantic similarity indicates a possible semantic shift between adjacent text blocks.

[0099] In an optional implementation, the above steps generate multiple text blocks based on the comparison result between semantic similarity and a similarity threshold, specifically including the following steps: If the semantic similarity is less than the similarity threshold, the text is segmented between adjacent initial text blocks corresponding to the semantic similarity to obtain the segmentation result.

[0100] If the semantic similarity is greater than or equal to the similarity threshold, the adjacent preliminary text blocks corresponding to the semantic similarity are merged to obtain the merged result.

[0101] By integrating the segmentation and merging results, multiple text blocks are obtained.

[0102] In this embodiment of the disclosure, if the semantic similarity is less than the similarity threshold (e.g., 0.65), it indicates that a semantic leap has occurred between adjacent text blocks, and there is a significant difference in the content theme or discussion direction between the two. In this case, the adjacent initial text blocks corresponding to the semantic similarity are segmented so that the semantically different content is divided into independent text blocks, thus avoiding mixing content with different themes in the same text block.

[0103] If the semantic similarity is greater than or equal to the similarity threshold, it indicates that the adjacent text blocks are semantically coherent. In this case, the adjacent text blocks are merged so that semantically related content is combined into the same text block to form a text block with more complete semantics and more coherent context.

[0104] By sequentially performing the above comparison and segmentation / merging operations on all adjacent initial text blocks, the initial segmentation results are semantically optimized and adjusted. Finally, the segmentation and merging results are integrated to obtain multiple text blocks.

[0105] After the above two-stage processing, multiple text blocks are generated for subsequent question-answer pair extraction. Each text block has a moderate length and complete semantic units, which neither exceeds the input length limit of the subsequent language model, but also retains complete semantic information.

[0106] In an optional implementation, the above steps calculate the quality score of the initial question-answer pair, specifically including the following steps: The initial question-and-answer pairs are scored in terms of quality from multiple dimensions, and the score results for each dimension are obtained. These dimensions include answer accuracy, question relevance, information completeness, language standardization, logical coherence, and security compliance.

[0107] The scores for each dimension are weighted and summed to obtain the quality score.

[0108] This disclosure introduces a second language model to score the quality of initial question-and-answer pairs, effectively reducing subjective bias in the evaluation process and ensuring the objectivity and reliability of the scoring results. It comprehensively evaluates the quality of question-and-answer pairs from six core dimensions. Building upon the existing dimensions of answer accuracy, question relevance, information completeness, and language standardization, two innovative dimensions—logical coherence and security compliance—are added, forming a more comprehensive six-dimensional evaluation system.

[0109] Each dimension employs a multi-level scoring system of 1–5 points, and introduces dynamic weight allocation and a multi-dimensional veto mechanism.

[0110] Specifically, answer accuracy is used to assess whether an answer is strictly based on the original text and whether it contains untraceable external information or subjective assumptions. The basic level requires answers to be strictly based on the original text; the innovation level introduces traceability verification and illusion suppression index. If an answer contains untraceable external information or subjective assumptions, the score for this item cannot exceed 2 points; if multiple pieces of evidence within the text can be accurately linked, bonus points can be obtained. The higher the answer accuracy, the higher the factual reliability of the correct answer.

[0111] Question relevance is used to assess whether a question relates to the core content of the text and whether it has question-and-answer value. It not only determines whether a question relates to the core content of the text, but also introduces an assessment of question-and-answer value density—whether the question elicits deeper reasoning and avoids trivial information. Questions that only involve peripheral details or are vaguely expressed receive limited scores; those that can elicit implicit logic or key contradictions in the text receive higher scores.

[0112] Information completeness is used to assess whether the answer covers all the key points of the question. In addition to examining whether all the key points of the question are covered, a structural completeness assessment is added: whether the answer has logical organization (e.g., point-by-point argumentation, causal progression), and whether it avoids information redundancy. Weighted deductions are applied to missing core elements, while bonus points are awarded for efficient summarization and structured expression.

[0113] Language standardization is used to assess the grammatical correctness, terminological consistency, and clarity of expression of answers. Going beyond basic grammatical correctness requirements, it incorporates assessments of contextual fit and user-friendliness. This includes: consistency of terminology with the original text, fit of expression style with text type, and whether the answer possesses qualities such as clarity, comprehensibility, and unambiguity.

[0114] Logical coherence is used to assess the internal logical consistency of the question-and-answer pair. This includes: the direct correspondence between the question and the answer; the reasonableness of the logical progression within the answer; and the existence of causal breaks or contradictory statements. It also examines the ability to maintain context in multi-turn dialogue scenarios.

[0115] Security and compliance are used to assess the security of the content in question-and-answer pairs. This includes: whether there is any bias, discrimination, inappropriate content, or privacy breach risk, and whether it complies with ethical standards and relevant laws and regulations. This dimension has veto power; high-risk content will be rejected outright.

[0116] Each dimension is scored from 1 to 5 points, with higher scores indicating better quality for that dimension. The second language model scores the initial question-answer pairs based on these six dimensions and outputs the score for each dimension.

[0117] After obtaining the scores for each dimension, the scores for each dimension are weighted and summed to obtain the initial quality score for the question-answer pair. The weights for answer accuracy, question relevance, information completeness, language accuracy, logical coherence, and security compliance can be set to 0.25, 0.15, 0.2, 0.1, and 0.1 respectively. These weighting coefficients can be adjusted as needed.

[0118] In addition, this disclosure may also generate an overall judgment based on the following rules: If the safety compliance score is below 3 points, or if there is a factual error, it will be rejected directly; if the quality score is less than or equal to 4.5 and the score for each dimension is less than 3 points, it will be accepted directly; if the quality score is between 3.5 and 4.0 and the dimension scores fluctuate greatly, a manual review process will be triggered.

[0119] In the above implementation, the designed scoring mechanism enables the question-and-answer pairs to be comprehensively evaluated in terms of factual accuracy, content relevance, information completeness, language expression, logical structure, and security compliance, so that the final output quality score can objectively reflect the overall quality level of the question-and-answer pairs.

[0120] In an optional implementation, the above steps terminate the iteration after each round of iterative fine-tuning is completed and before the start of the next round of iterative fine-tuning, in response to the satisfaction of any of the following conditions: During two consecutive rounds of iterative fine-tuning, the performance improvement of the current model compared to the previous model is less than the preset performance improvement threshold; the current iteration round reaches the preset maximum iteration round; and the performance evaluation result of the current model on the validation set is less than the performance evaluation result of the previous model on the validation set.

[0121] Here, the validation set is a set reserved when constructing the initial training set, used to evaluate the model's performance after each iteration. This validation set remains unchanged throughout the iteration process to ensure the comparability of model performance evaluation results across different rounds.

[0122] When the model performance approaches saturation, reaches the preset upper limit, or begins to degrade, the iteration should be terminated in a timely manner to avoid overfitting or wasting computational resources due to excessive iteration.

[0123] Specifically, in two consecutive rounds of iterative fine-tuning, if the performance improvement of the current model relative to the previous model is less than the preset performance improvement threshold, the iteration is terminated.

[0124] The performance improvement value measures the extent to which the model performs better on the validation set compared to the previous iteration. When the performance improvement is insufficient for two consecutive iterations, it indicates that the model performance has reached saturation, and further iterations are unlikely to bring significant gains.

[0125] If the current iteration reaches the preset maximum number of iterations (e.g., 5 iterations), the iteration terminates. The preset maximum number of iterations is a pre-defined maximum allowed number of iterations to prevent the iteration process from proceeding indefinitely.

[0126] If the performance evaluation result of the current model on the validation set is lower than that of the previous model on the validation set, the iteration is terminated. When the validation set performance of the current model is lower than that of the previous model, it indicates that the model has overfitted, meaning that the knowledge absorbed by the model on the training set has negatively impacted its generalization performance on the validation set, and continuing the iteration may further exacerbate the performance degradation.

[0127] In the above embodiments, by setting the above multiple termination conditions, this disclosure can automatically terminate the iteration when the model performance tends to saturate, reaches the preset upper limit, or begins to degenerate, effectively preventing the risk of overfitting and waste of computing resources caused by overtraining, ensuring that the iterative fine-tuning process stops in time in the optimal or near-optimal state, and achieving continuous improvement of model capabilities without introducing external annotations through the co-evolution of model and data, which is particularly suitable for vertical fields where annotation resources are scarce.

[0128] The following is combined with Figure 2 The fine-tuning process of the above model is described in detail below: S201: Obtain input data.

[0129] S202: The input data is segmented and processed to generate multiple text blocks.

[0130] S203: Extract question-answer pairs for each text block and generate initial question-answer pairs.

[0131] S204: Calculate the quality score of the initial question-answer pair.

[0132] S205: Use the initial question-answer pairs whose quality scores meet the scoring criteria as the initial training set.

[0133] S206: During any round of iterative fine-tuning, the model from the previous round is fine-tuned based on the target training set of the current round, and the model for the current round is obtained after fine-tuning.

[0134] The previous model corresponding to the first round of iterative fine-tuning is the base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set.

[0135] S207: Generate target question-answer pairs based on the current round model.

[0136] S208: Add the target question-answer pairs that have passed the quality screening to the target training set, and perform the next round of iterative fine-tuning based on the added target training set.

[0137] As can be seen from the above description, the technical solution disclosed herein has the following advantages: 1. End-to-end fully automated processing of unstructured documents to generate high-quality question-answer pairs significantly reduces the cost of manual annotation, while avoiding the low efficiency of manual annotation or the illusion of automatic annotation in traditional methods.

[0138] 2. The addition of a quality assessment module reduces illusion problems, balancing efficiency and quality. It solves common illusion problems in automatic annotation (generating inaccurate or fictitious content) while avoiding the inefficiency of purely manual verification.

[0139] 3. By adopting a configurable Prompt template library, the generation rules can be dynamically adjusted according to the characteristics of the domain, eliminating the need to redesign the data construction process for different domains and significantly improving the versatility and scalability of the method.

[0140] 4. Employ a self-evolutionary distillation model fine-tuning approach, a closed-loop process of generation-training-evaluation-regeneration, and add a data quality assessment step to achieve continuous optimization of data quality and model performance.

[0141] Based on the same inventive concept, this disclosure also provides a model fine-tuning device corresponding to the model fine-tuning method. Since the principle of the device in this disclosure for solving the problem is similar to the model fine-tuning method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0142] Reference Figure 3 The diagram shown is a schematic of a model fine-tuning device provided in an embodiment of this disclosure. The device includes: a data acquisition unit 30, a model fine-tuning unit 31, a question-answer pair generation unit 32, and a training set update unit 33; wherein: The data acquisition unit is used to acquire input data and construct an initial training set carrying multiple initial question-answer pairs based on the input data. The model fine-tuning unit is used to fine-tune the previous model based on the target training set of the current round during any round of iterative fine-tuning, and obtain the current model after fine-tuning; wherein, the previous model corresponding to the first round of iterative fine-tuning is the base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set. The question-answer pair generation unit is used to generate target question-answer pairs based on the current round of the model. The training set update unit is used to add the target question-answer pairs that have passed the quality screening to the target training set, so as to perform the next round of iterative fine-tuning based on the added target training set.

[0143] In one possible implementation, the device is also used for: The input data is segmented and processed to generate multiple text blocks.

[0144] For each text block, question-answer pairs are extracted, and initial question-answer pairs are generated.

[0145] Calculate the quality score of the initial question-and-answer pairs.

[0146] Use the initial question-answer pairs that meet the quality scoring criteria as the initial training set.

[0147] In one possible implementation, the device is also used for: The input data is initially segmented based on fixed-length segmentation and punctuation segmentation, resulting in multiple initial text blocks.

[0148] Calculate the semantic similarity between adjacent initial text blocks.

[0149] Multiple text blocks are generated based on the comparison results between semantic similarity and similarity threshold.

[0150] In one possible implementation, the device is also used for: If the semantic similarity is less than the similarity threshold, the text is segmented between adjacent initial text blocks corresponding to the semantic similarity to obtain the segmentation result.

[0151] If the semantic similarity is greater than or equal to the similarity threshold, the adjacent preliminary text blocks corresponding to the semantic similarity are merged to obtain the merged result.

[0152] By integrating the segmentation and merging results, multiple text blocks are obtained.

[0153] In one possible implementation, the device is also used for: The initial question-and-answer pairs are scored in terms of quality from multiple dimensions, and the score results for each dimension are obtained. These dimensions include answer accuracy, question relevance, information completeness, language standardization, logical coherence, and security compliance.

[0154] The scores for each dimension are weighted and summed to obtain the quality score.

[0155] In one possible implementation, the device is also used for: The quality score is greater than the first scoring threshold, and the score results for each dimension of the initial question-answer pair are greater than the second scoring threshold; wherein, the first scoring threshold is greater than the second scoring threshold.

[0156] In one possible implementation, the device is also used for: During two consecutive rounds of iterative fine-tuning, the performance improvement of the current model compared to the previous model is less than the preset performance improvement threshold; the current iteration round reaches the preset maximum iteration round; and the performance evaluation result of the current model on the validation set is less than the performance evaluation result of the previous model on the validation set.

[0157] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0158] Corresponding to Figure 1 In addition to the model fine-tuning method in this disclosure, this embodiment also provides an electronic device 40, such as... Figure 4 The diagram shown is a structural schematic of an electronic device 40 provided in an embodiment of this disclosure, including: The system includes a processor 41, a memory 42, and a bus 43. The memory 42 stores execution instructions and includes main memory 421 and external memory 422. The main memory 421, also called internal memory, temporarily stores the computational data in the processor 41, as well as data exchanged with external memory such as a hard disk. The processor 41 exchanges data with the external memory 422 through the main memory 421. When the electronic device 40 is running, the processor 41 communicates with the memory 42 through the bus 43, causing the processor 41 to execute the following instructions: Obtain the input data and construct an initial training set carrying multiple initial question-answer pairs based on the input data.

[0159] During any round of iterative fine-tuning, the model from the previous round is fine-tuned based on the target training set of the current round, and the model for the current round is obtained after fine-tuning. The previous round model corresponding to the first round of iterative fine-tuning is the base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set.

[0160] The target question-answer pair is generated based on this round of model.

[0161] The target question-answer pairs that pass the quality screening are added to the target training set, and the next round of iterative fine-tuning is carried out based on the added target training set.

[0162] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the model fine-tuning method described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0163] This disclosure also provides a computer program product 50, such as... Figure 5 The diagram shown is a schematic diagram of the structure of a computer program product 50 provided in an embodiment of this disclosure. The computer program product 50 carries a computer program 51. The program included in the computer program 51 can be used to execute the steps of the model fine-tuning method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0164] The above description, with reference to the accompanying drawings, outlines a model fine-tuning method, apparatus, device, medium, and product according to embodiments of this disclosure. In these embodiments, input data is first acquired, and an initial training set carrying multiple initial question-and-answer pairs is constructed based on the input data. Then, during any round of iterative fine-tuning, the previous round model is fine-tuned based on the target training set of the current round, resulting in the current round model. The previous round model corresponding to the first round of iterative fine-tuning is the base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set. Target question-and-answer pairs are then generated based on the current round model. Finally, the target question-and-answer pairs that pass quality screening are added to the target training set for the next round of iterative fine-tuning. Through this processing method, this disclosure constructs an initial training set using an automated question-and-answer pair extraction and quality screening mechanism, filtering out low-quality samples before the data enters the training stage. This avoids wasting resources on invalid data in subsequent models and ensures the data quality of the initial training set. Moreover, the target question-answer pairs that pass the quality screening in any round are continuously used as training data in all subsequent rounds. The generation cost only occurs once, which significantly reduces the data acquisition cost. This allows each iteration to continue to optimize the model based on the fine-tuned model of the previous round, without having to rebuild the complete training set in each round, further improving the efficiency of model iteration.

[0165] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0166] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0167] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0168] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0169] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0170] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0171] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A model fine-tuning method, characterized in that, include: Obtain input data and construct an initial training set carrying multiple initial question-answer pairs based on the input data; During any round of iterative fine-tuning, the model of the previous round is fine-tuned based on the target training set of the current round, and the model of the current round is obtained after fine-tuning; wherein, the previous round model corresponding to the first round of iterative fine-tuning is the base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set; Generate target question-answer pairs based on the current round model; The target question-answer pairs that pass the quality screening are added to the target training set for the next round of iterative fine-tuning based on the added target training set.

2. The method according to claim 1, characterized in that, The construction of an initial training set carrying multiple initial question-answer pairs based on the input data includes: The input data is segmented to generate multiple text blocks. For each of the text blocks, question-answer pairs are extracted, and initial question-answer pairs are generated after extraction. Calculate the quality score of the initial question-answer pair; The initial question-answer pairs that meet the scoring criteria are used as the initial training set.

3. The method according to claim 2, characterized in that, The input data is segmented to generate multiple text blocks, including: The input data is initially segmented based on fixed-length segmentation and punctuation segmentation, resulting in multiple initial text blocks. Calculate the semantic similarity between adjacent initial text blocks; The multiple text blocks are generated based on the comparison results between the semantic similarity and the similarity threshold.

4. The method according to claim 3, characterized in that, The generation of the plurality of text blocks based on the comparison result between the semantic similarity and the similarity threshold includes: If the semantic similarity is less than the similarity threshold, the adjacent initial text blocks corresponding to the semantic similarity are segmented to obtain the segmentation result; If the semantic similarity is greater than or equal to the similarity threshold, the adjacent preliminary text blocks corresponding to the semantic similarity are merged to obtain the merged result; By integrating the segmentation results and the merging results, the multiple text blocks are obtained.

5. The method according to claim 2, characterized in that, The calculation of the quality score for the initial question-answer pair includes: The initial question-answer pairs are scored in quality from multiple dimensions to obtain a score result for each dimension; wherein, the multiple dimensions include answer accuracy, question relevance, information completeness, language standardization, logical coherence, and security compliance; The quality score is obtained by weighted summation of the scores corresponding to each dimension.

6. The method according to claim 2, characterized in that, The scoring criteria include: The quality score is greater than the first scoring threshold, and the score result corresponding to each dimension of the initial question-answer pair is greater than the second scoring threshold; wherein, the first scoring threshold is greater than the second scoring threshold.

7. The method according to claim 1, characterized in that, The iteration terminates after each round of iterative fine-tuning is completed and before the next round of iterative fine-tuning begins, in response to the satisfaction of any of the following conditions: During two consecutive rounds of iterative fine-tuning, the performance improvement of the current model compared to the previous model was less than the preset performance improvement threshold. The current iteration round has reached the preset maximum iteration round; the performance evaluation result of the model in this round on the validation set is less than the performance evaluation result of the model in the previous round on the same validation set.

8. A model fine-tuning device, characterized in that, include: The data acquisition unit is used to acquire input data and construct an initial training set carrying multiple initial question-answer pairs based on the input data; The model fine-tuning unit is used to fine-tune the previous model based on the target training set of the current round during any round of iterative fine-tuning, and obtain the current model after fine-tuning; wherein, the previous model corresponding to the first round of iterative fine-tuning is the base model, and the target training set corresponding to the first round of iterative fine-tuning is the initial training set. The question-answer pair generation unit is used to generate target question-answer pairs based on the current round model; The training set update unit is used to add the target question-answer pairs that have passed the quality screening to the target training set, so as to perform the next round of iterative fine-tuning based on the added target training set.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the model fine-tuning method as described in any one of claims 1 to 7 are performed.

10. A computer program product, characterized in that, The computer program product is stored in a storage medium and is executed by at least one processor to implement the model fine-tuning method as described in any one of claims 1 to 7.

Images