A self-guided workflow-based instruction fine-tuning data end-to-end synthesis system

Abstract

The application discloses a kind of instruction fine-tuning data end-to-end synthesis system based on self-guided workflow, the system includes: task automatic discovery module, for based on preset prompt word and generation model, the original corpus data of target field is carried out semantic analysis, obtain multiple task modes in original corpus data, and generate instruction template for each task mode;Closed loop self-optimizing workflow module, through instruction data generation model, based on instruction template output multiple instruction data of target field;Use these data to carry out field fine-tuning and effect evaluation to target large language model;Then, according to the evaluation result, the preset prompt word is optimized, and the updated instruction template is obtained to drive the next round of iteration. Fine-tuned model can be used for the question and answer, abstract, reasoning, comparison and other tasks of target field.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence and natural language processing, and specifically relates to an end-to-end instruction fine-tuning data synthesis system based on a self-guided workflow. Background Technology

[0002] Fine-tuning instructions for large language models is a crucial step in improving their task execution capabilities, but obtaining high-quality instruction data has always been a core bottleneck restricting the development of this technology. While traditional manual annotation methods can ensure data quality, they face problems such as high costs, long annotation cycles, and difficulty in ensuring annotation consistency, making it difficult to meet the data requirements for large-scale model training.

[0003] In recent years, academia and industry have proposed various automated data generation schemes to alleviate the aforementioned problems. The Self-Instruct method generates instruction data through model bootstrapping, but still relies on 175 manually designed seed tasks as initiation conditions, failing to achieve full automation. The Evol-Instruct method increases instruction complexity through evolutionary strategies, but its evolutionary rules require expert experience for design, and difficulty control lacks objective standards. The Magpie method utilizes pre-query templates from aligned models to trigger instruction generation, reducing reliance on manual intervention, but its general template mechanism results in weak domain adaptability, making it difficult to generate specialized data for specific vertical domains.

[0004] In terms of data quality control, existing solutions generally adopt a "generate first, then filter" approach. Methods such as Cherry Data use perplexity scoring to filter generated data post-processing, while superfiltering methods utilize the consistency between small and large models for filtering. Although these post-processing strategies can improve the quality of the final dataset, they generate a large amount of low-quality data during the generation stage, resulting in a serious waste of computational resources, and the filtering process itself also requires additional computational overhead.

[0005] In summary, the relevant technologies have the following problems: relying on manual seed tasks or expert-designed rules cannot achieve full automation, and the general template mechanism lacks domain adaptability. Summary of the Invention

[0006] The purpose of this application is to provide an end-to-end synthesis system for instruction fine-tuning data based on a self-guided workflow. This system is designed for Large Language Models (LLMs) and instruction fine-tuning. Starting from the original corpus data of the target domain, it can automatically discover task patterns and generate high-quality instruction data of the target domain for model fine-tuning without the need for manual seed tasks or external API dependencies, thereby solving the above-mentioned problems.

[0007] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide an end-to-end instruction fine-tuning data synthesis system based on a self-guided workflow, the system comprising: The automatic task discovery module is used to perform semantic analysis on the content and structural features of the original corpus data in the target domain based on preset prompt words and through the instruction template generation model, to obtain multiple task patterns in the original corpus data of the target domain, generate a task pattern library through unsupervised clustering, and generate instruction templates for each task pattern. The closed-loop self-optimizing workflow module is used to generate a model through instruction data, generate multiple instruction data for the target domain according to the instruction template, fine-tune the target large language model and evaluate its performance in the target domain based on the instruction data, and optimize the preset prompt words based on the performance evaluation results to obtain an updated instruction template. The updated instruction template is used to generate instruction data for the next round of fine-tuning. The fine-tuned target large language model is used to perform at least the question-answering task, summarizing task, reasoning task, comparison task, and translation task in the target domain.

[0008] Optionally, the system further includes: A generative quality constraint module is used to impose quality constraints on the instruction data according to a preset constraint mechanism during the process of the closed-loop self-optimizing workflow module generating multiple instruction data of the target domain based on the instruction template; the preset constraint mechanism includes at least: detecting the content coherence of the instruction data, verifying the self-consistency of the instruction data, and detecting the repetition of the generated instruction data. The closed-loop self-optimizing workflow module is also used to optimize the preset constraint mechanism based on the results of the effect evaluation.

[0009] Optionally, the continuity of the instruction data is detected, including: During the generation of instructions, inputs, and outputs in the instruction data, the perplexity score of each generated token is calculated in real time. If a token sequence with a perplexity higher than a preset perplexity threshold is detected, the instructions, inputs, and outputs of the instruction data are regenerated.

[0010] Optionally, the self-consistency verification of the instruction data includes: For each generated instruction data, the instruction data generation model is guided to regenerate the corresponding instruction based on the output in the instruction data, and the semantic similarity between the regenerated instruction and the instruction in the instruction data is calculated. If the semantic similarity is greater than a preset semantic similarity threshold, the instruction, input, and output of the instruction data are retained. If the semantic similarity is not greater than a preset semantic similarity threshold, the instructions, inputs, and outputs of the instruction data are regenerated.

[0011] Optionally, the generated instruction data is subjected to repeatability detection, including: After generating new instruction data, a similarity test is performed between the new instruction data and the existing instruction data. When the similarity between new instruction data and existing instruction data indicates low diversity and rigid, singular expression in the generated instruction data, the sampling temperature of the instruction data generation model should be increased. When the similarity between new instruction data and existing instruction data is detected, and the vocabulary distribution in the generated instruction data is concentrated and the candidate word space is limited, the top-p sampling threshold of the instruction data generation model is increased. When the similarity between new instruction data and existing instruction data indicates that n consecutive phrases are reused, the n-gram repetition penalty coefficient of the instruction data generation model is increased.

[0012] Optionally, the self-consistency verification of the instruction data includes: For each instruction, input, and output in the generated instruction data, verify in the knowledge graph of the target domain whether there are supporting paths between the instructions, inputs, and outputs in the instruction data; If a supporting path exists between the instructions, inputs, and outputs in the instruction data, the instructions, inputs, and outputs of the instruction data are retained. If there is no supporting path between the instructions, inputs, and outputs in the instruction data, the instructions, inputs, and outputs of the instruction data are regenerated.

[0013] Optionally, based on preset prompt words, a model is generated using an instruction template to perform semantic analysis on the content and structural features of the original corpus data in the target domain, thereby obtaining multiple task patterns in the original corpus data of the target domain, including: Preprocess the raw corpus data of the target domain; Based on preset prompt words, a model is generated through instruction templates to analyze the content and structural features of the preprocessed target domain's original corpus data, thereby obtaining multiple task instances contained in the preprocessed target domain's original corpus data. Multiple task instances are vectorized and clustering algorithms are used to group similar task instances into the same task pattern, resulting in multiple task patterns. The automatic task discovery module is also used to analyze the syntactic structure of the original corpus data in the target domain according to preset sentence rules, so as to obtain multiple task patterns in the original corpus data in the target domain.

[0014] Secondly, embodiments of this application provide an end-to-end method for instruction fine-tuning data synthesis based on a self-guided workflow, the method comprising: Based on preset prompt words, a model is generated by generating instruction templates to perform semantic analysis on the content and structural features of the original corpus data in the target domain, thereby obtaining multiple task patterns in the original corpus data of the target domain. A task pattern library is generated through unsupervised clustering, and an instruction template is generated for each task pattern. The model generates multiple instruction data for the target domain based on the instruction template, and fine-tunes and evaluates the performance of the target large language model in the target domain based on the instruction data. Furthermore, it optimizes the preset prompt words based on the evaluation results to obtain an updated instruction template. The updated instruction template is used to generate instruction data for the next round of fine-tuning. The fine-tuned target large language model is used to perform at least the question-answering task, summarizing task, reasoning task, and comparison task in the target domain.

[0015] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps in the method described in the second aspect.

[0016] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps in the method described in the second aspect.

[0017] The technical solution of this application firstly achieves complete automation of instruction fine-tuning data generation. Through the automatic task discovery module, multiple task patterns can be automatically extracted from the raw corpus data, and instruction templates corresponding to each task pattern can be generated without manual seed tasks or external API dependencies, significantly reducing the human and time costs of data acquisition. Compared with traditional methods that require manually designed seed tasks, this achieves fully automated startup with zero manual seeding. Secondly, it enhances the model's adaptability to vertical domains. By automatically discovering tasks based on the raw corpus data of the target domain, the generated instruction data has relevance to the target domain. Compared with the use of general pre-query templates in traditional methods, the technical solution of this application can better adapt to the application needs of vertical domains, improving the domain relevance and professionalism of the instruction data. Finally, this application constructs a closed-loop optimization workflow between instruction data generation and model training, which can automatically adjust the instruction data generation strategy according to the performance of the target large language model in downstream tasks, thereby achieving continuous improvement in the quality of instruction data generation and enabling iterative improvement of the target large language model without manual intervention. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the framework of an end-to-end instruction fine-tuning data synthesis system based on a self-guided workflow, provided in one embodiment of this application. Figure 2 This is a flowchart illustrating an end-to-end instruction fine-tuning data synthesis method based on a self-guided workflow, according to an embodiment of this application. Figure 3 This is a schematic diagram of the framework of an end-to-end instruction fine-tuning data synthesis device based on a self-guided workflow, provided in an embodiment of this application. Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0021] The following description, in conjunction with the accompanying drawings, details an end-to-end instruction fine-tuning data synthesis system based on a self-guided workflow, provided by the embodiments of this application, through specific implementations and application scenarios.

[0022] Figure 1 This is a schematic diagram of the framework of an end-to-end instruction fine-tuning data synthesis system based on a self-guided workflow, provided in an embodiment of this application. (Refer to...) Figure 1 This application provides an end-to-end instruction fine-tuning data synthesis system based on a self-guided workflow, the system comprising: The automatic task discovery module is used to perform semantic analysis on the content and structural features of the original corpus data in the target domain by using preset prompt words generated through prompt word engineering and through the instruction template generation model. This results in multiple task patterns in the original corpus data of the target domain. The module generates a task pattern library through unsupervised clustering and generates an instruction template for each task pattern.

[0023] In this embodiment, the system includes an automatic task discovery module and a closed-loop self-optimizing workflow module. The automatic task discovery module is used to acquire the original corpus data of the target domain, and then drive the instruction template generation model to perform unsupervised analysis on the content features and structural features of the original corpus data through preset prompt words to obtain multiple task patterns in the original corpus data of the target domain. Then, a task pattern library is generated through unsupervised clustering, and an instruction template is generated for each task pattern in the task pattern library. Finally, an instruction template library corresponding to the task pattern library is constructed based on the instruction templates of all task patterns.

[0024] For example, the preset prompts are as follows: "Please analyze the content and structural features of this text (original corpus data), identify the task instances that can be used for translation training, and generate a translation instruction template adapted to the current scenario. The instruction template must include instructions, input, and output. The input is English text related to the target domain, and the output is the corresponding translation. Ensure that the translation accurately matches the professional expression of the target domain scenario and has no semantic deviation." The target domain's original corpus data refers to corpus data in a specific sub-domain that has not been manually processed or labeled. Content features refer to the knowledge content and semantic information contained in the original corpus data, while structural features refer to the sentence organization, paragraph logic, and layout of the original corpus data. The instruction template generation model is a pre-trained large language model with powerful semantic analysis capabilities. It is used to automatically identify task instances in the target domain's original corpus data and generate corresponding instruction templates for each task mode. Task modes refer to the task categories in the original corpus data that can be used for model training, such as question answering, summarizing, reasoning, and comparison. Instruction templates are standardized instruction data formats with fixed sentence structures corresponding to various task modes, including input, output, and instructions.

[0025] For example, the instruction template for translation tasks is as follows: Instructions: Please translate the following English text into fluent and standard Chinese text, without adding or deleting semantics or changing the original meaning; Input: A machine learning model can automatically analyze text data; Output: Machine learning models can automatically analyze text data.

[0026] Optionally, based on preset prompt words, a model is generated using an instruction template to perform semantic analysis on the content and structural features of the original corpus data in the target domain, thereby obtaining multiple task patterns in the original corpus data of the target domain, including: Preprocess the raw corpus data of the target domain; Based on preset prompt words, a model is generated through instruction templates to analyze the content and structural features of the preprocessed target domain's original corpus data, thereby obtaining multiple task instances contained in the preprocessed target domain's original corpus data. Multiple task instances are vectorized and clustering algorithms are used to group similar task instances into the same task pattern, resulting in multiple task patterns. The automatic task discovery module is also used to analyze the syntactic structure of the original corpus data in the target domain according to preset sentence rules, so as to obtain multiple task patterns in the original corpus data in the target domain.

[0027] Specifically, in another embodiment, the task automatic discovery module preprocesses the original corpus data of the target domain, including text cleaning, sentence segmentation, and paragraph division. Then, it inputs the preset prompt words and the original corpus data into the instruction template generation model. The instruction template generation model analyzes the content and structural features of the preprocessed original corpus data of the target domain, identifies multiple task instances contained in the preprocessed original corpus data of the target domain, and then uses an unsupervised clustering strategy to vectorize these task instances. The clustering algorithm groups similar task instances into the same task pattern, thereby obtaining multiple task patterns.

[0028] Optionally, in addition to using a large language model such as an instruction template generation model to perform semantic analysis on the raw corpus data, the task automatic discovery module can also analyze the syntactic structure of the raw corpus data in the target domain according to preset sentence pattern rules to obtain multiple task patterns in the raw corpus data of the target domain. For example, by identifying sentence features such as interrogative sentences and imperative sentences in the raw corpus data, potential task instances in the raw corpus data can be discovered, thereby obtaining multiple task patterns.

[0029] The closed-loop self-optimizing workflow module is used to generate a model through instruction data, generate multiple instruction data for the target domain according to the instruction template, fine-tune the target large language model and evaluate its performance in the target domain according to the instruction data, and optimize the preset prompt words according to the performance evaluation results to obtain an updated instruction template. The updated instruction template is used to generate instruction data for the next round of fine-tuning. The fine-tuned target large language model is used to perform at least the question-answering task, summarizing task, reasoning task and comparison task in the target domain.

[0030] In this embodiment, the closed-loop self-optimizing workflow module is used to call the instruction data generation model. Based on the instruction template corresponding to each task mode, it generates multiple instruction data that meet the target domain in batches. The instruction data also includes inputs, outputs, and instructions. Then, these instruction data are divided into training sets, validation sets, and test sets and input into the target large language model in sequence. The target large language model is a basic large language model that has not been adapted to the domain and needs to be fine-tuned to make it follow the instructions to complete the target domain task. The module completes the fine-tuning of the instruction data in the target domain and evaluates the effect of the fine-tuned target large language model, that is, tests the performance of the fine-tuned model and collects indicators such as the accuracy and fluency of the model in performing various tasks. Then, the policy optimizer analyzes the shortcomings of the instruction dataset in the current round based on the evaluation results.

[0031] For example, to optimize the preset prompts, if the instruction data generated by the instruction template generation model has "poor scene adaptability" (such as not reflecting relevant terms of the target domain), then "must include the core terms of the target domain scene" will be added to the preset prompts; if the instruction template is monotonous, then "generate instruction templates with 3 or more different sentence structures" will be added to the preset prompts as guidance.

[0032] For example, if the instruction template generation model performs poorly on inference tasks, the optimizer will increase the proportion of inference-related instruction data in subsequent generation; if the instruction template generation model has problems in generating long texts, the optimizer will adjust the distribution parameters of the length of the output in the instruction data, and then iterate bidirectionally based on the effect evaluation results, updating the instruction template by optimizing the preset prompt words, and using the updated instruction template for the next round of instruction data generation.

[0033] Optionally, refer to Figure 1 In another embodiment, the system further includes: a generative quality constraint module, used to impose quality constraints on the instruction data according to a preset constraint mechanism during the process of the closed-loop self-optimizing workflow module generating multiple instruction data of the target domain according to the instruction template; the preset constraint mechanism includes at least: detecting the content coherence of the instruction data, verifying the self-consistency of the instruction data, and detecting the repetition of the generated instruction data; Specifically, in this embodiment, the generative quality constraint module operates throughout the entire process of instruction data generation. It employs pre-emptive real-time constraints rather than post-emptive filtering. During the process of the closed-loop self-optimizing workflow module generating multiple domain instruction data based on the instruction template, the generative quality constraint module executes real-time quality constraints on the generated instruction data according to a preset constraint mechanism. It sequentially checks the content coherence of the instruction data, verifies the self-consistency of the instruction data, and performs repetition checks on the generated instruction data. This improves the quality of the instruction data during the generation process. The preset constraint mechanism is a real-time quality control rule for the entire instruction data generation process. The content coherence detection, i.e., the perplexity-based sentence fluency detection, is used to determine whether the generated text is fluent and logically sound. The self-consistency verification is used to verify whether the instructions and answers semantically match and whether the knowledge is true and reliable. The repetition detection is used to identify whether the generated content is highly repetitive and homogeneous.

[0034] The closed-loop self-optimizing workflow module is also used to optimize the preset constraint mechanism based on the results of the effect evaluation.

[0035] Specifically, based on the results of the performance evaluation, the thresholds and sampling parameters within the preset constraint mechanism can be optimized, ultimately enabling the finely tuned target large language model to stably perform tasks such as question answering, summarizing, reasoning, and comparison.

[0036] For example, if the generated instruction data is not grammatically correct, the preset perplexity threshold can be increased to ensure the coherence of the instruction data.

[0037] Optionally, the continuity of the instruction data is detected, including: During the generation of instructions, inputs, and outputs in the instruction data, the perplexity score of each generated token is calculated in real time. If a token sequence with a perplexity higher than a preset perplexity threshold is detected, the instructions, inputs, and outputs of the instruction data are regenerated.

[0038] Specifically, in this embodiment, the generative quality constraint module performs content coherence detection on the instruction data by first detecting coherence during the generation of each instruction data, rather than after the instruction data is generated. Specifically, during the generation of each token word by word by the instruction data generation model, the perplexity score of the currently generated token is calculated in real time. Perplexity is used to characterize the logical fluency and semantic rationality of the current text statement in the instruction data generated by the instruction data generation model. Higher perplexity indicates a more incoherent, illogical, and semantically fragmented statement. By pre-setting a reasonable perplexity threshold, when the perplexity of a consecutive token sequence is detected to be higher than the preset perplexity threshold, it is determined that the current text statement has problems such as incoherence and logical inconsistency. The generation process of that instruction data is then directly stopped, and a resampling or rollback mechanism is triggered to regenerate the complete instruction, input, and output of the instruction data, avoiding the generation of low-quality or incoherent content. The preset perplexity threshold for this mechanism can be configured according to specific application scenarios; for example, the value range can be set to 10 to 50.

[0039] Optionally, the self-consistency verification of the instruction data includes: For each generated instruction data, the instruction data generation model is guided to regenerate the corresponding instruction based on the output in the instruction data, and the semantic similarity between the regenerated instruction and the instruction in the instruction data is calculated. If the semantic similarity is greater than a preset semantic similarity threshold, the instruction, input, and output of the instruction data are retained. If the semantic similarity is not greater than a preset semantic similarity threshold, the instructions, inputs, and outputs of the instruction data are regenerated.

[0040] Specifically, in this embodiment, a self-consistency check is performed after each instruction data is generated. During this process, the instruction data generation model uses the content corresponding to the output in the currently generated instruction data as a basis to reconstruct and regenerate a corresponding instruction. Subsequently, the semantic similarity between the instruction reconstructed by the instruction data generation model and the original instruction in the instruction data is calculated. This semantic similarity is used to determine whether the original instruction, input, and output are semantically matched and the question and answer correspond. When the semantic similarity between the instruction reconstructed by the instruction data generation model and the original instruction in the instruction data is greater than a preset semantic similarity threshold (for example, a value of 0.7 to 0.9), it indicates that the original instruction and output are highly matched and logically self-consistent, and the instruction data will be retained. Conversely, when the semantic similarity is not greater than the preset threshold, it indicates that the instruction and output do not correspond or there are problems such as irrelevant answers, and the current instruction data will be discarded and new instruction data will be generated.

[0041] Optionally, the self-consistency verification of the instruction data includes: For each instruction, input, and output in the generated instruction data, verify in the knowledge graph of the target domain whether there are supporting paths between the instructions, inputs, and outputs in the instruction data; If a supporting path exists between the instructions, inputs, and outputs in the instruction data, the instructions, inputs, and outputs of the instruction data are retained. If there is no supporting path between the instructions, inputs, and outputs in the instruction data, the instructions, inputs, and outputs of the instruction data are regenerated.

[0042] Specifically, in this embodiment, in addition to using semantic similarity, the self-consistency verification of instruction data can also be performed by verifying the overall relationship between the instruction, input, and output of each instruction data. This involves mapping the instruction, input, and output to the knowledge graph of the target domain and verifying whether there is a valid knowledge support path in the knowledge graph for the semantic association and knowledge correspondence between the three. If a complete and valid knowledge support path exists, it indicates that the instruction data is knowledge-based and without model illusion errors, and it is retained. If no corresponding support path exists, the instruction data is deemed unqualified and the instruction, input, and output of the instruction data need to be regenerated.

[0043] Optionally, the generated instruction data is subjected to repeatability detection, including: After generating new instruction data, a similarity test is performed between the new instruction data and the existing instruction data. When the similarity between new instruction data and existing instruction data indicates low diversity and rigid, singular expression in the generated instruction data, the sampling temperature of the instruction data generation model should be increased. When the similarity between new instruction data and existing instruction data is detected, and the vocabulary distribution in the generated instruction data is concentrated and the candidate word space is limited, the top-p sampling threshold of the instruction data generation model is increased. When the similarity between new instruction data and existing instruction data indicates that n consecutive phrases are reused, the n-gram repetition penalty coefficient of the instruction data generation model is increased.

[0044] Specifically, in this embodiment, after each new instruction data is generated, the similarity of the instruction data is compared with all previously stored instruction data. When it is detected that the new instruction data has extremely high similarity with historical instructions, exhibits low diversity, and has a fixed and singular expression, it indicates that the expression pattern of the instruction data generation model is rigid. Therefore, it is necessary to increase the sampling temperature of the instruction data generation model to improve the randomness and expression diversity of instruction data generation. When it is detected that the vocabulary distribution of the new instruction data is highly concentrated and the candidate vocabulary selection space is limited, resulting in content homogenization, the top-p sampling threshold of the instruction data generation model will be increased to expand the candidate vocabulary distribution space of the instruction data generation model. When it is detected that there is continuous reuse of n-gram phrases and copying of sentence patterns within the new instruction data (n is greater than 0), the n-gram repetition penalty coefficient of the instruction data generation model will be increased to avoid the instruction data generation model continuously generating repetitive phrases.

[0045] The technical solution of this application has the following advantages and positive effects: First, it achieves complete automation of instruction fine-tuning data generation. Through the automatic task discovery module, it can automatically extract multiple task patterns from the raw corpus data and generate instruction templates corresponding to each task pattern, without requiring manual seed tasks or external API dependencies, significantly reducing the human and time costs of data acquisition. Compared with traditional methods that require manually designed seed tasks, it achieves fully automated startup with zero manual seeding.

[0046] Second, it significantly improves the efficiency of computing resource utilization. By innovatively embedding a quality constraint mechanism into the instruction data generation process, and through a triple mechanism of real-time perplexity monitoring, self-consistency verification, and diversity dynamic control, data quality is guaranteed during the instruction data generation stage. Compared with the traditional "generate first, then filter" approach, the method in this application can reduce the generation of invalid data by approximately 70%, avoiding the waste of computing resources.

[0047] Third, it enhances domain adaptability by automatically discovering tasks based on raw corpus data of the target domain, ensuring that the generated instruction data is relevant to the target domain. Compared with traditional methods that use general pre-query templates, the technical solution of this application can better adapt to the application needs of vertical domains, improving the domain relevance and professionalism of the instruction data.

[0048] Fourth, it supports continuous optimization of instruction data generation strategies. This application constructs a closed-loop optimization workflow between instruction data generation and model training, which can automatically adjust the instruction data generation strategy according to the performance of the target large language model in downstream tasks, thereby achieving continuous improvement in the quality of instruction data generation and enabling iterative improvement of the target large language model without manual intervention.

[0049] Figure 2 This is a flowchart illustrating an end-to-end instruction fine-tuning data synthesis method based on a self-guided workflow, according to an embodiment of this application. (Refer to...) Figure 2 .

[0050] One embodiment of this application provides an end-to-end method for instruction fine-tuning data synthesis based on a self-guided workflow, the method comprising steps S11 and S12: Step S11: Based on preset prompt words, a model is generated by generating an instruction template to perform semantic analysis on the content features and structural features of the original corpus data in the target domain, to obtain multiple task modes in the original corpus data of the target domain, to generate a task mode library through unsupervised clustering, and to generate an instruction template for each task mode. Step S12: Generate a model using instruction data, generate multiple instruction data for the target domain based on the instruction template, fine-tune the target large language model in the target domain based on the instruction data, evaluate the effect, and optimize the preset prompt words based on the effect evaluation results to obtain an updated instruction template. The updated instruction template is used to generate instruction data for the next round of fine-tuning. The fine-tuned target large language model is used to perform at least the question-answering task, summarizing task, reasoning task, and comparison task in the target domain.

[0051] Optionally, the method further includes: Step S21: During the process of the closed-loop self-optimizing workflow module generating multiple instruction data of the target domain according to the instruction template, the instruction data is subjected to quality constraints according to a preset constraint mechanism; the preset constraint mechanism includes at least: detecting the content coherence of the instruction data, verifying the self-consistency of the instruction data, and detecting the repetition of the generated instruction data. Step S22: Optimize the preset constraint mechanism based on the results of the effect evaluation.

[0052] Optionally, the "detecting the content coherence of the instruction data" in step S21 includes steps S21-1-1 and S21-1-2: Step S21-1-1: During the generation of instructions, inputs and outputs in the instruction data, the perplexity score of each generated token is calculated in real time. Step S21-1-2: If a token sequence with a perplexity higher than a preset perplexity threshold is detected, the instruction, input, and output of the instruction data are regenerated.

[0053] Optionally, the "performing self-consistency verification on the instruction data" in step S21 includes steps S21-2-1 to S21-2-3: Step S21-2-1: For each generated instruction data, based on the output in the instruction data, guide the instruction data generation model to regenerate the corresponding instruction, and calculate the semantic similarity between the regenerated instruction and the instruction in the instruction data; Step S21-2-2: If the semantic similarity is greater than a preset semantic similarity threshold, retain the instruction, input, and output of the instruction data; Step S21-2-3: If the semantic similarity is not greater than the preset semantic similarity threshold, regenerate the instruction, input, and output of the instruction data.

[0054] Optionally, the "perform repeatability detection on the generated instruction data" in step S21 includes steps S21-3-1 to S21-3-4, including: Step S21-3-1: After generating new instruction data, perform similarity detection between the new instruction data and the existing instruction data. Step S21-3-2: When the similarity between the new instruction data and the existing instruction data indicates low diversity and fixed and singular expression in the generated instruction data, the sampling temperature of the instruction data generation model is increased. Step S21-3-3: When the similarity between the new instruction data and the existing instruction data is detected, indicating that the vocabulary distribution in the generated instruction data is concentrated and the candidate word space is limited, the top-p sampling threshold of the instruction data generation model is increased. Step S21-3-4: When the similarity between new instruction data and existing instruction data indicates that n consecutive word groups are reused, increase the n-gram repetition penalty coefficient of the instruction data generation model.

[0055] Optionally, the "performing self-consistency verification on the instruction data" in step S21 includes steps S21-4-1 to S21-4-3, including: Step S21-4-1: For each instruction, input, and output in the generated instruction data, verify in the knowledge graph of the target domain whether there is a supporting path between the instructions, inputs, and outputs in the instruction data; Step S21-4-2: If there is a supporting path between the instructions, inputs and outputs in the instruction data, retain the instructions, inputs and outputs of the instruction data; Step S21-4-3: If there is no supporting path between the instructions, inputs and outputs in the instruction data, regenerate the instructions, inputs and outputs of the instruction data.

[0056] Optionally, step S11, "based on preset prompt words, generating a model through instruction templates, performing semantic analysis on the content and structural features of the original corpus data in the target domain, and obtaining multiple task patterns in the original corpus data of the target domain," includes steps S11-1 to S11-4: Step S11-1: Preprocess the original corpus data of the target domain; Step S11-2: Based on preset prompt words, a model is generated through instruction templates to analyze the content and structural features of the preprocessed target domain's original corpus data, thereby obtaining multiple task instances contained in the preprocessed target domain's original corpus data. Step S11-3: Vectorize multiple task instances and use a clustering algorithm to group similar task instances into the same task pattern to obtain multiple task patterns. Step S11-4: Analyze the syntactic structure of the original corpus data in the target domain according to preset sentence rules to obtain multiple task patterns in the original corpus data of the target domain.

[0057] It should be noted that the end-to-end instruction fine-tuning data synthesis method based on a self-guided workflow provided in this application embodiment can be executed by an end-to-end instruction fine-tuning data synthesis device based on a self-guided workflow, or by a control module within that device for executing the method. This application embodiment uses the execution of the end-to-end instruction fine-tuning data synthesis method based on a self-guided workflow by an end-to-end instruction fine-tuning data synthesis device as an example to illustrate the end-to-end instruction fine-tuning data synthesis method based on a self-guided workflow provided in this application embodiment.

[0058] Figure 3 This is a schematic diagram of the framework of an end-to-end instruction fine-tuning data synthesis device based on a self-guided workflow, provided in an embodiment of this application. (Refer to...) Figure 3.

[0059] One embodiment of this application provides an end-to-end instruction fine-tuning data synthesis apparatus based on a self-guided workflow, the apparatus comprising: The instruction template generation module 11 is configured in the task automatic discovery module. It is used to perform semantic analysis on the content features and structural features of the original corpus data in the target domain based on preset prompt words and through the instruction template generation model, to obtain multiple task patterns in the original corpus data of the target domain, generate a task pattern library through unsupervised clustering, and generate instruction templates for each task pattern. The model fine-tuning module 12, configured in the closed-loop self-optimization workflow module, is used to generate a model through instruction data, generate multiple instruction data for the target domain according to the instruction template, fine-tune the target large language model in the target domain and evaluate its performance based on the instruction data, and optimize the preset prompt words based on the results of the performance evaluation to obtain an updated instruction template. The updated instruction template is used to generate instruction data for the next round of fine-tuning. The fine-tuned target large language model is used to perform at least the question-answering task, summarizing task, reasoning task and comparison task in the target domain.

[0060] The instruction data quality constraint module 13 is configured in the generative quality constraint module and is used to perform quality constraints on the instruction data according to a preset constraint mechanism during the process of the closed-loop self-optimization workflow module generating multiple instruction data of the target domain according to the instruction template; the preset constraint mechanism includes at least: detecting the content coherence of the instruction data, verifying the self-consistency of the instruction data, and detecting the repetition of the generated instruction data. An optimization module, configured within the closed-loop self-optimizing workflow module, is used to optimize the preset constraint mechanism based on the results of the effect evaluation.

[0061] The perplexity score determination module, configured within the generative quality constraint module, is used to calculate the perplexity score of each generated token in real time during the generation process of instructions, inputs, and outputs in the instruction data. The first instruction data regeneration module is configured in the closed-loop self-optimizing workflow module. It is used to regenerate the instructions, inputs and outputs of the instruction data when a token sequence with a perplexity higher than a preset perplexity threshold is detected.

[0062] The semantic similarity determination module, configured in the generative quality constraint module, is used to guide the instruction data generation model to regenerate the corresponding instruction based on the output in each generated instruction data, and to calculate the semantic similarity between the regenerated instruction and the instruction in the instruction data. The second instruction data regeneration module, configured in the closed-loop self-optimization workflow module, is used to retain the instructions, inputs, and outputs of the instruction data when the semantic similarity is greater than a preset semantic similarity threshold. The third instruction data regeneration module, configured in the closed-loop self-optimizing workflow module, is used to regenerate the instructions, inputs, and outputs of the instruction data when the semantic similarity is not greater than a preset semantic similarity threshold.

[0063] The similarity detection module, configured in the closed-loop self-optimizing workflow module, is used to perform similarity detection between the new instruction data and the existing instruction data after the new instruction data is generated. The first parameter optimization module, configured within the generative quality constraint module, is used to improve the sampling temperature of the instruction data generation model when the similarity between new instruction data and existing instruction data indicates low diversity and rigid, singular expression in the generated instruction data. The second parameter optimization module, configured in the generative quality constraint module, is used to improve the top-p sampling threshold of the instruction data generation model when the similarity between the new instruction data and the existing instruction data indicates that the vocabulary distribution in the generated instruction data is concentrated and the candidate word space is limited. The third parameter optimization module, configured within the generative quality constraint module, is used to improve the n-gram repetition penalty coefficient of the instruction data generation model when the similarity representation between new instruction data and existing instruction data indicates that n consecutive word groups are reused.

[0064] The support path determination module, configured within the generative quality constraint module, is used to verify, within the target domain's knowledge graph, whether a support path exists between the instructions, inputs, and outputs in each generated instruction data. The fourth instruction data regeneration module, configured in the closed-loop self-optimizing workflow module, is used to retain the instructions, inputs, and outputs of the instruction data when there is a supporting path between the instructions, inputs, and outputs in the instruction data. The fifth instruction data regeneration module, configured in the closed-loop self-optimizing workflow module, is used to regenerate the instructions, inputs, and outputs of the instruction data when there is no supporting path between the instructions, inputs, and outputs in the instruction data.

[0065] The instruction template generation module 11 includes: The preprocessing unit is used to preprocess the original corpus data of the target domain; The task instance determination unit is used to analyze the content and structural features of the preprocessed target domain original corpus data based on preset prompt words and through the instruction template to generate a model, thereby obtaining multiple task instances contained in the preprocessed target domain original corpus data. The first task mode determination unit is used to vectorize multiple task instances and group similar task instances into the same task mode through a clustering algorithm to obtain multiple task modes. The second task mode determination unit is used to analyze the syntactic structure of the original corpus data in the target domain according to preset sentence rules, and obtain multiple task modes in the original corpus data in the target domain.

[0066] The instruction fine-tuning data end-to-end synthesis device based on self-guided workflow in this application embodiment can be a device, or it can be a component, integrated circuit, or chip in a terminal. This device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not specifically limit the specific implementation.

[0067] The instruction fine-tuning data end-to-end synthesis device based on a self-guided workflow in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0068] The instruction fine-tuning data end-to-end synthesis device based on a self-guided workflow provided in this application embodiment can achieve... Figure 2 The various processes implemented by the instruction fine-tuning data end-to-end synthesis device based on the self-guided workflow in the method embodiment will not be described again here to avoid repetition.

[0069] Optionally, Figure 4 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. This application also provides an electronic device; it should be noted that the electronic device in this application includes the mobile electronic device and non-mobile electronic device described above.

[0070] The electronic device includes, but is not limited to, components such as: radio frequency unit, network module, audio output unit, input unit, sensor, display unit, user input unit, interface unit, memory, and processor.

[0071] Those skilled in the art will understand that electronic devices may also include power supplies (such as batteries) that supply power to various components. The power supply may be connected to the processor logic through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0072] As an example, such as Figure 4 As shown, the electronic device 600 includes a memory 610 and a processor 620. The memory 610 and the processor 620 are connected via a bus for communication. The memory 610 stores a computer program that can run on the processor 620 to implement the steps of the method disclosed in the above embodiments of this application.

[0073] As the apparatus and method are basically similar to the system embodiments, the description is relatively simple, and relevant parts can be found in the description of the system embodiments.

[0074] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0075] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects.

[0076] Furthermore, this application embodiment also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of the instruction fine-tuning data end-to-end synthesis method based on a self-bootstrapping workflow, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0077] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0078] This application also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described embodiment of the instruction fine-tuning data end-to-end synthesis method based on self-guided workflow, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0079] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0080] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0082] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An end-to-end instruction fine-tuning data synthesis system based on a self-guided workflow, characterized in that, The system includes: The automatic task discovery module is used to perform semantic analysis on the content and structural features of the original corpus data in the target domain based on preset prompt words and through the instruction template generation model, to obtain multiple task patterns in the original corpus data of the target domain, generate a task pattern library through unsupervised clustering, and generate instruction templates for each task pattern. The closed-loop self-optimizing workflow module is used to generate a model through instruction data, generate multiple instruction data for the target domain according to the instruction template, fine-tune the target large language model and evaluate its performance in the target domain according to the instruction data, and optimize the preset prompt words according to the performance evaluation results to obtain an updated instruction template. The updated instruction template is used to generate instruction data for the next round of fine-tuning. The fine-tuned target large language model is used to perform at least the question-answering task, summarizing task, reasoning task and comparison task in the target domain.

2. The instruction fine-tuning data end-to-end synthesis system based on self-guided workflow according to claim 1, characterized in that, The system also includes: A generative quality constraint module is used to impose quality constraints on the instruction data according to a preset constraint mechanism during the process of the closed-loop self-optimizing workflow module generating multiple instruction data of the target domain based on the instruction template; the preset constraint mechanism includes at least: detecting the content coherence of the instruction data, verifying the self-consistency of the instruction data, and detecting the repetition of the generated instruction data. The closed-loop self-optimizing workflow module is also used to optimize the preset constraint mechanism based on the results of the effect evaluation.

3. The instruction fine-tuning data end-to-end synthesis system based on self-guided workflow according to claim 2, characterized in that, The detection of the content coherence of the instruction data includes: During the generation of instructions, inputs, and outputs in the instruction data, the perplexity score of each generated token is calculated in real time. If a token sequence with a perplexity higher than a preset perplexity threshold is detected, the instructions, inputs, and outputs of the instruction data are regenerated.

4. The instruction fine-tuning data end-to-end synthesis system based on self-guided workflow according to claim 2, characterized in that, Perform self-consistency verification on the instruction data, including: For each generated instruction data, the instruction data generation model is guided to regenerate the corresponding instruction based on the output in the instruction data, and the semantic similarity between the regenerated instruction and the instruction in the instruction data is calculated. If the semantic similarity is greater than a preset semantic similarity threshold, the instruction, input, and output of the instruction data are retained. If the semantic similarity is not greater than a preset semantic similarity threshold, the instructions, inputs, and outputs of the instruction data are regenerated.

5. The instruction fine-tuning data end-to-end synthesis system based on self-guided workflow according to claim 2, characterized in that, Perform duplicate detection on the generated instruction data, including: After generating new instruction data, a similarity test is performed between the new instruction data and the existing instruction data. When the similarity between new instruction data and existing instruction data indicates low diversity and rigid, singular expression in the generated instruction data, the sampling temperature of the instruction data generation model should be increased. When the similarity between new instruction data and existing instruction data is detected, and the vocabulary distribution in the generated instruction data is concentrated and the candidate word space is limited, the top-p sampling threshold of the instruction data generation model is increased. When the similarity between new instruction data and existing instruction data indicates that n consecutive phrases are reused, the n-gram repetition penalty coefficient of the instruction data generation model is increased.

6. The instruction fine-tuning data end-to-end synthesis system based on self-guided workflow according to claim 2, characterized in that, Perform self-consistency verification on the instruction data, including: For each instruction, input, and output in the generated instruction data, verify in the knowledge graph of the target domain whether there are supporting paths between the instructions, inputs, and outputs in the instruction data; If a supporting path exists between the instructions, inputs, and outputs in the instruction data, the instructions, inputs, and outputs of the instruction data are retained. If there is no supporting path between the instructions, inputs, and outputs in the instruction data, the instructions, inputs, and outputs of the instruction data are regenerated.

7. The instruction fine-tuning data end-to-end synthesis system based on self-guided workflow according to claim 1, characterized in that, Based on preset prompts, a model is generated using instruction templates to perform semantic analysis on the content and structural features of the original corpus data in the target domain, resulting in multiple task patterns in the original corpus data of the target domain, including: Preprocess the raw corpus data of the target domain; Based on preset prompt words, a model is generated through instruction templates to analyze the content and structural features of the preprocessed target domain's original corpus data, thereby obtaining multiple task instances contained in the preprocessed target domain's original corpus data. Multiple task instances are vectorized and clustering algorithms are used to group similar task instances into the same task pattern, resulting in multiple task patterns. The automatic task discovery module is also used to analyze the syntactic structure of the original corpus data in the target domain according to preset sentence rules, so as to obtain multiple task patterns in the original corpus data in the target domain.

8. A method for end-to-end synthesis of instruction fine-tuning data based on a self-guided workflow, characterized in that, The method includes: Based on preset prompt words, a model is generated by generating instruction templates to perform semantic analysis on the content and structural features of the original corpus data in the target domain, thereby obtaining multiple task patterns in the original corpus data of the target domain. A task pattern library is generated through unsupervised clustering, and an instruction template is generated for each task pattern. The model generates multiple instruction data for the target domain based on the instruction template, and fine-tunes and evaluates the performance of the target large language model in the target domain based on the instruction data. Furthermore, it optimizes the preset prompt words based on the evaluation results to obtain an updated instruction template. The updated instruction template is used to generate instruction data for the next round of fine-tuning. The fine-tuned target large language model is used to perform at least the question-answering task, summarizing task, reasoning task, and comparison task in the target domain.

9. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as described in claim 8.

10. A readable storage medium, characterized in that, A program or instructions are stored on the readable storage medium, which, when executed by a processor, implement the steps of the method as described in claim 8.

Images