Instruction Data Construction Method and System for Fine-tuning Large Language Models

Abstract

This invention provides a method and system for constructing instruction data for fine-tuning large language models, comprising: collecting semi-structured and unstructured data; cleaning the semi-structured data to obtain a first instruction dataset; cleaning the unstructured data and segmenting the cleaned unstructured data into text blocks; inputting the text blocks into a preset large language model to generate a second instruction dataset; inputting the first and second instruction datasets into the large language model to generate a third instruction dataset; and filtering the first, second, and third instruction datasets to obtain a fourth instruction dataset. This invention can quickly construct large-scale, high-quality instruction data, effectively supporting the fine-tuning of large language models and saving labor costs.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method and system for constructing instruction data for fine-tuning large language models. Background Technology

[0002] The training process for large language models generally consists of two stages: pre-training and fine-tuning (instruction fine-tuning). The pre-training stage uses unlabeled text data to train the language model, while the fine-tuning stage uses supervised instruction data to further adjust the model's parameters to adapt it to a specific task. However, high-quality instruction data often requires significant human input for annotation, and the high cost and complexity of data annotation limit the effectiveness of fine-tuning for large language models.

[0003] To address the aforementioned issues, current solutions include Self-Instruct and Self-QA, which automate the construction of instruction datasets based on GPT-4. However, these methods have high requirements for data cleaning and preparation, the quality of the generated data is difficult to guarantee, and they struggle to cover all possible tasks and scenarios.

[0004] Therefore, how to reduce labor costs while ensuring the diversity and high quality of instruction data, thereby improving the instruction compliance capability of large language models, is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This invention provides a method and system for constructing instruction data for fine-tuning large language models, which addresses the shortcomings of existing technologies such as high requirements for data preparation, difficulty in guaranteeing the quality of generated data, and difficulty in covering all possible tasks and scenarios. It enables the automatic construction of large-scale, high-quality instruction datasets, improves the instruction compliance performance of large language models, and saves labor costs.

[0006] This invention provides a method for constructing instruction data for fine-tuning large language models, comprising the following steps.

[0007] Collect semi-structured and unstructured data; The semi-structured data is cleaned to obtain the first instruction dataset; The unstructured data is cleaned, and the cleaned unstructured data is segmented into text blocks; The text block is input into a preset large language model, and a second instruction dataset is generated through the large language model. The first instruction dataset and the second instruction dataset are input into the large language model, and a third instruction dataset is generated through the large language model. The first instruction dataset, the second instruction dataset, and the third instruction dataset are filtered to obtain the fourth instruction dataset.

[0008] According to the instruction data construction method for fine-tuning of large language models provided by the present invention, each piece of semi-structured data includes two parts: a question and an answer, and each piece of unstructured data consists of text paragraphs, each of which is composed of free text without predefined relationships.

[0009] According to the instruction data construction method for fine-tuning large language models provided by the present invention, the step of cleaning the semi-structured data specifically includes: S31. Preprocess the semi-structured data to remove data with excessively short text length or too many illegal characters. S32. Use BERT to calculate the text perplexity of each piece of preprocessed semi-structured data; S33. Remove semi-structured data whose text perplexity exceeds the preset perplexity first threshold.

[0010] According to the instruction data construction method for fine-tuning large language models provided by the present invention, the step of cleaning the unstructured data specifically includes: S41. Preprocess the unstructured data to remove data with excessively short text length or too many illegal characters. S42. Remove personal privacy data from the preprocessed unstructured data; S43. Use BERT to calculate the text perplexity of each piece of unstructured data; S44. Remove unstructured data whose text perplexity exceeds the preset second perplexity threshold.

[0011] According to the instruction data construction method for fine-tuning large language models provided by the present invention, the step of segmenting cleaned unstructured data into text blocks specifically includes: Each piece of unstructured data after cleaning is segmented into paragraphs.

[0012] According to the instruction data construction method for fine-tuning a large language model provided by the present invention, the step of inputting the text block into a preset large language model and generating a second instruction dataset through the large language model specifically includes: S61. Input the text block into a preset large language model, and generate the question corresponding to the text block through the preset large language model; S62. Based on the question and the text block, generate the answer to the question using a preset large language model; S63. Concatenate the questions and their corresponding answers to form a first question-answer pair, and combine the first question-answer pairs into a second instruction dataset.

[0013] The instruction data construction method for fine-tuning a large language model provided by the present invention includes the following step: generating a third instruction dataset based on the first instruction dataset and the second instruction dataset using the large language model. S71. Combine the first instruction dataset and the second instruction dataset into a seed instruction set; S72. Randomly select several seed instructions from the seed instruction set, and generate questions and answers from the seed instructions using a preset large language model to form a second question-answer pair; S73. Determine the score of each newly generated second question-and-answer pair using a preset large language model, and filter each second question-and-answer pair based on a preset score threshold to remove second question-and-answer pairs with scores lower than the preset score threshold; S74. Expand the filtered second question-and-answer pairs into the seed instruction set; S75. Repeat S72~S74 until the preset number of seed instruction expansions is reached.

[0014] According to the instruction data construction method for fine-tuning a large language model provided by the present invention, the step of filtering the first instruction dataset, the second instruction dataset, and the third instruction dataset to obtain a fourth instruction dataset specifically includes: S81, using an embedding model to convert the instruction data in the first instruction dataset, the second instruction dataset, and the third instruction dataset into embedding vectors; S82, calculate the cosine similarity between each of the embedded vectors; S83, remove instruction data corresponding to embedding vectors with cosine similarity higher than a preset similarity threshold.

[0015] According to the instruction data construction method for fine-tuning large language models provided by the present invention, the embedding model is one of BERT, Sentence Transformers and Word2Vec models.

[0016] According to the instruction data construction method for fine-tuning large language models provided by the present invention, after obtaining the fourth instruction dataset, the method further includes: Based on the aforementioned fourth instruction dataset, the target large language model is fine-tuned.

[0017] According to the instruction data construction method for fine-tuning large language models provided by the present invention, the LoRa algorithm is used to fine-tune the target large language model; The Lora algorithm adjusts specific layers of a pre-trained target large language model by inserting a learnable low-rank fitting matrix through low-rank decomposition.

[0018] This invention also provides an instruction data construction system for fine-tuning large language models, comprising the following modules: The collection module is used to collect semi-structured and unstructured data; The first instruction dataset acquisition module is used to clean the semi-structured data to obtain the first instruction dataset; The text segmentation module is used to clean the unstructured data and segment the cleaned unstructured data into text blocks; The second instruction dataset acquisition module is used to input the text block into a preset large language model and generate a second instruction dataset through the large language model. The third instruction dataset acquisition module is used to input the first instruction dataset and the second instruction dataset into the large language model and generate the third instruction dataset through the large language model. The fourth instruction dataset acquisition module is used to filter the first instruction dataset, the second instruction dataset, and the third instruction dataset to obtain the fourth instruction dataset.

[0019] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the instruction data construction method for fine-tuning a large language model as described above.

[0020] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the instruction data construction method for fine-tuning a large language model as described above.

[0021] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the instruction data construction method for fine-tuning a large language model as described above.

[0022] The present invention provides a method and system for constructing instruction data for fine-tuning large language models. This method involves collecting semi-structured and unstructured data; cleaning the semi-structured data to obtain a first instruction dataset; cleaning the unstructured data and segmenting it into text blocks; inputting the text blocks into a preset large language model to generate a second instruction dataset; inputting the first and second instruction datasets into the large language model to generate a third instruction dataset; and filtering the first, second, and third instruction datasets to obtain a fourth instruction dataset. Compared with existing technologies, this solution can quickly construct large-scale, high-quality instruction data with less manpower, effectively supporting the fine-tuning of large language models. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the instruction data construction method for fine-tuning large language models provided by the present invention.

[0025] Figure 2 This is one of the structural diagrams of the instruction data construction system for fine-tuning large language models provided by the present invention.

[0026] Figure 3 This is the second schematic diagram of the instruction data construction system for fine-tuning large language models provided by the present invention.

[0027] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] The present invention will now be described in detail with reference to the accompanying drawings. The specific operation methods in the method embodiments can also be applied to the device embodiments or system embodiments. In the description of the present invention, unless otherwise stated, "at least one" includes one or more. "Multiple" refers to two or more. For example, at least one of A, B, and C includes: A existing alone, B existing alone, A and B existing simultaneously, A and C existing simultaneously, B and C existing simultaneously, and A, B, and C existing simultaneously. In the present invention, " / " means "or". For example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0030] The present invention will now be described in detail with reference to specific embodiments.

[0031] In some specific embodiments of the present invention, such as Figure 1 As shown, this solution provides a method for constructing instruction data for fine-tuning large language models, including: Step 100: Collect semi-structured and unstructured data; Step 200: Clean the semi-structured data to obtain the first instruction dataset; Step 300: Clean the unstructured data and segment the cleaned unstructured data into text blocks; Step 400: Input the text block into a preset large language model, and generate a second instruction dataset through the large language model; Step 500: Input the first instruction dataset and the second instruction dataset into the large language model, and generate a third instruction dataset through the large language model; Step 600: Filter the first instruction dataset, the second instruction dataset, and the third instruction dataset to obtain the fourth instruction dataset.

[0032] It should be noted that existing instruction data construction schemes have high requirements for data cleaning and preparation, the quality of the generated data is difficult to guarantee, and it is difficult to cover all possible tasks and scenarios.

[0033] Therefore, by integrating semi-structured and unstructured data, this invention generates diverse instruction data using a pre-set powerful language model, and by filtering high-quality data for fine-tuning the large language model, it can automatically construct a large-scale, high-quality instruction dataset with reduced human intervention, significantly improving the instruction compliance performance of the large language model.

[0034] The above steps will be explained in detail below through specific embodiments.

[0035] Step 100: Collect semi-structured and unstructured data; In some possible embodiments of the present invention, each piece of semi-structured data includes two parts: a question and an answer, and each piece of unstructured data consists of text paragraphs, each of which is composed of free text without predefined relationships.

[0036] Specifically, this embodiment provides an implementation method for semi-structured and unstructured data. The semi-structured data consists of two parts: questions and answers, while the unstructured data consists of text paragraphs. By comprehensively collecting both semi-structured and unstructured data, a rich source of corpus is provided for the pre-training of the target large language model to be fine-tuned and the construction of the instruction dataset.

[0037] In a possible embodiment, the semi-structured data includes question and answer data from publicly accessible internet question and answer platforms, and the unstructured data includes pages from Wikipedia.

[0038] Specifically, the first step involves a comprehensive collection of semi-structured and unstructured data. Semi-structured data primarily includes question-and-answer data collected from publicly accessible internet question-and-answer platforms. This data typically has a relatively clear structure, with each piece of data consisting of a question and a corresponding answer. The advantage of this data structure is that it preserves the user's authentic questioning style while including direct answer information, making it highly suitable for training language models' question-and-answer capabilities in real-world scenarios. To collect this data, this embodiment selects several mainstream internet question-and-answer platforms, such as Quora, Stack Overflow, and Zhihu. These platforms cover a wide range of topics and user questions, including but not limited to fields such as technology, medicine, law, education, and daily life. By crawling and filtering the data from these platforms, a large number of representative question-and-answer pairs can be obtained, providing a foundation for subsequent data processing and model training. Regarding unstructured data, all pages from Wikipedia are primarily collected. Wikipedia covers various knowledge from natural sciences to humanities and social sciences, characterized by its rich content, timely updates, and broad coverage. Unstructured data typically exists in the form of continuous text paragraphs without strict structure or labels. These text paragraphs include various content elements such as article titles, body text, references, and external links. A comprehensive crawl of Wikipedia pages can yield a large amount of free text data, providing a rich corpus source for the pre-training of large language models and the construction of instruction datasets.

[0039] Step 200: Clean the semi-structured data to obtain the first instruction dataset; In some possible embodiments of the present invention, the step of cleaning the semi-structured data specifically includes: S31. Preprocess the semi-structured data to remove data with excessively short text length or too many illegal characters. S32. Use BERT to calculate the text perplexity of each piece of preprocessed semi-structured data; S33. Remove semi-structured data whose text perplexity exceeds the preset perplexity first threshold.

[0040] Specifically, this embodiment provides an implementation method for cleaning semi-structured data. The semi-structured data is cleaned using rule-based and semantic methods to obtain cleaned semi-structured data.

[0041] In a possible implementation, when removing data that is too short, a minimum text length threshold is first set to ensure that each data entry contains sufficient information to be of practical value for training the language model. For question-and-answer data, the number of characters in both the question and the answer must reach a certain length to ensure that they can express clear semantics and complete content. For example, the minimum length of the question portion can be set to 10 characters, and the minimum length of the answer portion to 20 characters.

[0042] In a possible implementation, when removing data with excessive illegal characters, natural language processing tools (such as spell checkers and grammar checkers) are used to automatically detect grammatical and spelling errors in the question-and-answer data. Minor errors can be corrected using automatic correction tools; question-and-answer pairs with too many errors or those that cannot be automatically corrected are removed from the dataset to ensure data quality. Then, illegal characters (such as non-standard language symbols, non-ASCII characters, etc.) are defined, and their presence is detected by writing regular expressions or other character filtering methods.

[0043] In a possible implementation, BERT is used to calculate the text perplexity of each data point. Specifically, this involves feeding each data point into the BERT model and calculating its perplexity score. A lower perplexity score indicates that the text is more similar to the corpus used to train the model, making it easier to understand and generate; while a higher perplexity score indicates that the text is more difficult to understand and may contain noise, ambiguity, or uncommon structures. Text with lower perplexity scores is retained.

[0044] Furthermore, a first perplexity threshold is preset, and data with a text perplexity score higher than the preset threshold is removed. Specifically, after calculating the text perplexity score for each piece of structured data, a preset perplexity threshold is used to filter the data. If the perplexity score of a piece of data is higher than the preset threshold (e.g., 50), it is considered unsuitable for model training and is removed from the dataset.

[0045] Step 300: Clean the unstructured data and segment the cleaned unstructured data into text blocks; In some possible embodiments of the present invention, the step of cleaning the unstructured data specifically includes: S41. Preprocess the unstructured data to remove data with excessively short text length or too many illegal characters. S42. Remove personal privacy data from the preprocessed unstructured data; S43. Use BERT to calculate the text perplexity of each piece of unstructured data; S44. Remove unstructured data whose text perplexity exceeds the preset second perplexity threshold.

[0046] Specifically, this embodiment provides an implementation method for cleaning unstructured data. The unstructured data is cleaned using a rule-based and semantic method to obtain cleaned unstructured data.

[0047] In a possible implementation, when removing data that is too short, a minimum text length threshold is first set to ensure that each data entry contains sufficient information to be of practical value for training the language model. For question-and-answer data, the number of characters in both the question and the answer must reach a certain length to ensure that they can express clear semantics and complete content. For example, the minimum length of the question portion can be set to 10 characters, and the minimum length of the answer portion to 20 characters.

[0048] In a possible implementation, when removing data with excessive illegal characters, illegal characters (including non-standard language symbols and non-ASCII characters) are defined, and their presence is detected by writing regular expressions and other character filtering methods.

[0049] In a possible implementation, when removing personal privacy data, entries that may contain personal privacy information are identified and removed. Personal privacy data includes information that can be used to identify an individual, such as name, ID number, phone number, email address, and home address. Named entity recognition technology is used to automatically detect personal information in the text through a pre-trained model. After identifying this sensitive information, it is either removed from the text or replaced with common terms (such as "someone" or "a place"). This step aims to protect user privacy, ensure compliance with data usage, and reduce potential legal and ethical risks.

[0050] In a possible implementation, BERT is used to calculate the text perplexity of each data item, specifically including: Each piece of data is fed into the BERT model, and its perplexity score is calculated. A lower perplexity score indicates that the text is highly similar to the corpus used to train the model, making it easier to understand and generate; while a higher perplexity score indicates that the text is more difficult to understand and may contain noise, ambiguity, or uncommon structures. Text with a lower perplexity score is retained.

[0051] Furthermore, a second perplexity threshold is preset to remove data whose text perplexity exceeds the preset threshold. Specifically, after calculating the text perplexity for each piece of structured data, a preset perplexity threshold is used to filter the data. In particular, if the perplexity score of a piece of data is higher than the set threshold (set to 50 in this example), it is considered unsuitable for model training and is removed from the dataset.

[0052] In some possible embodiments of the present invention, the step of segmenting the cleaned unstructured data into text blocks specifically includes: Each piece of unstructured data after cleaning is segmented into paragraphs.

[0053] Specifically, the cleaned unstructured data is segmented into paragraphs and divided into multiple text blocks to avoid each data item being too long to meet the maximum context length requirement of a pre-defined large language model such as GPT-4.

[0054] Step 400: Input the text block into a preset large language model, and generate a second instruction dataset through the large language model; In some possible embodiments of the present invention, the step of inputting the text block into a preset large language model and generating a second instruction dataset through the large language model specifically includes: S61. Input the text block into a preset large language model, and generate the question corresponding to the text block through the preset large language model; S62. Based on the question and the text block, generate the answer to the question using a preset large language model; S63. Concatenate the questions and their corresponding answers to form a first question-answer pair, and combine the first question-answer pairs into a second instruction dataset.

[0055] Specifically, this embodiment provides an implementation method for generating a second instruction dataset by means of a preset large language model. The segmented text blocks are input into the preset large language model, and question-answer pairs are generated by the large language model. The generated question-answer pairs are then combined to form the second instruction dataset.

[0056] In a possible implementation, the pre-defined large language model can be a strong language model, such as the GPT-4 model. A second instruction dataset is generated from the text blocks based on GPT-4.

[0057] Specifically, a second instruction dataset is generated using the GPT-4 model. Specifically, the cleaned structured data is segmented into multiple text blocks, and GPT-4 is used to generate questions and answers based on these text blocks. The generated second instruction dataset contains new question-answer pairs that reflect the key information in the original text blocks, expressed in natural language. In this way, a high-quality instruction dataset covering a wide range of content can be constructed for training and fine-tuning large language models, enhancing their ability to understand and answer various real-world questions.

[0058] In a possible implementation, GPT-4 is used to generate questions based on the text blocks. For each text block, the GPT-4 model generates relevant questions. Each text block is input into GPT-4, which automatically generates a question based on the content, topic, and contextual information of the text block. The generated questions aim to reflect the main concepts, facts, or information points in the text block, enabling the model to extract key information from the text. When generating questions, GPT-4 considers the question format, semantic clarity, and contextual relevance of natural language to ensure that the generated questions are reasonable, easy to understand, and accurately match the content of the text block. This step provides a targeted foundation for subsequent answer generation.

[0059] In a possible embodiment, GPT-4 is used to generate answers based on the text blocks of questions and responses generated in step S301. In this step, GPT-4 is used to generate answers based on the questions and corresponding text blocks generated in step S301. Each generated question is input into the GPT-4 model along with the original text block. GPT-4 automatically generates answers corresponding to the questions by analyzing the semantics of the questions and the content of the text blocks. The generated answers are based on the information in the text blocks, directly answering the questions and providing relevant details. This step ensures that each question has a suitable and accurate answer, thus forming a valid question-answer pair. The generated answer content should be logical and informative, fully expressing the information contained in the original text blocks and maintaining consistency with the questions.

[0060] In a possible implementation, the process of concatenating questions and their corresponding answers to form question-answer pairs involves combining each previously generated question with its corresponding answer to create a complete question-answer pair. Constructing question-answer pairs helps transform information from text blocks into instruction data that can be directly used by natural language processing models. By systematically organizing the question-answer pairs into a well-structured second instruction dataset, a standardized input data format is provided for subsequent large language model training. These question-answer pairs reflect the information in the original text blocks, and their generation and filtering by GPT-4 ensure high quality and relevance, thereby enhancing the effectiveness of the instruction dataset.

[0061] Step 500: Input the first instruction dataset and the second instruction dataset into the large language model, and generate a third instruction dataset through the large language model; In some possible embodiments of the present invention, the step of generating a third instruction dataset based on the first instruction dataset and the second instruction dataset using the large language model specifically includes: S71. Combine the first instruction dataset and the second instruction dataset into a seed instruction set; S72. Randomly select several seed instructions from the seed instruction set, and generate questions and answers from the seed instructions using a preset large language model to form a second question-answer pair; S73. Determine the score of each newly generated second question-and-answer pair using a preset large language model, and filter each second question-and-answer pair based on a preset score threshold to remove second question-and-answer pairs with scores lower than the preset score threshold; S74. Expand the filtered second question-and-answer pairs into the seed instruction set; S75. Repeat S72~S74 until the preset number of seed instruction expansions is reached.

[0062] Specifically, this embodiment provides an implementation method for generating a third instruction dataset through a large language model. By expanding the first instruction dataset and the second instruction dataset, large-scale, high-quality instruction data can be obtained quickly.

[0063] In a possible embodiment, the process of generating a third instruction dataset based on the first and second instruction datasets using a pre-defined large language model such as GPT-4 is described in this embodiment, where the third instruction dataset is generated using the GPT-4 model. Specifically, the third instruction dataset is expanded and generated based on the aforementioned first and second instruction datasets. This process aims to further enrich the content of the instruction dataset and improve the model's ability to handle diverse user questions. The GPT-4 model leverages its powerful language generation and understanding capabilities to generate new question-and-answer pairs by analyzing and synthesizing existing instruction data. These newly generated question-and-answer pairs constitute the third instruction dataset, expanding the coverage and depth of the original dataset.

[0064] In a possible embodiment, the first and second instruction datasets are used as seed instructions. Specifically, in this step, the cleaned first and second instruction datasets are merged to form the initial seed instruction set. The first instruction dataset originates from the cleaning results of semi-structured data, while the second instruction dataset is instruction data generated using GPT-4 based on structured data blocks. These seed instruction sets provide the foundation for subsequent instruction generation, covering various topics and question-and-answer types. Merging these two datasets improves the diversity and representativeness of instruction generation, laying the foundation for generating a high-quality third instruction dataset.

[0065] In a possible embodiment, several seed instructions are randomly selected from the seed instruction set, and questions and answers are generated from these seed instructions using a preset large language model to form a second question-answer pair. For example, five seed instructions can be randomly selected, and GPT-4 is used to generate questions and answers based on these five seed instructions to construct a question-answer pair. Specifically, in this step, five instructions are randomly selected from the merged seed instruction set. Then, the GPT-4 model is used with these selected seed instructions as input to generate new questions and answers based on these five seed instructions, constructing a new question-answer pair. When generating a new question-answer pair, GPT-4 analyzes the content, grammatical structure, and semantic relationships of these five seed instructions to generate questions and answers that are similar or related to them in terms of topic and style. These newly generated question-answer pairs aim to expand the coverage of instruction data, making the dataset contain a wider range of scenarios and question types, increasing the diversity and practical application value of instruction data.

[0066] In a possible embodiment, the process of removing second question-and-answer pairs with scores below the preset score threshold involves, for example, using GPT-4 to determine the score of newly generated question-and-answer pairs and removing those with scores below the set threshold. Specifically, to ensure the quality of newly generated question-and-answer pairs, in this step, GPT-4 is used to score each newly generated question-and-answer pair. The scoring criteria may include multiple dimensions such as semantic relevance, grammatical correctness, accuracy of the answer, and information completeness. GPT-4 generates a comprehensive score for each question-and-answer pair and compares this score with the preset threshold. Question-and-answer pairs with scores below the threshold are considered substandard and will be removed from the dataset. This process ensures that only high-quality content is retained in the expanded third instruction dataset, avoiding the impact of low-quality or irrelevant content on the model training effect.

[0067] In a possible embodiment, the process of expanding the filtered second question-and-answer pairs into the seed instruction set involves adding the remaining question-and-answer pairs to the seed instructions. Specifically, after the scoring and filtering process, all remaining question-and-answer pairs that meet the quality criteria are added to the seed instruction set. This operation not only increases the number of seed instructions but also enriches the diversity of the instruction set. After each expansion, the types of question-and-answer pairs and the range of topics included in the seed instruction set are improved, providing more reference samples for subsequent instruction generation. This gradual expansion method helps accumulate high-quality instruction data, providing a solid foundation for further generation processes.

[0068] Furthermore, the above process can be repeated until the set number of seed instruction expansions is reached. In other words, instruction data expansion is an iterative process, continuously executing the above steps. Each time, five new seed instructions are randomly selected from the expanded seed instruction set, new question-answer pairs are generated using GPT-4, scored, and filtered, and qualified question-answer pairs are added to the seed instruction set. This process is repeated until the pre-set number of seed instruction expansions is reached (10,000 in this example). Through repeated iterations, the third instruction dataset gradually grows into a high-quality instruction dataset with broad coverage and stable quality. This dataset will be used to further optimize and fine-tune the large language model, enabling it to handle more complex and diverse user needs in practical applications.

[0069] Step 600: Filter the first instruction dataset, the second instruction dataset, and the third instruction dataset to obtain the fourth instruction dataset; In some possible embodiments of the present invention, the step of filtering the first instruction dataset, the second instruction dataset, and the third instruction dataset to obtain a fourth instruction dataset specifically includes: S81, using an embedding model to convert the instruction data in the first instruction dataset, the second instruction dataset, and the third instruction dataset into embedding vectors; S82, calculate the cosine similarity between each of the embedded vectors; S83, remove instruction data corresponding to embedding vectors with cosine similarity higher than a preset similarity threshold.

[0070] Specifically, this embodiment provides an implementation method for obtaining a fourth instruction dataset. By filtering the obtained instruction dataset, the filtered instruction data is combined to form a fourth instruction dataset.

[0071] Specifically, an embedding model can be used to filter the first, second, and third instruction datasets to obtain a fourth instruction dataset. In this embodiment, the fourth instruction dataset is constructed by filtering the first, second, and third instruction datasets using an embedding model. The purpose of this filtering is to ensure the diversity and high quality of the instruction dataset, reduce redundant information, and improve the effectiveness of the instruction set in training a large language model. The specific process includes converting the instruction data into embedding vectors, calculating the cosine similarity between instruction data, and removing instruction data with excessively high similarity. The resulting fourth instruction dataset will contain semantically unique and information-rich question-answer pairs, providing high-quality data input for training the large language model.

[0072] In a possible implementation, during the process of converting instruction data into embedding vectors using an embedding model, each instruction data in the first, second, and third instruction datasets is converted into an embedding vector using the embedding model. The embedding model employs Sentence-BERT to map natural language text into a high-dimensional continuous vector space. Each question-answer pair (including the question and answer) is used as input, and the embedding model generates a corresponding embedding vector. This vector representation preserves the semantic information of the original text, allowing the similarity between different texts to be measured by geometric distance in the vector space. This step provides the foundation for subsequent similarity calculations and filtering of redundant data.

[0073] In possible embodiments, the embedding model is one of BERT, Sentence Transformers, and Word2Vec models.

[0074] In a possible embodiment, the cosine similarity between each of the embedded vectors is calculated, i.e., the cosine similarity between the instruction data is calculated. In this step, the cosine similarity between each pair of instruction data is calculated based on the generated embedded vectors. Cosine similarity is a measure of the similarity between the directions of two vectors, commonly used in text similarity analysis, and its value ranges from -1 to 1. By calculating the cosine similarity between the embedded vectors, the semantic similarity of different instruction data can be determined. Specifically, the embedded vectors are arranged into a matrix, and the similarity between each pair of vectors in the matrix is ​​calculated using the cosine similarity formula. High cosine similarity indicates that the corresponding instruction data are highly similar semantically, while low similarity indicates significant semantic differences. This step aims to identify semantically redundant instruction data, providing a basis for the next screening step.

[0075] In a possible implementation, instruction data corresponding to embedding vectors with a cosine similarity higher than a preset similarity threshold can be removed, thus removing instruction data with a cosine similarity higher than the set threshold. In this step, a similarity threshold (set to 0.8 in this example) is set based on the calculated cosine similarity to filter instruction data. If the cosine similarity of a pair of instruction data exceeds the set threshold, one of the instruction data is considered redundant and can be removed. Specifically, this involves traversing the cosine similarity matrix, identifying instruction pairs with similarity exceeding the threshold, marking one of the instruction data as redundant, and deleting it from the dataset. In this way, redundant information in the instruction dataset can be effectively reduced, ensuring that the final fourth instruction dataset only contains semantically independent and information-rich question-answer pairs. This process helps improve the training quality and efficiency of large language models and avoids performance degradation caused by redundant data.

[0076] In some possible embodiments of the present invention, after obtaining the fourth instruction dataset, the target large language model is fine-tuned based on the fourth instruction dataset.

[0077] In this step, the target large language model is fine-tuned using the fourth instruction dataset. The purpose of fine-tuning is to further optimize the large language model, enabling it to answer user questions more accurately and effectively, and to better adapt to specific task requirements. The fourth instruction dataset contains selected and optimized high-quality question-answer pairs covering a wide range of topics and complexities, ensuring that the model learns rich semantic information and diverse language structures during fine-tuning. The fine-tuning process adjusts the model's weights so that it can generate more accurate, relevant, and meaningful answers when dealing with questions similar to those in the training data.

[0078] In some possible embodiments of the present invention, the LoRa algorithm is used to fine-tune the target large language model; The Lora algorithm adjusts specific layers of a pre-trained target large language model by inserting a learnable low-rank fitting matrix through low-rank decomposition.

[0079] In this embodiment, the fine-tuning process employs a technique called Lora (Low-Rank Adaptation). The Lora method achieves fine-tuning by inserting learnable low-rank adaptation matrices into specific layers of the model. This method not only effectively reduces the computational resources required during fine-tuning but also significantly reduces memory consumption while maintaining the overall performance and expressive power of the model. Lora's low-rank decomposition mechanism allows for model optimization with only a small number of parameters adjusted, making the fine-tuning process more efficient and adaptable.

[0080] The instruction data construction method for fine-tuning large language models provided in this invention integrates semi-structured data (such as internet question-and-answer platforms) and unstructured data (such as Wikipedia pages), uses GPT-4 to generate diverse instruction data, and filters high-quality data through an embedding model, ultimately using it to fine-tune large language models. Compared with existing technologies, this method can automatically construct large-scale, high-quality instruction datasets with less manpower and reduced human intervention, effectively supporting the fine-tuning of large language models and significantly improving the instruction compliance performance of large language models.

[0081] In some specific embodiments of the present invention, such as Figure 2 As shown, this solution provides an instruction data construction system for fine-tuning large language models, including the following modules: Collection module 21 is used to collect semi-structured data and unstructured data; The first instruction dataset acquisition module 22 is used to clean the semi-structured data to obtain the first instruction dataset; The text segmentation module 23 is used to clean the unstructured data and segment the cleaned unstructured data into text blocks; The second instruction dataset acquisition module 24 is used to input the text block into a preset large language model and generate a second instruction dataset through the large language model. The third instruction dataset acquisition module 25 is used to input the first instruction dataset and the second instruction dataset into the large language model and generate a third instruction dataset through the large language model. The fourth instruction dataset acquisition module 26 is used to filter the first instruction dataset, the second instruction dataset, and the third instruction dataset to obtain the fourth instruction dataset.

[0082] In some possible embodiments of the present invention, the instruction data construction system for fine-tuning a large language model further includes: a fine-tuning module 27, used to fine-tune the target large language model based on the fourth instruction dataset.

[0083] The instruction data construction system for fine-tuning large language models provided in this embodiment of the invention has a similar implementation principle and beneficial effects to the instruction data construction method for fine-tuning large language models shown in the above embodiments. Please refer to the implementation principle and beneficial effects of the instruction data construction method for fine-tuning large language models shown in the above embodiments, which will not be repeated here.

[0084] In some possible embodiments of the present invention, such as Figure 3As shown, the instruction data construction system for fine-tuning large language models can be divided into different functional units. Through the collaborative work of meticulously designed specific modules, efficient training and optimization of large language models in vertical domains can be achieved. This system can specifically include: Data collection module 31 is used to collect semi-structured data and unstructured data, wherein the semi-structured data includes question and answer data from publicly accessible Internet question and answer platforms, and the unstructured data includes all pages of Wikipedia; Data cleaning module 32 is used to clean the semi-structured and unstructured data. The cleaning method for semi-structured data includes removing data with excessively short text length, removing data with excessively many illegal characters, and removing data with text perplexity exceeding a set threshold. The cleaning method for unstructured data includes removing data with excessively short text length, removing data with excessively many illegal characters, removing personal privacy data, and removing data with text perplexity exceeding a set threshold. The resulting data is the first instruction dataset. Data generation module 33 is used to generate larger-scale instruction data based on the instruction data constructed by the aforementioned module. It divides the cleaned structured data into text blocks, where each text block refers to a segment obtained from each data segment. It uses GPT-4 to generate a second instruction dataset based on the text blocks. Specifically, the second instruction dataset involves generating questions using GPT-4 based on the text blocks, generating answers using GPT-4 based on the generated questions and response text blocks, and concatenating the questions and corresponding answers to form question-answer pairs. Finally, it uses GPT-4 to generate a third instruction dataset based on the first and second instruction datasets. Specifically, the third instruction dataset is generated by combining the first and second instruction datasets. Five seed instructions are randomly selected from the instruction dataset. GPT-4 is used to generate questions and answers based on these five seed instructions, constructing question-answer pairs. GPT-4 is then used to evaluate the scores of the newly generated question-answer pairs, removing those with scores below a set threshold. The remaining question-answer pairs are then added to the seed instructions. This process is repeated until the set number of seed instruction additions is reached. An embedding model is used to filter the first, second, and third instruction datasets to obtain a fourth instruction dataset. Specifically, the fourth instruction dataset involves converting instruction data into embedding vectors using an embedding model, calculating the cosine similarity between instruction data, and removing instruction data with cosine similarity exceeding a set threshold. Model training module 34 is used to fine-tune the target large language model using the instruction dataset. Specifically, the fine-tuning method involves adjusting specific layers of the pre-trained language model by inserting a learnable low-rank adaptation matrix through low-rank decomposition.

[0085] Through the above-described configuration in this embodiment, instruction data construction utilizes multiple specially designed modules working collaboratively to efficiently train and optimize large language models using high-quality instruction datasets, thereby improving the overall performance and application value of the model. The functions of each module are closely integrated, collectively forming a complete framework for instruction data construction and model optimization.

[0086] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute an instruction data construction method for fine-tuning a large language model. This method includes: collecting semi-structured and unstructured data; cleaning the semi-structured data to obtain a first instruction dataset; cleaning the unstructured data and segmenting the cleaned unstructured data into text blocks; inputting the text blocks into a preset large language model to generate a second instruction dataset; inputting the first and second instruction datasets into the large language model to generate a third instruction dataset; and filtering the first, second, and third instruction datasets to obtain a fourth instruction dataset.

[0087] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0088] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the instruction data construction method for fine-tuning a large language model provided by the above methods. The method includes: collecting semi-structured data and unstructured data; cleaning the semi-structured data to obtain a first instruction dataset; cleaning the unstructured data and segmenting the cleaned unstructured data into text blocks; inputting the text blocks into a preset large language model to generate a second instruction dataset through the large language model; inputting the first instruction dataset and the second instruction dataset into the large language model to generate a third instruction dataset through the large language model; filtering the first instruction dataset, the second instruction dataset, and the third instruction dataset to obtain a fourth instruction dataset; and based on the fourth instruction dataset...

[0089] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the instruction data construction method for fine-tuning a large language model provided by the above methods. This method includes: collecting semi-structured and unstructured data; cleaning the semi-structured data to obtain a first instruction dataset; cleaning the unstructured data and segmenting the cleaned unstructured data into text blocks; inputting the text blocks into a preset large language model to generate a second instruction dataset; inputting the first and second instruction datasets into the large language model to generate a third instruction dataset; filtering the first, second, and third instruction datasets to obtain a fourth instruction dataset; and based on the fourth instruction dataset...

[0090] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing instruction data for fine-tuning large language models, characterized in that, include: Collect semi-structured and unstructured data, wherein the semi-structured data comes from real question-and-answer interaction data of publicly accessible Internet question-and-answer platforms, and the unstructured data comes from encyclopedic texts; The semi-structured data is cleaned to obtain the first instruction dataset; The unstructured data is cleaned, and the cleaned unstructured data is segmented into text blocks; The text block is input into a preset large language model, and a second instruction dataset is generated through the large language model, specifically including: S61. Input the text block into a preset large language model, and generate questions that reflect the main concepts, facts or information points in the text block based on the content, topic and context information of the text block through the preset large language model; S62. Input the question and the text block into a preset large language model, and generate an answer that is consistent with the information in the text block and fully expresses the information contained in the text block through the preset large language model; S63. Concatenate each question and its corresponding answer to form a first question-answer pair, and combine each first question-answer pair into a second instruction dataset; The first instruction dataset and the second instruction dataset are input into the large language model, and a third instruction dataset is generated through the large language model, specifically including: S71. Combine the first instruction dataset and the second instruction dataset into a seed instruction set; S72. Randomly select several seed instructions from the seed instruction set, and generate topic- and style-related questions and answers based on the content, grammatical structure and semantic relationship of the seed instructions through a preset large language model to form a second question-and-answer pair. S73. The second question-and-answer pairs are scored in multiple dimensions using a preset large language model, and a comprehensive score is generated for each second question-and-answer pair. The scoring dimensions include semantic relevance, grammatical correctness, answer accuracy, and information completeness. The second question-and-answer pairs are then filtered based on a preset score threshold to remove second question-and-answer pairs with a comprehensive score lower than the preset score threshold. S74. Expand the filtered second question-and-answer pairs into the seed instruction set; S75. Repeat S72~S74 until the preset number of seed instruction expansions is reached; The first instruction dataset, the second instruction dataset, and the third instruction dataset are filtered to obtain the fourth instruction dataset.

2. The instruction data construction method for fine-tuning large language models according to claim 1, characterized in that, Each piece of semi-structured data includes two parts: a question and an answer. Each piece of unstructured data consists of text paragraphs, and each text paragraph is composed of free text without predefined relationships.

3. The instruction data construction method for fine-tuning large language models according to claim 2, characterized in that, The steps for cleaning the semi-structured data specifically include: S31. Preprocess the semi-structured data to remove data with excessively short text length or too many illegal characters. S32. Use BERT to calculate the text perplexity of each piece of preprocessed semi-structured data; S33. Remove semi-structured data whose text perplexity exceeds the preset perplexity first threshold.

4. The instruction data construction method for fine-tuning large language models according to claim 2, characterized in that, The steps for cleaning the unstructured data specifically include: S41. Preprocess the unstructured data to remove data with excessively short text length or too many illegal characters. S42. Remove personal privacy data from the preprocessed unstructured data; S43. Use BERT to calculate the text perplexity of each piece of unstructured data; S44. Remove unstructured data whose text perplexity exceeds the preset second perplexity threshold.

5. The instruction data construction method for fine-tuning a large language model according to claim 4, characterized in that, The steps of segmenting the cleaned unstructured data into text blocks specifically include: Each piece of unstructured data after cleaning is segmented into paragraphs.

6. The instruction data construction method for fine-tuning a large language model according to claim 1, characterized in that, The step of filtering the first instruction dataset, the second instruction dataset, and the third instruction dataset to obtain the fourth instruction dataset specifically includes: S81, using an embedding model to convert the instruction data in the first instruction dataset, the second instruction dataset, and the third instruction dataset into embedding vectors; S82, calculate the cosine similarity between each of the embedded vectors; S83, remove instruction data corresponding to embedding vectors with cosine similarity higher than a preset similarity threshold.

7. The instruction data construction method for fine-tuning a large language model according to claim 6, characterized in that, The embedding model is one of BERT, Sentence Transformers, and Word2Vec.

8. The method for constructing instruction data for fine-tuning a large language model according to any one of claims 1-7, characterized in that, After obtaining the fourth instruction dataset, the method further includes: Based on the aforementioned fourth instruction dataset, the target large language model is fine-tuned.

9. The instruction data construction method for fine-tuning a large language model according to claim 8, characterized in that, The LoRa algorithm was used to fine-tune the target large language model; The Lora algorithm adjusts specific layers of a pre-trained target large language model by inserting a learnable low-rank fitting matrix through low-rank decomposition.

10. A system for constructing instruction data for fine-tuning large language models, characterized in that, include: The collection module is used to collect semi-structured data and unstructured data, wherein the semi-structured data comes from real question-and-answer interaction data of publicly accessible Internet question-and-answer platforms, and the unstructured data comes from encyclopedic texts; The first instruction dataset acquisition module is used to clean the semi-structured data to obtain the first instruction dataset; The text segmentation module is used to clean the unstructured data and segment the cleaned unstructured data into text blocks; The second instruction dataset acquisition module is used to input the text block into a preset large language model and generate a second instruction dataset through the large language model, specifically for: The text block is input into a preset large language model, and the preset large language model generates questions that reflect the main concepts, facts or information points in the text block based on the content, topic and context information of the text block. The question and the text block are input into a preset large language model, and an answer that is consistent with the information in the text block and fully expresses the information contained in the text block is generated by the preset large language model. The questions and their corresponding answers are concatenated to form a first question-answer pair, and the first question-answer pairs are combined to form a second instruction dataset. The third instruction dataset acquisition module is used to input the first instruction dataset and the second instruction dataset into the large language model, and generate a third instruction dataset through the large language model, specifically for: Combine the first instruction dataset and the second instruction dataset into a seed instruction set; Several seed instructions are randomly selected from the seed instruction set. Based on the content, syntactic structure and semantic relationship of the seed instructions, a preset large language model is used to generate topic- and style-related questions and answers to form a second question-and-answer pair. Each second question-and-answer pair is scored in multiple dimensions using a pre-defined large language model, generating a comprehensive score for each pair. The scoring dimensions include semantic relevance, grammatical correctness, answer accuracy, and information completeness. Each second question-and-answer pair is then filtered based on a pre-defined score threshold to remove pairs with a comprehensive score lower than the pre-defined score threshold. The filtered second question-and-answer pairs are then added to the seed instruction set. Repeat the process until the preset number of seed instruction expansions is reached; The fourth instruction dataset acquisition module is used to filter the first instruction dataset, the second instruction dataset, and the third instruction dataset to obtain the fourth instruction dataset.

Images