High-quality data set construction method and system for large model in vertical field

Abstract

The invention discloses a vertical field large model-oriented high-quality data set construction method and system, and relates to the technical field of text processing, and the method comprises the steps: selecting seed text data, carrying out long text processing, segmenting the seed text data into a plurality of text segments, processing the text segments by using a large model tool to generate seed question and answer pairs, and constructing a seed question and answer pair; auditing and optimizing are carried out according to a preset auditing standard; making an instruction generation data set for the cue word, the text fragment and the seed question and answer pair according to a preset organization rule, performing fine adjustment on a pre-selected base model by using the instruction generation data set, and performing iterative optimization on the large model by testing the performance of the large model after fine adjustment and adjusting and optimizing parameters, thereby obtaining an intelligent extraction large model IELM; and based on the intelligent extraction big model, processing the text data by utilizing a preset script code, automatically generating knowledge question and answer pair data corresponding to the text data, and performing format conversion processing to obtain an instruction data set which is used for a training data set of the vertical domain knowledge big model.

Description

Technical Field

[0001] The present invention relates to the fields of LLM large language model technology and text processing technology, and more specifically, to a method and system for constructing a high-quality data set for a large model in a vertical field. Background Art

[0002] High-quality datasets come from various types of long texts, such as papers, textbooks, and books, and cover a wide range of fields, including but not limited to materials science, medicine, physics, chemistry, law, and financial knowledge. In the application of large vertical domain knowledge models, the quality of the dataset is a key factor that directly affects the performance of the model. However, the construction of high-quality datasets usually faces these problems:

[0003] Traditional methods of extracting high-quality domain knowledge from various types of long texts usually rely on a large amount of manual annotation. This method is not only inefficient and costly, but also difficult to fully cover the complex knowledge in vertical fields. In addition, the current pre-trained text large models are generally transformer architectures, and the length of tokens processed at a time does not exceed the preset length. For example, the glm4-9B model has a single token length of no more than 8192. Therefore, when processing long texts exceeding the preset token length, the existing generative text model structure may not be able to fully extract the information in it, resulting in incomplete information extraction.

[0004] Therefore, how to extract knowledge question-answer pairs from vertical field papers, textbooks, and book text data to build high-quality data sets in a more efficient, accurate, and cost-effective way is a key technology to improve the performance of large vertical field knowledge models. Summary of the invention

[0005] In view of the fact that the existing dataset construction methods have the problems of low efficiency and high cost in manually annotating large-scale text data, and the problem of incomplete information extraction of long text data by large models, the present invention is proposed.

[0006] Therefore, the problem to be solved by the present invention is how to provide a more efficient, accurate and lower-cost method for constructing a data set.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In a first aspect, an embodiment of the present invention provides a method for constructing a high-quality dataset for a large model in a vertical field, comprising the following steps:

[0009] S1: Select seed text data and process long texts into multiple text segments. Use large model tools to process the text segments to generate seed question-answer pairs, and review and optimize them according to preset review standards.

[0010] S2: Prompt words, text fragments, and seed question-answer pairs are used to generate instruction data sets according to preset organizational rules. The pre-selected base model is fine-tuned using the instruction data sets. The performance of the fine-tuned large model is tested, and the optimization parameters are adjusted to iteratively optimize the large model, thereby obtaining the intelligent large model IELM.

[0011] S3: Based on the Zhicui big model, use the preset script code to process the text data, automatically generate the knowledge question and answer data corresponding to the text data, and perform format conversion processing to obtain the instruction data set, which can be used as the training data set of the vertical field knowledge big model.

[0012] As a preferred solution of the method for constructing a high-quality dataset for a large vertical model described in the present invention, step S1 includes the following steps:

[0013] Set the selection ratio of seed texts, and select representative texts from the vertical field text data as seed texts according to the selection ratio, so as to be used for obtaining seed question-answer pairs in the future;

[0014] The seed text processing length limit information is set according to the maximum token processing length information of the large model tool, and the seed text is divided into multiple text segments according to the seed text processing length limit information, wherein the length of the text segment is the number of words converted from a single word as a primitive. Starting from the second text segment, the starting position of each subsequent text segment overlaps with the end of the previous text segment according to a preset overlap ratio;

[0015] Inputting the multiple text segments into the large model tool in sequence according to the segmentation order, and guiding the large model tool to generate initial seed question-answer pairs related to the text segments in combination with the designed structured prompt words, wherein the initial seed question-answer pairs are generated according to the output format preset in the structured prompt words, so as to obtain the initial seed question-answer pairs using a preset script code;

[0016] The review level of the initial seed question and answer pairs is set, and corresponding review standards are formulated based on the review level. Vertical field experts review and optimize the content of the seed question and answer pairs according to the review standards.

[0017] As a preferred solution of the method for constructing a high-quality dataset for a vertical field big model described in the present invention, the step S2 method includes: setting an organization rule for an instruction generation dataset to associate text fragments, seed question and answer pairs related to the text fragments, and designed structured prompt words, organizing the text fragments, seed question and answer pairs related to the text fragments, and designed structured prompt words according to the organization rule to form an instruction generation dataset for training the instruction generation big model, wherein the structured prompt words in the instruction generation dataset contain output format information, and the data format of the seed question and answer pairs is organized according to the output format information so that the output format information of the seed question and answer pairs in the instruction generation dataset is consistent with that in the structured prompt words; selecting a base model for training, and fine-tuning the base model using the instruction generation dataset; testing the performance of the fine-tuned big model, adjusting the optimization parameters according to the test results, and iteratively optimizing the big model to obtain the instruction generation big model, the instruction generation big model is a special knowledge question and answer generation big model, and the knowledge question and answer generation big model is named the Intelligent Big Model IELM.

[0018] As a preferred solution of the method for constructing a high-quality dataset for a vertical field big model described in the present invention, the step S3 method includes: inputting the vertical field text data into the Zhicui big model, and using a pre-set script code to automatically generate high-quality question and answer pairs IELM (prompt, text) = QA related to the vertical field text data, wherein prompt represents a structured prompt word, text represents a text fragment, and QA represents a question and answer pair generated using the Zhicui big model IELM; performing format conversion processing on the generated high-quality question and answer pairs according to the vertical field big model training dataset format to obtain an instruction dataset, which is used as a training dataset for the vertical field knowledge big model.

[0019] In the second aspect, an embodiment of the present invention provides a high-quality dataset construction system for a vertical field big model, which includes: a seed question and answer pair generation module, which selects seed text data and processes long text to split it into multiple text segments, uses a big model tool to process the text segments to generate seed question and answer pairs, and reviews and optimizes them according to preset review standards; an intelligent big model development module, which generates an instruction generation dataset for prompt words, text segments and seed question and answer pairs according to preset organizational rules, uses the instruction generation dataset to fine-tune the pre-selected base model, and adjusts the optimization parameters to iteratively optimize the big model by testing the performance of the fine-tuned big model, thereby obtaining the intelligent big model IELM; an instruction dataset construction module, based on the intelligent big model, uses a preset script code to process text data, automatically generates knowledge question and answer pair data corresponding to the text data, and performs format conversion processing to obtain an instruction dataset for use as a training dataset for the vertical field knowledge big model.

[0020] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, the steps of a method for constructing a high-quality dataset for a large vertical field model as described in the first aspect of the present invention are implemented.

[0021] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, the steps of a method for constructing a high-quality dataset for a large vertical field model as described in the first aspect of the present invention are implemented.

[0022] The beneficial effects of the present invention are as follows: the present invention generates seed question and answer pairs based on a big model tool, and after review and optimization, uses the seed question and answer to develop a wisdom big model, a special knowledge question and answer generation big model, which automatically generates knowledge question and answer data corresponding to text data, thereby efficiently constructing vertical field data sets; compared with traditional manual labeling methods, it avoids the need for a large amount of manual labeling, reduces costs and improves efficiency; compared with existing big models, it can fully extract long text information; the method of the present invention is suitable for any scientific field, including but not limited to materials science, medicine, physics, chemistry, law, and finance. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0024] Figure 1 A flowchart of a method for constructing a high-quality dataset for a large vertical domain knowledge model provided in an embodiment of this specification. DETAILED DESCRIPTION

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Example 1: See Figure 1 As shown, this embodiment provides a method for constructing a high-quality dataset for a large model in a vertical field, including:

[0027] S1: Select seed text data and process long texts into multiple text segments. Use large model tools to process the text segments to generate seed question-answer pairs, and review and optimize them according to preset review standards.

[0028] Specifically, seed texts are selected from vertical fields, which include but are not limited to materials science, medicine, physics, chemistry, law, and finance. The seed texts can be selected by obtaining text data from papers, journals, and book websites, and representative texts covering the core knowledge of the field are selected as seed texts according to a set selection ratio, wherein the selection ratio is in the range of 5% to 10% of the total amount of text data obtained in the vertical field.

[0029] Furthermore, the seed text processing length limit information is set according to the maximum token processing length information of the large model tool, and the seed text is processed as a long text according to the seed text processing length limit information, and is divided into multiple text segments. The large model tool here can be GPT4 or GLM4.

[0030] Among them, the length of the text segment is the word length converted from a single word as the primitive. Starting from the second text segment, the starting position of each subsequent text segment overlaps with the end of the previous text segment according to a preset overlapping ratio. The overlapping ratio here is 10% to 20% of the length of the text segment.

[0031] Multiple text fragments are input into the big model tool in sequence according to the segmentation order, and the API interface of the big model tool is called by pre-written script code to process the text fragments. Combined with the designed structured prompt words, the big model tool is guided to generate initial seed question-answer pairs related to the text fragments. The initial seed question-answer pairs refer to questions and answers related to the seed text content generated by the big model tool under the guidance of the structured prompt words.

[0032] Among them, the initial seed question and answer pairs are generated by using the large model tool GPT4 or GLM4 according to the output format preset in the structured prompt words. The output format can be a json format organized by Q and A, where Q represents the question and A represents the answer. The structured prompt words here are used to guide the large model tool to generate seed question and answer pairs, which can include specified task roles, such as text knowledge miners, task requirements, such as designed questions covering central ideas and key technologies, and explaining material properties and key indicators, as well as output formats, such as [{"Q":"","A":""},{"Q":"","A":""}], where the task requirements are related to technical details in a specific technical field, such as material properties and mechanism analysis in materials science.

[0033] Furthermore, an audit level is set for the initial seed question and answer pairs, and corresponding audit standards are formulated based on the audit level. Vertical field experts review and optimize the content of the seed question and answer pairs according to the audit standards.

[0034] Among them, the audit level is divided into three quality audit levels, and the three quality audit levels are based on the following audit standards: logical clarity and information completeness are the first quality level, which is the most important audit standard; the second quality level is the appropriateness of the expression method, whether it focuses on the knowledge content itself; the third quality level is the question and answer pairs that are less related to the vertical field knowledge. From the first to the third quality level, the importance decreases successively. Vertical field experts review and optimize the initial seed question and answer pairs according to the above audit standards.

[0035] S2: Generate an instruction data set based on preset organizational rules using prompt words, text fragments, and seed question-answer pairs. Fine-tune the pre-selected base model using the instruction data set. Test the performance of the fine-tuned large model and adjust the optimization parameters to iteratively optimize the large model, thereby obtaining the intelligent large model IELM.

[0036] Specifically, first set the instruction to generate the dataset organization rules, for example

[0037] {"input":"prompt+text","output":"I"}, where prompt is a pre-designed structured prompt word, text is text fragment information, and I is a set of instructions, which includes seed question-answer pairs corresponding to the text fragments. According to the organization rule, the text fragments, the seed question-answer pairs related to the text fragments, and the designed structured prompt words are organized to form an instruction generation dataset for training the instruction generation large model, wherein the structured prompt words in the instruction generation dataset include output format information, and the data format of the seed question-answer pairs contained in the instruction I is organized according to the output format information to ensure that the output format information of the seed question-answer pairs in the instruction generation dataset is consistent with that in the structured prompt words.

[0038] Furthermore, a base model is selected for training. The selection method is to compare multiple open source large models, such as the qwen2 series open source large models, the llama series open source large models, and the chatglm series open source large models, and compare the performance of these models in question and answer generation tasks. The large model with better performance is selected as the base model, and the instruction generation data set is used to fine-tune the base model.

[0039] Furthermore, the performance of the fine-tuned big model is tested, the question and answer generation task results of the big model are tested, the optimization parameters are adjusted according to the test results, and the big model is iteratively optimized to obtain the instruction generation big model. The instruction generation big model is a special knowledge question and answer generation big model. The knowledge question and answer generation big model is named IELM, Intelligent Extraction Large Model.

[0040] S3: Based on the Zhicui big model, use the preset script code to process the text data, automatically generate the knowledge question and answer data corresponding to the text data, and perform format conversion processing to obtain the instruction data set, which can be used as the training data set of the vertical field knowledge big model.

[0041] Specifically, the vertical field text data is input into the Zhicui big model. The vertical field text data here may include but are not limited to material science, physics, chemistry, biology, and medicine text data. The text data format can be PDF, docx, and txt. For example, physical chemistry journal text in PDF format, using pre-set python script code, automatically generate high-quality question and answer pairs related to the text IELM (prompt, text) = QA, where prompt represents a structured prompt word, text represents a text fragment, and QA represents a question and answer pair generated using the Zhicui big model IELM.

[0042] Furthermore, the generated high-quality question-answer pairs are formatted and processed according to the format of the vertical field big model training dataset to obtain an instruction dataset, which is used as a training dataset for the vertical field knowledge big model. The vertical field big model here can be a big language model in the field of materials science, a big language model in the field of biological science, or a big language model in the field of finance.

[0043] Furthermore, the present embodiment also provides a high-quality dataset construction system for vertical field big models, including: a seed question and answer pair generation module, which selects seed text data and processes long text to split it into multiple text segments, uses a big model tool to process the text segments to generate seed question and answer pairs, and reviews and optimizes them according to preset review standards; an intelligent big model development module, which generates an instruction generation dataset for prompt words, text segments and seed question and answer pairs according to preset organizational rules, and uses the instruction generation dataset to fine-tune the pre-selected base model, and adjusts the optimization parameters to iteratively optimize the big model by testing the performance of the fine-tuned big model, thereby obtaining the intelligent big model IELM; an instruction dataset construction module, based on the intelligent big model, uses a preset script code to process text data, automatically generates knowledge question and answer pair data corresponding to the text data, and performs format conversion processing to obtain an instruction dataset for use as a training dataset for the vertical field knowledge big model.

[0044] This embodiment also provides a computer device, which is suitable for a method for constructing a high-quality dataset for a large model in a vertical field, including a processor and a memory communicatively connected to the processor; wherein the memory is used to store computer-executable instructions, and the executor is used to execute the computer-executable instructions to implement a method for constructing a high-quality dataset for a large model in a vertical field as proposed above.

[0045] The present embodiment also provides a storage medium having a computer program stored thereon, which implements the following steps when executed by a processor: selecting seed text data and performing long text processing to split it into multiple text segments, using a large model tool to process the text segments to generate seed question-answer pairs, and reviewing and optimizing them according to preset review standards; generating instruction generation data sets based on preset organizational rules for prompt words, text segments, and seed question-answer pairs, using the instruction generation data set to fine-tune a pre-selected base model, adjusting optimization parameters to iteratively optimize the large model by testing the performance of the fine-tuned large model, thereby obtaining the IELM large model; based on the IELM large model, using a preset script code to process text data, automatically generating knowledge question-answer pair data corresponding to the text data, and performing format conversion processing to obtain an instruction data set for use as a training data set for the vertical domain knowledge large model.

[0046] In summary, the present invention generates seed question and answer pairs based on the big model tool, and after review and optimization, uses the seed question and answer pairs to make instructions according to preset organizational rules to generate data sets and develop the Zhicui big model. The Zhicui big model is a special knowledge question and answer generation big model. Through pre-set script codes, the Zhicui big model is used to automatically generate knowledge question and answer data corresponding to text data, thereby efficiently constructing data sets in vertical fields.

[0047] It should be noted that the above contents are merely examples and explanations of the present invention rather than limitations. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the invention or exceed the scope defined by the claims, they shall all fall within the protection scope of the present invention.

Claims

1. A method for constructing a high-quality dataset for a large vertical model, characterized in that: The following steps are involved: S1: Select seed text data and process long texts into multiple text segments. Use large model tools to process the text segments to generate seed question-answer pairs, and review and optimize them according to preset review standards. S2: Prompt words, text fragments, and seed question-answer pairs are used to generate instruction data sets according to preset organizational rules. The pre-selected base model is fine-tuned using the instruction data sets. The performance of the fine-tuned large model is tested, and the optimization parameters are adjusted to iteratively optimize the large model, thereby obtaining the intelligent large model IELM. S3: Based on the Zhicui big model, use the preset script code to process the text data, automatically generate the knowledge question and answer data corresponding to the text data, and perform format conversion processing to obtain the instruction data set, which can be used as the training data set of the vertical field knowledge big model.

2. According to claim 1, a method for constructing a high-quality dataset for a large vertical domain model is characterized in that: The step S1 specifically includes: Set the selection ratio of seed texts, and select representative texts from the vertical field text data as seed texts according to the selection ratio, so as to be used for obtaining seed question-answer pairs in the future; The seed text processing length limit information is set according to the maximum token processing length information of the large model tool, and the seed text is divided into multiple text segments according to the seed text processing length limit information, wherein the length of the text segment is the number of words converted from a single word as a primitive. Starting from the second text segment, the starting position of each subsequent text segment overlaps with the end of the previous text segment according to a preset overlap ratio; Inputting the multiple text segments into the large model tool in sequence according to the segmentation order, and guiding the large model tool to generate initial seed question-answer pairs related to the text segments in combination with the designed structured prompt words, wherein the initial seed question-answer pairs are generated by the large model tool according to the output format preset in the structured prompt words; The review level of the initial seed question and answer pairs is set, and corresponding review standards are formulated based on the review level. Vertical field experts review and optimize the content of the seed question and answer pairs according to the review standards.

3. A method for constructing a high-quality dataset for a large vertical domain model according to claim 2, characterized in that: The step S2 specifically includes: An organization rule of an instruction generation data set is set to associate text segments, seed question-answer pairs related to the text segments, and designed structured prompt words. According to the organization rule, the text segments, seed question-answer pairs related to the text segments, and designed structured prompt words are organized to form an instruction generation data set for training an instruction generation large model, wherein the structured prompt words in the instruction generation data set include output format information, and the data format of the seed question-answer pairs is organized according to the output format information so as to make the seed question-answer pairs in the instruction generation data set consistent with the output format information in the structured prompt words; Select the base model for training and use the instruction to generate a dataset to fine-tune the base model; The performance of the fine-tuned big model is tested, and the optimization parameters are adjusted according to the test results. The big model is iteratively optimized to obtain the instruction generation big model. The instruction generation big model is a dedicated knowledge question and answer generation big model. The knowledge question and answer generation big model is named the Intelligent Big Model IELM.

4. The method for constructing a high-quality dataset for a large vertical domain model according to claim 3, characterized in that: The step S3 specifically includes: Input vertical field text data into the intelligent big model, and use the pre-set script code to automatically generate high-quality question-answer pairs related to the vertical field text data IELM (prompt, text) = QA, where prompt represents a structured prompt word, text represents a text fragment, and QA represents a question-answer pair generated by the intelligent big model IELM; The generated high-quality question-answer pairs are formatted and processed according to the vertical domain big model training dataset format to obtain an instruction dataset, which is used as a training dataset for the vertical domain knowledge big model.

5. A high-quality dataset construction system for a vertical field large model, based on a high-quality dataset construction method for a vertical field large model according to any one of claims 1 to 4, characterized in that: include: The seed question-answer pair generation module selects seed text data and processes long texts into multiple text segments. It uses a large model tool to process the text segments to generate seed question-answer pairs, and then reviews and optimizes them according to preset review standards. The intelligent large model development module generates instruction data sets according to preset organizational rules for prompt words, text fragments, and seed question-answer pairs. The instruction-generated data sets are used to fine-tune the pre-selected base model. By testing the performance of the fine-tuned large model, the optimization parameters are adjusted to iteratively optimize the large model, thereby obtaining the intelligent large model IELM. The instruction data set construction module, based on the Zhicui big model, uses preset script codes to process text data, automatically generates knowledge question and answer data corresponding to the text data, and performs format conversion processing to obtain the instruction data set for use as a training data set for the vertical domain knowledge big model.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the computer program is executed by a processor, the steps of a method for constructing a high-quality dataset for a large vertical field model as described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for constructing a high-quality dataset for a large vertical field model as described in any one of claims 1 to 4 are implemented.

Images