Multi-agent collaboration system for constructing data set for vertical field large language model training

Abstract

The invention discloses a multi-agent collaboration system for constructing a data set for vertical field large language model training. The system integrates four levels of data acquisition, data processing, data generation and data optimization, and can automatically complete full-process processing from multi-source corpus acquisition, structured analysis to instruction and preference data construction after receiving a field demand and a retrieval intention proposed by a user. Through the cooperative effect of picking, structured reconstruction, co-citation analysis, expert generation and screening optimization intelligent agents, automatic construction of a pre-training data set, an instruction fine tuning data set and human feedback reinforcement learning RLHF data is realized, and the consistency, traceability and high quality of the data are ensured; therefore, the training effect and the application performance of the vertical field large language model are effectively improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence and natural language processing, and particularly relates to a multi-agent collaborative system for constructing a vertical field large language model training data set. BACKGROUND

[0002] In the training process of a large language model, data quality directly affects the understanding and generation performance of the model. Existing training usually includes three stages of pre-training, fine-tuning and human feedback reinforcement learning (RLHF). The pre-training data is mostly derived from open web pages, academic papers and books, which are often integrated by public resources or processed by self-crawling; the fine-tuning and RLHF data mainly rely on manual annotation in the early stage, and with the development of large language models and prompt word engineering, although part of the data can be automatically generated, the quality stability and controllability are insufficient, and there is much noise and redundancy. In order to enhance the knowledge mastery and reasoning ability of the model in a specific professional field, the industry attempts to introduce professional corpus for secondary training on the basis of general models. However, the acquisition and processing of high-quality professional data is still limited: first, how to efficiently gather and retain the structured information (such as tables, formulas and figure captions) in the literature in multiple sources; second, how to systematically construct high-quality instructions and preference data to reduce the cost of manual work and ensure consistency; third, there is a lack of integrated process and traceable metadata management throughout the "acquisition-processing-generation-optimization" process. Therefore, it has become an urgent problem to construct a technical solution for automatically obtaining and optimizing training data for vertical fields. SUMMARY

[0003] Based on the problems existing in the prior art, the application ensures high fidelity and traceability of corpus through structure preservation analysis and metadata through management, strengthens the relevance and knowledge density of the field through co-citation analysis, and improves the quality of instruction samples through expert agents and quality closed loops, and constructs high-reliable RLHF data through the joint mechanism of reward modeling and rejection sampling, thereby realizing end-to-end high-quality data automation production for vertical fields as a whole.

[0004] To achieve the above purpose, the application adopts the following technical solutions: The system comprises a crawling agent, a structured reconstruction agent, a data preprocessing agent, a co-citation analysis agent, a sub-field expert agent, a screening agent, an optimization agent, a reward modeling agent and a rejection sampling agent. The system is used to realize the following steps: S1: receiving user input, calling the crawling agent to obtain multi-source corpus according to the sub-field strategy, and constructing an original field corpus library; calling the structured reconstruction agent to analyze and reconstruct the corpus in the original field corpus library, and generating a structured field corpus library.

[0005] S2: Based on the structured domain corpus, call the data preprocessing intelligent agent to perform data filtering and data deduplication, and construct the pre-training data set.

[0006] S3: Based on the structured domain corpus, call the co-citation analysis intelligent agent to calculate the local cited quantity and perform co-citation analysis to filter out high-value literature and construct high-quality corpus of each sub-field.

[0007] S4: Based on the high-quality corpus of each sub-field, call the corresponding sub-field expert intelligent agent to generate instruction data in batches; call the screening intelligent agent and optimization intelligent agent to construct the field instruction fine-tuning data set.

[0008] S5: Based on the field instruction fine-tuning data set, call the reward modeling intelligent agent to construct the reward model training data set; call the rejection sampling intelligent agent to form the fine-tuning data set, and merge to form the human feedback reinforcement learning data set.

[0009] In the above technical solution, further, S1 includes the following steps: S11, the harvesting intelligent agent receives the keyword set and the target field intent input by the user, calls the large language model to analyze and expand the target field intent, obtains a plurality of sub-fields, expands and refines the keyword set in combination with the prior knowledge of the large language model and the domain ontology, generates a fine-grained field keyword table, and generates corresponding retrieval combinations and crawling strategies.

[0010] S12, based on the retrieval combinations and crawling strategies obtained in S11, the harvesting intelligent agent crawls publicly available academic papers from academic preprint databases, publicly available journal paper full texts and their metadata from journal databases, and publicly available monographs and reports from professional knowledge platforms, and stores them according to sources and sub-fields to form an original domain corpus. PDF format files are used as collection objects to ensure the structural integrity and consistency of the literature.

[0011] S13, the structured reconstruction intelligent agent analyzes and reconstructs each corpus in the original domain corpus, including: the structured reconstruction intelligent agent calls the built-in PDF analysis module to process each page of the original domain corpus, performs layout segmentation, identifies paragraphs, chapter titles, tables, mathematical formulas, image placeholders, footnotes, and reference elements, and reconstructs the natural reading order according to the layout rules, thereby eliminating multi-column layout, header and footer, and page break interference.

[0012] S14, the structured reconstruction agent performs semantic-level structured conversion on each type of element obtained by parsing; specifically including: performing cell griding and cross-row and cross-column parsing on the table to reconstruct editable table text; performing symbol recognition and standardized conversion on the mathematical formula to unify the standard formula expression syntax, preferably in LaTeX form; establishing a bidirectional association relationship between the image placeholder, table and footnote respectively; extracting and field standardizing the reference item; and automatically generating a table of contents outline according to the hierarchical relationship of the chapter titles.

[0013] To ensure that the conversion result can retain the structural elements in the original literature as much as possible and support subsequent readability and processability, the structured reconstruction agent converts the parsing result into an intermediate representation and stores it. Preferably, the intermediate representation adopts the Markdown format to retain the structural characteristics of elements such as tables, formulas, and figure captions, avoiding formula loss or structural damage when directly converted to pure text format. The intermediate representation generated in this way has high human readability and is convenient for subsequent corpus cleaning, quality detection, and machine processing, thereby forming a structured domain corpus input that retains the original structural characteristics.

[0014] S15, the structured reconstruction agent performs structure consistency verification on the intermediate representation output by S14; wherein the structure consistency score is defined as:

[0015] wherein, represents the table cell reconstruction accuracy, represents the formula parsing rate, represents the association completeness rate of the image placeholder, table and footnote respectively, represents the chapter level matching degree. When the structure consistency score is lower than the preset threshold, the structured reconstruction agent calls a large language model to perform structure repair, specifically including table alignment correction, formula symbol and logic consistency repair, and chart numbering and cross-reference completion operation. If it still does not reach the preset threshold after three consecutive repairs, the corpus is marked as an abnormal sample and removed from the original domain corpus. Finally, the corpus that passes the consistency verification is included in the structured domain corpus as the basis for subsequent pre-training dataset construction.

[0016] Further, S2 includes the following steps: S21. Based on the structured domain corpus, the data preprocessing agent first performs data filtering on the corpus, including quality filtering and sensitive content filtering. Preferably, a large language model is invoked as a text classification discriminator to sequentially read the content of the structured domain corpus, identify and remove low-quality text with formatting defects, garbled symbols, incomplete content, or logical incoherence, and detect and remove content containing privacy information or that does not meet compliance requirements, thereby ensuring that the corpus data entering subsequent processing stages has high integrity and compliance.

[0017] S22. Based on the data filtering in S21, the data preprocessing agent calls a high-efficiency deduplication module built on Bert-base to cluster the text of the corpus in the vector space; preferably, each text... Inputting the BERT-base model yields a vector representation. In vector space The K-Means algorithm is used for clustering, and the objective function is defined as:

[0018] in, For the first One cluster, This is the centroid of the cluster.

[0019] Within each cluster, adopt - Spherical neighborhood deduplication strategy: For any two text vectors If the following conditions are met:

[0020] Then determine and The text is highly semantically redundant; a large language model is used to determine the redundant text, retaining only those with higher quality scores and discarding the rest. Through the above clustering and... The spherical neighborhood deduplication strategy can effectively remove duplicate or highly similar data, thereby avoiding the bias caused by duplicate samples in model training.

[0021] S23. After completing the data filtering in S21 and the data deduplication in S22, the data preprocessing agent performs fusion processing on the retained corpus and converts it into a unified JSON file format to form a pre-training dataset, which serves as the direct input for subsequent model pre-training.

[0022] S3 includes the following steps: S31. The method for the co-citation analysis agent to screen high-quality documents based on co-citation analysis is as follows: the co-citation analysis agent, based on the metadata of journal articles obtained in S12, selects documents from its respective sub-domain's literature set. Establish a directed citation graph above, and let the indicator function... Documents References Otherwise, it is 0; define local references:

[0023] Based on this, a one-time filtering is performed to obtain the desired result. candidate set The goal is to screen literature that is strongly relevant to the target subfield and has a certain level of influence, with the best being selected. .

[0024] S32, in the candidate set Calculate the co-reference matrix :

[0025] Then normalize them to obtain similarity:

[0026] With similarity matrix For measurement, a density-based clustering method is used. For unsupervised partitioning, preferably using Several clusters were obtained ,in For similarity threshold, The minimum density parameter; in each cluster Cluster centrality of internally computed documents:

[0027] according to Sort by size from largest to smallest, select These documents constitute the core set of this cluster. The The size can be adaptively set according to the cluster size to avoid oversampling of a single topic. Finally, the union of the core sets of each cluster is calculated.

[0028] Finally, the union of the core sets of each cluster will be calculated. This serves as a refined collection of core literature, upon which high-quality corpora for each subfield are constructed to support subsequent instruction data generation and quality optimization. Preferably, the parameters can be... , .

[0029] Further, S4 comprises the following steps: S41, based on the high-quality corpus of each sub-field constructed in S3, calling the corresponding sub-field expert agent to read each literature fragment in the high-quality corpus of the sub-field and perform downstream task rewriting; The sub-field expert agent first determines whether the current literature fragment is suitable for generating instruction data; when it is determined that it is not suitable, the fragment is skipped and the next fragment is read; when it is determined that it is suitable, instruction data is generated on at least one downstream task according to the pre-set task template, and the downstream task includes fact verification, reasoning, named entity recognition, text summarization, explanation of nouns, question and answer, text classification and explainability; The instruction data is represented in the form of a "instruction-input-output" triple 〈Instruct, Input, Output〉, and saved as a JSON file according to the downstream task category.

[0030] S42, based on the instruction data saved in S41, calling a filtering agent and an optimization agent to evaluate and optimize the triple of each instruction data; the filtering agent scores the instruction data from the accuracy , completeness , relevance and practicality from four dimensions (0-10 points), and calculates the average comprehensive score:

[0031] When and meet the requirements, preferably, meet and , the instruction data is admitted to the field instruction fine-tuning data set; otherwise, the instruction data is handed over to the optimization agent for automatic repair according to the scoring opinion, and the automatic repair includes error correction, information completion and expression optimization, and the revised instruction data is returned to the filtering agent for re-evaluation; The closed-loop process of "scoring-optimization-re-evaluation" is performed at most rounds, preferably, the ; instruction data that does not meet the admission threshold within the limited rounds is determined to be irreparable and is removed; the admitted instruction data set constitutes the field instruction fine-tuning data set, which is used for subsequent instruction alignment training of the model.

[0032] Further, S5 comprises the following steps: S51, the reward modeling agent extracts the I-way subjective question from the field instruction fine-tuning data set constructed in S4, denoted as . For each subjective question , calling a large language model with human writing style characteristics and a pre-trained large language model with different performance levels to generate multi-source candidate answers; the answer quality is ranked as ; on this basis, a pair of adjacent positive and negative sample pairs is constructed, and

[0033] Thus, the reward model training data set is constructed:

[0034] S52, the rejection sampling agent filters a plurality of subjective questions not overlapping with the question set extracted by S51 based on the domain instruction fine-tuning data set constructed by S4, and the plurality of subjective questions are denoted as . For each question , a large language model trained by a pre-training data set and a domain instruction fine-tuning data set is called to generate a candidate answer set , , represents the tth candidate answer of the question , and the candidate answer set generated by each question constitutes a rejection sampling fine-tuning stage data set: .

[0035] Further, the vertical domain is one or more of the energy, medical, education and financial fields.

[0036] Beneficial effects: The application provides a vertical domain large language model training data set generation system integrating four layers of data collection, data processing, data generation and data optimization. The system relies on a multi-agent collaborative mechanism to analyze the field requirements and retrieval intentions proposed by users, automatically completes the full-process data set generation from corpus acquisition, structured processing to instruction and preference data construction, and provides consistent and traceable high-quality data for pre-training, instruction fine-tuning and human feedback reinforcement learning RLHF.

[0037] At the data collection and processing level, the application first identifies user requirements and generates a retrieval strategy using a large language model, and calls a scraping agent to batch capture open available domain literature and materials; then a structured reconstruction agent performs structure preservation analysis and reconstruction on the PDF source file to form a structured corpus containing tables, formulas and figure annotation elements. Further, a data preprocessing agent is called to implement quality and sensitive content filtering using a large language model as a classification discriminator, and combines embedding space clustering and epsilon-neighborhood determination for deduplication and fusion, thereby generating a pre-training data set for model pre-training.

[0038] ​At the data generation level, the application calls a co-citation analysis agent based on the scraped metadata, filters a core literature set highly relevant to the target subfield and with academic influence through a two-stage process of "local citation amount screening-co-citation similarity clustering-centrality sorting", and constructs a high-quality field corpus. Based on this, the subfield expert agent generates "instruction-input-output" triples in batches according to the task template, and the screening agent and optimization agent implement the quality closed loop of "scoring-optimization-revaluation" to form the field instruction fine-tuning data set for model instruction alignment training.

[0039] At the data optimization level, in order to improve the consistency of model output and human preference, the application constructs two types of data required for RLHF based on the field instruction fine-tuning data set: first, a reward modeling agent is called to generate multiple source candidate answers for a first question set extracted from the field instruction data set and construct adjacent positive and negative sample pairs with expert answers, forming a reward model training data set; second, a rejection sampling agent is called to generate candidate answers for a second question set that does not overlap with the first question set from the pre-trained and instruction fine-tuned field model, and only high-score samples are retained according to the reward model score, forming a rejection sampling fine-tuning data set. The two types of data are combined to generate a human feedback reinforcement learning data set to further optimize the preference alignment and actual availability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 A flow chart of a data set construction method for a vertical field large language model based on a multi-agent collaborative system according to an embodiment of the application; Figure 2 A vertical field large language model training principle diagram according to an embodiment of the application. DETAILED DESCRIPTION

[0041] The technical solutions of the embodiments of the application are explained and described below, but the following embodiments are only preferred embodiments of the application, not all. Based on the embodiments in the embodiments, other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0042] The application will be further described below in conjunction with the drawings and specific embodiments.

[0043] A multi-agent collaborative system for constructing a data set for training a vertical field large language model according to the application, the construction process and the vertical field large language model training principle can be referred to Figure 1 and Figure 2As shown, the system comprises a harvesting agent, a structured reconstruction agent, a data preprocessing agent, a co-citation analysis agent, a sub-field expert agent, a screening agent, an optimization agent, a reward modeling agent, and a rejection sampling agent. The system is used to implement the following steps: S1, receiving user input, calling the harvesting agent to obtain multi-source corpus according to the sub-field strategy, and constructing the original field corpus library; calling the structured reconstruction agent to parse and reconstruct the corpus, and generating the structured field corpus library.

[0044] S2, based on the structured field corpus library, calling the data preprocessing agent to perform data filtering and data deduplication, and constructing the pre-training data set.

[0045] S3, based on the structured field corpus library, calling the co-citation analysis agent to calculate the local citation quantity and perform co-citation analysis to screen out high-value literature, and constructing the high-quality corpus library.

[0046] S4, based on the high-quality corpus library, calling the field expert agent to generate instruction data in batches; calling the screening agent and the optimization agent to construct the field instruction fine-tuning data set.

[0047] S5, based on the field instruction fine-tuning data set, calling the reward modeling agent to construct the reward model training data set; calling the rejection sampling agent to form the fine-tuning data set, and merging to form the training data of human feedback reinforcement learning.

[0048] Embodiment: The embodiment provides a multi-agent collaborative system for constructing a vertical field large language model training data set, which is used to generate three types of training data sets of a wisdom energy field large language model, including a pre-training data set, a field instruction fine-tuning data set, and a human feedback reinforcement learning (RLHF) data set. Based on the data set, the pre-training, instruction fine-tuning, and RLHF three-stage training are sequentially completed on the Qwen2.5-7B base model to obtain the wisdom energy field large language model Helios-7B. In the evaluation of objective tasks and subjective tasks in the energy field, the model significantly outperforms the comparative baseline on similar parameter scale models (such as Qwen3-8B and Llama3-8B), and approaches a larger parameter scale general-purpose model in several indicators.

[0049] A multi-agent collaborative system for constructing a vertical field large language model training data set, the system comprising a harvesting agent, a structured reconstruction agent, a data preprocessing agent, a co-citation analysis agent, a sub-field expert agent, a screening agent, an optimization agent, a reward modeling agent, and a rejection sampling agent; The system is used to realize the following steps: S1, receiving wisdom energy field intention, calling the crawling intelligent agent to obtain multi-source corpus according to the sub-field strategy, and constructing the original field corpus; calling the structured reconstruction intelligent agent to parse and reconstruct the corpus, and generating the structured field corpus.

[0050] S2, based on the structured field corpus, calling the data preprocessing intelligent agent, performing data filtering and data deduplication, and constructing the pre-training data set.

[0051] S3, based on the metadata of the crawling, calling the co-citation analysis intelligent agent, calculating the local cited quantity and performing co-citation analysis to filter out high-value literature, and constructing the high-quality corpus.

[0052] S4, based on the high-quality corpus, calling the field expert intelligent agent to generate instruction data in batches; calling the screening intelligent agent and the optimization intelligent agent to construct the field instruction fine-tuning data set.

[0053] S5, based on the field instruction fine-tuning data set, calling the reward modeling intelligent agent to construct the reward model training data set; calling the rejection sampling intelligent agent to form the fine-tuning data set, and merging to form the human feedback reinforcement learning data set.

[0054] In the above technical solution, further, S1 includes the following steps: S11, the crawling intelligent agent accepts the input wisdom energy field intention, calls GPT-4o to analyze and expand the intention, and obtains 14 sub-fields (energy management system, clean energy, cogeneration, combined heat and power, distributed energy, energy hub, energy optimization, energy storage, energy transformation, integrated energy, load forecasting, low-carbon energy, smart energy, and smart grid), and uses the 14 sub-fields as keywords as the retrieval strategy.

[0055] S12, based on the retrieval strategy, the crawling intelligent agent obtains open source academic preprints, academic journals and monograph data from arXiv, WoS and ACEmap respectively, and stores them according to the source and sub-field. Through the above steps, 173,541 preprints, 32,549 open source journal papers and their metadata, and 363 monographs are obtained, which are stored in PDF format to form the original field corpus.

[0056] S13, the structured reconstruction intelligent agent receives the original field corpus, calls the built-in PDF parsing module (mainly composed of Python components, see Table 1). The PDF documents in the original field corpus are processed page by page. First, perform layout segmentation, identify paragraphs, chapter titles, tables, mathematical formulas, image placeholders, footnotes and reference elements, and reconstruct the reading order according to the layout rules, so as to eliminate multi-column layout, header and footer and page break interference.

[0057] Table 1 PDF parsing module details

[0058] S14, based on S13, after completing the understanding of the layout, the structured reconstruction agent performs semantic-level structured conversion on the parsed elements of each type, and stores them in Markdown form. Specifically, it includes: cell gridization and cross-row and cross-column parsing of tables, reconstruction into editable table text; symbol recognition and standardized conversion of mathematical formulas, unified into standard LaTex form expression; bidirectional association between image placeholders, tables, and footnotes; extraction and field standardization of reference entries; and automatic generation of table of contents outline according to the hierarchical relationship of chapter titles.

[0059] S15, based on 14, after the structure conversion, the structured reconstruction agent performs structure consistency test on the output results. Preferably, the structure consistency score is defined as:

[0060] wherein, represents the table cell reconstruction accuracy, represents the formula parsing rate, represents the image placeholder, table, and footnote association completeness, represents the chapter level matching degree. When the structure consistency score is lower than the threshold value 0.75, the structured reconstruction agent calls a large language model to perform structure repair, including table alignment correction, formula symbol and logic consistency repair, figure numbering and cross-reference completion, etc. If it still fails to reach the preset threshold value after three consecutive repairs, the document is marked as an abnormal sample and removed from the original corpus. Finally, the documents that pass the consistency test are included in the structured field corpus, a total of 153,165 preprints, 30,249 open source journal papers, and 342 monographs, which serve as the basis for subsequent pre-training data set construction.

[0061] S2 includes the following steps: S21, based on the structured field corpus obtained in S1, the data preprocessing agent first performs data filtering on the corpus, which includes quality filtering and sensitive content filtering. GPT-4o mini is called as a text classification discriminator to read the content of the structured corpus in sequence, identify and remove low-quality texts with format damage, symbol garbled code, content defects or logical discontinuity, and detect and remove private information and harmful content, so as to ensure that the data entering the subsequent processing link has high integrity and compliance.

[0062] S22, based on S21 data filtering, the data preprocessing agent calls an efficient deduplication module based on Bert-base construction, which clusters large-scale text in vector space. Each piece of text Input BERT-base model to get vector representation In the vector space , K-Means algorithm is used for clustering, and the objective function is defined as:

[0063] Wherein, is the th cluster, is the centroid of the cluster.

[0064] Within each cluster, a - ball neighborhood deduplication strategy is used, where is the similarity threshold, and is selected as 0.3: for any two text vectors , if it satisfies:

[0065] then it is determined that and are highly redundant in semantics. Call GPT-4o mini to determine redundant text, and only keep the one with higher quality score, and the rest are deleted. Through the above clustering and - ball neighborhood deduplication method, repeated or highly similar corpus can be effectively removed, thereby avoiding the impact of repeated samples on model training performance.

[0066] S23, after completing S21 data filtering and S22 data deduplication, the data preprocessing agent fuses the retained corpus and uniformly converts it into a JSON file format, forming a pre-training corpus as a direct input for subsequent model pre-training. The scale statistics are: about 2.315B Tokens of preprints, about 0.57B Tokens of open source journal papers, and about 0.038B Tokens of monographs. Based on this data, pre-training is carried out on the Qwen2.5-7B base model to obtain a pre-trained model in the field of smart energy.

[0067] S3 includes the following steps: S31, the co-citation analysis agent establishes a directed citation graph on the literature set in each sub-field based on the journal paper metadata obtained in S1, and defines the indicator function , where indicates that the literature cites the literature , otherwise it is 0; define the local citation amount:

[0068] and a one-off filtering is performed to obtain a candidate set satisfying to filter out literatures strongly related to the target sub-field and with certain influence. Among them can be adaptively set according to the size of the sub-field, and here it is taken as 1.

[0069] S32, a co-citation matrix is calculated on the candidate set :

[0070] and normalized to obtain the similarity:

[0071] With the similarity matrix as a measure, a density-based clustering method is used to perform unsupervised division on , preferably using to obtain a plurality of clusters , wherein is a similarity threshold, is a minimum density parameter; the cluster centrality of the literatures is calculated in each cluster :

[0072] According to from large to small, the core set of literatures of the cluster is selected ; the , and can be adaptively set according to the size of the cluster to avoid over-sampling of a single topic. Finally, the union of the core sets of each cluster:

[0073] The number of high-quality corpus libraries of each sub-field finally constructed is shown in Table 2, which is used to support subsequent instruction data generation and quality optimization.

[0074] Table 2 Statistics of literatures after refinement of each sub-field

[0075] S4 includes the following steps: ​S41, based on the sub-field high-quality corpus constructed in S3, the sub-field expert agent (constructed based on GPT o1 combined with prompt word engineering fine-tuning) is called to read and rewrite each piece of literature fragment into downstream tasks. The sub-field expert agent first determines whether the current fragment is suitable for instruction data generation; when it is determined that it is not suitable, the fragment is skipped and the next fragment is read; when it is determined that it is suitable, instruction data is generated on at least one downstream task according to the pre-set task template, such as objective task (fact verification), subjective task (reasoning, named entity recognition, text summary, noun explanation, question and answer, text classification and explainability), the instruction data is represented as a triple 〈Instruct, Input, Output〉, and saved as a JSON file according to the task category.

[0076] S42, based on the instruction data saved in S41, a filtering agent and an optimization agent are called to evaluate and optimize the quality of each triple. The filtering agent (GPT 4o mini) scores the sample from the four dimensions of accuracy , completeness , relevance and practicality (0-10 points), and calculates the average comprehensive score:

[0077] When and are met, the sample is admitted to the training set; otherwise, the sample is handed over to the optimization agent (GPT o1) to implement automatic repair according to the scoring opinions, including but not limited to error correction, information completion and expression optimization, and the revised sample is returned to the filtering agent for re-evaluation. The closed-loop process of "scoring-optimization-re-evaluation" is performed at most rounds; samples that still do not meet the admission threshold within the limited rounds are determined to be irreparable and are removed. The admitted sample set constitutes the field instruction fine-tuning dataset, and the number of instruction fine-tuning datasets obtained is shown in Table 3. Based on the instruction fine-tuning dataset, the pre-trained model in the field of intelligent energy obtained in S2 is trained using the instruction fine-tuning dataset to obtain a fine-tuned large language model in the field of intelligent energy.

[0078] S5 includes the following steps: S51, the reward modeling agent extracts 5,000 subjective questions from the field instruction fine-tuning dataset constructed in S42, denoted as . For each question , a large language model with human writing style characteristics and pre-trained large language models of different performance levels are called to generate multiple source candidate answers, and the answer quality is ranked as On this basis, adjacent two pairs of positive and negative sample pairs are constructed, and the following is obtained:

[0079] The reward model training data set is constructed as follows:

[0080] S52, the rejection sampling intelligent agent filters 10,000 subjective questions that do not overlap with the question set extracted in S51 based on the domain instruction fine-tuning data set constructed in S42, and records them as For each question , a large language model trained by a pre-training data set and a domain instruction fine-tuning data set is called to generate five candidate answers: The above candidate answers constitute the rejection sampling fine-tuning phase data set:

[0081] Based on the human feedback reinforcement learning data set, human feedback reinforcement learning training (including two stages of reward model training and rejection sampling fine-tuning) is carried out, so that the large language model in the field of smart energy is aligned with human preferences. Through the above multi-agent collaborative process, pre-training, instruction fine-tuning, and human feedback reinforcement learning three-stage training data are obtained, and the large language model in the field of smart energy is trained based on the training data.

[0082] Model performance is one of the important indicators reflecting data quality. In the field of objective task evaluation (single choice, multiple choice, fact verification, and questions from publicly available smart energy related test questions), the trained large language model achieved an accuracy of 79.09%, significantly exceeding the similar parameter size models Qwen3-8B-Instruct (41.87%) and Llama3-8B-Instruct (54.93%), and the performance level was close to that of GPT-4 (83.57%) with a trillion parameter size. In the field of subjective tasks (modeling, explanation, and question answering, and questions from field-specific evaluation sets), human experts and large language models jointly score, and the trained large language model still significantly outperforms similar parameter size models and approaches the level of GPT-4 with a trillion parameter size.

[0083] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Those skilled in the art should understand that the present invention includes but is not limited to the content described in the above specific embodiments. Any modification that does not deviate from the functional and structural principles of the present invention shall be included in the scope of the claims.

Claims

1. A multi-agent collaborative system for constructing training datasets for large language models in vertical domains, characterized in that, The system includes: a data acquisition agent, a structured reconstruction agent, a data preprocessing agent, a co-citation analysis agent, a subdomain expert agent, a screening agent, an optimization agent, a reward modeling agent, and a rejection sampling agent; The system is used to implement the following steps: S1. Receive user input, call the data acquisition agent to acquire multi-source corpora according to the sub-domain strategy, and construct the original domain corpus; call the structured reconstruction agent to parse and reconstruct the corpora in the original domain corpus to generate a structured domain corpus; S2. Based on a structured domain corpus, call the data preprocessing agent to perform data filtering and deduplication, and build a pre-training dataset; S3. Based on a structured domain corpus, call the co-citation analysis agent to calculate local citations and perform co-citation analysis to screen out high-value documents and build a high-quality corpus for each subdomain. S4. Based on high-quality corpora of each subdomain, call the corresponding expert agents in the subdomain to generate instruction data in batches; call the filtering agent and the optimization agent to build a domain instruction fine-tuning dataset. S5. Fine-tune the dataset based on domain instructions, call the reward modeling agent to construct the reward model training dataset; call the rejection sampling agent to form the fine-tuning dataset, and merge them to form the human feedback reinforcement learning dataset; the pre-training dataset, the pre-training dataset and the human feedback reinforcement learning dataset constitute the training dataset for the vertical domain large language model.

2. The multi-agent collaborative system for constructing a training dataset for a large language model in a vertical domain, as described in claim 1, is characterized in that... S1 specifically includes the following steps: S11. The intelligent agent receives the set of keywords and the target domain intent input by the user, calls the large language model to parse and expand the target domain intent, obtains several sub-domains, combines the prior knowledge of the large language model and the domain ontology, expands and refines the set of keywords, generates a subdivided domain keyword table, and generates corresponding retrieval combinations and crawling strategies. S12. Based on the retrieval combination and crawling strategy obtained in S11, the crawling agent takes PDF format files as the collection object, crawls publicly available academic papers from academic preprint databases, crawls publicly available full texts of journal articles and their metadata from journal databases, and crawls publicly available monographs and reports from professional knowledge platforms, and stores them in partitions according to source and sub-domain to form an original domain corpus. S13, the structured reconstruction agent parses and reconstructs each text in the original domain corpus, including: the structured reconstruction agent calls the built-in PDF parsing module to process the text in the original domain corpus page by page, performs page segmentation, identifies paragraph, chapter title, table, mathematical formula, image placeholder, footnote and reference elements, and reconstructs the natural reading order according to the typesetting rules; S14. The structured reconstruction agent performs semantic-level structured transformation on the parsed elements, specifically: it performs cell gridding and cross-row / column parsing on the table to reconstruct editable table text; it performs symbol recognition and standardization on the mathematical formulas to unify them into standardized formula expression syntax; it establishes bidirectional associations between the image placeholders, tables, and footnotes; it extracts and standardizes the fields of the reference entries; and it automatically generates a table of contents outline based on the hierarchical relationship of the chapter titles; the structured reconstruction agent converts the parsing results into an intermediate representation and stores it. S15. The structured reconstruction agent performs a structure consistency check on the intermediate representation output in S14 using a structure consistency score, whereby the structure consistency score is: ； in, This indicates the accuracy of table cell reconstruction. This represents the analyticity of a mathematical formula. This indicates the completeness of the image placeholder and table, respectively, in relation to the footnotes. Indicates the degree of matching between chapter levels; when the structural consistency score is... When the value is below a preset threshold, the structured reconstruction agent calls the large language model to perform structure repair. The structure repair includes table alignment correction, formula symbol and logic consistency repair, and chart numbering and cross-reference completion. If the preset threshold is still not reached after n consecutive repairs, the corpus is marked as an abnormal sample and removed from the original domain corpus. Finally, the corpus that passes the consistency test is included in the structured domain corpus.

3. The multi-agent collaborative system for constructing a training dataset for a large language model in a vertical domain, as described in claim 1, is characterized in that... S2 specifically includes the following steps: S21. Based on the structured domain corpus, the data preprocessing agent first performs data filtering on the corpus in the structured domain corpus. The data filtering includes quality filtering and sensitive content filtering. Specifically, it calls a large language model as a text classification discriminator, reads the content of the structured domain corpus in sequence, identifies and removes low-quality text with format corruption, garbled symbols, incomplete content or logical incoherence, and detects and removes content containing privacy information or that does not meet compliance requirements. S22. Based on the structured domain corpus filtered by S21, the data preprocessing agent calls an efficient deduplication module built on Bert-base to cluster and deduplicat the text of the corpus in vector space; S23. After completing the data filtering in S21 and the data deduplication in S22, the data preprocessing agent performs fusion processing on the retained corpus and converts it into a unified JSON file format to form a pre-training dataset.

4. The multi-agent collaborative system for constructing a training dataset for a large language model in a vertical domain, as described in claim 3, is characterized in that... Specifically, S22 is: Each text Inputting the BERT-base model yields a vector representation. In vector space The K-Means algorithm is used for clustering, and the objective function is defined as: ； in, For the first One cluster, Let be the centroid of this cluster; Within each cluster, adopt - Spherical neighborhood deduplication strategy: For any two text vectors If the following conditions are met: ； Then determine and The text is highly semantically redundant. A large language model is used to judge the redundant text, and only those with higher quality scores are retained, while the rest are discarded.

5. The multi-agent collaborative system for constructing a training dataset for a large language model in a vertical domain, as described in claim 2, is characterized in that... S3 specifically includes the following steps: S31. The co-citation analysis agent, based on the metadata of journal articles obtained in S12, analyzes the literature sets in their respective sub-fields. Establish a directed citation graph above, and let the indicator function... Documents References Otherwise, it is 0; define local references: ； Based on this, a one-time filtering is performed to obtain the desired result. candidate set ; S32, in the candidate set Calculate the co-reference matrix : ； Then normalize them to obtain similarity: ； With similarity matrix For measurement, a density-based clustering method is used. Unsupervised partitioning is performed, specifically by using... Several clusters were obtained ,in For similarity threshold, The minimum density parameter; In each cluster Cluster centrality of internally computed documents : ； Based on cluster centrality Sort by size from largest to smallest, select These documents constitute the core set of this cluster. ; Core sets of each cluster Union of: ； Finally, the core sets of each cluster will be determined. union As a refined collection of core literature, we construct high-quality corpora for each subfield.

6. The multi-agent collaborative system for constructing a training dataset for a large language model in a vertical domain, as described in claim 1, is characterized in that... S4 specifically includes the following steps: S41. Based on the high-quality corpus of each subdomain constructed by S3, call the corresponding expert intelligent agent of the subdomain to read the document fragments in the high-quality corpus of the subdomain one by one and perform downstream task-oriented rewriting. The subdomain expert agent first determines whether the current document fragment is suitable for generating instruction data; if it is determined to be unsuitable, it skips the fragment and reads the next fragment; if it is determined to be suitable, it generates instruction data on at least one downstream task according to a preset task template. The downstream tasks include fact verification, reasoning, named entity recognition, text summarization, terminology explanation, question answering, text classification, and interpretability description. The instruction data is represented by a "instruction-input-output" triple and saved as a JSON file according to the downstream task category. S42. Based on the instruction data saved in S41, the screening agent and the optimization agent are invoked to perform quality evaluation and closed-loop optimization on the triples of each instruction data; the screening agent focuses on accuracy. Completeness Correlation and practicality The instruction data is scored across four dimensions, and an average comprehensive score is calculated: ； when , , , and When all scores meet the required criteria, the instruction data is admitted to the domain instruction fine-tuning dataset; otherwise, the instruction data is transferred to the optimization agent for automatic repair based on the scoring opinions. This automatic repair includes error correction, information completion, and expression optimization. The revised instruction data is then returned to the screening agent for re-evaluation. The aforementioned closed-loop process of "scoring—optimization—re-evaluation" is performed at most once. Rounds; instruction data that fails to reach the admission threshold within a limited number of rounds is deemed irreparable and removed; all admitted instruction data constitutes the domain instruction fine-tuning dataset.

7. The multi-agent collaborative system for constructing a training dataset for a large language model in a vertical domain, as described in claim 1, is characterized in that... S5 specifically includes the following steps: S51. The reward modeling agent fine-tunes the dataset based on the domain instructions, extracting I subjective questions from it, denoted as... For each subjective question The system invokes a large language model with human writing style characteristics and pre-trained large language models of different performance levels to generate multi-source candidate answers. These candidate answers are then ranked according to the performance of the large language model, and pairwise positive and negative sample pairs are constructed to obtain the subjective question. Multi-source candidate answer set All subjective issues Construct the training dataset for the reward model; S52. The rejection sampling agent fine-tunes the dataset based on the domain instructions, and filters out Z subjective questions that do not overlap with the question set extracted in S51, denoted as... For each problem The large language model, trained on a pre-trained dataset and a domain instruction fine-tuning dataset, is used to generate a set of candidate answers. Each of the above issues The generated candidate answer set Together they constitute the dataset for the rejection sampling fine-tuning stage: 。 8. The multi-agent collaborative system for constructing a training dataset for a large language model in a vertical domain, as described in claim 7, is characterized in that... In S51, a large language model with human writing style characteristics and pre-trained large language models GPT-o3, GPT-4, and GPT-3.5 were specifically selected to generate multi-source candidate answers, respectively. , , and The quality of the answers is ranked as follows: Based on this, pairs of positive and negative samples are constructed to obtain the subjective question. Multi-source candidate answer set : ； Therefore, a training dataset for the reward model is constructed. : .

9. The multi-agent collaborative system for constructing a training dataset for a large language model in a vertical domain, as described in claim 1, is characterized in that... The vertical sectors are one or more of the energy, healthcare, education, and finance sectors.

Images