Dataset construction method and device for large language model, and medium

Abstract

The present application relates to the technical field of data processing. Disclosed are a dataset construction method and device for a large language model, and a medium. The method comprises: acquiring professional file data in a specified professional field, so as to determine large-language-model dataset information on the basis of the professional file data, wherein a target task type comprises professional knowledge question-answering and specific case analysis; by means of a preset hierarchical progressive construction strategy, constructing question-answer pairs for the professional file data, so as to generate a knowledge question-answering-type data subset corresponding to the professional file data; analyzing the professional file data, and determining analysis case information corresponding to the professional file data, so as to construct a case-analysis-type data subset on the basis of the probability distribution characteristics and the analysis case information; and on the basis of the large-language-model dataset information, the knowledge question-answering-type data subset and the case-analysis-type data subset, constructing a dedicated large-language-model dataset corresponding to the specified professional field.

Description

A dataset construction method, device and medium for large language models

[0001] Cross-reference to Related Applications

[0002] This application claims priority to the Chinese patent application No. 202410804921.7, filed on June 21, 2024, and entitled "A dataset construction method, device and medium for large language models", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the technical field of data processing, and in particular to a dataset construction method, device and medium for large language models. BACKGROUND

[0004] In diversified fields such as medical treatment, education, transportation, finance, media and government affairs, artificial intelligence plays a decisive role. The generative artificial intelligence represented by the Transformer architecture, i.e., the large language model, has shown great application value in complex tasks such as natural language processing, information retrieval and decision support. Data, algorithms and computing power are the three cores of large language models, and high-quality corpus data is the key to improving the performance of large language models. In order to improve the application value of large language models in professional fields, it is necessary to construct high-quality professional datasets.

[0005] Currently, document materials and knowledge graphs are the main sources for constructing professional datasets. Among them, it is necessary to have a certain threshold to extract information from knowledge graphs to construct professional datasets, and it is required to have specific knowledge graph support in the professional field. Therefore, extracting information from document materials to construct professional datasets is the main way. At present, information is mainly extracted from document materials to construct datasets by manual methods and large language models. Artificially constructing datasets is time-consuming and laborious and has extremely low efficiency, especially when dealing with large-scale documents. The large language model assisted construction of datasets has problems such as incomplete information, narrow information, and weak information hierarchy. Incomplete information may lead to the lack of key information in the dataset, narrow information may limit the universality and diversity of the dataset, and weak information hierarchy may affect the depth and quality of the dataset. Therefore, the current dataset construction is limited by the construction method, and the construction efficiency is low, and there are problems such as incomplete information, narrow information, and weak information hierarchy in the dataset, which leads to the fact that the large language model cannot meet the professional needs of the professional field. SUMMARY

[0006] Some embodiments of the present application provide a dataset construction method, device and medium for a large language model, to solve the technical problem that the current dataset construction is limited by the construction method, has low construction efficiency, and has problems such as incomplete information, narrow information, and weak information hierarchy in the dataset, which leads to the large language model being unable to meet the professional needs of the professional field.

[0007] Some embodiments of the present application adopt the following technical solutions:

[0008] Some embodiments of the present application provide a dataset construction method for a large language model, which includes: obtaining professional file data in a specified professional field, and determining large language model dataset information based on the professional file data, wherein the large language model dataset information includes a plurality of dataset target task types and a target task proportion of the plurality of dataset target task types, and the target task types include professional knowledge question answering and specific case analysis; constructing question and answer pairs for the professional file data through a pre-set hierarchical progressive construction strategy to generate a knowledge question and answer data subset corresponding to the professional file data; analyzing the professional file data to determine analysis case information corresponding to the professional file data, and constructing a case analysis data subset according to the probability distribution characteristics and the analysis case information, wherein the analysis case information includes variable type information corresponding to the analysis case; and constructing a large language model special dataset corresponding to the specified professional field based on the large language model dataset information, the knowledge question and answer data subset, and the case analysis data subset.

[0009] In some embodiments, the professional file data is constructed into question and answer pairs through a pre-set hierarchical progressive construction strategy to generate a knowledge question and answer data subset corresponding to the professional file data, specifically including: obtaining large language model input restriction information and natural delimiter information in the professional file data; processing the professional file data according to the large language model input restriction information and the natural delimiter information in the professional file data to generate a plurality of segments; taking each segment as an input of the large language model, generating a segment question corresponding to each segment through a pre-set prompt word, expanding the segment question through synonym replacement to generate a plurality of expanded segment questions; generating a segment answer corresponding to each expanded segment question through the prompt word engineering, and constructing a segment question and answer pair; and generating a knowledge question and answer data subset corresponding to the professional file data based on the plurality of segment question and answer pairs.

[0010] In some embodiments, the professional file data is processed according to the large language model input restriction information and the natural separator information in the professional file data to generate a plurality of segments, specifically including: segmenting the professional file data according to the natural separator information in the professional file data to determine a plurality of first segments; determining a first segment order corresponding to the plurality of first segments through the order of the natural separator information in the professional file data; according to the large language model input restriction information, the first segment order is used to sequentially accumulate the segment word number information of the first segment to obtain accumulated accumulated segment word number information; and the plurality of specified first segments are merged through the accumulated segment word number information and the large language model input restriction information to determine a plurality of second segments.

[0011] In some embodiments, the case analysis data subset is constructed according to the probability distribution characteristics and the analysis case information, specifically including: obtaining the variable type information of each variable corresponding to the analysis case in the analysis case information; constructing at least one case analysis question and answer pair corresponding to each variable type according to a predetermined uniform distribution rule; determining the total number of historical case variables corresponding to the analysis case and the number of historical case variable occurrences corresponding to each variable type, and determining the occurrence probability of each variable type through the number of historical case variable occurrences and the total number of historical case variables; according to the occurrence probability of each variable type, each case analysis question and answer pair is expanded according to the normal distribution principle to construct the case analysis data subset.

[0012] In some embodiments, at least one case analysis question and answer pair corresponding to each variable type is constructed according to a predetermined uniform distribution rule, specifically including: designing a question for each variable type to generate at least one case analysis question corresponding to each variable type; determining the case analysis answer corresponding to each case analysis question through the large language model according to the professional file data; and based on the at least one case analysis question corresponding to each variable type and the case analysis answer corresponding to each case analysis question, at least one case analysis question and answer pair corresponding to each variable type is constructed.

[0013] In some embodiments, the large language model dataset information is determined based on the professional file data, specifically including: obtaining historical large language model input and output information corresponding to a specified professional field of the professional file data; identifying the historical large language model input and output information to determine a plurality of historical task types to determine a plurality of dataset target task types; counting the number of tasks of each historical task type, and determining the target task proportion of the plurality of dataset target task types according to the ratio between the number of tasks of the plurality of historical task types.

[0014] In some embodiments, based on the large language model dataset information, the large language model special dataset corresponding to the specified professional field is constructed according to the knowledge question and answer type data subset and the case analysis type data subset, specifically including: obtaining the target task proportion of the target task type of the plurality of datasets in the large language model dataset information; respectively determining the number of knowledge question and answer pairs in the knowledge question and answer type data subset and the number of case analysis type question and answer pairs; according to the number of knowledge question and answer pairs and the number of case analysis type question and answer pairs, it is judged whether the knowledge question and answer type data subset and the case analysis type data subset meet the target task proportion; when the knowledge question and answer type data subset and the case analysis type data subset do not meet the target task proportion, the case analysis type data subset is expanded through the target task proportion to obtain an expanded case analysis type data subset, so as to construct the large language model special dataset corresponding to the specified professional field according to the knowledge question and answer type data subset and the expanded case analysis type data subset according to the target task proportion.

[0015] In some embodiments, after the large language model special dataset corresponding to the specified professional field is constructed, the method further includes: obtaining the large language model general dataset and determining the current data amount in the large language model special dataset; determining the general data amount to be mixed according to the pre-set mixing ratio and the current data amount, wherein the mixing ratio is used to represent the mixing ratio of the large language model special dataset and the general dataset; through the pre-set script code, data extraction is performed in the large language model general dataset according to the general data amount to be mixed to determine the general dataset to be mixed; the general dataset to be mixed is mixed into the large language model special dataset to construct the large language model dataset.

[0016] Some embodiments of the present application provide a data set construction device for a large language model, comprising:

[0017] at least one processor; and,

[0018] a memory in communication connection with the at least one processor; wherein,

[0019] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0020] Some embodiments of the present application provide a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are configured to: acquire professional file data in a specified professional field, and determine large language model data set information based on the professional file data, wherein the large language model data set information includes a plurality of data set target task types and target task proportions of the plurality of data set target task types, and the target task types include professional knowledge question answering and specific case analysis; perform question and answer pair construction on the professional file data through a pre-set hierarchical progressive construction strategy to generate a knowledge question and answer type data subset corresponding to the professional file data; analyze the professional file data to determine analysis case information corresponding to the professional file data, and construct a case analysis type data subset according to probability distribution characteristics and the analysis case information, wherein the analysis case information includes variable type information corresponding to the analysis case; and construct a large language model special data set corresponding to the specified professional field based on the large language model data set information, the knowledge question and answer type data subset, and the case analysis type data subset.

[0021] The above at least one technical solution adopted by some embodiments of the present application can achieve the following beneficial effects: According to the above technical solution, the knowledge question and answer type data subset and the case analysis type data subset constructed by directly acquiring data from professional file data in a specified professional field will directly reflect the professional knowledge and application scenarios of the field, so that the large language model special data set is closer to actual needs and business scenarios, thereby improving the pertinence and professionalism of the model; The plurality of data set target task types and the corresponding target task proportions included in the large language model data set information can ensure that the data set contains both general professional knowledge questions and answers and specific case analysis, and this mixed data set helps the model to learn more diverse features and patterns during the training process, thereby improving the generalization ability and adaptability of the model; The knowledge question and answer type data subset generated through the hierarchical progressive construction strategy can ensure that the quality and quantity of the question and answer pairs meet the needs of model training, and can improve the hierarchical sense of professional field data set construction and the ability of the large model to answer micro and macro problems in the professional field; The case analysis type data subset constructed based on the probability distribution characteristics and the analysis case information not only improves the breadth of professional field data set construction, but also improves the depth, further improving the comprehensiveness and accuracy of the large model in the professional field; Using the pre-set hierarchical progressive construction strategy and the probability distribution characteristics of the analysis case information can automatically or semi-automatically construct the data set, reduce manual intervention and errors, and improve the quality and construction efficiency of the data set; By constructing the specific case analysis type data subset, the model can combine specific case information when generating predictions or suggestions to provide more detailed and accurate explanations, thereby enhancing the interpretability and credibility of the model, making the results of the model more easily accepted and understood by users. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to make the technical solutions of some embodiments of the present application or the prior art clearer, the drawings needed to be used in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art without creative labor. In the drawings:

[0023] FIG. 1 is a flowchart of a method for constructing a data set for a large language model according to some embodiments of the present application;

[0024] FIG. 2 is a flowchart of constructing a knowledge question and answer data subset according to some embodiments of the present application;

[0025] FIG. 3 is a flowchart of constructing a case analysis data subset according to some embodiments of the present application;

[0026] FIG. 4 is a normal distribution diagram according to some embodiments of the present application;

[0027] FIG. 5 is a structural diagram of a data set construction device for a large language model according to some embodiments of the present application. DETAILED DESCRIPTION

[0028] In order to make the technical solutions of some embodiments of the present application or the prior art clearer, the drawings needed to be used in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art without creative labor. In the drawings:

[0029] Some embodiments of the present application provide a method for constructing a data set for a large language model. It should be noted that the execution subject in the embodiments of the present application can be a server or any device with data processing capability. FIG. 1 is a flowchart of a method for constructing a data set for a large language model according to some embodiments of the present application, as shown in FIG. 1, which mainly includes the following steps:

[0030] Step S101, obtaining professional file data in a specified professional field, to determine large language model data set information based on the professional file data.

[0031] The large language model data set information includes a plurality of data set target task types and a target task proportion of the plurality of data set target task types, and the target task type includes professional knowledge question and answer and specific case analysis.

[0032] In some embodiments of the present application, professional file data in a specified professional field is obtained, where the specified professional field refers to the professional field corresponding to the business of the large language model, and the professional file data can be a professional PDF (Portable Document Format) file, such as an air conditioner maintenance manual PDF file or an enterprise reimbursement system PDF file.

[0033] In some embodiments, based on the professional file data, large language model dataset information is determined, specifically including: obtaining historical large language model input and output information corresponding to the specified professional field of the professional file data; identifying the historical large language model input and output information to determine a plurality of historical task types, to determine the plurality of dataset target task types; counting the number of tasks for each historical task type, and determining the target task proportion of the plurality of dataset target task types according to the ratio between the number of tasks of the plurality of historical task types.

[0034] In some embodiments of the present application, the professional file data is analyzed, and the target task proportion is determined by analyzing the professional PDF file. Generally, a professional field includes two types of tasks, namely professional knowledge question and answer type and specific case analysis type. For example, a special dataset is constructed from an air conditioner maintenance manual PDF file of a certain model. The professional knowledge question and answer type dataset can be constructed for the professional knowledge clearly visible in the manual, while the case analysis type dataset needs to be constructed for logical reasoning and analysis of air conditioner component damage and expected life.

[0035] In the analysis of professional file data, the historical question and answer information of the large language model using the professional file data, that is, the historical input and output information of the large language model, can be referred to. In the system for storing the historical data of the large language model, the historical file data corresponding to the professional file data in the specified professional field is retrieved and obtained. The input (user question or instruction) and output (model generated answer or execution result) of the large language model are extracted from the historical file data. According to the characteristics of the professional field, a series of possible task types are defined in advance, such as knowledge question and answer type and case analysis type. The extracted input and output information is identified using natural language processing (NLP) techniques such as text classification and named entity recognition to determine the task type it belongs to. Through multiple historical task types, the target task type of the data set corresponding to the professional file data is determined. The number of occurrences of each historical task type in the total number of tasks is counted, and the ratio between them is calculated according to the number of occurrences of each task type, that is, the target task ratio of multiple data set target tasks is determined. Generally, the ratio of knowledge question and answer type target task to case analysis type target task is generally between 1:1 and 1:5. By determining the target task ratio, the data can be more targetedly collected, sorted and labeled, improving the quality and efficiency of the data; adjusting the distribution of training data according to the target task ratio can make the model pay more attention to important task types during training, thereby improving the performance and generalization ability of the model; as the professional field develops and changes, the target task ratio can also be adjusted accordingly, making data collection and model training more consistent with the development trend of the field; by confirming the target task type corresponding to the professional file data through the historical question and answer information of the large language model using the professional file data, the matching of the professional file data and the task type is ensured; in addition, by determining the target task ratio through the ratio between the number of historical tasks, the corresponding data set ratio can be determined according to the target task ratio, avoiding subjective interference in the process of manually setting the ratio and ensuring the accuracy of the data set ratio of each task type.

[0036] In step S102, the professional file data is constructed into a question and answer pair through a pre-set hierarchical progressive construction strategy to generate a knowledge question and answer type data subset corresponding to the professional file data.

[0037] In some embodiments, the professional file data is constructed into a question and answer pair through a pre-set hierarchical progressive construction strategy to generate a knowledge question and answer type data subset corresponding to the professional file data, specifically including: obtaining large language model input restriction information and natural delimiter information in the professional file data; processing the professional file data according to the large language model input restriction information and the natural delimiter information in the professional file data to generate a plurality of segments; taking each segment as an input of a large language model, generating a segment question corresponding to each segment through a pre-set prompt word, expanding the segment question through synonym replacement to generate a plurality of expanded segment questions; through prompt word engineering, generating a segment answer corresponding to each expanded segment question and constructing a segment question and answer pair from the plurality of expanded segment questions and the professional file data; and generating a knowledge question and answer type data subset corresponding to the professional file data based on a plurality of segment question and answer pairs.

[0038] In some embodiments, the professional file data is processed according to the large language model input restriction information and the natural delimiter information in the professional file data to generate a plurality of segments, specifically including: segmenting the professional file data according to the natural delimiter information in the professional file data to determine a plurality of first segments; determining a first segment order corresponding to the plurality of first segments through the order of the natural delimiter information in the professional file data; according to the large language model input restriction information, sequentially adding segment word number information of the first segment according to the first segment order to obtain accumulated accumulated segment word number information; and merging a plurality of specified first segments through the accumulated segment word number information and the large language model input restriction information to determine a plurality of second segments.

[0039] In some embodiments of the present application, FIG. 2 is a construction process schematic diagram of a knowledge question and answer type data subset provided by some embodiments of the present application, as shown in FIG. 2, a progressive strategy is used to generate question and answer pairs with a hierarchical sense, sequentially containing information from small to large, and the process includes segmenting a PDF file according to word number and content relevance setting rules, a large language model generating sentence small paragraph level questions according to a PDF file, a large language model generating sentence small paragraph level question and answer pairs and auditing question and answer pairs, a large language model generating large paragraph level questions according to a PDF file, and a large language model generating large paragraph level question and answer pairs and auditing question and answer pairs.

[0040] In some embodiments of the present application, professional file data is segmented according to word count and content relevance to meet the input word limit requirements of large language models. First, the input limit information of the large language model is obtained, which can be the input word size limit. The input word size of the large language model is generally about 32K, and the set limit word size is about 14K. Natural delimiters in the professional file data are obtained, such as titles and other natural delimiters. The PDF file is divided into many small segments, i.e., the first segment. The first segment order corresponding to the plurality of first segments is determined by the order of the natural delimiter information in the professional file data. Then, from the first small segment, the total word count is accumulated step by step until the total word count exceeds 14K. The small segments with a total word count less than 14K are combined into large segments, which are the second segments. Then, the small segments exceeding 14K are combined as new first small segments to continue to accumulate with the following small segments. The operation is repeated until the professional PDF file is divided into many large segments with content correlation and a total word count not exceeding 14K, i.e., a plurality of second segments are obtained.

[0041] In some embodiments of the present application, the large language model generates the first segment corresponding to the level problem according to the PDF file through the design of the prompt word. The prompt word is "generate a question for each sentence or small paragraph, and the question should be as many as possible". The large language model understands the input PDF file information and then extracts the question. Each sentence or small paragraph extracts a question. Then, the large language model expands the question by 2-3 times according to the synonym replacement. For example, the original PDF file has the following content: "The reimbursing person should not initiate a new reimbursement process if the paper reimbursement document is not signed and delivered within 10 days from the date of completion of the audit". The question generated by the large language model is "What are the consequences of not processing paper reimbursement documents in a timely manner?". Then, the large language model performs synonym replacement and expands the new question: "When should the reimbursing person sign and deliver paper documents in the reimbursement process?". The large language model generates sentence and small paragraph level question and answer pairs and audits the question and answer pairs. The generated sentence and small paragraph level questions and professional PDF file content are input into the large language model through the prompt word engineering. The large language model generates answers according to the professional PDF file, and then the question and answer constitute a plurality of first segment corresponding question and answer pair datasets. The question and answer pairs generated by the large language model are screened through setting word limits, sensitive word checks, and other conditions. When the question and answer pair does not meet the condition, the question and answer pair that does not meet the condition is modified through sensitive word deletion, word replacement, and other methods.

[0042] The large language model generates a large paragraph level question according to the PDF file, that is, a second sub-paragraph level question corresponding to a second sub-paragraph. The prompt word engineering is still used, and the key of the prompt word is "generate a question summarizing the content of a large paragraph as much as possible, and the question should be as few as possible". The large language model generates a large paragraph level question by summarizing the content of a large paragraph. In this way, the question generated by the large language model contains not only micro questions at the sentence and small paragraph level, but also macro questions at the large paragraph level, thereby improving the comprehensiveness and hierarchy of the professional dataset construction. The large language model generates a large paragraph level question and answer pair, and still inputs the large paragraph level question and the content of the professional PDF file into the large language model through prompt word engineering. The large language model generates an answer according to the professional PDF file, and then a plurality of second sub-paragraph corresponding question and answer pair datasets are formed by the question and the answer. The question and answer pair data are still screened by setting word limit, sensitive word, content relevance and other conditions. Data that do not meet the conditions are deleted or modified by word replacement. It should be noted that the sensitive word here refers to a statement that violates the regulations, which can be pre-set according to the needs. The knowledge question and answer dataset is constructed by the progressive strategy, which can improve the hierarchy of the professional dataset construction and improve the ability of the large language model to answer micro and macro questions in the professional field.

[0043] In step S103, the professional file data is analyzed to determine the analysis case information corresponding to the professional file data, so as to construct a case analysis type data subset according to the probability distribution characteristics and the analysis case information.

[0044] The analysis case information includes variable type information corresponding to the analysis case.

[0045] In some embodiments, the case analysis type data subset is constructed according to the probability distribution characteristics and the analysis case information, specifically including: obtaining the variable type information of each variable corresponding to the analysis case in the analysis case information; constructing at least one case analysis question and answer pair corresponding to each variable type according to a pre-determined uniform distribution rule; determining the total number of historical case variables corresponding to the analysis case and the number of historical case variables corresponding to each variable type, and determining the occurrence probability of each variable type by the number of historical case variables and the total number of historical case variables; according to the occurrence probability of each variable type, each case analysis question and answer pair is expanded according to the normal distribution principle, and the case analysis type data subset is constructed.

[0046] In some embodiments, at least one case analysis question and answer pair corresponding to each variable type is constructed according to a predetermined uniform distribution rule, specifically including: designing a question for each variable type, generating at least one case analysis question corresponding to each variable type; answering each case analysis question according to the professional file data through a large language model, determining the case analysis answer corresponding to each case analysis question; based on at least one case analysis question corresponding to each variable type and the case analysis answer corresponding to each case analysis question, constructing at least one case analysis question and answer pair corresponding to each variable type.

[0047] In some embodiments of the present application, Figure 3 is a flowchart of constructing a case analysis data subset provided by some embodiments of the present application. As shown in Figure 3, the data set is constructed and expanded according to the characteristics of the probability distribution, the flow includes the steps of analyzing the variable type and quantity information involved in the case according to the PDF file, constructing at least one question and answer pair for each variable according to the principle of uniform distribution, sorting the variables according to the probability size, expanding the question and answer pair according to the principle of normal distribution according to the variable probability size, etc.

[0048] The variable type and quantity information involved in the analysis case information corresponding to the PDF file is obtained, that is, when constructing the question and answer pair that requires logical reasoning of the large language model, the variable type and quantity in the question need to be analyzed first. For example, in the professional business trip expense reimbursement business, the trip personnel go to a city on a business trip, and need to consider the accommodation environment and reimbursement expenses, the variables should be the level of the trip personnel, the city type, and the trip days. According to the principle of uniform distribution, at least one question and answer pair is constructed for each variable, which means that the breadth of the data set is considered, and the question and answer pair is constructed according to the principle that each variable is involved. Taking the business trip expense reimbursement business as an example, the level of the trip personnel usually includes chairman, general manager, manager, supervisor, director, and implementer, etc., each company contains different categories, and it is generally assumed to have 3 major categories; the city type is generally divided into municipalities, provincial capitals, and ordinary prefecture-level cities, etc., and it is also assumed to have three major categories; the trip days are assumed to be up to 15 days. Each type contains at least 3x3x15=135 data sets. The total number of historical case variables corresponding to the analysis case and the number of historical case variable occurrences corresponding to each variable type are obtained, and the occurrence probability of each variable type is calculated through the number of historical case variable occurrences and the total number of historical case variables. All variables are sorted according to the probability size. For example, the number of times the level of the trip personnel, the trip city type, and the trip days appear in the existing business trip expense reimbursement form is sorted.

[0049] According to the principle of normal distribution, the question and answer pairs are expanded according to the variable probability. It should be noted that most real problems are approximately or close to normal distribution. FIG. 4 is a normal distribution diagram provided by some embodiments of the present application. As shown in FIG. 4, the standard normal distribution probability density function is where the mean μ = 0 and the standard deviation σ = 1. Considering the symmetry of the normal distribution, only one side needs to be calculated. The probability in the [0, 3] interval is divided into several zones according to the number of variables, and the probability in each zone is calculated. Then, given a total data amount, the number of each variable probability is calculated to expand the data set. Assuming that the most frequent business trip personnel are level three personnel, followed by level two personnel, and finally level one personnel, the one-sided normal distribution probability is 0.341, 0.136, and 0.021, respectively. Assuming the total number is 25, the number of expanded question and answer pairs corresponding to level three personnel in the expanded data set is 25*0.341*2, which is 17.05, and the value is 17. According to the above method, the number of expanded question and answer pairs corresponding to level two personnel and level one personnel is 7 and 1, respectively. The characteristics of the normal distribution are used to construct the data, which not only ensures the breadth of the data but also ensures the depth of the data, further improving the comprehensiveness and accuracy of the large language model in the professional field.

[0050] Step S104, based on the large language model data set information, according to the knowledge question and answer type data subset and the case analysis type data subset, a large language model special data set corresponding to the specified professional field is constructed.

[0051] In some embodiments, based on the large language model data set information, according to the knowledge question and answer type data subset and the case analysis type data subset, the large language model special data set corresponding to the specified professional field is constructed, specifically including: obtaining the target task proportion of the plurality of data set target task types in the large language model data set information; respectively determining the number of knowledge question and answer pairs and the number of case analysis question and answer pairs in the knowledge question and answer type data subset and the case analysis type data subset; according to the number of knowledge question and answer pairs and the number of case analysis question and answer pairs, it is judged whether the knowledge question and answer type data subset and the case analysis type data subset meet the target task proportion; when the knowledge question and answer type data subset and the case analysis type data subset do not meet the target task proportion, the case analysis type data subset is expanded through the target task proportion to obtain an expanded case analysis type data subset, so as to construct the large language model special data set corresponding to the specified professional field according to the knowledge question and answer type data subset and the expanded case analysis type data subset according to the target task proportion.

[0052] In some embodiments of the present application, first, the number of knowledge and question pairs in the knowledge and question data subset obtained in the above steps and the number of case analysis and question pairs in the case analysis data subset are determined. Assuming that the number of knowledge and question pairs is 100, the number of case analysis and question pairs is 150, and the target task ratio of the knowledge and question class to the case analysis class is 1:4. According to the number of knowledge and question pairs and the number of case analysis and question pairs, it is determined whether the knowledge and question data subset and the case analysis data subset meet the target task ratio. In the above example, the knowledge and question data subset and the case analysis data subset do not meet the target task ratio. In this case, according to the target task ratio of 1:4, the case analysis data subset is expanded so that the ratio between the number of knowledge and question pairs and the number of expanded case analysis and question pairs in the expanded case analysis data subset meets the ratio of 1:4. It should be noted that when expanding, the expansion can be performed by synonym expansion or other expansion methods. According to the knowledge and question data subset and the expanded case analysis data subset, the large language model special data set corresponding to the specified professional field is constructed.

[0053] After constructing the large language model special data set corresponding to the specified professional field, the method further includes: obtaining a large language model general data set and determining a current data amount in the large language model special data set; determining a general data amount to be mixed according to a pre-set mixing ratio and the current data amount, wherein the mixing ratio is used to represent the mixing ratio of the large language model special data set and the general data set; according to the general data amount to be mixed, data extraction is performed in the large language model general data set through a pre-set script code to determine a general data set to be mixed; and the general data set to be mixed is mixed into the large language model special data set to construct a large language model data set.

[0054] In some embodiments of the present application, data is extracted from the general data set in a mixed proportion by designing script code and mixed into the constructed knowledge question and answer type and case analysis type data set to form a large language model data set. The proportion of the large language model special data set and the general data set can be a value between 1:5 and 1:10, which can be set according to empirical data. Specifically, the following steps are included. First, obtain the general data set of the large language model from public data sources or internal databases. The general data set should contain diversified text data, covering multiple topics and contexts. Count the number of existing text data or data volume in the special data set. According to the need, the theme distribution and quality of the data can be further analyzed. According to the pre-set mixing ratio (such as special data set: general data = 1:5) and the current data volume of the special data set, calculate the amount of general data to be mixed in. The mixing ratio can be adjusted according to the specific needs and performance of the model. Use the pre-set script code to extract data from the general data set randomly or according to a certain strategy (such as according to the theme distribution) according to the calculated amount of general data to be mixed in. The extracted data should ensure diversity and quality, avoiding the introduction of noise or duplicate data. Merge the extracted general data set to be mixed in with the special data set. The original labels of the data can be preserved or necessary preprocessing (such as de-duplication, text cleaning, etc.) can be performed during the merging process. After merging and possible preprocessing, the final large language model data set is obtained. The final data set should meet the needs of model training and have good generalization ability and adaptability.

[0055] By mixing in the general data set, the diversity and richness of the data set can be increased, thereby improving the generalization ability of the large language model, enabling it to better handle various contexts and topics; mixing the data set can better simulate the data distribution in the real world, enabling the model to learn more useful features and patterns during the training process, thereby optimizing the performance of the model. By introducing general data of different topics and contexts, the adaptability of the large language model to different fields and scenarios can be enhanced, improving its practicality and application range. Using the pre-set mixing ratio and script code for automatic extraction and mixing of data can improve the efficiency of data utilization, reduce manual intervention and errors, and allow the mixing ratio and data extraction strategy to be adjusted according to actual needs, with high flexibility and scalability.

[0056] According to the technical solution, the knowledge question and answer type data subset and the case analysis type data subset constructed by directly obtaining data from professional file data in a specified professional field directly reflect the professional knowledge and application scenarios in the field, so that the large language model special data set is closer to actual needs and business scenarios, thereby improving the pertinence and professionalism of the model; the multiple data set target task types and corresponding target task proportions contained in the large language model data set information can ensure that the data set contains both general professional knowledge questions and answers and specific case analysis, and such a mixed data set helps the model to learn more diverse features and patterns during the training process, thereby improving the generalization ability and adaptability of the model; the knowledge question and answer type data subset generated by the hierarchical progressive construction strategy can ensure that the quality and quantity of the question and answer pairs meet the needs of model training, and can improve the hierarchical sense of professional field data set construction and the ability of the large model to answer micro and macro problems in the professional field; and the case analysis type data subset constructed based on the probability distribution characteristics and analysis case information improves the breadth and depth of professional field data set construction, and further improves the comprehensiveness and accuracy of the large model in the professional field; the use of the pre-set hierarchical progressive construction strategy and the probability distribution characteristics of the analysis case information can automatically or semi-automatically construct the data set, reduce manual intervention and errors, and improve the quality and construction efficiency of the data set; by constructing the specific case analysis type data subset, the model can combine specific case information when generating predictions or suggestions to provide more detailed and accurate explanations, thereby enhancing the interpretability and credibility of the model, and making the results of the model more easily accepted and understood by users.

[0057] Some embodiments of the present application also provide a data set construction device for a large language model, as shown in FIG. 5, the device comprises: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0058] Some embodiments of the present application also provide a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are configured to: acquire professional file data in a specified professional field, and determine large language model data set information based on the professional file data, wherein the large language model data set information includes a plurality of data set target task types and target task proportions of the plurality of data set target task types, and the target task types include professional knowledge question answering and specific case analysis; perform question and answer pair construction on the professional file data through a pre-set hierarchical progressive construction strategy, to generate a knowledge question and answer type data subset corresponding to the professional file data; analyze the professional file data to determine analysis case information corresponding to the professional file data, and construct a case analysis type data subset according to probability distribution characteristics and the analysis case information, wherein the analysis case information includes variable type information corresponding to the analysis case; and based on the large language model data set information, construct a large language model special data set corresponding to the specified professional field according to the knowledge question and answer type data subset and the case analysis type data subset.

[0059] Each of the embodiments in the present application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments mainly describes the difference from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0060] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or equipment. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of another identical element in the process, method, article, or equipment including the element.

[0061] The above is only one or more embodiments of the present application, and is not intended to limit the present application. One or more embodiments of the present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of one or more embodiments of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for dataset construction of a large language model, characterized in that, The method comprises: obtaining professional file data in a specified professional field to determine large language model dataset information based on the professional file data, wherein the large language model dataset information comprises a plurality of dataset target task types and target task proportions of the plurality of dataset target task types, and the target task types comprise professional knowledge question answering and specific case analysis; constructing question and answer pairs of the professional file data through a pre-set hierarchical progressive construction strategy to generate a knowledge question and answer data subset corresponding to the professional file data; analyzing the professional file data to determine analysis case information corresponding to the professional file data, and constructing a case analysis data subset according to probability distribution characteristics and the analysis case information, wherein the analysis case information comprises variable type information corresponding to the analysis case; based on the large language model dataset information, constructing a large language model special dataset corresponding to the specified professional field according to the knowledge question and answer data subset and the case analysis data subset.

2. The data set construction method for a large language model according to claim 1, characterized in that, The method comprises: obtaining large language model input restriction information and natural delimiter information in the professional file data; processing the professional file data according to the large language model input restriction information and the natural delimiter information in the professional file data to generate a plurality of segments; taking each segment as an input of a large language model, generating a segment question corresponding to each segment through a pre-set prompt word, expanding the segment question through synonym replacement to generate a plurality of expanded segment questions; generating a segment answer corresponding to each expanded segment question through a prompt word engineering, and constructing a segment question and answer pair; based on a plurality of segment question and answer pairs, generating a knowledge question and answer data subset corresponding to the professional file data.

3. The data set construction method for a large language model according to claim 2, characterized in that, Before the method generates the knowledge question and answer data subset corresponding to the professional file data based on the plurality of segment question and answer pairs, the method further comprises: determining whether the segment question and answer pair meets a pre-set screening condition, wherein the pre-set screening condition at least comprises a set word limit and a sensitive word check; if the question and answer pair does not meet the pre-set screening condition, modifying the question and answer pair, and the modification method at least comprises sensitive word deletion and word replacement.

4. The data set construction method for a large language model according to claim 2, characterized in that, The method comprises: segmenting the professional file data according to the natural delimiter information in the professional file data to determine a plurality of first segments; determining a first segment order corresponding to the plurality of first segments through an order of the natural delimiter information in the professional file data; According to the large language model input restriction information, the segment word number information of the first segment is accumulated in sequence according to the first segment order, and accumulated segment word number information after accumulation is obtained; By the accumulated segment word number information and the large language model input restriction information, a plurality of specified first segments are merged to determine a plurality of second segments.

5. The data set construction method for a large language model according to claim 2, characterized in that, The large language model input restriction information is an input word size limit.

6. The data set construction method for a large language model according to claim 1, characterized in that, The case analysis type information corresponding to each variable in the analysis case information is obtained; According to a predetermined uniform distribution rule, at least one case analysis question and answer pair corresponding to each variable type is constructed; The total number of historical case variables corresponding to the analysis case and the number of historical case variable occurrences corresponding to each variable type are determined, and the occurrence probability of each variable type is determined by the number of historical case variable occurrences and the total number of historical case variables. According to the normal distribution principle, each case analysis question and answer pair is expanded according to the occurrence probability of each variable type to construct a case analysis data subset. The analysis case information at least includes variable type and quantity information.

7. The data set construction method for a large language model according to claim 6, characterized in that, The case analysis question and answer pair corresponding to each variable type is constructed according to a predetermined uniform distribution rule, specifically including:

8. The data set construction method for a large language model according to claim 6, characterized in that, Each variable type is designed to generate at least one case analysis question corresponding to each variable type; Each case analysis question is answered by a large language model according to the professional file data to determine the case analysis answer corresponding to each case analysis question; Based on at least one case analysis question corresponding to each variable type and the case analysis answer corresponding to each case analysis question, at least one case analysis question and answer pair corresponding to each variable type is constructed. The professional file data is used to determine the large language model dataset information, specifically including:

9. The data set construction method for a large language model according to claim 1, characterized in that, The historical large language model input and output information corresponding to the specified professional field of the professional file data is obtained; The historical large language model input and output information is identified to determine a plurality of historical task types to determine the plurality of dataset target task types; The number of tasks of each historical task type is counted, and the target task proportion of the plurality of dataset target task types is determined according to the ratio between the number of tasks of the plurality of historical task types. The historical large language model input and output information is identified to determine a plurality of historical task types to determine the plurality of dataset target task types, specifically including:

10. The data set construction method for a large language model according to claim 9, characterized in that, The historical large language model input and output information is identified by natural language processing technology to determine a plurality of historical task types corresponding to the historical large language model input and output information; According to the plurality of historical task types, a plurality of dataset target task types corresponding to the professional file data are determined. ​ 11. The data set construction method for a large language model according to claim 1, characterized in that, The method comprises the following steps: Obtaining the target task proportion of the plurality of data set target task types in the large language model data set information; Respectively determining the number of knowledge question and answer pairs in the knowledge question and answer type data subset and the number of case analysis type question and answer pairs in the case analysis type data subset; According to the number of knowledge question and answer pairs and the number of case analysis type question and answer pairs, it is judged whether the knowledge question and answer type data subset and the case analysis type data subset meet the target task proportion; When the knowledge question and answer type data subset and the case analysis type data subset do not meet the target task proportion, the case analysis type data subset is expanded through the target task proportion to obtain an expanded case analysis type data subset, so as to construct the large language model special data set corresponding to the specified professional field according to the knowledge question and answer type data subset and the expanded case analysis type data subset according to the target task proportion.

12. The data set construction method for a large language model according to claim 11, wherein, The method comprises the following steps: The case analysis type data subset is expanded in the form of synonym expansion through the target task proportion to obtain the expanded case analysis type data subset.

13. The data set construction method for a large language model according to claim 1, characterized in that, After the large language model special data set corresponding to the specified professional field is constructed, the method further comprises the following steps: Obtaining a large language model general data set and determining the current data amount in the large language model special data set; According to the pre-set mixing proportion and the current data amount, determining the general data amount to be mixed, wherein the mixing proportion is used to represent the mixing proportion of the large language model special data set and the general data set; According to the general data amount to be mixed, data extraction is performed in the large language model general data set through the pre-set script code to determine the general data set to be mixed; The general data set to be mixed is mixed into the large language model special data set to construct a large language model data set.

14. The data set construction method for a large language model according to claim 13, wherein, The method comprises the following steps: The large language model general data set is obtained from a public data source or an internal database, and the large language model general data set comprises text data of multiple topics and contexts.

15. The data set construction method for a large language model according to claim 13, wherein, The method comprises the following steps: According to the general data amount to be mixed, data extraction is performed in the large language model general data set through the pre-set script code to determine the general data set to be mixed; 16. The data set construction method for a large language model according to claim 13, wherein, According to the general data amount to be mixed, data is extracted from the large language model general data set randomly or according to a pre-set strategy through the pre-set script code to determine the general data set to be mixed, and the pre-set strategy at least comprises extracting data according to topic distribution. The pre-set mixing proportion is adjusted according to the demand and performance of the large language model.

17. The data set construction method for a large language model according to claim 13, wherein, The method comprises the following steps: The method comprises the following steps:

18. The data set construction method for a large language model according to claim 1, wherein, The specified professional field is a professional field of a business corresponding to the large language model.

19. A dataset construction device for a large language model, comprising: The device comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-18.

20. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer executable instructions are configured to: obtain professional file data in a specified professional field, and determine large language model dataset information based on the professional file data, wherein the large language model dataset information comprises a plurality of dataset target task types and target task proportions of the plurality of dataset target task types, and the target task types comprise professional knowledge question answering and specific case analysis; construct a question and answer pair for the professional file data through a pre-set hierarchical progressive construction strategy to generate a knowledge question and answer type data subset corresponding to the professional file data; analyze the professional file data to determine analysis case information corresponding to the professional file data, and construct a case analysis type data subset according to probability distribution characteristics and the analysis case information, wherein the analysis case information comprises variable type information corresponding to an analysis case; construct a large language model special dataset corresponding to the specified professional field based on the large language model dataset information, the knowledge question and answer type data subset, and the case analysis type data subset.

Images