Method, system and equipment for constructing model training data and medium

Abstract

The invention discloses a method, a system, equipment and a medium for constructing model training data, and belongs to the technical field of data processing. The method comprises the steps of determining model information of a target domain model for the target domain model, and determining a data source of model training data for training the target domain model; performing error data cleaning processing, chapter-level and sentence-level duplicate removal processing and security data filtering processing on the initial model training data to obtain to-be-evaluated model training data; constructing an evaluation index for evaluating the quality of the model training data; and evaluating the quality of the to-be-evaluated model training data, determining whether an evaluation result conforms to an evaluation index threshold, and outputting the to-be-evaluated model training data conforming to the evaluation index threshold as the model training data. According to the method, the data set required by the model can be constructed, the manpower and time cost is saved, and the construction difficulty of the data set is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a method, system, device and medium for constructing model training data. Background Art

[0002] With the continuous development of technologies such as big data and artificial intelligence, financial and tax management is transforming towards intelligence and automation. Enterprises need to use advanced technical means to improve the efficiency and accuracy of financial and tax management and reduce labor costs. The general large model demonstrates extraordinary understanding, reasoning and decision-making capabilities in complex interactive environments by pre-training on large-scale unsupervised data.

[0003] The existing general models are extremely inadequate in the vertical field of finance and taxation, so it is necessary to develop targeted industry big models for the finance and taxation industry. Industry big models are reshaping thousands of industries, and big model technology will inevitably bring profound technological changes to the finance and taxation industry. However, big models still have certain limitations in tasks in specific fields. These tasks often require specialized tools or domain knowledge to be effectively solved. In addition, big models also face risks such as compliance of training data sources, bias in data use, and security of generated content. Summary of the invention

[0004] In view of the above problems, the present invention proposes a method for constructing model training data, comprising:

[0005] For a target domain model, determine the model information of the target domain model, and determine the data source of the model training data used to train the target domain model;

[0006] Collect data from the data source to obtain initial model training data, clean the initial model training data for erroneous data, remove duplicates at the chapter and sentence levels, and filter security data to obtain model training data to be evaluated;

[0007] For historical model training data, determining factors affecting the quality of historical model training data, and constructing evaluation indicators for evaluating the quality of model training data based on the influencing factors;

[0008] Based on the evaluation index, the quality of the model training data to be evaluated is evaluated to determine whether the evaluation result meets the evaluation index threshold, and the model training data to be evaluated that meets the evaluation index threshold is output as model training data.

[0009] Optional, model information, including: the amount of data required to train the target domain model;

[0010] Among them, determine the amount of data required to train the target domain model, including:

[0011] Determine the vertical domain tasks that the target domain model needs to support in the future, generate a task list based on the vertical domain tasks, and mark each vertical domain task in the task list according to its importance and difficulty. According to the marking results, determine the amount of data required for each vertical domain task, and take the sum of the data amounts described in all vertical domain tasks as the amount of data required to train the target domain model.

[0012] Optional data sources include: public data sources in the field, business data sources, and open source high-quality SFT dataset sources.

[0013] Optionally, perform cleaning of erroneous data, including:

[0014] The initial model training data is processed by removing duplicate identical data, completing missing values, removing erroneous values, and unifying the data format.

[0015] Optionally, perform paragraph-level and sentence-level deduplication processing, including:

[0016] After cleaning the erroneous data, the content fields of the chapter-level data and sentence-level data in the initial model training data are deduplicated, including:

[0017] Determine the similarity value of the content field content of two chapter-level data, if the similarity value exceeds a preset threshold, determine that the two chapter-level data are duplicate chapter-level data, and delete one of the chapter-level data;

[0018] Determine the content field content of a sentence-level data, if there are repeated sentences in the content field content of the sentence-level data, determine that there are repeated sentences in the sentence-level data, and delete the repeated sentences in the sentence-level data.

[0019] Optionally, perform security data filtering, including:

[0020] The text classification model based on the quantized neural network QDNN uses a keyword combination matching algorithm to identify the initial model training data after the paragraph-level and sentence-level deduplication processing. If sensitive words that affect data security are identified, the sensitive words are filtered.

[0021] Optional, evaluation indicators, including: data distribution index, diversity index, complexity index, usefulness index and input-output repeatability index;

[0022] The diversity indicators include: task diversity indicators, instruction expression diversity indicators, topic diversity indicators and task difficulty indicators.

[0023] Optionally, when evaluating the quality of the model training data to be evaluated, the ROUGE-L similarity of the input and output of the target domain model is calculated for the model training data to be evaluated, and repeated input data in the model training data to be evaluated is filtered out based on the ROUGE-L similarity.

[0024] In another aspect, the present invention further provides a system for constructing model training data, comprising:

[0025] An initialization unit, used to determine, for a target domain model, model information of the target domain model and determine a data source of model training data used to train the target domain model;

[0026] A data processing unit is used to collect data from the data source to obtain initial model training data, clean the initial model training data for erroneous data, remove duplicates at the chapter level and sentence level, and filter security data to obtain model training data to be evaluated;

[0027] An evaluation unit, used to determine, with respect to the historical model training data, factors affecting the quality of the historical model training data, and construct an evaluation index for evaluating the quality of the model training data based on the factors;

[0028] The output unit is used to evaluate the quality of the model training data to be evaluated based on the evaluation index, determine whether the evaluation result meets the evaluation index threshold, and output the model training data to be evaluated that meets the evaluation index threshold as model training data.

[0029] Optional, model information, including: the amount of data required to train the target domain model;

[0030] Among them, determine the amount of data required to train the target domain model, including:

[0031] Determine the vertical domain tasks that the target domain model needs to support in the future, generate a task list based on the vertical domain tasks, and mark each vertical domain task in the task list according to its importance and difficulty. According to the marking results, determine the amount of data required for each vertical domain task, and take the sum of the data amounts described in all vertical domain tasks as the amount of data required to train the target domain model.

[0032] Optional data sources include: public data sources in the field, business data sources, and open source high-quality SFT dataset sources.

[0033] Optionally, perform cleaning of erroneous data, including:

[0034] The initial model training data is processed by removing duplicate identical data, completing missing values, removing erroneous values, and unifying the data format.

[0035] Optionally, perform paragraph-level and sentence-level deduplication processing, including:

[0036] After cleaning the erroneous data, the content fields of the chapter-level data and sentence-level data in the initial model training data are deduplicated, including:

[0037] Determine the similarity value of the content field content of two chapter-level data, if the similarity value exceeds a preset threshold, determine that the two chapter-level data are duplicate chapter-level data, and delete one of the chapter-level data;

[0038] Determine the content field content of a sentence-level data, if there are repeated sentences in the content field content of the sentence-level data, determine that there are repeated sentences in the sentence-level data, and delete the repeated sentences in the sentence-level data.

[0039] Optionally, perform security data filtering, including:

[0040] The text classification model based on the quantized neural network QDNN uses a keyword combination matching algorithm to identify the initial model training data after the paragraph-level and sentence-level deduplication processing. If sensitive words that affect data security are identified, the sensitive words are filtered.

[0041] Optional, evaluation indicators, including: data distribution index, diversity index, complexity index, usefulness index and input-output repeatability index;

[0042] The diversity indicators include: task diversity indicators, instruction expression diversity indicators, topic diversity indicators and task difficulty indicators.

[0043] Optionally, when evaluating the quality of the model training data to be evaluated, the ROUGE-L similarity of the input and output of the target domain model is calculated for the model training data to be evaluated, and repeated input data in the model training data to be evaluated is filtered out based on the ROUGE-L similarity.

[0044] In yet another aspect, the present invention further provides a computing device, comprising: one or more processors;

[0045] a processor for executing one or more programs;

[0046] When the one or more programs are executed by the one or more processors, the above-described method is implemented.

[0047] In yet another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, the method described above is implemented.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The present invention provides a method for constructing model training data, comprising: for a target domain model, determining the model information of the target domain model, and determining the data source of the model training data for training the target domain model; through the data source, collecting data, obtaining initial model training data, cleaning the initial model training data with wrong data, removing duplicates at the chapter level and sentence level, and filtering the security data, and obtaining the model training data to be evaluated; for historical model training data, determining the influencing factors affecting the quality of historical model training data, and constructing an evaluation index for evaluating the quality of model training data according to the influencing factors; based on the evaluation index, evaluating the quality of the model training data to be evaluated, determining whether the evaluation result meets the evaluation index threshold, and outputting the model training data to be evaluated that meets the evaluation index threshold as the model training data. The present invention can construct the data set required for the model, saves manpower and time costs, and reduces the difficulty of constructing the data set. The accuracy of the constructed data set is high, so the accuracy rate of the model constructed subsequently is also high. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of the method of the present invention;

[0051] Figure 2 A flowchart of an implementation case of the method of the present invention;

[0052] Figure 3 A safety filtering flow chart of an implementation case of the method of the present invention;

[0053] Figure 4 It is a structural diagram of the system of the present invention. DETAILED DESCRIPTION

[0054] Now, exemplary embodiments of the present invention are described with reference to the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to disclose the present invention in detail and completely and to fully convey the scope of the present invention to those skilled in the art. The terms used in the exemplary embodiments shown in the accompanying drawings are not intended to limit the present invention. In the accompanying drawings, the same units / elements are marked with the same reference numerals.

[0055] Unless otherwise specified, the terms (including technical terms) used herein have the commonly understood meanings to those skilled in the art. In addition, it is understood that the terms defined in commonly used dictionaries should be understood to have the same meanings as those in the context of the relevant fields, and should not be understood as idealized or overly formal meanings.

[0056] Embodiment 1:

[0057] The present invention proposes a method for constructing model training data, such as Figure 1 As shown, including:

[0058] Step 1: for a target domain model, determine the model information of the target domain model, and determine the data source of the model training data used to train the target domain model;

[0059] Step 2: Collect data from the data source to obtain initial model training data, clean the initial model training data for erroneous data, remove duplicates at the chapter and sentence levels, and filter security data to obtain model training data to be evaluated;

[0060] Step 3: for the historical model training data, determine the factors affecting the quality of the historical model training data, and construct an evaluation index for evaluating the quality of the model training data based on the influencing factors;

[0061] Step 4: Based on the evaluation index, the quality of the model training data to be evaluated is evaluated to determine whether the evaluation result meets the evaluation index threshold, and the model training data to be evaluated that meets the evaluation index threshold is output as model training data.

[0062] Among them, model information includes: the amount of data required to train the target domain model;

[0063] Among them, determine the amount of data required to train the target domain model, including:

[0064] Determine the vertical domain tasks that the target domain model needs to support in the future, generate a task list based on the vertical domain tasks, and mark each vertical domain task in the task list according to its importance and difficulty. According to the marking results, determine the amount of data required for each vertical domain task, and take the sum of the data amounts described in all vertical domain tasks as the amount of data required to train the target domain model.

[0065] Among them, data sources include: public data sources in the field, business data sources, and open source high-quality SFT data set sources.

[0066] Among them, cleaning of erroneous data includes:

[0067] The initial model training data is processed by removing duplicate identical data, completing missing values, removing erroneous values, and unifying the data format.

[0068] Among them, the duplicate elimination processing at the chapter level and sentence level includes:

[0069] After cleaning the erroneous data, the content fields of the chapter-level data and sentence-level data in the initial model training data are deduplicated, including:

[0070] Determine the similarity value of the content field content of two chapter-level data, if the similarity value exceeds a preset threshold, determine that the two chapter-level data are duplicate chapter-level data, and delete one of the chapter-level data;

[0071] Determine the content field content of a sentence-level data, if there are repeated sentences in the content field content of the sentence-level data, determine that there are repeated sentences in the sentence-level data, and delete the repeated sentences in the sentence-level data.

[0072] Among them, the security data filtering process includes:

[0073] The text classification model based on the quantized neural network QDNN uses a keyword combination matching algorithm to identify the initial model training data after the paragraph-level and sentence-level deduplication processing. If sensitive words that affect data security are identified, the sensitive words are filtered.

[0074] Among them, the evaluation indicators include: data distribution index, diversity index, complexity index, usefulness index and input and output repeatability index;

[0075] The diversity indicators include: task diversity indicators, instruction expression diversity indicators, topic diversity indicators and task difficulty indicators.

[0076] When evaluating the quality of the model training data to be evaluated, the ROUGE-L similarity of the input and output of the target domain model is calculated for the model training data to be evaluated, and repeated input data in the model training data to be evaluated is filtered out according to the ROUGE-L similarity.

[0077] The present invention is further described below with reference to specific cases:

[0078] Specific case flow: Figure 2 As shown, including:

[0079] Data demand analysis includes: mainly confirming the vertical tasks that the domain model needs to support in the future, forming a task list, and confirming the amount of data required based on the importance and difficulty of the tasks. At the same time, confirming the data sources that can be obtained.

[0080] Prepare and extract data, including: collect relevant domain data based on the domain tasks accumulated during the previous communication process, including relevant domain public data, internally collated and accessible business data, and whether there are high-quality SFT data sets that are already open source.

[0081] Data cleaning, including: Data cleaning is a fundamental aspect of pre-processing before training large language models (LLMs). This technique involves identifying and correcting inaccuracies, inconsistencies, and extraneous elements in raw text data. Data cleaning steps include removing duplicates, handling missing or erroneous values, and resolving formatting irregularities.

[0082] Data deduplication includes: in addition to deduplication of the same data in the same data source in the cleaning step, it also involves chapter-level deduplication and sentence-level deduplication. Chapter-level deduplication is performed on the content field in the standardized data. That is, if the similarity value of the content field of two data exceeds the threshold, they are judged as duplicates. Sentence-level deduplication refers to deduplication of sentences in the content field of a piece of data. If there are repeated sentences in the content, the repeated parts will be deleted.

[0083] Data security filtering, such as Figure 3 As shown, it includes: data security filtering including toxic content screening and data political content filtering.

[0084] Data quality analysis includes: Data quality analysis is mainly conducted from two dimensions: diversity assessment and quality assessment, to gain an in-depth understanding of the distribution, complexity, and diversity of data sets, and to help screen high-quality SFT data.

[0085] The diversity assessment includes the following:

[0086] Task diversity: refers to the types of tasks that the model will encounter in the production environment (task classification). The implementation method is based on rule mapping and large language models to label the data with task labels. There are 8 task categories: multi-round rewriting, multi-round clarification, information extraction, question generation, reading comprehension, chart drawing, table analysis, and text summarization.

[0087] Instruction expression diversity: refers to the fact that questions or instructions contain multiple language styles, expressions, and vocabulary (instruction diversity, main verb-object phrases). The implementation method is to identify instruction data with similar expressions based on edit distance, and perform streaming clustering on the data set to obtain multiple instruction clusters. The diversity of the instruction data set is defined as the number of clusters after clustering / the total amount of the data set. When the diversity index of the data set exceeds 0.9, we believe that its diversity is acceptable. Secondly, based on the syntactic analysis model, identify the verb-noun structure in the instruction, extract the verb closest to the root node and its first direct noun object as the main verb-object phrase, and count the instruction templates with a frequency greater than or equal to 5 in the data set, and display them visually.

[0088] Topic diversity: Contains a wealth of topics and knowledge points (keywords and key phrases). Based on the key phrase extraction algorithm, the top 200 keywords and key phrases in the instruction data are extracted and visualized to assist in understanding the topic distribution in the data set.

[0089] Task difficulty: covers tasks of various difficulty levels (input and output length). Count the input and output lengths of each piece of data and visualize the length distribution, which reflects the diversity and complexity of the dataset to a certain extent. Count the conversation rounds of the statistics. Given the training requirements, you may need to focus on specific rounds of conversations or ensure that the number of different rounds of conversations in the dataset is relatively balanced to ensure that the model can fully cope with various conversation scenarios.

[0090] Quality assessment refers to the analysis of usefulness and input-output repetitiveness. By calculating the ROUGE-L similarity between input and output, we filter out low-quality data where the model fails to understand human intent and generates repeated output.

[0091] This implementation case starts from the pre-training processing flow of the finance and taxation scenarios, realizes the automated data cleaning process and the automated data quality analysis process. The combination of the two forms a complete pre-training data processing method, and successfully constructs pre-training data under the eight major capabilities in the finance and taxation field (multi-round rewriting, multi-round clarification, information extraction, question generation, reading comprehension, chart drawing, table analysis, and text summarization).

[0092] This implementation case provides a basic guarantee for the production of high-quality datasets in the field, and lays a data foundation for training large-scale language models with professional knowledge and high accuracy in this field. At the same time, the datasets processed by the pre-training data processing process will greatly reduce the manpower and time costs of subsequent manual labeling, verification and modification, and reduce the difficulty of constructing specific classification data required for different categories of tasks.

[0093] The present invention introduces Python's Pandas library and Simhash algorithm, combines the characteristics of the data set, cleans the data, and realizes the identification and correction of inaccurate, inconsistent, repeated and irrelevant elements in the original text data.

[0094] The present invention introduces a QDNN (Quantized Deep Neural Network) text classification model and constructs a harmful politically sensitive word list to achieve toxic content screening and data political content filtering.

[0095] The present invention designs an automated process for data quality analysis to assist in understanding data characteristics and distribution, help screen and construct high-quality SFT data, and improve the performance of large language models.

[0096] The present invention combines the data cleaning process with the automated data quality analysis process to form a complete pre-training data processing method, and completes the construction of a pre-training data set for eight capabilities of a large language model in the finance and taxation field.

[0097] Embodiment 2:

[0098] In another aspect, the present invention also provides a system 200 for constructing model training data, such as Figure 4 As shown, including:

[0099] The initialization unit 201 is used to determine the model information of the target domain model and determine the data source of the model training data used to train the target domain model.

[0100] The data processing unit 202 is used to collect data from the data source to obtain initial model training data, clean the initial model training data for erroneous data, remove duplicates at the chapter level and sentence level, and filter security data to obtain model training data to be evaluated;

[0101] An evaluation unit 203 is used to determine the factors affecting the quality of the historical model training data, and to construct an evaluation index for evaluating the quality of the model training data according to the factors.

[0102] The output unit 204 is used to evaluate the quality of the model training data to be evaluated based on the evaluation index, determine whether the evaluation result meets the evaluation index threshold, and output the model training data to be evaluated that meets the evaluation index threshold as model training data.

[0103] Among them, model information includes: the amount of data required to train the target domain model;

[0104] Among them, determine the amount of data required to train the target domain model, including:

[0105] Determine the vertical domain tasks that the target domain model needs to support in the future, generate a task list based on the vertical domain tasks, and mark each vertical domain task in the task list according to its importance and difficulty. According to the marking results, determine the amount of data required for each vertical domain task, and take the sum of the data amounts described in all vertical domain tasks as the amount of data required to train the target domain model.

[0106] Among them, data sources include: public data sources in the field, business data sources, and open source high-quality SFT data set sources.

[0107] Among them, cleaning of erroneous data includes:

[0108] The initial model training data is processed by removing duplicate identical data, completing missing values, removing erroneous values, and unifying the data format.

[0109] Among them, the duplicate elimination processing at the chapter level and sentence level includes:

[0110] After cleaning the erroneous data, the content fields of the chapter-level data and sentence-level data in the initial model training data are deduplicated, including:

[0111] Determine the similarity value of the content field content of two chapter-level data, if the similarity value exceeds a preset threshold, determine that the two chapter-level data are duplicate chapter-level data, and delete one of the chapter-level data;

[0112] Determine the content field content of a sentence-level data, if there are repeated sentences in the content field content of the sentence-level data, determine that there are repeated sentences in the sentence-level data, and delete the repeated sentences in the sentence-level data.

[0113] Among them, the security data filtering process includes:

[0114] The text classification model based on the quantized neural network QDNN uses a keyword combination matching algorithm to identify the initial model training data after the paragraph-level and sentence-level deduplication processing. If sensitive words that affect data security are identified, the sensitive words are filtered.

[0115] Among them, the evaluation indicators include: data distribution index, diversity index, complexity index, usefulness index and input and output repeatability index;

[0116] The diversity indicators include: task diversity indicators, instruction expression diversity indicators, topic diversity indicators and task difficulty indicators.

[0117] When evaluating the quality of the model training data to be evaluated, the ROUGE-L similarity of the input and output of the target domain model is calculated for the model training data to be evaluated, and repeated input data in the model training data to be evaluated is filtered out according to the ROUGE-L similarity.

[0118] The present invention can construct the data set required for the model, saving manpower and time costs, and reducing the difficulty of constructing the data set. The constructed data set has high accuracy, so the accuracy of the subsequently constructed model is also high.

[0119] Embodiment 3:

[0120] Based on the same inventive concept, the present invention also provides a computer device, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding functions, so as to implement the steps of the method in the above embodiment.

[0121] Embodiment 4:

[0122] Based on the same inventive concept, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both a built-in storage medium in a computer device and an extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of the method in the above embodiment.

[0123] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0124] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0127] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0128] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for constructing model training data, characterized in that: include: For a target domain model, determine the model information of the target domain model, and determine the data source of the model training data used to train the target domain model; Collect data from the data source to obtain initial model training data, clean the initial model training data for erroneous data, remove duplicates at the chapter and sentence levels, and filter security data to obtain model training data to be evaluated; For historical model training data, determining factors affecting the quality of historical model training data, and constructing evaluation indicators for evaluating the quality of model training data based on the influencing factors; Based on the evaluation index, the quality of the model training data to be evaluated is evaluated to determine whether the evaluation result meets the evaluation index threshold, and the model training data to be evaluated that meets the evaluation index threshold is output as model training data.

2. The method according to claim 1, characterized in that: The model information includes: the amount of data required to train the target domain model; Among them, determine the amount of data required to train the target domain model, including: Determine the vertical domain tasks that the target domain model needs to support in the future, generate a task list based on the vertical domain tasks, and mark each vertical domain task in the task list according to its importance and difficulty. According to the marking results, determine the amount of data required for each vertical domain task, and take the sum of the data amounts described in all vertical domain tasks as the amount of data required to train the target domain model.

3. The method according to claim 1, characterized in that The data sources include: public data sources in the field, business data sources, and open source high-quality SFT data set sources.

4. The method according to claim 1, characterized in that The cleaning process of the erroneous data includes: The initial model training data is processed by removing duplicate identical data, completing missing values, removing erroneous values, and unifying the data format.

5. The method according to claim 1, characterized in that: The paragraph-level and sentence-level duplicate elimination processing includes: After cleaning the erroneous data, the content fields of the chapter-level data and sentence-level data in the initial model training data are deduplicated, including: Determine the similarity value of the content field content of two chapter-level data, if the similarity value exceeds a preset threshold, determine that the two chapter-level data are duplicate chapter-level data, and delete one of the chapter-level data; Determine the content field content of a sentence-level data, if there are repeated sentences in the content field content of the sentence-level data, determine that there are repeated sentences in the sentence-level data, and delete the repeated sentences in the sentence-level data.

6. The method according to claim 1, characterized in that The security data filtering process includes: The text classification model based on the quantized neural network QDNN uses a keyword combination matching algorithm to identify the initial model training data after the paragraph-level and sentence-level deduplication processing. If sensitive words that affect data security are identified, the sensitive words are filtered.

7. The method according to claim 1, characterized in that The evaluation indicators include: data distribution index, diversity index, complexity index, usefulness index and input-output repeatability index; The diversity indicators include: task diversity indicators, instruction expression diversity indicators, topic diversity indicators and task difficulty indicators.

8. A system for constructing model training data, characterized in that: include: An initialization unit, used to determine, for a target domain model, model information of the target domain model and determine a data source of model training data used to train the target domain model; A data processing unit is used to collect data from the data source to obtain initial model training data, clean the initial model training data for erroneous data, remove duplicates at the chapter level and sentence level, and filter security data to obtain model training data to be evaluated; An evaluation unit, used to determine, with respect to the historical model training data, factors affecting the quality of the historical model training data, and construct an evaluation index for evaluating the quality of the model training data based on the factors; The output unit is used to evaluate the quality of the model training data to be evaluated based on the evaluation index, determine whether the evaluation result meets the evaluation index threshold, and output the model training data to be evaluated that meets the evaluation index threshold as model training data.

9. A computer device, characterized in that: include: one or more processors; a processor for executing one or more programs; When the one or more programs are executed by the one or more processors, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images