Method and system for generating fine tuning data set based on data consanguinity and large model
By collecting and generating business data, building fine-tuning data sets, and using data ties to achieve rapid comparison and evaluation of data, it solves the problem of low evaluation efficiency of existing traditional Chinese medicine and patient dialogue data and admission medical record data, and improves data evaluation efficiency and fine-tuning training quality.
Patent Information
- Application Number
- CN202411536141.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-25
AI Technical Summary
The evaluation efficiency of the doctor-patient dialogue data and corresponding admission record data is low, and it is impossible to effectively realize the comparative evaluation of data.
By collecting business data from various regions and hospitals, generating raw data, derive data, publishing data, training data and verification data, using data blood relationship to build a fine-tuned data set, and quickly comparing and evaluating the data directory and file names.
This improves the efficiency of data evaluation, improves the quality of data sets generated by medical records, shortens the time for fine-tuning data sets to be built on demand, improves the efficiency of fine-tuning training, and ultimately achieves the overall improvement of fine-tuning training quality.
Smart Images
Figure CN119513596A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and system for generating a fine-tuning data set based on data lineage and a large model. Background Art
[0002] Large model fine-tuning refers to retraining the pre-trained large model for specific domain tasks. In the medical field, hospital admission records can be generated based on doctor-patient conversations, and then a fine-tuning model can be generated based on the admission records. By collecting a large number of doctor-patient conversations and corresponding patient medical records, the large model can be fine-tuned to obtain an enhanced vertical application fine-tuning model.
[0003] Currently, the existing technology is to continuously collect standard data sets and derive and construct other dialect data sets based on the standard data sets.
[0004] However, the existing technology generates a fine-tuned model by collecting a large amount of fine-tuned annotated data from target customers, but is unable to achieve comparative evaluation of doctor-patient conversation data and corresponding hospital admission medical record data, which results in low efficiency of data evaluation. Summary of the invention
[0005] The embodiment of the present invention provides a method and system for generating a fine-tuning data set based on data lineage and a large model, which can improve the efficiency of data evaluation. The method includes:
[0006] A1: Collecting business data of at least one region and hospital, wherein the business data includes: pending doctor-patient conversations and pending hospital admission medical records;
[0007] A2: Based on the business data and the original data definition file, the original data is generated and stored in the original data directory, wherein the original data definition file is an XML format file including the original directory classification management rules, the original directory naming rules and the original file naming rules, and the original data directory includes: the first doctor-patient dialogue directory and the first hospitalization medical record directory;
[0008] A3: Based on the original data and the derived data definition file, generate derived data corresponding to at least one dialect version of the original data, and store them in a derived data directory, wherein the derived data definition file is an XML format file including derived directory classification management rules, derived directory naming rules and derived file naming rules, and the derived data directory includes: a second doctor-patient dialogue directory and a second hospitalization medical record directory;
[0009] A4: Generate release data based on the derived data, release data definition file and project requirements, wherein the release data is a fine-tuning data set including training data and verification data, the training data is used to train the large model, and the verification data is used to fine-tune the parameters of the large model based on the training results.
[0010] Preferably,
[0011] The original data definition file also includes: original data verification rules;
[0012] The original data directory also includes: an original data verification result directory;
[0013] After A2 and before A3, further comprising:
[0014] Based on the original data verification rule, the original data is verified, an original data verification result file is generated, and the original data verification result file is stored in the original data verification result directory file, wherein the original data verification result file is in XML format;
[0015] The derived data definition file also includes: derived data verification rules;
[0016] The derived data directory also includes: a derived data verification result directory;
[0017] After A3 and before A4, further comprising:
[0018] Based on the derived data verification rule, the derived data is verified, a derived data verification result file is generated, and stored in a derived data verification result directory file, wherein the derived data verification result file is in XML format.
[0019] Preferably,
[0020] After A4, it further includes:
[0021] Based on the published data and the training data definition file, the training data is generated and stored in a training data directory file, wherein the training data is an XML format file, the training data definition file includes: a training data directory naming rule and a training data file naming rule, and the training data directory includes: a third doctor-patient dialogue directory and a third hospitalization medical record directory;
[0022] Based on the published data and the verification data definition file, the verification data is generated and stored in a verification data directory file, wherein the verification data is an XML format file, the verification data definition file includes: a verification data directory naming rule, a verification data file naming rule and a verification data verification rule, and the verification data directory includes: a fourth doctor-patient dialogue directory, a fourth admission medical record directory and a first admission medical record reasoning directory;
[0023] Based on the verification data verification rule, the first admission medical record reasoning data is verified, a verification data verification result file is generated, and stored in the verification data verification result directory file, wherein the verification data verification result file is in XML format;
[0024] Based on the verification result of the validation data, it is determined whether it is necessary to fine-tune the parameters of the large model.
[0025] Preferably,
[0026] The release data also includes: test data;
[0027] After performing data verification on the first admission medical record reasoning data based on the verification data verification rule, generating a verification data verification result file, and storing it in the verification data verification result directory file, before determining whether it is necessary to fine-tune the parameters of the large model based on the verification data, further comprising:
[0028] Based on the release data and the test data definition file, the test data is generated and stored in a test data directory file, wherein the test data is an XML format file, the test data definition file includes: a test data directory naming rule, a test data file naming rule and a test data verification rule, and the test data directory includes: a fifth doctor-patient dialogue directory, a fifth admission medical record directory and a second admission medical record reasoning directory;
[0029] After determining whether it is necessary to fine-tune the parameters of the large model based on the verification data verification result, the method further includes:
[0030] When it is determined that the parameters of the large model need to be fine-tuned, fine-tuning the parameters of the large model to generate a target large model;
[0031] Based on the test data verification rules, the second admission medical record inference data is verified, a test data verification result file is generated, and stored in the test data verification result directory file, wherein the test data verification result file is in XML format.
[0032] In a second aspect, an embodiment of the present invention provides a system for generating a fine-tuning dataset based on data lineage and a large model, the system comprising:
[0033] Collection module: used to collect business data of at least one region and hospital, wherein the business data includes: pending doctor-patient dialogues and pending hospital admission medical records;
[0034] The original data processing module is used to generate original data based on the business data and the original data definition file collected by the collection module, and store the original data in the original data directory, wherein the original data definition file is an XML format file including original directory classification management rules, original directory naming rules and original file naming rules, and the original data directory includes: the first doctor-patient dialogue directory and the first hospitalization medical record directory;
[0035] A derived data processing module: used for generating derived data corresponding to at least one dialect version of the original data based on the original data and the derived data definition file generated by the original data processing module, and storing the derived data in a derived data directory, wherein the derived data definition file is an XML format file including derived directory classification management rules, derived directory naming rules and derived file naming rules, and the derived data directory includes: a second doctor-patient dialogue directory and a second hospitalization medical record directory;
[0036] Release data processing module: used to generate release data based on the derived data generated by the derived data processing module, the release data definition file and the project requirements, wherein the release data is a fine-tuning data set including training data and verification data, the training data is used to train the large model, and the verification data is used to fine-tune the parameters of the large model based on the training results.
[0037] Preferably,
[0038] After the original data processing module and before the derived data processing module, it further includes: an original data verification module;
[0039] The original data verification module is used to perform data verification on the original data based on the original data verification rules, generate an original data verification result file, and store it in the original data verification result directory file, wherein the original data verification result file is in XML format;
[0040] After the derived data processing module and before the published data processing module, it further includes: a derived data verification module;
[0041] The derived data verification module is used to perform data verification on the derived data based on the derived data verification rules, generate a derived data verification result file, and store it in a derived data verification result directory file, wherein the derived data verification result file is in XML format.
[0042] Preferably,
[0043] After the release data processing module, it further includes: a training data processing module, a verification data processing module, a verification data checking module and a determination module;
[0044] The training data processing module is used to generate the training data based on the published data and the training data definition file, and store it in the training data directory file, wherein the training data is an XML format file, the training data definition file includes: a training data directory naming rule and a training data file naming rule, and the training data directory includes: a third doctor-patient dialogue directory and a third admission medical record directory;
[0045] The verification data processing module is used to generate the verification data based on the published data and the verification data definition file, and store it in the verification data directory file, wherein the verification data is an XML format file, the verification data definition file includes: verification data directory naming rules, verification data file naming rules and verification data verification rules, and the verification data directory includes: a fourth doctor-patient dialogue directory, a fourth admission medical record directory and a first admission medical record reasoning directory;
[0046] The verification data verification module is used to perform data verification on the first admission medical record reasoning data based on the verification data verification rule, generate a verification data verification result file, and store it in the verification data verification result directory file, wherein the verification data verification result file is in XML format;
[0047] The determination module is used to determine whether it is necessary to fine-tune the parameters of the large model based on the verification data verification result.
[0048] Preferably,
[0049] After the verification data checking module and before the determination module, it further includes: a test data processing module;
[0050] The test data processing module is used to generate the test data based on the release data and the test data definition file, and store it in the test data directory file, wherein the test data is an XML format file, the test data definition file includes: a test data directory naming rule, a test data file naming rule and a test data verification rule, and the test data directory includes: a fifth doctor-patient dialogue directory, a fifth admission medical record directory and a second admission medical record reasoning directory;
[0051] After the determination module, it further includes: a fine-tuning module and a test data verification module;
[0052] The fine-tuning module is used to fine-tune the parameters of the large model to generate a target large model when it is determined that the parameters of the large model need to be fine-tuned;
[0053] The test data verification module is used to perform data verification on the second admission medical record inference data based on the test data verification rules, generate a test data verification result file, and store it in the test data verification result directory file, wherein the test data verification result file is in XML format.
[0054] In a third aspect, an embodiment of the present invention provides a system for generating a fine-tuning data set based on data lineage and a large model, including: at least one memory and at least one processor;
[0055] The at least one memory is used to store a machine-readable program;
[0056] The at least one processor is used to call the machine-readable program to execute any method described in the first aspect of claim 1.
[0057] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having computer instructions stored thereon, wherein when the computer instructions are executed by a processor, the processor executes any one of the methods described in the first aspect of claim 1.
[0058] The embodiment of the present invention provides a method and system for generating a fine-tuning data set based on data lineage and a large model. By collecting business data from various regions and hospitals to generate original data, derived data, published data, training data, and verification data in the fine-tuning training process, the data management problem in the fine-tuning training of medical records generation is solved, including a series of data quality management problems and data utilization efficiency problems such as the collection and management of original data, the generation and tracking of derived data, and the construction and tracking of fine-tuning data sets. At the same time, by classifying by directory, continuously tracking and optimizing data by directory files, and constructing fine-tuning data sets according to project requirements, the quality of the data set generated by medical records is improved, the time for on-demand construction of fine-tuning data sets is shortened, the efficiency of fine-tuning training is improved, and ultimately the overall improvement of the quality of fine-tuning training is achieved. In the above manner, the doctor-patient conversation data and the corresponding admission medical record data can be quickly compared and evaluated according to the directory and file name, thereby improving the efficiency of data evaluation. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0060] Figure 1 is a flow chart of a method for generating a fine-tuning data set based on data lineage and a large model provided by an embodiment of the present invention;
[0061] Figure 2 is a flowchart of another method for generating a fine-tuning data set based on data lineage and a large model provided by an embodiment of the present invention;
[0062] Figure 3 is a schematic diagram of a system for generating a fine-tuning data set based on data lineage and a large model provided by an embodiment of the present invention;
[0063] Figure 4 is a schematic diagram of another system for generating a fine-tuning data set based on data lineage and a large model provided by an embodiment of the present invention;
[0064] Figure 5 is a schematic diagram of another system for generating a fine-tuning data set based on data lineage and a large model provided by an embodiment of the present invention;
[0065] Figure 6 It is a schematic diagram of another system for generating a fine-tuning data set based on data lineage and a large model provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0067] like Figure 1 As shown, an embodiment of the present invention provides a method for generating a fine-tuning data set based on data lineage and a large model. The method may include the following steps:
[0068] Step 101: collecting business data of at least one region and hospital, wherein the business data includes: pending doctor-patient conversations and pending hospital admission medical records;
[0069] Step 102: Generate original data based on the business data and the original data definition file, and store it in the original data directory, wherein the original data definition file is an XML format file including original directory classification management rules, original directory naming rules and original file naming rules, and the original data directory includes: the first doctor-patient dialogue directory and the first hospitalization medical record directory;
[0070] Step 103: Based on the original data and the derived data definition file, generate derived data corresponding to at least one dialect version of the original data, and store them in a derived data directory, wherein the derived data definition file is an XML format file including derived directory classification management rules, derived directory naming rules and derived file naming rules, and the derived data directory includes: a second doctor-patient dialogue directory and a second inpatient medical record directory; Step 104: Based on the derived data, the published data definition file and project requirements, generate published data, wherein the published data is a fine-tuning data set including training data and verification data, the training data is used to train the large model, and the verification data is used to fine-tune the parameters of the large model based on the training results.
[0071] In an embodiment of the present invention, a method for generating a fine-tuning data set based on data lineage and a large model is provided. By collecting business data from various regions and hospitals to generate original data, derived data, published data, training data, and verification data in the fine-tuning training process, the data management problem in the fine-tuning training of medical record generation is solved, including a series of data quality management problems and data utilization efficiency problems such as the collection and management of original data, the generation and tracking of derived data, and the construction and tracking of fine-tuning data sets. At the same time, by classifying by directory, continuously tracking and optimizing data by directory files, and constructing fine-tuning data sets according to project requirements, the quality of the data set generated by medical records is improved, the time for on-demand construction of fine-tuning data sets is shortened, the efficiency of fine-tuning training is improved, and ultimately the overall improvement of the quality of fine-tuning training is achieved. In the above manner, the doctor-patient conversation data and the corresponding admission medical record data can be quickly compared and evaluated according to the directory and file name, thereby improving the efficiency of data evaluation.
[0072] Specifically, the original data is the annotated data obtained by blurring the business data of different regions and hospitals. It is a pair of doctor-patient conversations and hospital admission records. These data need to be continuously optimized and improved. The original data needs to support tracking to derived data according to directories and files in order to improve data quality. Therefore, the "Original Data Definition.xml" file can be defined to achieve quality management and classification management of the original data; each original data directory is classified according to administrative regions, hospital departments and dialect types, so that it can be extracted later as needed.
[0073] Derived data is a dataset of various dialect versions derived from the original data, such as Jinan dialect, Qingdao dialect, Weifang dialect, and Northeast dialect. Derived data needs to support tracking to the original data by directory and file in order to optimize the data uniformly. At the same time, derived data needs to support extracting one or more derived versions to build a personalized project dataset. Therefore, the "derived data definition.xml" file can be defined to implement quality management and classification management of derived data.
[0074] Release data is a subset of derived data extracted and constructed based on project requirements. Release data needs to support tracking management according to the release data directory name and file name. Therefore, you can define the "Release Data Definition.xml" file to implement tracking management of release data.
[0075] For example, the original data can be set as follows:
[0076] Naming rules for raw data directories:
[0077] (1) The root directory is named [01-Original Data];
[0078] (2) Create two project collection directories under the above directory to store the original doctor-patient conversation data and the corresponding admission medical record data. (The naming rule of the project collection directory is: directory flow + hospital department + dialect flow + dialect description, such as: [D0001-xx hospital_xx department-P001-Mandarin], [D0002-yy hospital_yy department-P001-Mandarin]);
[0079] (3) Under the above-mentioned project collection directory, two sub-directories are planned: [01-First doctor-patient conversation data] and [02-First hospital admission medical record data] (file naming convention: serial number + disease name, for doctor-patient conversation data and corresponding hospital admission medical record data, the same-name files are used to store them in the project collection directory, such as: [F00001-xxaa disease.txt], [F00002-xxbb disease.txt]). The doctor-patient conversation and the corresponding hospital admission medical record can be quickly compared according to the directory name and file name, thereby improving the efficiency of data evaluation and data optimization.
[0080] Derived data can be set as follows:
[0081] Derived data directory naming convention:
[0082] (1) Name the root directory of derived data [02-derived data];
[0083] (2) Create project collection directories under the above two sub-directories respectively. The directory naming rule adopts directory sequence + hospital department + dialect sequence + dialect description to track and trace the data source, such as: [D0001-xx hospital_xx department-P002-Jinan dialect], [D0002-yy hospital_yy department-P002-Jinan dialect];
[0084] (3) Under the above-mentioned project collection directory, two sub-directories are planned: [01-Second doctor-patient conversation data] and [02-Second hospital admission medical record data], which respectively store the derived doctor-patient conversation data and the corresponding hospital admission medical record data.
[0085] The following settings can be made for the published data:
[0086] Naming rules for the published data directory:
[0087] (1) The root directory of the published data is named [03 - Published Data];
[0088] (2) Under the above root directory, create a project publication directory. The directory naming rule is the publication description + publication date to facilitate subsequent tracking and tracing of the source of the published data. For example: [Shandong xx Hospital - Mandarin_Jinan Dialect - 20240510], [Tibet yy Hospital - Mandarin_Tibetan - 20240610];
[0089] (3) Under the project publication directory, create two subdirectories: [01 - Doctor-Patient Conversation Data] and [02 - Admission Medical Record Data] to store doctor-patient conversation data and the corresponding admission medical records respectively.
[0090] File naming rules:
[0091] For doctor-patient conversation data and the corresponding admission medical record data, use the same file name. The file naming rule is: directory serial number + dialect serial number + file serial number + disease name, which can quickly compare and evaluate doctor-patient conversation data and the corresponding admission medical record data according to the directory and file name, thereby improving the efficiency of data evaluation and data modification and optimization. For example:
[0092]
D0001-P001-F00001-Male_66_xxaa disease.txt
[0093]
D0001-P001-F00002-Female_54_xxbb disease.txt
[0094]
D0002-P002-F00001-Male_77_xxcc disease.txt
[0095] In order to perform data verification on the original data, in an embodiment of the present invention, after step 102 and before step 103 in the above embodiment, it further includes:
[0096] Based on the original data verification rules, perform data verification on the original data, generate an original data verification result file, and store it in the original data verification result directory file, where the original data verification result file is in xml format;
[0097] In an embodiment of the present invention, the original data verification rules can be set in advance, and the original data can be verified according to the original data verification rules to generate an "original data verification result.xml" file. The verification result can be used to mark erroneous data, guide data quality improvement, and avoid misuse in the fine-tuning training link, which affects the quality of fine-tuning training.
[0098] For example, the raw data verification rules describe the composition and constraints of the raw data files (doctor-patient conversations and hospital admission records). Taking hospital admission records as an example, the following raw data verification rules can be set: chief complaint (not empty), current medical history (not empty), past medical history (not empty), menstrual and reproductive history (optional for men), marital history (optional), family history (not empty), etc.
[0099] In order to perform data verification on the derived data, in one embodiment of the present invention, the derived data definition file further includes: derived data verification rules; the derived data directory further includes: derived data verification result directory; after step 103 and before step 104, further including:
[0100] Based on the derived data verification rule, the derived data is verified, a derived data verification result file is generated, and stored in a derived data verification result directory file, wherein the derived data verification result file is in XML format.
[0101] In an embodiment of the present invention, derived data verification rules can be set in advance, and the derived data can be verified according to the derived data verification rules to generate a "derived data verification result.xml" file. The verification result can be used to mark erroneous data, guide data quality improvement, and avoid misuse in the fine-tuning training link, which affects the quality of fine-tuning training.
[0102] In order to fine-tune the large model, in one embodiment of the present invention, after step 104, the above embodiment further includes:
[0103] Based on the published data and the training data definition file, the training data is generated and stored in a training data directory file, wherein the training data is an XML format file, the training data definition file includes: a training data directory naming rule and a training data file naming rule, and the training data directory includes: a third doctor-patient dialogue directory and a third hospitalization medical record directory;
[0104] Based on the published data and the verification data definition file, the verification data is generated and stored in a verification data directory file, wherein the verification data is an XML format file, the verification data definition file includes: a verification data directory naming rule, a verification data file naming rule and a verification data verification rule, and the verification data directory includes: a fourth doctor-patient dialogue directory, a fourth admission medical record directory and a first admission medical record reasoning directory;
[0105] Based on the verification data verification rule, the first admission medical record reasoning data is verified, a verification data verification result file is generated, and stored in the verification data verification result directory file, wherein the verification data verification result file is in XML format;
[0106] Based on the verification result of the validation data, it is determined whether the parameters of the large model need to be fine-tuned.
[0107] In the embodiment of the present invention, the training data is a data subset extracted proportionally from the same-name published data directory, and needs to support tracking management according to the published data directory name and file name. Therefore, a "training data definition.xml" file can be defined to implement tracking management of the training data.
[0108] Verification data is a subset of data extracted proportionally from the published data directory with the same name, and needs to support tracking and management according to the published data directory name and file name. Therefore, a "Verification Data Definition.xml" file can be defined to implement tracking and management of verification data. During fine-tuning training, the inference results of the verification data (first admission medical record data) are verified according to the verification data verification rules to generate "Verification Data Verification Result.xml". The verification result can be used to evaluate the current training effect and determine whether it is necessary to adjust parameters to continue training.
[0109] Specifically, the naming rules of the training data directory included in the training data definition can be set as follows:
[0110] (1) The root directory of the training data is named [04-training data];
[0111] (2) Under the above root directory, inherit the project release directory of the release data, such as:
[0112] [Shandong xx Hospital-Mandarin_Jinan dialect-20240510];
[0113]
Tibet YY Hospital-Mandarin_Tibetan-20240610
[0114] (3) Under the project release directory, two sub-directories are inherited: [01-Third Doctor-Patient Conversation Data] and [02-Third Hospital Admission Medical Record Data], which store the doctor-patient conversation data and the corresponding hospital admission medical record respectively.
[0115] Specifically, the directory naming rules included in the verification data definition can be set as follows:
[0116] Verify the data directory naming rules:
[0117] (1) The root directory of the verification data is named [05-Verification Data]
[0118] (2) Under the above root directory, inherit the project release directory of the release data, such as:
[0119] [Shandong xx Hospital-Mandarin_Jinan dialect-20240510]
[0120]
Tibet YY Hospital-Mandarin_Tibetan-20240610
[0121] (3) Under the project release directory, two sub-directories are inherited: [01-Fourth doctor-patient dialogue data] and [02-Fourth hospital admission medical record data], and a new sub-directory [03, first hospital admission medical record data (from inference results)] is added to store the doctor-patient dialogue data, the corresponding hospital admission medical record, and the corresponding hospital admission medical record-verification data inference results.
[0122] In order to evaluate the training effect, in one embodiment of the present invention, in the above embodiment, after the step of performing data verification on the first admission medical record inference data based on the verification data verification rule, generating a verification data verification result file, and storing it in the verification data verification result directory file, before the step of determining whether it is necessary to fine-tune the parameters of the large model based on the verification data, further includes:
[0123] Based on the release data and the test data definition file, the test data is generated and stored in a test data directory file, wherein the test data is an XML format file, the test data definition file includes: a test data directory naming rule, a test data file naming rule and a test data verification rule, and the test data directory includes: a fifth doctor-patient dialogue directory, a fifth admission medical record directory and a second admission medical record reasoning directory;
[0124] After determining whether it is necessary to fine-tune the parameters of the large model based on the verification data verification result in step 1, the method further includes:
[0125] When it is determined that the parameters of the large model need to be fine-tuned, fine-tuning the parameters of the large model to generate a target large model;
[0126] Based on the test data verification rules, the second admission medical record inference data is verified, a test data verification result file is generated, and stored in the test data verification result directory file, wherein the test data verification result file is in XML format.
[0127] In the embodiment of the present invention, the test data is a subset of data extracted proportionally from the same-name published data directory, and needs to support tracking and management according to the published data directory name and file name. Therefore, a "test data definition.xml" file can be defined to implement tracking and management of the test data. After the fine-tuning training is completed, the inference results of the test data can be verified according to the test data verification rules to generate a "test data verification result.xml" file, and the verification results can be used to evaluate the current training effect and evaluate the training effect.
[0128] For example, the test data can be set as follows;
[0129] Test data directory naming rules:
[0130] (1) The root directory is named [05-Verification Data];
[0131] (2) Under the above root directory, inherit the project release directory of the release data, such as:
[0132] [Shandong xx Hospital-Mandarin_Jinan dialect-20240510];
[0133]
Tibet YY Hospital-Mandarin_Tibetan-20240610
[0134] (3) Under the project release directory, two sub-directories are inherited: [01-Fifth doctor-patient dialogue data] and [02-Fifth hospital admission medical record data], and a new sub-directory [03, Second hospital admission medical record data (from inference results)] is added to store the doctor-patient dialogue data, the corresponding hospital admission medical record, and the corresponding hospital admission medical record-test data inference results.
[0135] like Figure 2 To more clearly illustrate the technical solution and advantages of the present invention, a method for generating a fine-tuning data set based on data lineage and a large model is described in detail below. Specifically, the method may include the following steps:
[0136] Step 201: collecting business data of at least one region and hospital, wherein the business data includes: pending doctor-patient dialogues and pending hospital admission medical records;
[0137] Step 202: Generate original data based on the business data and the original data definition file, and store it in the original data directory, wherein the original data definition file is an XML format file including original directory classification management rules, original directory naming rules, original file naming rules and original data verification rules, and the original data directory includes: original data verification result directory, first doctor-patient dialogue directory and first admission medical record directory;
[0138] Step 203: Based on the original data verification rule, the original data is verified, an original data verification result file is generated, and the original data verification result file is stored in the original data verification result directory file, wherein the original data verification result file is in XML format;
[0139] Step 204: Based on the original data and the derived data definition file, generate derived data corresponding to at least one dialect version of the original data, and store them in a derived data directory, wherein the derived data definition file is an XML format file including derived directory classification management rules, derived directory naming rules, derived file naming rules and derived data verification rules, and the derived data directory includes: a derived data verification result directory, a second doctor-patient dialogue directory and a second admission medical record directory;
[0140] Specifically, when there is a need to serve medical record generation projects in many different regions, it is necessary to select certain types of derived data (such as Mandarin and Tibetan, serving hospitals in Tibetan areas) according to the requirements of different regions or hospitals for fine-tuning training, which creates the need to build a personalized project data set.
[0141] Step 205: Based on the derived data verification rule, perform data verification on the derived data, generate a derived data verification result file, and store it in the derived data verification result directory file, wherein the derived data verification result file is in XML format;
[0142] Step 206: Generate release data based on the derived data, the release data definition file and the project requirements, wherein the release data is a fine-tuning data set including training data, verification data and test data, the training data is used to train the large model, and the verification data is used to fine-tune the parameters of the large model based on the training results;
[0143] Specifically, the published data can be split into training set, validation set and test set in a certain ratio (such as 96:2:2) for use in large model training.
[0144] Step 207: Generate training data based on the published data and the training data definition file, and store it in a training data directory file, wherein the training data is an XML format file, the training data definition file includes: a training data directory naming rule and a training data file naming rule, and the training data directory includes: a third doctor-patient dialogue directory and a third hospital admission medical record directory;
[0145] Step 208: Based on the published data and the verification data definition file, verification data is generated and stored in a verification data directory file, wherein the verification data is an XML format file, the verification data definition file includes: a verification data directory naming rule, a verification data file naming rule and a verification data verification rule, and the verification data directory includes: a fourth doctor-patient dialogue directory, a fourth admission medical record directory and a first admission medical record reasoning directory;
[0146] Step 209: Based on the verification data verification rule, perform data verification on the first admission medical record reasoning data, generate a verification data verification result file, and store it in the verification data verification result directory file, wherein the verification data verification result file is in XML format;
[0147] Step 210: Generate test data based on the published data and the test data definition file, and store it in the test data directory file, wherein the test data is an XML format file, the test data definition file includes: a test data directory naming rule, a test data file naming rule and a test data verification rule, and the test data directory includes: a fifth doctor-patient dialogue directory, a fifth admission medical record directory and a second admission medical record reasoning directory;
[0148] Step 211: Determine whether it is necessary to fine-tune the parameters of the large model based on the verification data verification result;
[0149] Step 212: When it is determined that the parameters of the large model need to be fine-tuned, the parameters of the large model are fine-tuned to generate a target large model;
[0150] Step 213: Based on the test data verification rules, perform data verification on the second admission medical record inference data, generate a test data verification result file, and store it in the test data verification result directory file, wherein the test data verification result file is in XML format.
[0151] like Figure 3 As shown, an embodiment of the present invention provides a system for generating a fine-tuning data set based on data lineage and a large model, the system comprising:
[0152] Collection module 301: used to collect business data of at least one region and hospital, wherein the business data includes: pending doctor-patient dialogues and pending hospital admission medical records;
[0153] The original data processing module 302 is used to generate original data based on the business data and the original data definition file collected by the collection module 301, and store it in the original data directory, wherein the original data definition file is an XML format file including original directory classification management rules, original directory naming rules and original file naming rules, and the original data directory includes: a first doctor-patient dialogue directory and a first hospitalization medical record directory;
[0154] The derived data processing module 303 is used to generate derived data corresponding to at least one dialect version of the original data based on the original data and the derived data definition file generated by the original data processing module 302, and store the derived data in a derived data directory, wherein the derived data definition file is an XML format file including derived directory classification management rules, derived directory naming rules and derived file naming rules, and the derived data directory includes: a second doctor-patient dialogue directory and a second hospitalization medical record directory;
[0155] Release data processing module 304: used to generate release data based on the derived data generated by the derived data processing module 303, the release data definition file and the project requirements, wherein the release data is a fine-tuning data set including training data and verification data, the training data is used to train the large model, and the verification data is used to fine-tune the parameters of the large model based on the training results.
[0156] based on Figure 3 A system for generating fine-tuning datasets based on data lineage and large models is shown in Figure 4 As shown, after the original data processing module 302 and before the derived data processing module 303, it further includes: an original data verification module 305;
[0157] The original data verification module 305 is used to perform data verification on the original data based on the original data verification rules, generate an original data verification result file, and store it in an original data verification result directory file, wherein the original data verification result file is in XML format.
[0158] After the derived data processing module 303 and before the published data processing module 304, it further includes: a derived data verification module 306;
[0159] The derived data verification module 306 is used to perform data verification on the derived data based on the derived data verification rules, generate a derived data verification result file, and store it in a derived data verification result directory file, wherein the derived data verification result file is in XML format.
[0160] based on Figure 4 A system for generating a fine-tuning data set based on data lineage and a large model, as shown in 5, further comprises, after the release data processing module 304, a training data processing module 307, a verification data processing module 308, a verification data checking module 309 and a determination module 310;
[0161] The training data processing module 307 is used to generate the training data based on the published data and the training data definition file, and store it in the training data directory file, wherein the training data is an XML format file, the training data definition file includes: a training data directory naming rule and a training data file naming rule, and the training data directory includes: a third doctor-patient dialogue directory and a third admission medical record directory;
[0162] The verification data processing module 308 is used to generate the verification data based on the published data and the verification data definition file, and store it in the verification data directory file, wherein the verification data is an XML format file, the verification data definition file includes: verification data directory naming rules, verification data file naming rules and verification data verification rules, and the verification data directory includes: the fourth doctor-patient dialogue directory, the fourth hospitalization medical record directory and the first hospitalization medical record reasoning directory;
[0163] The verification data verification module 309 is used to perform data verification on the first admission medical record reasoning data based on the verification data verification rule, generate a verification data verification result file, and store it in the verification data verification result directory file, wherein the verification data verification result file is in XML format;
[0164] The determination module 310 is used to determine whether it is necessary to fine-tune the parameters of the large model based on the verification data verification result.
[0165] based on Figure 5 A system for generating fine-tuning datasets based on data lineage and large models is shown in Figure 6 As shown,
[0166] After the verification data checking module 309 and before the determination module 310, it further includes: a test data processing module 311;
[0167] The test data processing module 311 is used to generate the test data based on the release data and the test data definition file, and store it in the test data directory file, wherein the test data is an XML format file, the test data definition file includes: a test data directory naming rule, a test data file naming rule and a test data verification rule, and the test data directory includes: a fifth doctor-patient dialogue directory, a fifth admission medical record directory and a second admission medical record reasoning directory;
[0168] After the determination module 310, it further includes: a fine-tuning module 312 and a test data verification module 313;
[0169] The fine-tuning module 312 is used to fine-tune the parameters of the large model to generate a target large model when it is determined that the parameters of the large model need to be fine-tuned.
[0170] The test data verification module 313 is used to perform data verification on the second admission medical record inference data based on the test data verification rules, generate a test data verification result file, and store it in the test data verification result directory file, wherein the test data verification result file is in XML format.
[0171] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on a system for generating a fine-tuning dataset based on data lineage and a large model. In other embodiments of the present invention, a system for generating a fine-tuning dataset based on data lineage and a large model may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0172] The information interaction, execution process and other contents between the units in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.
[0173] An embodiment of the present invention also provides a system for generating a fine-tuning data set based on data lineage and a large model, comprising: at least one memory and at least one processor;
[0174] at least one memory for storing a machine-readable program;
[0175] At least one processor is used to call a machine-readable program to execute a method for generating a fine-tuning data set based on data lineage and a large model in any embodiment of the present invention.
[0176] An embodiment of the present invention further provides a computer-readable medium having computer instructions stored thereon. When the computer instructions are executed by a processor, the processor executes a method for generating a fine-tuning data set based on data lineage and a large model in any embodiment of the present invention.
[0177] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.
[0178] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
[0179] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.
[0180] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.
[0181] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.
[0182] Each embodiment of the present invention has at least the following beneficial effects:
[0183] 1. In an embodiment of the present invention, a method for generating a fine-tuning dataset based on data lineage and a large model is provided. By collecting business data from various regions and hospitals to generate original data, derived data, published data, training data, and verification data in the fine-tuning training process, the data management problem in the fine-tuning training of medical record generation is solved, including a series of data quality management issues and data utilization efficiency issues such as the collection and management of original data, the generation and tracking of derived data, and the construction and tracking of fine-tuning datasets. At the same time, by classifying by directory, continuously tracking and optimizing data by directory files, and constructing fine-tuning datasets according to project requirements, the quality of the dataset generated by medical records is improved, the time for on-demand construction of fine-tuning datasets is shortened, the efficiency of fine-tuning training is improved, and ultimately the overall improvement of the quality of fine-tuning training is achieved. In the above manner, the doctor-patient conversation data and the corresponding admission medical record data can be quickly compared and evaluated according to the directory and file name, thereby improving the efficiency of data evaluation;
[0184] 2. In the embodiment of the present invention, the original data verification rules can be pre-set, and the original data can be verified according to the original data verification rules to generate "original data verification result.xml". The verification result can be used to mark erroneous data, guide data quality improvement, and avoid misuse in the fine-tuning training link, which affects the quality of fine-tuning training;
[0185] 3. In an embodiment of the present invention, derived data verification rules can be set in advance, and according to the derived data verification rules, the derived data can be verified to generate "derived data verification results.xml". The verification results can be used to mark erroneous data, guide data quality improvement, and avoid misuse in the fine-tuning training link, which affects the quality of fine-tuning training.
[0186] It should be noted that not all steps and modules in the above-mentioned processes and system structure diagrams are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.
[0187] In the above embodiments, the hardware unit can be realized by mechanical means or electrical means. For example, a hardware unit can include permanent dedicated circuits or logic (such as special processors, FPGA or ASIC) to complete the corresponding operation. The hardware unit can also include programmable logic or circuits (such as general-purpose processors or other programmable processors), which can be temporarily set by software to complete the corresponding operation. Concrete implementation (mechanical means or dedicated permanent circuits or temporarily set circuits) can be determined based on cost and time considerations.
[0188] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for generating a fine-tuning dataset based on data lineage and a large model, characterized in that: The method includes: A1: Collecting business data of at least one region and hospital, wherein the business data includes: pending doctor-patient conversations and pending hospital admission medical records; A2: Based on the business data and the original data definition file, the original data is generated and stored in the original data directory, wherein the original data definition file is an XML format file including the original directory classification management rules, the original directory naming rules and the original file naming rules, and the original data directory includes: the first doctor-patient dialogue directory and the first hospitalization medical record directory; A3: Based on the original data and the derived data definition file, generate derived data corresponding to at least one dialect version of the original data, and store them in a derived data directory, wherein the derived data definition file is an XML format file including derived directory classification management rules, derived directory naming rules and derived file naming rules, and the derived data directory includes: a second doctor-patient dialogue directory and a second hospitalization medical record directory; A4: Generate release data based on the derived data, release data definition file and project requirements, wherein the release data is a fine-tuning data set including training data and verification data, the training data is used to train the large model, and the verification data is used to fine-tune the parameters of the large model based on the training results.
2. The method according to claim 1, characterized in that The original data definition file also includes: original data verification rules; The original data directory also includes: an original data verification result directory; After A2 and before A3, further comprising: Based on the original data verification rule, the original data is verified, an original data verification result file is generated, and the original data verification result file is stored in the original data verification result directory file, wherein the original data verification result file is in XML format; The derived data definition file also includes: derived data verification rules; The derived data directory also includes: a derived data verification result directory; After A3 and before A4, further comprising: Based on the derived data verification rule, the derived data is verified, a derived data verification result file is generated, and stored in a derived data verification result directory file, wherein the derived data verification result file is in XML format.
3. The method according to claim 1, characterized in that After A4, it further includes: Based on the published data and the training data definition file, the training data is generated and stored in a training data directory file, wherein the training data is an XML format file, the training data definition file includes: a training data directory naming rule and a training data file naming rule, and the training data directory includes: a third doctor-patient dialogue directory and a third hospitalization medical record directory; Based on the published data and the verification data definition file, the verification data is generated and stored in a verification data directory file, wherein the verification data is an XML format file, the verification data definition file includes: a verification data directory naming rule, a verification data file naming rule and a verification data verification rule, and the verification data directory includes: a fourth doctor-patient dialogue directory, a fourth admission medical record directory and a first admission medical record reasoning directory; Based on the verification data verification rule, the first admission medical record reasoning data is verified, a verification data verification result file is generated, and stored in the verification data verification result directory file, wherein the verification data verification result file is in XML format; Based on the verification result of the validation data, it is determined whether it is necessary to fine-tune the parameters of the large model.
4. The method according to claim 3, characterized in that The release data also includes: test data; After performing data verification on the first admission medical record reasoning data based on the verification data verification rule, generating a verification data verification result file, and storing it in the verification data verification result directory file, before determining whether it is necessary to fine-tune the parameters of the large model based on the verification data, further comprising: Based on the release data and the test data definition file, the test data is generated and stored in a test data directory file, wherein the test data is an XML format file, the test data definition file includes: a test data directory naming rule, a test data file naming rule and a test data verification rule, and the test data directory includes: a fifth doctor-patient dialogue directory, a fifth admission medical record directory and a second admission medical record reasoning directory; After determining whether it is necessary to fine-tune the parameters of the large model based on the verification data verification result, the method further includes: When it is determined that the parameters of the large model need to be fine-tuned, fine-tuning the parameters of the large model to generate a target large model; Based on the test data verification rules, the second admission medical record inference data is verified, a test data verification result file is generated, and stored in the test data verification result directory file, wherein the test data verification result file is in XML format.
5. A system for generating fine-tuning datasets based on data lineage and large models, characterized by: The system includes: Collection module: used to collect business data of at least one region and hospital, wherein the business data includes: pending doctor-patient dialogues and pending hospital admission medical records; The original data processing module is used to generate original data based on the business data and the original data definition file collected by the collection module, and store the original data in the original data directory, wherein the original data definition file is an XML format file including original directory classification management rules, original directory naming rules and original file naming rules, and the original data directory includes: the first doctor-patient dialogue directory and the first hospitalization medical record directory; A derived data processing module: used for generating derived data corresponding to at least one dialect version of the original data based on the original data and the derived data definition file generated by the original data processing module, and storing the derived data in a derived data directory, wherein the derived data definition file is an XML format file including derived directory classification management rules, derived directory naming rules and derived file naming rules, and the derived data directory includes: a second doctor-patient dialogue directory and a second hospitalization medical record directory; Release data processing module: used to generate release data based on the derived data generated by the derived data processing module, the release data definition file and the project requirements, wherein the release data is a fine-tuning data set including training data and verification data, the training data is used to train the large model, and the verification data is used to fine-tune the parameters of the large model based on the training results.
6. The system according to claim 5, characterized in that After the original data processing module and before the derived data processing module, it further includes: an original data verification module; The original data verification module is used to perform data verification on the original data based on the original data verification rules, generate an original data verification result file, and store it in the original data verification result directory file, wherein the original data verification result file is in XML format; After the derived data processing module and before the published data processing module, it further includes: a derived data verification module; The derived data verification module is used to perform data verification on the derived data based on the derived data verification rules, generate a derived data verification result file, and store it in a derived data verification result directory file, wherein the derived data verification result file is in XML format.
7. The system according to claim 5, characterized in that After the release data processing module, it further includes: a training data processing module, a verification data processing module, a verification data checking module and a determination module; The training data processing module is used to generate the training data based on the published data and the training data definition file, and store it in the training data directory file, wherein the training data is an XML format file, the training data definition file includes: a training data directory naming rule and a training data file naming rule, and the training data directory includes: a third doctor-patient dialogue directory and a third admission medical record directory; The verification data processing module is used to generate the verification data based on the published data and the verification data definition file, and store it in the verification data directory file, wherein the verification data is an XML format file, the verification data definition file includes: verification data directory naming rules, verification data file naming rules and verification data verification rules, and the verification data directory includes: a fourth doctor-patient dialogue directory, a fourth admission medical record directory and a first admission medical record reasoning directory; The verification data verification module is used to perform data verification on the first admission medical record reasoning data based on the verification data verification rule, generate a verification data verification result file, and store it in the verification data verification result directory file, wherein the verification data verification result file is in XML format; The determination module is used to determine whether it is necessary to fine-tune the parameters of the large model based on the verification data verification result.
8. The system according to claim 7, characterized in that After the verification data checking module and before the determination module, it further includes: a test data processing module; The test data processing module is used to generate the test data based on the release data and the test data definition file, and store it in the test data directory file, wherein the test data is an XML format file, the test data definition file includes: a test data directory naming rule, a test data file naming rule and a test data verification rule, and the test data directory includes: a fifth doctor-patient dialogue directory, a fifth admission medical record directory and a second admission medical record reasoning directory; After the determination module, it further includes: a fine-tuning module and a test data verification module; The fine-tuning module is used to fine-tune the parameters of the large model to generate a target large model when it is determined that the parameters of the large model need to be fine-tuned; The test data verification module is used to perform data verification on the second admission medical record inference data based on the test data verification rules, generate a test data verification result file, and store it in the test data verification result directory file, wherein the test data verification result file is in XML format.
9. A system for generating fine-tuning datasets based on data lineage and large models, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 4.
10. A computer readable medium, characterized in that The computer readable medium stores computer instructions, which, when executed by a processor, cause the processor to perform any one of the methods of claims 1 to 4.