Automatic iteration method and system of medical identification and diagnosis model and storage medium

Abstract

The invention provides an automatic iteration method and system for a medical identification and diagnosis model and a storage medium, and the method comprises the steps: obtaining medical medical record data used for model iteration and updating in response to the detection that a preset iteration period is met, generating high-quality medical instruction data according to the medical medical record data and a large language model, and carrying out the automatic iteration of the medical identification and diagnosis model. The method comprises the following steps: performing fine tuning training on a to-be-iterated medical identification and diagnosis model on the basis of medical instruction data to obtain an iterated medical identification and diagnosis model, then scoring the medical identification and diagnosis models before and after iteration on the basis of a large language model, and selecting a target model from model performance scores so as to facilitate medical identification and diagnosis by using the target model. And the target model is used as the to-be-iterated medical identification and diagnosis model in the next iteration process, so that automatic iteration optimization of the model is realized, the availability and practicability of the model are improved, the model can quickly respond to dynamic changes of business requirements, errors gradually generated in the iteration process can be avoided, and the question and answer effect of the model is ensured.

Description

Technical Field

[0001] The present application relates to the field of medical question-answering technology, and in particular to an automatic iteration method, system and storage medium of a medical diagnosis model. Background Art

[0002] In recent years, with the rapid development of artificial intelligence and big model technology, various fields have gradually adopted data-driven methods to solve complex problems. In the medical field, artificial intelligence-assisted diagnosis has gradually been applied to doctors' diagnosis and treatment, and intelligent diagnosis and treatment technology based on big models has continued to develop.

[0003] However, when using large models to solve problems in the medical field, complex variables are often involved, such as the patient's medical history, clinical symptoms, examination results, etc., and the relationship between these variables may be nonlinear. The model obtained by one training may not be perfect, especially when dealing with a small number of samples, rare diseases or complex clinical conditions, which leads to low accuracy of the large model's answers and cannot meet the needs of real applications.

[0004] In addition, the development cycle of traditional model training usually requires manual adjustments and cross-team collaboration, which has a long development cycle, affecting model performance and making it impossible to quickly respond to dynamic changes in business needs. Summary of the invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, the present application aims to provide an automatic iteration method, system and storage medium for a medical diagnosis model to solve the problems in the prior art such as low accuracy of model answers, long development cycle, and inability to quickly respond to dynamic changes in business needs.

[0006] The present application embodiment provides an automatic iteration method of a medical diagnosis model, the method comprising:

[0007] In response to detecting that a preset iteration cycle is met, obtaining medical record data for iterative updating of the model, and generating medical instruction data based on the medical record data and the large language model;

[0008] Acquire a medical diagnosis model to be iterated, and fine-tune the medical diagnosis model to be iterated based on the medical instruction data to obtain an iterated medical diagnosis model;

[0009] Determine the model performance scores of the medical diagnosis model to be iterated and after the iteration based on the large language model, and select a target model from the medical diagnosis model to be iterated and after the iteration according to the model performance scores of the medical diagnosis model to be iterated and after the iteration;

[0010] The target model is used for medical diagnosis and is used as a medical diagnosis model to be iterated in the next iteration process.

[0011] Optionally, generating medical instruction data based on the medical record data and the large language model includes:

[0012] Based on the medical case record data and the large language model, obtain medical instruction data and diagnostic evidence corresponding to each answer text in the medical instruction data;

[0013] The diagnostic thought process text corresponding to the answer text is extracted from the diagnostic basis, and the answer text is updated based on the diagnostic thought process text.

[0014] Optionally, based on the medical case record data and the large language model, obtaining medical instruction data and diagnostic evidence corresponding to each answer text in the medical instruction data includes:

[0015] Get the preset model prompt text;

[0016] Inputting the preset model prompt text and the medical record data into the large language model, so that the large language model generates medical instruction data that meets the preset model prompt text, and outputs the diagnosis basis corresponding to each answer text in the medical instruction data;

[0017] The preset model prompt text includes that false information cannot be constructed, the number of words in the answer text exceeds the preset number of words, and the format of the answer text meets the preset format.

[0018] Optionally, after generating the medical instruction data based on the medical record data and the large language model, the method further includes:

[0019] For each question-answer pair in the medical instruction data, based on a pre-trained medical term extraction model, medical term location is performed on the answer text and the corresponding medical record data in the question-answer pair to obtain generated data corresponding to the medical term in the answer text and real data corresponding to the medical term in the medical record data;

[0020] Determine whether the generated data is consistent with the real data; if not, remove the question-answer pair.

[0021] Optionally, fine-tuning the medical diagnosis model to be iterated based on the medical instruction data includes:

[0022] Determining the complexity of each question-answer pair in the medical instruction data, and dividing the medical instruction data into a plurality of sample sets according to the complexity of each question-answer pair;

[0023] Sort all sample sets in order of complexity from low to high, and use the first sample set as the current training set;

[0024] The medical diagnosis model to be iterated is fine-tuned using the current training set, and after the fine-tuning training is completed, the next sample set of the current training set is used as the new current training set, and the step of fine-tuning the medical diagnosis model to be iterated using the current training set is returned to, until the current training set is the last sample set.

[0025] Optionally, determining model performance scores of the medical diagnosis model to be iterated and after iteration based on the large language model includes:

[0026] Generate model prompt text based on the answer text output by the medical diagnosis model to be iterated and after iteration, and the template text;

[0027] The model prompt text is input into the large language model to obtain the correctness score, adequacy score, authenticity score and scientificity score of the medical diagnosis model to be iterated and after iteration.

[0028] Optionally, selecting a target model according to the model performance scores of the medical diagnosis model to be iterated and after iteration includes:

[0029] Determine whether the correctness scores of the medical diagnosis model to be iterated and after iteration are lower than a preset score threshold;

[0030] If the correctness scores of the medical diagnosis model to be iterated and after the iteration are both higher than the preset score threshold, or the correctness scores of the medical diagnosis model to be iterated and after the iteration are both lower than the preset score threshold, then based on the adequacy score, authenticity score and scientificity score of the medical diagnosis model to be iterated and after the iteration, determine the fusion score of the medical diagnosis model to be iterated and after the iteration;

[0031] The target model is determined according to the fusion score of the medical diagnosis model to be iterated and the iterated model.

[0032] Optionally, after selecting a target model based on the model performance scores of the medical diagnosis model to be iterated and the iterated model, the following further includes:

[0033] Extract the model reinforcement test set from all medical record data used in historical iterations;

[0034] Inputting the model reinforcement test set into the target model, and determining the correctness score, adequacy score, authenticity score and scientificity score of the target model based on the output of the target model and the large language model;

[0035] If there is at least one score lower than the corresponding set threshold, a feedback data set is constructed based on the model reinforcement test set, wherein the feedback data set is used for the next model iteration update.

[0036] The present application also provides an automatic iteration system for a medical diagnosis model, the system comprising a data processing module, a model fine-tuning module, and a model evaluation module, wherein:

[0037] The data processing module is used to obtain medical record data for model iteration and update in response to detecting that a preset iteration cycle is met, and generate medical instruction data based on the medical record data and the large language model;

[0038] The model fine-tuning module is used to obtain the medical diagnosis model to be iterated, and fine-tune the medical diagnosis model to be iterated based on the medical instruction data to obtain the iterated medical diagnosis model;

[0039] The model evaluation module is used to determine the model performance scores of the medical diagnosis model to be iterated and after the iteration based on the large language model, and select a target model from the medical diagnosis model to be iterated and after the iteration according to the model performance scores of the medical diagnosis model to be iterated and after the iteration;

[0040] The target model is used for medical diagnosis and is used as a medical diagnosis model to be iterated in the next iteration process.

[0041] The present application also provides an electronic device, the electronic device comprising:

[0042] Processor and memory;

[0043] The processor is used to execute the steps of the automatic iteration method of the medical diagnosis model provided in any embodiment of the present application by calling the program or instruction stored in the memory.

[0044] An embodiment of the present application also provides a computer-readable storage medium, which stores a program or instruction, and the program or instruction enables a computer to execute the steps of the automatic iteration method of the medical diagnosis model provided in any embodiment of the present application.

[0045] In summary, the present application proposes an automatic iteration method for a medical diagnosis model. The method responds to a model iteration instruction, obtains medical medical record data for model iteration and updates, and constructs a high-quality medical instruction set based on the medical medical record data and a large language model, thereby obtaining a medical diagnosis model to be iterated, and fine-tuning the medical diagnosis model to be iterated based on the medical instruction data to obtain an iterated medical diagnosis model, thereby determining the model performance scores of the medical diagnosis models to be iterated and after iteration, and selecting a target model from the model performance scores, so as to use the target model for medical diagnosis, and use the target model as the medical diagnosis model to be iterated in the next iteration process. This method can be achieved The integration of training and evaluation of large models meets the application requirements of automatic iterative optimization of models, significantly reduces the cost of manual intervention, improves the availability and practicality of models, and enables models to quickly respond to dynamic changes in business needs. In addition, this method generates medical instruction data through large language models and medical record data, which can solve the problem of scarce high-quality medical data, improve the quality of the model fine-tuning data set, and thus improve the fine-tuning effect of the model and ensure the accuracy of the model's question and answer. In addition, after iterative optimization, this method determines the question and answer scores of the model before and after iteration, thereby selecting the target model for question and answer and participating in the next iteration, which can avoid errors gradually generated during the iteration process and ensure the question and answer effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 It is a flowchart of an automatic iteration method of a medical diagnosis model provided in an embodiment of the present application;

[0048] Figure 2 It is an automatic iteration flow chart of a medical diagnosis model provided in an embodiment of the present application;

[0049] Figure 3 It is a structural schematic diagram of an automatic iteration system of a medical diagnosis model provided in an embodiment of the present application;

[0050] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0052] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0053] Before introducing the method provided in the embodiment of the present application in detail, the technical problem solved by the method is first described.

[0054] In existing technologies, the first trained model is not perfect, especially when dealing with a small number of samples, rare diseases, or clinically complex conditions. The development cycle of traditional model training and evaluation usually requires manual adjustments and cross-team collaboration, which poses the following challenges:

[0055] The surge in data volume: The quality and processing efficiency of large-scale data directly affect model performance;

[0056] Long iteration cycle and low development efficiency: From data preprocessing to model deployment, multiple links are involved, requiring a lot of manual intervention and repeated operations, and the development cycle is long;

[0057] Unable to meet the needs of automated iteration: Unable to quickly respond to dynamic changes in business needs;

[0058] Therefore, in order to solve the above problems, the embodiment of the present application provides an automatic iteration method of a medical diagnosis model, which can realize automatic iteration optimization of the model through an integrated model development process, significantly reducing the cost of manual intervention, improving the availability and practicality of the system, and providing strong technical support for intelligent applications in professional fields. In addition, it solves the problem of the scarcity of high-quality medical field data, effectively improves the quality of medical fine-tuning data, and thus improves the fine-tuning effect of the model.

[0059] As mentioned in the background technology, in response to the problems in the prior art, this application proposes an automatic iteration method for a medical diagnosis model. Figure 1 : is a flowchart of an automatic iteration method of a medical differential diagnosis model provided in an embodiment of the present application, which is suitable for generating a model for performing medical differential diagnosis by question-answering. Figure 1 , the automatic iteration method of the medical diagnosis model specifically includes:

[0060] S110. In response to detecting that a preset iteration cycle is met, obtaining medical case record data for iterative updating of the model, and generating medical instruction data based on the medical case record data and the large language model.

[0061] Among them, the preset iteration cycle can be a pre-set cycle of automatic iteration of the model, which can be determined based on the size of the data increment generated by the outpatient information management system within a set time.

[0062] Exemplarily, the more data increments the outpatient information management system generates within a set time, the shorter the preset iteration cycle. The preset iteration cycle can be dynamically adjusted based on the data increments of the outpatient information management system. In addition to triggering model iteration when the preset iteration cycle is met, the subsequent model iteration process can also be triggered when a new disease is detected in the outpatient information management system.

[0063] Specifically, if it is detected that the preset iteration cycle is met, the medical record data used for model iteration and updating can be obtained first, so as to generate medical instruction data for model training through the medical record data. The medical record data can come from the hospital's outpatient information management system. For example, the server of the hospital's outpatient information management system can be communicated with through HTTP (Hypertext Transfer Protocol), and a request can be sent to the back-end system to obtain the medical record data.

[0064] Exemplarily, obtaining medical record data for iterative updating of the model includes: sending an HTTP request to the outpatient information management system to obtain the medical record data generated by the outpatient information management system within a set period, or obtaining the medical record data of newly added diseases in the outpatient information management system.

[0065] Among them, the requests library can be used to send HTTP requests to the outpatient information management system to obtain medical record data. This process mainly involves API (Application Programming Interface) authentication and authorization and request hyperparameter settings. In the embodiment of the present application, the interface of the connected outpatient information management system can be authenticated and authorized using the API Key method. Therefore, authentication can be performed by passing the API key in the URL of the HTTP request. The setting of the request hyperparameters mainly includes the interface service code and the time period for obtaining medical record data.

[0066] In the embodiment of the present application, considering that the medical record data obtained through the interface may be chaotic, before generating medical instruction data through the medical record data, the medical record data may be screened and formatted. For example, data that is invalid for model fine-tuning, such as no patient complaints, no prescription information, and no preliminary diagnosis information, may be screened out first; further, the screened medical record data may be formatted, for example, the medical record data may be converted into data in JSON (JavaScript Object Notation) format, as shown below:

[0067] {

[0068] "deptName": "",

[0069] "doctext": {

[0070] "Chief complaint": "",

[0071] "Current medical history": "",

[0072] "History": "",

[0073] "Physical Examination": "",

[0074] "Prescription": "",

[0075] "Auxiliary inspection": "",

[0076] "Preliminary diagnosis": ""

[0077] }

[0078] }

[0079] Furthermore, medical instruction data for model fine-tuning can be generated through medical medical record data. In an embodiment of the present application, a large language model can be used to generate medical instruction data. The reason is that in the current medical field, medical tasks are highly professional and complex, involving a large number of medical terms, clinical pathways, treatment plans, symptom diagnosis and other contents. This requires fine-tuning instructions to not only cover a wide range of medical fields, but also have strong context understanding and reasoning capabilities. Therefore, the embodiment of the present application utilizes the excellent language understanding and generation capabilities of the large language model, as well as the medical knowledge in the real medical medical record data, to generate high-quality fine-tuning data. Among them, the large language model can be the DeepSeek-R1 large model, and FastAPI can be used to configure the corresponding interface for this.

[0080] Specifically, medical record data can be input into the large language model so that the large language model can extract multiple question-answer pairs from it to obtain medical instruction data. The question-answer pair consists of a question text and a corresponding answer text. For example, medical record data can be used as the medical knowledge basis, the identity role of a medical expert can be given to the large language model, the analysis steps for constructing question-answer pairs can be provided, and corresponding instruction requirements can be set to drive the large language model to construct question-answer pairs for the consultation scenario.

[0081] In the embodiment of the present application, in order to make the generated answer text more logical and explainable, the large language model can also generate intermediate reasoning step texts during the process of generating question-answer pairs by the large language model, so as to avoid the medical diagnosis model directly giving unreasonable or jumpy answer texts, and make the answers generated by the medical diagnosis model more logical and explainable.

[0082] In a specific implementation, generating medical instruction data based on medical record data and a large language model includes the following steps:

[0083] Step 11: Based on the medical record data and the large language model, obtain the medical instruction data and the diagnostic basis corresponding to each answer text in the medical instruction data;

[0084] Step 12: extract the diagnostic thinking process text corresponding to the answer text from the diagnostic basis, and update the answer text based on the diagnostic thinking process text.

[0085] Among them, in step 11, the medical record data can be first input into the large language model, and the large language model can perform semantic analysis on the medical record data to generate multiple question-answer pairs to constitute medical instruction data; and while the large language model generates question-answer pairs, the large language model can also extract the diagnostic basis corresponding to each answer text from the medical record data.

[0086] Taking into account the possibility that the question-answer pairs output by the large language model may contain false information, in order to ensure the accuracy of subsequent training of the medical diagnosis model, corresponding instruction requirements can be set for the large language model during the process of generating medical instruction data.

[0087] In one example, based on medical case record data and a large language model, obtaining medical instruction data and diagnostic evidence corresponding to each answer text in the medical instruction data includes the following steps:

[0088] Step 111, obtaining a preset model prompt text;

[0089] Step 112: input the preset model prompt text and medical record data into the large language model, so that the large language model generates medical instruction data that meets the preset model prompt text, and outputs the diagnosis basis corresponding to each answer text in the medical instruction data.

[0090] Specifically, in step 111, the preset model prompt text includes that false information cannot be constructed, the number of words in the answer text exceeds the preset number of words, and the format of the answer text meets the preset format. Among them, false information can be content that does not conform to the medical record data, the preset number of words can be a preset minimum number of words in the answer text, such as 800 words, and the preset format can be the format of the medical record data input to the large language model, such as JSON format.

[0091] In step 112, the preset model prompt text can be spliced ​​with the medical record data and then input into the large language model. The large language model performs reasoning to generate medical instruction data that meets the preset model prompt text, and at the same time, outputs the diagnostic basis corresponding to each answer text.

[0092] Through the above steps 111 and 112, the pre-set model prompt conditions can be used to stipulate that the large language model avoids generating false information while generating medical instruction data, thereby ensuring the accuracy of fine-tuning of the subsequent medical diagnosis model.

[0093] After obtaining the medical instruction data and the diagnostic basis corresponding to each answer text in the medical instruction data through the large language model, further, in step 12, the diagnostic thinking process text corresponding to the answer text can be extracted from the diagnostic basis. For example, the diagnostic thinking process text can be extracted from the diagnostic basis through the large language model. Among them, the diagnostic thinking process text can describe the reasoning process of obtaining the answer text from the question text. This thinking chain is formed through the deep thinking, continuous exploration and backtracking of the large language model.

[0094] Furthermore, the answer text can be updated according to the diagnostic thinking process text so that the answer text includes the diagnostic thinking process, that is, the diagnostic thinking process is integrated into the answer text to construct a diagnostic thinking chain.

[0095] Through the above steps 11 and 12, the diagnostic thinking can be integrated into the generated answer text, so that the generated answer text has a reasoning process, and the fine-tuned medical diagnosis model can have stronger reasoning ability.

[0096] Considering that the output of the large language model is standardized by the pre-set model prompt text, the question and answer pairs synthesized by the large language model may still contain false information. In order to further ensure the authenticity of the answer text generated by the large language model and avoid the hallucination problem of the large language model, after obtaining the medical instruction data, the medical instruction data can be further screened to eliminate the question and answer pairs containing false information.

[0097] In a specific implementation, after generating medical instruction data based on medical record data and a large language model, the following steps are also included:

[0098] Step 13: for each question-answer pair in the medical instruction data, based on the pre-trained medical term extraction model, the answer text in the question-answer pair and the corresponding medical record data are respectively located for medical terminology, so as to obtain generated data corresponding to the medical terminology in the answer text and real data corresponding to the medical terminology in the medical record data;

[0099] Step 14: Determine whether the generated data is consistent with the real data. If not, eliminate the question and answer pair.

[0100] The medical term extraction model may be a model for locating medical terms from input text.

[0101] Specifically, in step 13, the answer text and the corresponding medical record data in the question-answer pair can be respectively input into the medical term extraction model, and the medical term extraction model retrieves medical terms in the answer text and the medical record data. The medical terms can be drug names, examination item names, or indicator item names.

[0102] After locating the medical terms, you can further query the generated data corresponding to the medical terms in the answer text, and query the real data corresponding to the medical terms in the medical record data. The generated data corresponding to the medical terms can be the dosage, value or value range corresponding to the medical terms in the answer text, such as the dosage of a certain drug, the part of a certain examination item, and the value range of a certain indicator item; the real data corresponding to the medical terms can be the dosage, value or value range corresponding to the medical terms in the medical record data.

[0103] After retrieving the medical terms and obtaining the generated data and the real data, in step 14, the two can be matched, that is, to determine whether the generated data and the real data are consistent for the same medical term. If not, it means that the answer text generated by the large language model contains false information. At this time, the question-answer pair can be discarded. At the same time, the question-answer pair can be regenerated for the corresponding medical record data.

[0104] Through the above steps 13 and 14, medical terms can be located and the corresponding data can be queried, and then the data can be matched to determine whether the question-answer pairs generated by the large language model contain false information. This can further avoid the model hallucination problem and ensure the accuracy of the medical instruction data, thereby further improving the reliability of subsequent iterative training.

[0105] In an embodiment of the present application, after the generated medical instruction data is screened, considering that in the process of self-generating data through a large language model, erroneous, redundant or invalid data may be generated, and there may be problems in data format and data matching, the medical instruction data can also be pre-processed to improve the quality of the medical instruction data so that the medical diagnosis model can better understand and perform tasks.

[0106] Exemplarily, the medical instruction data is preprocessed, including data cleaning, data partitioning and data matching. Among them, data cleaning can be to identify and remove dirty data (empty data, formatted data, garbled data, etc.), and use hash functions and text embedding to identify and remove redundant data, and then format the medical instruction data. Data partitioning can be to divide the cleaned medical instruction data into training sets and test sets in proportion. Data matching can be to ensure that the medical diagnosis model maintains general task capabilities during the process of professional fine-tuning, and a part of general data is supplemented in the training set to prevent the medical diagnosis model from overfitting. The source of general data can be BELLE, and the ratio of general data to medical instruction data generated based on medical record data can be 1:1.

[0107] S120, obtaining a medical diagnosis model to be iterated, and fine-tuning the medical diagnosis model to be iterated based on the medical instruction data to obtain an iterated medical diagnosis model.

[0108] Specifically, after obtaining the medical instruction data, the medical diagnosis model can be iteratively trained using the medical instruction data. It should be noted that in the process of this iterative training, the medical diagnosis model to be iterated is the target model determined after the last iterative training and evaluation. If this round of iteration is the first iteration, the medical diagnosis model to be iterated used in this iteration can be a general basic model.

[0109] In an embodiment of the present application, the medical diagnosis model to be iterated can be fine-tuned and trained based on medical instruction data through efficient parameter fine-tuning technology, which can reduce the consumption of computing resources, alleviate the catastrophic forgetting problem, and improve the generalization ability of the model.

[0110] For example, using efficient parameter fine-tuning technology can be to fine-tune some parameters of the medical diagnosis model to be iterated, rather than fine-tune all parameters of the model, which can significantly reduce computing resource consumption. For example, LoRA (Low-Rank Adaptation) can be used to achieve efficient parameter updates by introducing low-rank matrices into model weights.

[0111] Among them, the efficient parameter fine-tuning technology effectively alleviates the problem of catastrophic forgetting by retaining most of the original model parameters and reducing the coverage of historical knowledge. Moreover, the model can still maintain its ability to handle historical tasks after fine-tuning, which is especially important in special domain models. In addition, since only some parameters are updated, the medical diagnosis model can be quickly loaded and deployed; in the subsequent reasoning process, seamless fine-tuning updates can be achieved by loading the fine-tuned LoRA weights without redeploying the entire model.

[0112] In an embodiment of the present application, in the same scenario, data of different task types can be collected over time. For example, in the medical field, data of different tasks such as diagnosis, treatment plan recommendation, and drug query can be collected, and model parameters can be dynamically updated through multiple rounds of iterations to ensure that the model can meet business needs in real time.

[0113] Moreover, in each iteration, the current weight of the medical diagnosis model to be iterated can be loaded first to ensure that the model can inherit the existing knowledge and perform iterative updates based on it. Through iterative fine-tuning, the model can gradually learn and adapt to various task types and improve its generalization ability. Each fine-tuning will be based on newly collected task data to further enhance the model's ability to handle diverse tasks. As new task data is continuously added, the model can dynamically adjust its parameters to adapt to new task requirements without losing its ability to handle old tasks.

[0114] In order to further improve the accuracy and efficiency of model iteration, for an iterative update, the medical instruction data can be classified according to the length or complexity of the medical instruction data, and then the model can be trained in multiple rounds in order of length from low to high or complexity from low to high, so as to prioritize simple medical instruction data to preheat the model, and then use complex medical instruction data to optimize the model, simulating the human learning process from easy to difficult, and achieving efficient model capability improvement.

[0115] In a specific implementation, fine-tuning the iterative medical diagnosis model based on medical instruction data includes the following steps:

[0116] Step 21: determining the complexity of each question-answer pair in the medical instruction data, and dividing the medical instruction data into multiple sample sets according to the complexity of each question-answer pair;

[0117] Step 22: Sort all sample sets in order of complexity from low to high, and use the first sample set as the current training set;

[0118] Step 23, use the current training set to fine-tune the iterative medical diagnosis model, and after the fine-tuning training is completed, use the next sample set of the current training set as the new current training set, and return to the step of using the current training set to fine-tune the iterative medical diagnosis model until the current training set is the last sample set.

[0119] In step 21, the complexity of each question-answer pair may be determined first according to the total length of the text, the number of paragraphs, and the number of medical terms of each question-answer pair. For example, when the total length of the text is greater than a set length threshold, the corresponding complexity is queried according to the total length of the text; when the total length of the text is less than the set length threshold, the corresponding complexity is queried according to the number of paragraphs and the number of medical terms.

[0120] Furthermore, the medical instruction data may be divided according to the complexity of each question-answer pair to obtain a plurality of sample sets of different complexity. In step 22, all sample sets may be sorted in order of complexity from low to high, and the first sample set is used as the current training set.

[0121] Furthermore, in step 23, the current training set can be used to perform fine-tuning training on the iterative medical diagnosis model. After the fine-tuning training is completed, on the basis of this fine-tuning training, the next sample set of the sorted current training set can be used as the current training set again, and the process returns to step 22 until all sample sets are used to complete the fine-tuning training of the model, thereby obtaining the iterative medical diagnosis model.

[0122] Through the above steps 21 to 23, the model can be fine-tuned and trained in rounds according to the sample complexity in one iteration, with simple samples being used to warm up the model first, and then the sample difficulty being gradually increased. This can improve the model's question-answering accuracy while improving the model iteration efficiency. This training method simulates the human learning process from easy to difficult, and improves the model performance in an orderly and effective progressive manner.

[0123] S130. Determine the model performance scores of the medical diagnosis model to be iterated and after the iteration based on the large language model, and select a target model from the model performance scores of the medical diagnosis model to be iterated and after the iteration.

[0124] Specifically, after obtaining the iterated medical forensic diagnosis model, the iterated and iterated medical forensic diagnosis models can be evaluated, such as evaluating the performance of the medical forensic diagnosis model in specific tasks and the generalization ability of the model. For example, in the medical field, medical forensic diagnosis models are widely used in tasks such as patient question answering, diagnosis assistance, and clinical decision support. The performance of the medical forensic diagnosis model can be evaluated in combination with the task requirements in the medical field.

[0125] In the embodiment of the present application, the medical diagnosis model to be iterated and the medical diagnosis model after iteration can be loaded, and the two models respond to and store each question text in the test set, and then the performance evaluation is performed based on the answer texts responded by the two models. For example, the medical diagnosis model before and after this iteration can be evaluated in terms of correctness, adequacy, authenticity and scientificity.

[0126] In a specific embodiment, determining the model performance scores of the medical diagnosis model to be iterated and after iteration includes the following steps:

[0127] Step 31: Generate a model prompt text based on the answer text output by the medical diagnosis model to be iterated and after iteration, and the template text;

[0128] Step 32: Input the model prompt text into the large language model to obtain the accuracy score, adequacy score, authenticity score and scientificity score of the medical diagnosis model to be iterated and after iteration.

[0129] The template text may describe the prompt word template for the large language model to perform a specific task, and the template prompt text may be a prompt word for judging the template to perform a characteristic task. In step 31, the answer text output by the medical diagnosis model to be iterated and after iteration may be filled into the template text to obtain the template prompt text.

[0130] For example, the template text may be: "For the question text '''***''', the answer text of model model1 is '''+response1+''', and the answer text of model model2 is '''+response2+'''. Please evaluate the answer texts of the two models from the perspectives of correctness, adequacy, authenticity and scientificity."

[0131] Furthermore, in step 32, the template prompt text can be input as a prompt into the large language model, and the large language model outputs the correctness score, adequacy score, authenticity score and scientificity score of the medical diagnosis model to be iterated and after iteration. The large language model can be DeepSeek-R1 or trained using other types of models.

[0132] Through the above steps 31 and 32, the medical diagnosis model before and after this iteration can be evaluated from the perspectives of correctness, sufficiency, authenticity and scientificity through the large language model. While ensuring the efficiency of model evaluation, the comprehensiveness of the evaluation can be ensured, thereby improving the accuracy of subsequent target model selection.

[0133] In addition to obtaining the above scores through a large language model, it is also possible to identify keywords in the answer text output by the medical diagnosis model before and after iteration, and determine whether the keywords are included in the corresponding medical record data, and determine the correctness score of the medical diagnosis model based on the number of included keywords.

[0134] Alternatively, the number of answered questions can be retrieved in the answer text, and whether the answer data is empty. The adequacy score of the medical diagnosis model can be determined according to the number of answered questions, the number of questions in the corresponding question text, and whether the answer data is empty.

[0135] Alternatively, similar text matching can be performed on the answer text and the corresponding medical record data to locate the difference text in the answer text that is not included in the medical record data, and then the difference text and the medical record data can be input into the large language model to determine whether the difference text can be derived from the medical record data through the large language model to obtain the authenticity score of the medical diagnosis model.

[0136] Alternatively, medical terms and colloquial words in the answer text may be identified, and the scientificity score of the medical diagnosis model may be determined according to the number of identified medical terms and the number of colloquial words.

[0137] After obtaining the scores of the medical diagnosis model to be iterated and the medical diagnosis model after iteration, a target model can be further selected from the two models, wherein the target model is used for medical diagnosis and is used as the medical diagnosis model to be iterated in the next iteration process.

[0138] Exemplarily, the template text may specify that the large language model outputs the target model according to the score, so that after the large language model outputs the correctness score, adequacy score, authenticity score, and scientificity score of the two models, it further selects the target model from the two models and outputs the basis for the selection.

[0139] For example, the template text may also include: "Please select a model finalmodel with better response according to the correctness score, adequacy score, authenticity score and scientificity score, and provide the comparative analysis process analyze_proess. The result is output in the following json format:

[0140] {{

[0141] "finalmodel":"A relatively good model, reply to model1 or model2"

[0142] "analyse_proess":"Analyze the model's answers from various angles"

[0143] }}”

[0144] In a specific implementation, selecting a target model from the medical diagnosis models to be iterated and after iteration according to the model performance scores includes the following steps:

[0145] Step 41, determining whether the correctness scores of the medical diagnosis model to be iterated and after iteration are lower than a preset score threshold;

[0146] Step 42: If the correctness scores of the medical diagnosis model to be iterated and after the iteration are both higher than the preset score threshold, or the correctness scores of the medical diagnosis model to be iterated and after the iteration are both lower than the preset score threshold, then based on the adequacy score, authenticity score and scientificity score of the medical diagnosis model to be iterated and after the iteration, determine the fusion score of the medical diagnosis model to be iterated and after the iteration;

[0147] Step 43: Determine the target model according to the fusion score of the medical diagnosis model to be iterated and the iterated model.

[0148] Specifically, considering that the correctness of the response of the medical diagnosis model is particularly important, a higher priority can be given to the correctness score. In step 41, it is first determined whether the correctness score of the medical diagnosis model before and after the iteration is lower than a preset score threshold.

[0149] Furthermore, if the correctness score of one of the medical diagnosis models is higher than the preset score threshold, the model can be directly used as the target model. Otherwise, the adequacy score, authenticity score and scientificity score of the two models can be further compared.

[0150] Specifically, in step 42, when the correctness scores of the medical diagnosis model to be iterated and after the iteration are both higher than the preset score threshold, or when the correctness scores of the medical diagnosis model to be iterated and after the iteration are both lower than the preset score threshold, the correctness score, adequacy score, authenticity score and scientificity score of the medical diagnosis model can be fused according to a set ratio to obtain a fusion score of the medical diagnosis model. The set ratio can be 3:3:2:2.

[0151] After calculating the fusion scores of the medical diagnosis models before and after the iteration, the medical diagnosis model with a high fusion score can be selected as the target model.

[0152] Through the above steps 41 to 43, the target model can be selected preferentially based on the correctness score. If the correctness scores of the two models before and after the iteration are both high or low, the sufficiency score, authenticity score and scientificity score can be further combined to calculate the fusion score, so as to compare the performance of the two models through the fusion score to ensure the question-answering accuracy of the target model and the accuracy of subsequent iterations.

[0153] In an embodiment of the present application, in order to further improve the performance of the model in subsequent iterations, after selecting the target model, a feedback data set can be constructed according to the performance of the target model to participate in the next iteration, so as to resist catastrophic forgetting of the model and maintain generalized medical diagnostic capabilities.

[0154] In some embodiments, after selecting a target model based on the model performance scores of the medical diagnosis model to be iterated and after iteration, the following steps are further included:

[0155] Step 51, extracting a model test set from all medical record data used in the historical iteration process;

[0156] Step 52: input the model test set into the target model, and determine the correctness score, adequacy score, authenticity score, and scientificity score of the target model based on the output of the target model and the large language model;

[0157] Step 53: If there is at least one score lower than the corresponding set threshold, it proves that the current model has lost this part of its ability. Such data is collected to construct a feedback reinforcement dataset, where the feedback reinforcement dataset is used for the next model iteration update.

[0158] In step 51, in order to evaluate whether the target model has catastrophic forgetting, all medical record data involved in the training in the previous iterations can be randomly sampled to construct a model reinforcement test set for the current iteration. For example, the model reinforcement test set can be extracted from the medical record test data in the previous iterations.

[0159] Furthermore, in step 52, the model reinforcement test set can be input into the target model to obtain the corresponding output, and then the output of the target model can be evaluated in multiple dimensions such as correctness, scientificity, authenticity and sufficiency using the large language model.

[0160] Specifically, if the score of a certain dimension is lower than the set threshold of that dimension, it means that the target model's response in that dimension may be imperfect. At this time, the model reinforcement test set can be used as a pre-feedback data set. At the same time, the large language model can also generate an evaluation basis for the imperfection of the model reinforcement test set in that dimension, and concatenate the evaluation basis with the pre-feedback data set to construct a feedback data set, which can be used to participate in the next iterative fine-tuning.

[0161] For example, during the next model iteration, the medical record data can be retrieved and the target model can be fine-tuned and updated using the medical record data and the feedback dataset.

[0162] Medical forensic models usually involve high-risk decisions, such as diagnosis, treatment plan recommendations, etc. Through the above steps 51 to 53, the performance of the model can be improved through feedback reinforcement, and its reliability, accuracy and safety in practical applications can be ensured, and catastrophic forgetting of the model can be resisted to maintain generalized medical diagnostic capabilities.

[0163] The automatic iteration method of the medical diagnosis model provided in the embodiment of the present application, in response to detecting that a preset iteration cycle is met, obtains medical case record data for model iteration and update, and generates medical instruction data based on the medical case record data and the large language model, thereby obtaining the medical diagnosis model to be iterated, and fine-tuning the medical diagnosis model to be iterated based on the medical instruction data to obtain the iterated medical diagnosis model, thereby determining the model performance scores of the medical diagnosis model to be iterated and after iteration, and selecting a target model from them according to the model performance scores, so as to use the target model for medical diagnosis, and use the target model as the medical diagnosis model to be iterated in the next iteration process, This method can realize automatic iterative optimization of the model, significantly reduce the cost of manual intervention, improve the availability and practicality of the model, and enable the model to quickly respond to dynamic changes in business needs. In addition, this method generates medical instruction data through a large language model and medical record data, which can solve the problem of scarce high-quality medical data, improve the quality of the model fine-tuning data set, and thus improve the fine-tuning effect of the model and ensure the accuracy of the model's question and answer. In addition, after iterative optimization, this method determines the question and answer scores of the model before and after iteration, thereby selecting the target model for question and answer and participating in the next iteration, which can avoid errors gradually generated during the iteration process and ensure the question and answer effect of the model.

[0164] Figure 2 is an automatic iteration flow chart of a medical diagnosis model provided in an embodiment of the present application, such as Figure 2 As shown, for each model iteration, medical case data can be obtained first, and then combined with the large language model to generate medical instruction data, and the medical instruction data can be preprocessed. Furthermore, the medical instruction data is used to fine-tune the current medical diagnosis model to be iterated, and the iterated model and the model before model fine-tuning are evaluated to select the target model.

[0165] After obtaining the target model, the target model can be used for model application, and a test set can be randomly sampled to perform model testing on the target model. A feedback data set is constructed based on the test results, and the feedback data set can be used for the next model iteration.

[0166] Through the above process, the automatic iterative optimization of the medical diagnosis model can be achieved, which significantly reduces the cost of manual intervention, improves the availability and practicality of the system, and provides strong technical support for intelligent applications in professional fields. In addition, the sample self-generation technology based on large language models and real medical records solves the problem of the scarcity of high-quality data in the medical field, effectively improves the quality of medical fine-tuning samples, and avoids the problem of "repeater" in the fine-tuned model caused by excessive repetition of text in the direct filling process of medical record data, thereby improving the fine-tuning effect of the model.

[0167] In addition, in the embodiment of the present application, by generating samples that include the corresponding diagnostic process and diagnostic basis, the sample can internalize the thinking chain, thereby improving the quality of the sample and enhancing the reasoning ability of the fine-tuning model.

[0168] In addition, a feedback reinforcement link is set up during the iteration process. By continuously using the test sets in all previous iteration rounds to test the current target model, the data with poor model response is screened to construct a feedback data set, and this feedback data set is used for the next round of iterative training, thereby effectively resisting catastrophic forgetting of the model and realizing continuous learning of the model.

[0169] Figure 3 3 is a schematic diagram of the structure of an automatic iteration system of a medical diagnosis model provided in an embodiment of the present application. The system includes a data processing module 310, a model fine-tuning module 320, and a model evaluation module 330, wherein:

[0170] The data processing module 310 is used to obtain medical record data for model iteration and update in response to detecting that a preset iteration cycle is met, and generate medical instruction data based on the medical record data and the large language model;

[0171] The model fine-tuning module 320 is used to obtain the medical diagnosis model to be iterated, and fine-tune the medical diagnosis model to be iterated based on the medical instruction data to obtain the iterated medical diagnosis model;

[0172] The model evaluation module 330 is used to determine the model performance scores of the medical diagnosis model to be iterated and after iteration based on the large language model, and select a target model from the medical diagnosis model to be iterated and after iteration according to the model performance scores of the medical diagnosis model to be iterated and after iteration.

[0173] The target model is used for medical diagnosis and is used as a medical diagnosis model to be iterated in the next iteration process.

[0174] On the basis of the above-mentioned implementation modes, optionally, the data processing module 310 is also used to obtain medical instruction data and diagnostic basis corresponding to each answer text in the medical instruction data based on the medical case record data and the large language model; extract the diagnostic thinking process text corresponding to the answer text from the diagnostic basis, and update the answer text based on the diagnostic thinking process text.

[0175] On the basis of the above-mentioned implementation modes, optionally, the data processing module 310 is also used to obtain a preset model prompt text; input the preset model prompt text and the medical record data into the large language model, so that the large language model generates medical instruction data that meets the preset model prompt text, and outputs the diagnosis basis corresponding to each answer text in the medical instruction data; wherein the preset model prompt text includes that false information cannot be constructed, the number of words in the answer text exceeds the preset number of words, and the format of the answer text meets the preset format.

[0176] On the basis of the above-mentioned implementation modes, optionally, the data processing module 310 is also used to locate the medical terms for each question-answer pair in the medical instruction data, based on a pre-trained medical term extraction model, respectively for the answer text in the question-answer pair and the corresponding medical record data, to obtain the generated data corresponding to the medical terms in the answer text and the real data corresponding to the medical terms in the medical record data; determine whether the generated data is consistent with the real data, and if not, eliminate the question-answer pair.

[0177] On the basis of the above-mentioned implementation modes, optionally, the model fine-tuning module 320 is also used to determine the complexity of each question-answer pair in the medical instruction data, and divide the medical instruction data into multiple sample sets according to the complexity of each question-answer pair; sort all sample sets in order of complexity from low to high, and use the first sample set as the current training set; use the current training set to perform fine-tuning training on the medical diagnosis model to be iterated, and after the fine-tuning training is completed, use the next sample set of the current training set as the new current training set, and return to the step of fine-tuning training on the medical diagnosis model to be iterated using the current training set, until the current training set is the last sample set.

[0178] On the basis of the above-mentioned implementation modes, optionally, the model evaluation module 330 is also used to generate a model prompt text based on the answer text output by the medical diagnosis model to be iterated and after iteration, and the template text; the model prompt text is input into the large language model to obtain the correctness score, adequacy score, authenticity score and scientificity score of the medical diagnosis model to be iterated and after iteration.

[0179] On the basis of the above-mentioned embodiments, optionally, the model evaluation module 330 is also used to determine whether the correctness scores of the medical diagnosis models to be iterated and after iteration are lower than a preset score threshold; if the correctness scores of the medical diagnosis models to be iterated and after iteration are both higher than the preset score threshold, or the correctness scores of the medical diagnosis models to be iterated and after iteration are both lower than the preset score threshold, then based on the adequacy score, authenticity score and scientificity score of the medical diagnosis models to be iterated and after iteration, determine the fusion score of the medical diagnosis models to be iterated and after iteration; determine the target model according to the fusion score of the medical diagnosis models to be iterated and after iteration.

[0180] Based on the above embodiments, optionally, the system also includes a model feedback module, which is used to extract a model reinforcement test set from all medical record data used in the historical iteration process; input the model reinforcement test set into the target model, and determine the correctness score, adequacy score, authenticity score and scientificity score of the target model based on the output of the target model and the large language model; if there is at least one score lower than the corresponding set threshold, a feedback data set is constructed based on the model reinforcement test set, wherein the feedback data set is used for the next model iteration update.

[0181] The above system can be run on a machine with a GPU. Two A100s can be used for fine-tuning, and other service interfaces can be called for large models required for testing.

[0182] The automatic iteration system of the medical diagnosis model provided in the embodiment of the present application can execute the steps in the automatic iteration method of the medical diagnosis model provided in the method embodiment of the present application, and the execution steps and beneficial effects are no longer repeated here.

[0183] Figure 4 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 400 includes one or more processors 401 and a memory 402 .

[0184] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0185] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may run the program instructions to implement the automatic iteration method of the medical diagnosis model of any embodiment of the present application described above and / or other desired functions. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage medium.

[0186] In one example, the electronic device 400 may further include: an input device 403 and an output device 404, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including early warning prompt information, braking force, etc. The output device 404 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0187] Of course, to simplify, Figure 4 Only some of the components related to the present application in the electronic device 400 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 400 may also include any other appropriate components.

[0188] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the automatic iteration method of the medical diagnosis model provided in any embodiment of the present application.

[0189] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0190] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the automatic iteration method of the medical diagnosis model provided in any embodiment of the present application.

[0191] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0192] It should be noted that the terms used in this application are only for describing specific embodiments, rather than limiting the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0193] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0194] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and its core ideas of this application. The above is only the preferred implementation method of this application. It should be pointed out that due to the limitations of textual expression and the objective existence of infinite specific structures, ordinary technicians in this technical field can make several improvements, modifications or changes without departing from the principles of this application, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes or combinations, or the direct application of the concept and technical solution of the invention to other occasions without improvement, should be regarded as the scope of protection of this application.

Claims

1. An automatic iteration method for a medical diagnosis model, characterized in that: include: In response to detecting that a preset iteration cycle is met, obtaining medical record data for iterative updating of the model, and generating medical instruction data based on the medical record data and the large language model; Acquire a medical diagnosis model to be iterated, and fine-tune the medical diagnosis model to be iterated based on the medical instruction data to obtain an iterated medical diagnosis model; Determine the model performance scores of the medical diagnosis model to be iterated and after the iteration based on the large language model, and select a target model from the medical diagnosis model to be iterated and after the iteration according to the model performance scores of the medical diagnosis model to be iterated and after the iteration; The target model is used for medical diagnosis and is used as a medical diagnosis model to be iterated in the next iteration process.

2. The method according to claim 1, characterized in that Generate medical instruction data based on the medical record data and the large language model, including: Based on the medical case record data and the large language model, obtain medical instruction data and diagnostic evidence corresponding to each answer text in the medical instruction data; The diagnostic thought process text corresponding to the answer text is extracted from the diagnostic basis, and the answer text is updated based on the diagnostic thought process text.

3. The method according to claim 2, characterized in that Based on the medical case record data and the large language model, medical instruction data and diagnostic evidence corresponding to each answer text in the medical instruction data are obtained, including: Get the preset model prompt text; Inputting the preset model prompt text and the medical record data into the large language model, so that the large language model generates medical instruction data that meets the preset model prompt text, and outputs the diagnosis basis corresponding to each answer text in the medical instruction data; The preset model prompt text includes that false information cannot be constructed, the number of words in the answer text exceeds the preset number of words, and the format of the answer text meets the preset format.

4. The method according to claim 1, characterized in that: After generating medical instruction data based on the medical record data and the large language model, the method further includes: For each question-answer pair in the medical instruction data, based on a pre-trained medical term extraction model, medical term location is performed on the answer text and the corresponding medical record data in the question-answer pair to obtain generated data corresponding to the medical term in the answer text and real data corresponding to the medical term in the medical record data; Determine whether the generated data is consistent with the real data; if not, remove the question-answer pair.

5. The method according to claim 1, characterized in that Fine-tuning and training the medical diagnosis model to be iterated based on the medical instruction data includes: Determining the complexity of each question-answer pair in the medical instruction data, and dividing the medical instruction data into a plurality of sample sets according to the complexity of each question-answer pair; Sort all sample sets in order of complexity from low to high, and use the first sample set as the current training set; The medical diagnosis model to be iterated is fine-tuned using the current training set, and after the fine-tuning training is completed, the next sample set of the current training set is used as the new current training set, and the step of fine-tuning the medical diagnosis model to be iterated using the current training set is returned to, until the current training set is the last sample set.

6. The method according to claim 1, characterized in that Determining model performance scores of the medical diagnosis model to be iterated and after iteration based on the large language model includes: Generate model prompt text based on the answer text output by the medical diagnosis model to be iterated and after iteration, and the template text; The model prompt text is input into the large language model to obtain the correctness score, adequacy score, authenticity score and scientificity score of the medical diagnosis model to be iterated and after iteration.

7. The method according to claim 6, characterized in that The step of selecting a target model according to the model performance scores of the medical diagnosis model to be iterated and the iterated model comprises: Determine whether the correctness scores of the medical diagnosis model to be iterated and after iteration are lower than a preset score threshold; If the correctness scores of the medical diagnosis model to be iterated and after the iteration are both higher than the preset score threshold, or the correctness scores of the medical diagnosis model to be iterated and after the iteration are both lower than the preset score threshold, then based on the adequacy score, authenticity score and scientificity score of the medical diagnosis model to be iterated and after the iteration, determine the fusion score of the medical diagnosis model to be iterated and after the iteration; The target model is determined according to the fusion score of the medical diagnosis model to be iterated and the iterated model.

8. The method according to claim 6, characterized in that After selecting the target model based on the model performance scores of the medical diagnosis model to be iterated and the iterated model, the following steps are also included: Extract the model reinforcement test set from all medical record data used in historical iterations; Inputting the model reinforcement test set into the target model, and determining the correctness score, adequacy score, authenticity score and scientificity score of the target model based on the output of the target model and the large language model; If there is at least one score lower than the corresponding set threshold, a feedback data set is constructed based on the model reinforcement test set, wherein the feedback data set is used for the next model iteration update.

9. An automatic iteration system for a medical diagnosis model, characterized in that: The system includes a data processing module, a model fine-tuning module, and a model evaluation module, wherein: The data processing module is used to obtain medical record data for model iteration and update in response to detecting that a preset iteration cycle is met, and generate medical instruction data based on the medical record data and the large language model; The model fine-tuning module is used to obtain the medical diagnosis model to be iterated, and fine-tune the medical diagnosis model to be iterated based on the medical instruction data to obtain the iterated medical diagnosis model; The model evaluation module is used to determine the model performance scores of the medical diagnosis model to be iterated and after the iteration based on the large language model, and select a target model from the medical diagnosis model to be iterated and after the iteration according to the model performance scores of the medical diagnosis model to be iterated and after the iteration; The target model is used for medical diagnosis and is used as a medical diagnosis model to be iterated in the next iteration process.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, which enables a computer to execute the steps of the automatic iteration method of the medical diagnosis model according to any one of claims 1 to 8.

Images