Automatic Iteration Method, System and Storage Medium of Medical Diagnosis Model

Abstract

The present application provides an automatic iteration method, system and storage medium for a medical diagnosis model. The method, in response to detecting that a preset iteration period is satisfied, obtains medical record data for model iteration and update, generates high-quality medical instruction data based on the medical record data and a large language model, fine-tunes and trains the medical diagnosis model to be iterated based on the medical instruction data to obtain an iterated medical diagnosis model, then scores the medical diagnosis models before and after iteration based on the large language model, and selects a target model from the model performance scores, so as to use the target model for medical diagnosis, and use the target model as the medical diagnosis model to be iterated in the next iteration process, realizing the automatic iteration and optimization of the model, improving the usability and practicality of the model, enabling the model to quickly respond to the dynamic changes of business requirements, being able to avoid the gradually generated errors in the iteration process, and ensuring the question-and-answer effect of the model.

Description

Technical Field

[0001] The present application relates to the technical field of medical Q&A, and specifically relates to an automatic iteration method, system, and storage medium for a medical diagnosis model. Background Art

[0002] In recent years, with the rapid development of artificial intelligence and large model technologies, data-driven methods have gradually been adopted in various fields to solve complex problems. In the medical field, artificial intelligence-assisted diagnosis has gradually been applied to the diagnosis and treatment process of doctors, and intelligent diagnosis and treatment technologies based on large models have been continuously developed.

[0003] However, when using large models to solve problems in the medical field, it often involves complex variables, such as the medical history, clinical symptoms, examination conditions, etc. of patients, and the relationships between these variables may be non-linear. The model obtained from one-time training may not be perfect, especially when dealing with a small number of samples, rare diseases, or clinically complex situations, which results in low answer accuracy of the large model and cannot meet the actual application requirements.

[0004] Moreover, in the development cycle of traditional model training, manual adjustment and cross-team collaboration are usually required, and the development cycle is relatively long, resulting in the impact on model performance and the inability to quickly respond to the dynamic changes of business requirements. Summary of the Invention

[0005] In view of the above defects or deficiencies in the prior art, the present application aims to provide an automatic iteration method, system, and storage medium for a medical diagnosis model to solve problems such as low answer accuracy of the existing model, long development cycle, and inability to quickly respond to the dynamic changes of business requirements in the prior art.

[0006] An embodiment of the present application provides an automatic iteration method for a medical diagnosis model, and the method includes:

[0007] In response to detecting that a preset iteration period is satisfied, obtain medical record data for model iteration and update, and generate medical instruction data based on the medical record data and a large language model;

[0008] Obtain the medical diagnosis model to be iterated, and perform fine-tuning training on the medical diagnosis model to be iterated based on the medical instruction data to obtain an iterated medical diagnosis model;

[0009] Determine the model performance scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model based on the large language model, and select a target model from them according to the model performance scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model;

[0010] Wherein, the target model is used for medical diagnosis and as the medical diagnosis model to be iterated in the next iteration process.

[0011] Optionally, based on the medical record data and the large language model, generate medical instruction data, including:

[0012] Based on the medical record data and the large language model, obtain medical instruction data and the diagnostic basis corresponding to each answer text in the medical instruction data;

[0013] Extract the diagnostic thinking process text corresponding to the answer text from the diagnostic basis, and update the answer text based on the diagnostic thinking process text.

[0014] Optionally, based on the medical record data and the large language model, obtaining medical instruction data and the diagnostic basis corresponding to each answer text in the medical instruction data includes:

[0015] Obtain the preset model prompt text;

[0016] Input the preset model prompt text and the medical record data into the large language model, so that the large language model generates medical instruction data that meets the preset model prompt text, and outputs the diagnostic basis corresponding to each answer text in the medical instruction data;

[0017] Wherein, the preset model prompt text includes not fabricating false information, the number of words in the answer text exceeding the preset number of words, and the format of the answer text meeting the preset format.

[0018] Optionally, after generating medical instruction data based on the medical record data and the large language model, it further includes:

[0019] For each question-and-answer pair in the medical instruction data, based on a pre-trained medical term extraction model, respectively perform medical term localization on the answer text and the corresponding medical record data in the question-and-answer pair, to obtain the generated data corresponding to the medical terms in the answer text and the real data corresponding to the medical terms in the medical record data;

[0020] Judge whether the generated data is consistent with the real data, if not, then eliminate the question-and-answer pair.

[0021] Optionally, fine-tuning and training the medical diagnosis model to be iterated based on the medical instruction data includes:

[0022] Determine the complexity of each question-and-answer pair in the medical instruction data, and divide the medical instruction data into multiple sample sets according to the complexity of each question-and-answer pair;

[0023] Sort all sample sets in ascending order of complexity, and use the first sample set as the current training set;

[0024] Fine-tune the medical diagnosis model to be iterated using the current training set, and after the fine-tuning training is completed, use the next sample set of the current training set as the new current training set, and return the step of fine-tuning the medical diagnosis model to be iterated using the current training set until the current training set is the last sample set.

[0025] Optionally, determining the model performance scores of the medical diagnosis model to be iterated and the iterated model based on the large language model includes:

[0026] Generating model prompt text based on the response texts output by the medical diagnosis model to be iterated and the iterated model, and the template text;

[0027] Input the model prompt text into the large language model to obtain the correctness score, sufficiency score, authenticity score, and scientificity score of the medical diagnosis model to be iterated and the iterated model.

[0028] Optionally, selecting a target model from the model performance scores of the medical diagnosis model to be iterated and the iterated model includes:

[0029] Judging whether the correctness score of the medical diagnosis model to be iterated and the iterated model is lower than a preset score threshold;

[0030] If the correctness scores of the medical diagnosis model to be iterated and the iterated model are both higher than the preset score threshold, or the correctness scores of the medical diagnosis model to be iterated and the iterated model are both lower than the preset score threshold, then based on the sufficiency score, authenticity score, and scientificity score of the medical diagnosis model to be iterated and the iterated model, determine the fusion score of the medical diagnosis model to be iterated and the iterated model;

[0031] Determine the target model according to the fusion score of the medical diagnosis model to be iterated and the iterated model.

[0032] Optionally, after selecting a target model from the model performance scores of the medical diagnosis model to be iterated and the iterated model, it further includes:

[0033] Extract a model reinforcement test set from all medical record data used in the historical iteration process;

[0034] Input the model reinforcement test set into the target model, and based on the output of the target model and the large language model, determine the correctness score, sufficiency score, authenticity score, and scientificity score of the target model;

[0035] If there is at least one score lower than the corresponding set threshold, then construct a feedback data set based on the model reinforcement test set, where the feedback data set is used for the next model iteration update.

[0036] An embodiment of the present application further provides an automatic iteration system for a medical diagnosis model, and the system includes a data processing module, a model fine-tuning module, and a model evaluation module, where:

[0037] The data processing module is configured to, in response to detecting that a preset iteration period is met, obtain medical record data for model iteration update, and generate medical instruction data based on the medical record data and a large language model;

[0038] The model fine-tuning module is configured to obtain a medical diagnosis model to be iterated, and perform fine-tuning training on the medical diagnosis model to be iterated based on the medical instruction data to obtain an iterated medical diagnosis model;

[0039] The model evaluation module is configured to determine model performance scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model based on the large language model, and select a target model therefrom according to the model performance scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model;

[0040] Wherein, the target model is used for medical diagnosis and as the medical diagnosis model to be iterated in the next iteration process.

[0041] An embodiment of the present application further provides an electronic device, and the electronic device includes:

[0042] A processor and a memory;

[0043] The processor is configured to execute the steps of the automatic iteration method for a medical diagnosis model provided in any embodiment of the present application by calling a program or instruction stored in the memory.

[0044] An embodiment of the present application further provides a computer-readable storage medium, and the computer-readable storage medium stores a program or instruction, and the program or instruction causes a computer to execute the steps of the automatic iteration method for a medical diagnosis model provided in any embodiment of the present application.

[0045] In summary, the present application proposes an automatic iteration method for a medical diagnosis model. In response to a model iteration instruction, the method obtains medical record data for model iteration and update, constructs a high-quality medical instruction set based on the medical record data and a large language model, and then obtains the medical diagnosis model to be iterated. The medical diagnosis model to be iterated is fine-tuned and trained based on the medical instruction data to obtain the iterated medical diagnosis model. Then, the model performance scores of the medical diagnosis models to be iterated and the iterated ones are determined, and a target model is selected from them for medical diagnosis and used as the medical diagnosis model to be iterated in the next iteration process. This method can achieve the integration of training and evaluation of large models, meet the application requirements of automatic iteration and optimization of the model, significantly reduce the cost of manual intervention, improve the usability and practicality of the model, enable the model to quickly respond to the dynamic changes of business requirements, and, by generating medical instruction data through the large language model and medical record data, this method can solve the problem of the lack of high-quality data in the medical field, improve the quality of the model fine-tuning dataset, and then enhance the fine-tuning effect of the model to ensure the accuracy of model answers. In addition, after iterative optimization, this method determines the answer scores of the models before and after iteration, selects the target model for answering questions and participates in the next iteration, which can avoid the gradually generated errors in the iteration process and ensure the answer effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0047] Figure 1 is a flowchart of an automatic iteration method for a medical diagnosis model provided by an embodiment of the present application;

[0048] Figure 2 is an automatic iteration flowchart of a medical diagnosis model provided by an embodiment of the present application;

[0049] Figure 3 is a schematic structural diagram of an automatic iteration system for a medical diagnosis model provided by an embodiment of the present application;

[0050] Figure 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention, rather than limiting the invention. Additionally, it should be noted that for the convenience of description, only the parts related to the invention are shown in the drawings.

[0052] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0053] Before introducing the method provided by the embodiments of the present application in detail, the technical problems solved by this method will be described first.

[0054] In the prior art, the initially trained model is not perfect, especially when dealing with a small number of samples, rare diseases, or clinical complex situations. In the development cycle of traditional model training and evaluation, manual adjustment and cross-team collaboration are usually required, and there are the following challenges:

[0055] Surge in data volume: The quality and processing efficiency of large-scale data directly affect the model performance;

[0056] Long iteration cycle time and low development efficiency: From data preprocessing to model deployment, it involves multiple links, requires a large amount of manual intervention and repetitive operations, and the development cycle is relatively long;

[0057] Unable to meet the demand for automated iteration: Unable to quickly respond to the dynamic changes in business requirements;

[0058] Therefore, in order to solve the above problems, the embodiments of the present application provide an automatic iteration method for a medical diagnosis model, which can achieve automatic iteration and optimization of the model through an integrated model development process, significantly reducing the manual intervention cost, improving the usability and practicality of the system, and providing strong technical support for the intelligent application in the professional field. Moreover, it solves the problem of the lack of high-quality medical data, effectively improves the quality of medical fine-tuning data, and further improves the fine-tuning effect of the model.

[0059] As mentioned in the background art, in view of the problems in the prior art, the present application proposes an automatic iteration method for a medical diagnosis model. Figure 1 It is a flowchart of an automatic iteration method for a medical diagnosis model provided by the embodiments of the present application. This method is applicable to generating a model for medical differential diagnosis through question and answer. Refer to Figure 1 This automatic iteration method for the medical diagnosis model specifically includes:

[0060] S110. In response to detecting that the preset iteration cycle is satisfied, obtain medical record data for model iteration and update, and generate medical instruction data based on the medical record data and the large language model.

[0061] Among them, the preset iteration period can be the period for the model to automatically iterate set in advance, and can be determined based on the size of the data increment generated by the outpatient information management system within a set time.

[0062] Exemplarily, the more data increment is generated by the outpatient information management system within a set time, the shorter the preset iteration period is. The preset iteration period can be dynamically adjusted based on the data increment of the outpatient information management system. In addition to triggering model iteration when the preset iteration period is satisfied, model iteration can also be triggered when a new disease condition appears in the outpatient information management system.

[0063] Specifically, if it is detected that the preset iteration period is satisfied, medical record data for model iteration update can be obtained first, so as to generate medical instruction data for model training through the medical record data. Among them, the medical record data can be sourced from the outpatient information management system of the hospital. For example, communication can be carried out with the server of the outpatient information management system of the hospital through HTTP (Hypertext Transfer Protocol), and a request can be sent to the backend system to obtain the medical record data.

[0064] Exemplarily, obtaining medical record data for model iteration update includes: sending an HTTP request to the outpatient information management system to obtain the medical record data generated by the outpatient information management system within a set period, or obtaining the medical record data of the new disease condition in the outpatient information management system.

[0065] Among them, the requests library can be used to send an HTTP request to the outpatient information management system to obtain the medical record data. This process mainly involves API (Application Programming Interface) authentication authorization and request hyperparameter settings. In the embodiments of the present application, for the docked outpatient information management system, its interface can be authenticated and authorized in the way of API Key. Therefore, authentication can be carried out by passing the API key in the URL of the HTTP request. The setting of request hyperparameters mainly includes the interface service code and the time period for obtaining medical record data.

[0066] In the embodiments of the present application, considering that the medical record data obtained through the interface may be in a state of data chaos, therefore, before generating medical instruction data from the medical record data, the medical record data can be first screened and formatted. Exemplarily, data that is ineffective for model fine-tuning, such as no patient chief complaint, no prescription information, no preliminary diagnosis information, etc., can be screened out first; further, the screened medical record data is formatted. For example, the medical record data is converted into data in JSON (JavaScript Object Notation) format, as shown below:

[0067] {

[0068] "deptName": "",

[0069] "doctext": {

[0070] "Chief Complaint": "",

[0071] "History of Present Illness": "",

[0072] "Past Medical History": "",

[0073] "Physical Examination": "",

[0074] "Prescription": "",

[0075] "Auxiliary Examinations": "",

[0076] "Preliminary Diagnosis": ""

[0077] }

[0078] }

[0079] Further, medical instruction data for model fine-tuning can be generated from the medical record data. In the embodiments of the present application, a large language model can be used to generate medical instruction data. The reason is that in the current medical field, medical tasks are highly professional and complex, involving a large amount of medical terms, clinical pathways, treatment plans, symptom diagnoses, etc. This makes the fine-tuning instructions not only need to cover a wide range of medical fields, but also need to have strong context understanding and reasoning abilities. Therefore, the embodiments of the present application utilize the excellent language understanding and generation capabilities of the large language model, as well as the medical knowledge in the real medical record data, to generate high-quality fine-tuning data. Among them, the large language model can be the DeepSeek-R1 large model, and FastAPI can be used to configure the corresponding interface for this.

[0080] Specifically, medical record data can be input into a large language model so that the large language model extracts multiple question-and-answer pairs from it to obtain medical instruction data. Among them, the question-and-answer pair consists of a question text and a corresponding answer text. For example, based on the medical record data as the medical knowledge base, the identity role of a medical expert can be given to the large language model, providing the analysis steps for constructing the question-and-answer pair, and setting corresponding instruction requirements to drive the large language model to construct the question-and-answer pair for the consultation scenario.

[0081] In the embodiment of the present application, in order to make the generated answer text more logical and interpretable, during the process of the large language model generating the question-and-answer pair, the large language model can also be made to generate intermediate reasoning step text, so as to avoid the medical diagnosis model directly giving unreasonable or jumping answer text, and making the answer generated by the medical diagnosis model more logical and interpretable.

[0082] In a specific implementation manner, based on the medical record data and the large language model, generating medical instruction data includes the following steps:

[0083] Step 11: Based on the medical record data and the large language model, obtain medical instruction data and the diagnostic basis corresponding to each answer text in the medical instruction data;

[0084] Step 12: Extract the diagnostic thinking process text corresponding to the answer text from the diagnostic basis, and update the answer text based on the diagnostic thinking process text.

[0085] Among them, in step 11, the medical record data can be first input into the large language model, and the large language model performs semantic analysis on the medical record data to generate multiple question-and-answer pairs, constituting medical instruction data; and while the large language model generates the question-and-answer pair, the large language model can also extract the diagnostic basis corresponding to each answer text from the medical record data.

[0086] Considering that there may be a situation where the question-and-answer pairs output by the large language model contain false information, in order to ensure the training accuracy of the subsequent medical diagnosis model, corresponding instruction requirements can be set for the large language model during the process of generating medical instruction data.

[0087] In an example, based on the medical record data and the large language model, obtaining medical instruction data and the diagnostic basis corresponding to each answer text in the medical instruction data includes the following steps:

[0088] Step 111: Obtain a preset model prompt text;

[0089] Step 112: Input the preset model prompt text and the medical record data into the large language model so that the large language model generates medical instruction data that meets the preset model prompt text and outputs the diagnostic basis corresponding to each answer text in the medical instruction data.

[0090] Specifically, in step 111, the preset model prompt text includes not being able to fabricate false information, the word count of the answer text exceeding the preset word count, and the format of the answer text meeting the preset format. Among them, false information can be content that does not conform to medical record data, the preset word count can be the minimum word count of the answer text set in advance, such as 800 words, and the preset format can be the format of the medical record data input into the large language model, such as JSON format.

[0091] In step 112, the preset model prompt text can be concatenated with the medical record data and then input into the large language model. The large language model performs reasoning to generate medical instruction data that meets the preset model prompt text. At the same time, the diagnostic basis corresponding to each answer text is output.

[0092] Through the above steps 111 - 112, the large language model can be regulated by the pre-set model prompt conditions to avoid generating false information while generating medical instruction data, ensuring the accuracy of the subsequent fine-tuning of the medical diagnosis model.

[0093] After obtaining the medical instruction data from the large language model and the diagnostic basis corresponding to each answer text in the medical instruction data, further, in step 12, the diagnostic thinking process text corresponding to the answer text can be extracted from the diagnostic basis. For example, the large language model can be used to extract the diagnostic thinking process text from the diagnostic basis. The diagnostic thinking process text can describe the reasoning process from the question text to the answer text, and this thinking chain is formed through the in-depth thinking, continuous exploration, and backtracking of the large language model.

[0094] Furthermore, the answer text can be updated according to the diagnostic thinking process text so that the answer text contains the diagnostic thinking process, that is, the diagnostic thinking process is integrated into the answer text to construct a diagnostic thinking chain.

[0095] Through the above steps 11 - 12, the diagnostic thinking can be integrated into the generated answer text, making the generated answer text have a reasoning process, and further enabling the fine-tuned medical diagnosis model to have stronger reasoning ability.

[0096] Considering that regulating the output of the large language model through the pre-set model prompt text (prompt) may still make the Q&A pairs synthesized by the large language model contain false information. To further ensure the authenticity of the answer text generated by the large language model and avoid the hallucination problem of the large language model, after obtaining the medical instruction data, the medical instruction data can be further screened to eliminate the Q&A pairs containing false information.

[0097] In a specific implementation, after generating medical instruction data based on medical record data and a large language model, the following steps are further included:

[0098] Step 13: For each Q&A pair in the medical instruction data, based on a pre-trained medical term extraction model, perform medical term localization on the answer text in the Q&A pair and the corresponding medical record data respectively, to obtain the generated data corresponding to the medical terms in the answer text and the real data corresponding to the medical terms in the medical record data;

[0099] Step 14: Determine whether the generated data is consistent with the real data. If not, then discard the Q&A pair.

[0100] Among them, the medical term extraction model can be a model used to locate medical terms from the input text.

[0101] Specifically, in Step 13, the answer text in the Q&A pair and the corresponding medical record data can be respectively input into the medical term extraction model, and the medical term extraction model retrieves medical terms in the answer text and the medical record data. Among them, the medical terms can be drug names, examination item names, or index item names.

[0102] After locating the medical terms, the generated data corresponding to the medical terms in the answer text and the real data corresponding to the medical terms in the medical record data can be further queried. Among them, the generated data corresponding to the medical terms can be the dosage, value, or value range corresponding to the medical terms in the answer text, such as the dosage of a certain drug, the location of a certain examination item, the value range of a certain index item; the real data corresponding to the medical terms can be the dosage, value, or value range corresponding to the medical terms in the medical record data.

[0103] After retrieving the medical terms and obtaining the generated data and the real data, in Step 14, the two can be matched, that is, it is determined whether the generated data is consistent with the real data for the same medical term. If not, it means that the answer text generated by the large language model contains false information. At this time, the Q&A pair can be discarded. At the same time, the generation of Q&A pairs can be performed again for the corresponding medical record data.

[0104] Through the above Step 13 - Step 14, by locating medical terms and querying the corresponding data, and then matching the data to determine whether the Q&A pairs generated by the large language model contain false information, the problem of model hallucination can be further avoided, the accuracy of the medical instruction data can be guaranteed, and thus the reliability of subsequent iterative training can be further improved.

[0105] In the embodiments of the present application, after screening the generated medical instruction data, considering that errors, redundancies, or invalid data may be generated during the process of self-generating data by the large language model, and at the same time, there may be problems in aspects such as data format and data ratio, therefore, the medical instruction data can also be preprocessed to improve the quality of the medical instruction data, so that the medical diagnosis model can better understand and execute tasks.

[0106] Exemplarily, preprocessing the medical instruction data includes data cleaning, data partitioning, and data ratio matching. Among them, data cleaning can be to identify and remove dirty data (such as empty data, data with incorrect formats, garbled data, etc.), and use hash functions and text embeddings to identify and remove redundant data, and then format the medical instruction data. Data partitioning can be to divide the cleaned medical instruction data into a training set and a test set according to a certain ratio. Data ratio matching can be to supplement a part of general data in the training set to prevent the medical diagnosis model from overfitting in order to ensure the general task ability of the medical diagnosis model during the process of professional fine-tuning. The source of the general data can be BELLE, and the ratio of the general data to the medical instruction data generated based on medical record data can be 1:1.

[0107] S120. Obtain the medical diagnosis model to be iterated, and perform fine-tuning training on the medical diagnosis model to be iterated based on the medical instruction data to obtain the iterated medical diagnosis model.

[0108] Specifically, after obtaining the medical instruction data, the medical instruction data can be used to perform iterative training on the medical diagnosis model. It should be noted that during this iterative training process, the medical diagnosis model to be iterated used is the target model determined after the previous iterative training and evaluation. If this round of iteration is the initial iteration, the medical diagnosis model to be iterated used in this iteration can be a general basic model.

[0109] In the embodiments of the present application, through the efficient parameter fine-tuning technology, fine-tuning training can be performed on the medical diagnosis model to be iterated based on the medical instruction data, which can reduce the consumption of computing resources, alleviate the catastrophic forgetting problem, and improve the generalization ability of the model.

[0110] Exemplarily, using the efficient parameter fine-tuning technology can be to fine-tune some parameters of the medical diagnosis model to be iterated instead of all parameters of the model, which can significantly reduce the consumption of computing resources. For example, LoRA (Low-Rank Adaptation) can be used to introduce a low-rank matrix into the model weights to achieve efficient parameter update.

[0111] Among them, the efficient parameter fine-tuning technology reduces the coverage of historical knowledge by retaining most of the original model parameters, thus effectively alleviating the catastrophic forgetting problem. Moreover, the model can still maintain the processing ability for historical tasks after fine-tuning, which is particularly important in domain-specific models. In addition, since only some parameters are updated, the rapid loading and deployment of the medical diagnosis model can be achieved; in the subsequent inference process, seamless fine-tuning update can be realized by loading the fine-tuned LoRA weights without redeploying the entire model.

[0112] In the embodiments of the present application, in the same scenario, data of different task types can be collected over time. For example, in the medical field, data of different tasks such as diagnosis, treatment plan recommendation, and drug query are collected, and the model parameters are dynamically updated through multiple rounds of iteration to ensure that the model can meet the business requirements in real time.

[0113] Moreover, in each iteration process, the current weights of the medical diagnosis model to be iterated can be loaded first to ensure that the model can inherit the existing knowledge and perform iterative updates on this basis. Through iterative fine-tuning, the model can gradually learn and adapt to multiple task types, improving its generalization ability. Each fine-tuning will be based on the newly collected task data to further enhance the model's processing ability for diverse tasks. As new task data is continuously added, the model can dynamically adjust its parameters to adapt to new task requirements without losing the processing ability for old tasks.

[0114] To further improve the accuracy and efficiency of model iteration, for a single iteration update, the medical instruction data can be classified according to the length or complexity of the medical instruction data, and then the model can be trained in multiple rounds of training in ascending order of length or complexity, so as to preferentially use simple medical instruction data to warm up the model and then use complex medical instruction data to optimize the model, simulating the human learning process from easy to difficult to achieve efficient improvement of model capabilities.

[0115] In a specific implementation manner, fine-tuning training the medical diagnosis model to be iterated based on the medical instruction data includes the following steps:

[0116] Step 21: Determine the complexity of each question-and-answer pair in the medical instruction data, and divide the medical instruction data into multiple sample sets according to the complexity of each question-and-answer pair;

[0117] Step 22: Sort all sample sets in ascending order of complexity, and use the first sample set as the current training set;

[0118] Step 23: Use the current training set to perform fine-tuning training on the medical diagnosis model to be iterated. After the fine-tuning training is completed, use the next sample set of the current training set as the new current training set, and return to the step of using the current training set to perform fine-tuning training on the medical diagnosis model to be iterated until the current training set is the last sample set.

[0119] Among them, in step 21, the complexity of each Q&A pair can be determined first according to the total text length, the number of paragraphs, and the number of medical terms of each Q&A pair. For example, when the total text length is greater than the set length threshold, query the corresponding complexity according to the total text length; when the total text length is lower than the set length threshold, query the corresponding complexity according to the number of paragraphs and the number of medical terms.

[0120] Furthermore, the medical instruction data can be divided according to the complexity of each Q&A pair to obtain multiple sample sets with different complexities. In step 22, all sample sets can be sorted in ascending order of complexity, and the first sample set among them can be used as the current training set.

[0121] Furthermore, in step 23, the medical diagnosis model to be iterated can be first fine-tuned using the current training set. After the fine-tuning training is completed, on the basis of this fine-tuning training, continue to use the next sample set of the sorted current training set as the current training set again, and return to step 22 until the fine-tuning training of the model is completed using all sample sets to obtain the iterated medical diagnosis model.

[0122] Through the above steps 21 - 23, in one iteration process, the model can be fine-tuned in rounds according to the sample complexity, and simple samples are preferentially used to warm up the model, and then the sample difficulty is gradually increased. While improving the model iteration efficiency, the Q&A accuracy of the model can be further improved. Such a training method simulates the human learning process from easy to difficult, and gradually and effectively improves the model performance in an orderly manner.

[0123] S130: Determine the model performance scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model based on the large language model, and select the target model from them according to the model performance scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model.

[0124] Specifically, after obtaining the iterated medical diagnosis model, the medical diagnosis model to be iterated and the iterated medical diagnosis model can be evaluated, such as evaluating the performance of the medical diagnosis model in specific tasks and the generalization ability of the model. For example, in the medical field, medical diagnosis models are widely used in tasks such as patient Q&A, diagnostic assistance, and clinical decision support, and the performance of the medical diagnosis model can be evaluated in combination with the task requirements of the medical field.

[0125] In the embodiments of the present application, the medical diagnosis model to be iterated and the iterated medical diagnosis model can be loaded. The two models respectively respond to each question text in the test set and store the responses. Then, the performance of the two models is evaluated based on the response texts. For example, the medical diagnosis models before and after this iteration can be evaluated in terms of correctness, sufficiency, authenticity, and scientificity respectively.

[0126] In a specific implementation manner, determining the model performance scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model includes the following steps:

[0127] Step 31: Generate model prompt texts based on the response texts output by the medical diagnosis model to be iterated and the iterated medical diagnosis model, and the template text.

[0128] Step 32: Input the model prompt texts into the large language model to obtain the correctness scores, sufficiency scores, authenticity scores, and scientificity scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model.

[0129] Among them, the template text can describe the prompt template for the large language model to perform specific tasks, and the template prompt text can be the prompt words for judging the execution of specific tasks by the template. In step 31, the response texts output by the medical diagnosis model to be iterated and the iterated medical diagnosis model can be filled into the template text to obtain the template prompt text.

[0130] Exemplarily, the template text can be: "For the question text '''***''', the response text of model model1 is '''+response1+''', and the response text of model model2 is '''+response2+'''. Please evaluate the response texts of the two models from the perspectives of correctness, sufficiency, authenticity, and scientificity respectively."

[0131] Further, in step 32, the template prompt text can be used as a prompt and input into the large language model, and the large language model outputs the correctness scores, sufficiency scores, authenticity scores, and scientificity scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model. Among them, the large language model can use DeepSeek-R1, or can also be trained by using other types of models.

[0132] Through the above steps 31-step 32, the medical diagnosis models before and after this iteration can be evaluated by the large language model in terms of correctness, sufficiency, authenticity, and scientificity respectively. While ensuring the efficiency of model evaluation, the comprehensiveness of the evaluation can be ensured, and thus the accuracy of subsequent target model selection is improved.

[0133] In addition to obtaining the above scores through the large language model, it is also possible to identify the keywords in the response text output by the medical diagnosis model to be iterated and the iterated one, and determine whether the keywords are included in the corresponding medical record data, and determine the correctness score of the medical diagnosis model according to the number of included keywords.

[0134] Alternatively, the number of questions answered in the response text and whether the response data is empty can be retrieved, and the sufficiency score of the medical diagnosis model can be determined according to the number of questions answered, the number of questions in the corresponding question text, and whether the response data is empty.

[0135] Alternatively, similar text matching can be performed on the response text and the corresponding medical record data to locate the different text not included in the medical record data in the response text, and then the different text and the medical record data are input into the large language model to determine whether the different text can be deduced from the medical record data through the large language model, so as to obtain the authenticity score of the medical diagnosis model.

[0136] Alternatively, medical terms and colloquial words in the response text can be identified, and the scientificity score of the medical diagnosis model can be determined according to the number of identified medical terms and the number of colloquial words.

[0137] After obtaining the scores of the medical diagnosis model to be iterated and the iterated one, further, a target model can be selected from the two models. Among them, the target model is used for medical diagnosis and as the medical diagnosis model to be iterated in the next iteration process.

[0138] Exemplarily, it can be stipulated in the template text that the large language model outputs the target model according to the scores, so that after the large language model outputs the correctness score, sufficiency score, authenticity score and scientificity score of the two models, it further selects the target model from the two models and outputs the selection basis.

[0139] For example, the template text can also include: "Please select a model finalmodel with a better response according to the correctness score, sufficiency score, authenticity score and scientificity score, and give the comparative analysis process analyse_proess. The result is output in the following json format:

[0140] {{

[0141] "finalmodel": "The better model, the response is model1 or model2"

[0142] "analyse_proess": "Analyze the responses of the models from various perspectives"

[0143] }}"

[0144] In a specific embodiment, according to the model performance scores of the medical diagnosis model to be iterated and the iterated model, the target model is selected therefrom, including the following steps:

[0145] Step 41: Determine whether the correctness scores of the medical diagnosis model to be iterated and the iterated model are lower than the preset score threshold;

[0146] Step 42: If the correctness scores of both the medical diagnosis model to be iterated and the iterated model are higher than the preset score threshold, or if the correctness scores of both the medical diagnosis model to be iterated and the iterated model are lower than the preset score threshold, then based on the sufficiency score, authenticity score, and scientificity score of the medical diagnosis model to be iterated and the iterated model, determine the fusion score of the medical diagnosis model to be iterated and the iterated model;

[0147] Step 43: Determine the target model according to the fusion score of the medical diagnosis model to be iterated and the iterated model.

[0148] Specifically, considering that the correctness of the response of the medical diagnosis model is particularly important, therefore, a higher priority can be given to the correctness score. In Step 41, first determine whether the correctness scores of the medical diagnosis model before and after iteration are lower than the preset score threshold.

[0149] Furthermore, if the correctness score of one of the medical diagnosis models is higher than the preset score threshold, then this model can be directly used as the target model. Otherwise, the sufficiency scores, authenticity scores, and scientificity scores of the two models can be further compared.

[0150] Specifically, in Step 42, when the correctness scores of both the medical diagnosis model to be iterated and the iterated model are higher than the preset score threshold, or when the correctness scores of both the medical diagnosis model to be iterated and the iterated model are lower than the preset score threshold, the correctness score, sufficiency score, authenticity score, and scientificity score of the medical diagnosis model can be fused according to the set ratio to obtain the fusion score of the medical diagnosis model. Among them, the set ratio can be 3:3:2:2.

[0151] After calculating the fusion scores of the medical diagnosis model before and after iteration, the medical diagnosis model with the higher fusion score can be selected as the target model.

[0152] Through the above Steps 41 - 43, the target model can be preferentially selected through the correctness score. If the correctness scores of the two models before and after iteration are both relatively high or both relatively low, then the sufficiency score, authenticity score, and scientificity score can be further combined to calculate the fusion score, so as to compare the performance of the two models through the fusion score, ensuring the accuracy of the answers of the target model and the accuracy of subsequent iterations.

[0153] In the embodiments of the present application, in order to further improve the performance of the model in subsequent iterations, after selecting the target model, a feedback dataset can also be constructed according to the performance of the target model to participate in the next iteration, so as to resist the catastrophic forgetting of the model and maintain the generalized medical diagnosis ability.

[0154] In some embodiments, after selecting the target model according to the model performance scores of the medical diagnosis model to be iterated and the iterated model, the following steps are further included:

[0155] Step 51: Extract a model test set from all medical record data used in the historical iteration process;

[0156] Step 52: Input the model test set into the target model, and based on the output of the target model and the large language model, determine the correctness score, sufficiency score, authenticity score, and scientificity score of the target model;

[0157] Step 53: If at least one score is lower than the corresponding set threshold, it proves that the current model has lost some of its capabilities in this part, and collect such data to construct a feedback reinforcement dataset, where the feedback reinforcement dataset is used for the next model iteration update.

[0158] Among them, in step 51, in order to evaluate whether the target model has catastrophic forgetting, all medical record data participating in training in each previous iteration process can be randomly sampled to construct a model reinforcement test set for the current iteration round. For example, a model reinforcement test set can be extracted from the medical record test data in each previous iteration process.

[0159] Furthermore, in step 52, the model reinforcement test set can be input into the target model to obtain the corresponding output, and then the large language model is used to evaluate the output of the target model in multiple dimensions such as correctness, scientificity, authenticity, and sufficiency.

[0160] Specifically, if the score of a certain dimension is lower than the set threshold of that dimension, it means that the response of the target model in that dimension may be imperfect. At this time, the model reinforcement test set can be used as a pre-feedback dataset. At the same time, the large language model can also generate an evaluation basis for the imperfection of the model reinforcement test set in that dimension, and splice the evaluation basis with the pre-feedback dataset to construct a feedback dataset, which can be used to participate in the next iteration fine-tuning.

[0161] For example, when the model is iterated next time, medical record data can be re-obtained, and the target model can be fine-tuned and updated using the medical record data and the feedback dataset.

[0162] Medical diagnosis models usually involve high-risk decisions, such as diagnosis and treatment plan recommendation. Through the above steps 51 - step 53, the performance of the model can be improved through feedback reinforcement, and its reliability, accuracy, and safety in practical applications can be ensured, resisting catastrophic forgetting of the model and maintaining the generalized medical diagnosis ability.

[0163] The automatic iteration method of the medical diagnosis model provided by the embodiments of the present application, in response to detecting that a preset iteration period is met, obtains medical record data for model iteration and update, generates medical instruction data based on the medical record data and a large language model, then obtains the medical diagnosis model to be iterated, fine-tunes and trains the medical diagnosis model to be iterated based on the medical instruction data to obtain the iterated medical diagnosis model, thereby determining the model performance scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model, and selecting a target model from them for medical diagnosis using the target model, and using the target model as the medical diagnosis model to be iterated in the next iteration process. This method can achieve automatic iteration and optimization of the model, significantly reducing the manual intervention cost, improving the usability and practicality of the model, enabling the model to quickly respond to the dynamic changes of business requirements. Moreover, this method generates medical instruction data through the large language model and medical record data, which can solve the problem of the lack of high-quality medical data, improve the quality of the model fine-tuning dataset, and then improve the fine-tuning effect of the model, ensuring the accuracy of the model's question answering. In addition, this method determines the question answering scores of the model before and after iteration after iterative optimization, thereby selecting a target model for question answering and participating in the next iteration, which can avoid the errors gradually generated during the iteration process and ensure the question answering effect of the model.

[0164] Figure 2 It is an automatic iteration flowchart of a medical diagnosis model provided by the embodiments of the present application, as Figure 2 shown. For each model iteration, medical record data can be obtained first, and then medical instruction data can be generated in combination with a large language model, and the medical instruction data is preprocessed. Further, the medical instruction data is used to fine-tune the current medical diagnosis model to be iterated, and the iterated model and the model before entering the model fine-tuning are evaluated to select a target model.

[0165] After obtaining the target model, the target model can be used for model application. Moreover, a test set can be randomly sampled to test the target model, and a feedback dataset can be constructed based on the test results. The feedback dataset can be used for the next model iteration.

[0166] Through the above process, the automatic iterative optimization of the medical diagnosis model can be achieved, significantly reducing the cost of manual intervention, improving the usability and practicality of the system, and providing strong technical support for the intelligent application in the professional field. Moreover, based on the sample self-generation technology of the large language model and real medical records, the problem of relatively scarce high-quality medical data is solved, the quality of medical fine-tuning samples is effectively improved, and the problem that the model after fine-tuning appears as a "parrot" due to excessive text repetition during the direct filling of medical record data is avoided, thereby enhancing the fine-tuning effect of the model.

[0167] In addition, in the embodiments of the present application, by generating samples containing corresponding diagnostic processes and diagnostic bases, the samples can internalize the thinking chain, thereby improving the quality of the samples and enhancing the reasoning ability of the fine-tuning model.

[0168] Furthermore, in the iterative process, a feedback reinforcement link is set. By continuously using the test sets in all previous iterative rounds to test the current target model, data with poor model responses are screened to construct a feedback data set, and this feedback data set is used for the next round of iterative training, thus effectively resisting the catastrophic forgetting of the model and realizing the continuous learning of the model.

[0169] Figure 3 It is a schematic structural diagram of an automatic iterative system for a medical diagnosis model provided by an embodiment of the present application. The system includes a data processing module 310, a model fine-tuning module 320, and a model evaluation module 330, where:

[0170] The data processing module 310 is configured to, in response to detecting that a preset iteration period is satisfied, obtain medical record data for model iterative update, and generate medical instruction data based on the medical record data and the large language model;

[0171] The model fine-tuning module 320 is configured to obtain the medical diagnosis model to be iterated, and perform fine-tuning training on the medical diagnosis model to be iterated based on the medical instruction data to obtain the iterated medical diagnosis model;

[0172] The model evaluation module 330 is configured to determine the model performance scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model based on the large language model, and select a target model from the model performance scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model.

[0173] Wherein, the target model is used for medical diagnosis and as the medical diagnosis model to be iterated in the next iteration process.

[0174] Based on the above embodiments, optionally, the data processing module 310 is further configured to obtain medical instruction data and the diagnostic basis corresponding to each answer text in the medical instruction data based on the medical record data and the large language model; extract the diagnostic thinking process text corresponding to the answer text from the diagnostic basis, and update the answer text based on the diagnostic thinking process text.

[0175] Based on the above embodiments, optionally, the data processing module 310 is further configured to obtain a preset model prompt text; input the preset model prompt text and the medical record data into the large language model, so that the large language model generates medical instruction data that meets the preset model prompt text, and output the diagnostic basis corresponding to each answer text in the medical instruction data; wherein, the preset model prompt text includes not fabricating false information, the number of words in the answer text exceeding a preset number of words, and the format of the answer text meeting a preset format.

[0176] Based on the above embodiments, optionally, the data processing module 310 is further configured to, for each question-and-answer pair in the medical instruction data, perform medical term localization on the answer text and the corresponding medical record data in the question-and-answer pair respectively based on a pre-trained medical term extraction model, to obtain the generated data corresponding to the medical terms in the answer text and the real data corresponding to the medical terms in the medical record data; determine whether the generated data is consistent with the real data, and if not, remove the question-and-answer pair.

[0177] Based on the above embodiments, optionally, the model fine-tuning module 320 is further configured to determine the complexity of each question-and-answer pair in the medical instruction data, and divide the medical instruction data into multiple sample sets according to the complexity of each question-and-answer pair; sort all the sample sets in ascending order of complexity, and use the first sample set as the current training set; perform fine-tuning training on the medical diagnosis model to be iterated using the current training set, and after the fine-tuning training is completed, use the next sample set of the current training set as the new current training set, and return to the step of performing fine-tuning training on the medical diagnosis model to be iterated using the current training set until the current training set is the last sample set.

[0178] Based on the above embodiments, optionally, the model evaluation module 330 is further configured to generate a model prompt text based on the answer texts output by the medical diagnosis model to be iterated and the iterated medical diagnosis model, and a template text; input the model prompt text into the large language model to obtain the correctness score, sufficiency score, authenticity score, and scientificity score of the medical diagnosis model to be iterated and the iterated medical diagnosis model.

[0179] Based on the above embodiments, optionally, the model evaluation module 330 is further configured to determine whether the correctness scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model are lower than a preset score threshold; if the correctness scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model are both higher than the preset score threshold, or if the correctness scores of the medical diagnosis model to be iterated and the iterated medical diagnosis model are both lower than the preset score threshold, then based on the sufficiency score, authenticity score, and scientificity score of the medical diagnosis model to be iterated and the iterated medical diagnosis model, determine the fusion score of the medical diagnosis model to be iterated and the iterated medical diagnosis model; determine the target model according to the fusion score of the medical diagnosis model to be iterated and the iterated medical diagnosis model.

[0180] Based on the above embodiments, optionally, the system further includes a model feedback module, configured to extract a model reinforcement test set from all medical record data used in the historical iteration process; input the model reinforcement test set into the target model, and based on the output of the target model and the large language model, determine the correctness score, sufficiency score, authenticity score, and scientificity score of the target model; if at least one score is lower than the corresponding set threshold, then construct a feedback data set based on the model reinforcement test set, where the feedback data set is used for the next model iteration update.

[0181] The above system can run on a machine with a GPU. For fine-tuning, two A100s can be used, and for testing the large model required, other service interfaces can be called.

[0182] The automatic iteration system of the medical diagnosis model provided by the embodiments of the present application can execute the steps in the automatic iteration method of the medical diagnosis model provided by the method embodiments of the present application, and the implementation steps and beneficial effects are not described herein again.

[0183] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 4 shown, the electronic device 400 includes one or more processors 401 and a memory 402.

[0184] The processor 401 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0185] The memory 402 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor 401 may run the program instructions to implement the automatic iteration method of the medical diagnosis model of any embodiment of the present application described above and / or other desired functions. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage media.

[0186] In one example, the electronic device 400 may further include: an input device 403 and an output device 404, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including warning prompt information, braking force, etc. The output device 404 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0187] Of course, for simplicity, Figure 4 only some of the components related to the present application in the electronic device 400 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 400 may further include any other appropriate components.

[0188] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, and when the computer program instructions are run by a processor, the processor is caused to execute the steps of the automatic iteration method of the medical diagnosis model provided by any embodiment of the present application.

[0189] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the embodiments of the present application. The programming languages include object-oriented programming languages, such as Java, C++, etc., and also include conventional procedural programming languages, such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0190] In addition, an embodiment of the present application may also be a computer-readable storage medium storing computer program instructions, which, when run by a processor, cause the processor to execute the steps of the automatic iteration method of the medical diagnosis model provided by any embodiment of the present application.

[0191] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0192] It should be noted that the terms used in the present application are only for describing specific embodiments and do not limit the scope of the present application. As shown in the specification and claims of the present application, unless the context clearly indicates an exception, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include plural. The term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such a process, method, or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, or device comprising the element.

[0193] It should also be noted that the orientation or positional relationship indicated by terms such as "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present application. Unless otherwise clearly specified and limited, terms such as "installed", "connected", "coupled", etc. should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0194] In this article, specific examples are used to illustrate the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. The above is only the preferred implementation manner of the present application. It should be noted that due to the limitation of literal expression, there are objectively infinite specific structures. For those of ordinary skill in the art, without departing from the principle of the present application, several improvements, refinements or changes can be made, or the above technical features can be combined in an appropriate manner; these improvements, refinements, changes or combinations, or directly applying the concept and technical solution of the invention to other occasions without improvement, shall all be regarded as the protection scope of the present application.

Claims

1. An automatic iteration method for a medical diagnosis model, characterized in that: include: In response to detecting that a preset iteration cycle is met, obtaining medical record data for iterative updating of the model, and generating medical instruction data based on the medical record data and the large language model, wherein the preset iteration cycle is dynamically adjusted based on the data increment of the outpatient information management system; Acquire a medical diagnosis model to be iterated, and fine-tune the medical diagnosis model to be iterated based on the medical instruction data to obtain an iterated medical diagnosis model, wherein the medical diagnosis model to be iterated is a target model determined after the last iterative training and evaluation; Determine the model performance scores of the medical diagnosis model to be iterated and after the iteration based on the large language model, and select a target model from the medical diagnosis model to be iterated and after the iteration according to the model performance scores of the medical diagnosis model to be iterated and after the iteration; The target model is used for medical diagnosis and is used as a medical diagnosis model to be iterated in the next iteration process; Among them, the medical forensic model to be iterated and after iteration is evaluated from the perspective of correctness, sufficiency, authenticity and scientificity; the keywords in the answer text output by the medical forensic model are identified, and the correctness score is determined according to the number of keywords contained in the corresponding medical record data; the sufficiency score is determined according to the number of questions answered by the medical forensic model, the number of questions in the corresponding question text and whether the answer data is empty; the difference text that is not included in the medical record data is located in the answer text output by the medical forensic model, the difference text and the medical record data are input into the large language model, and it is determined whether the difference text can be derived from the medical record data to obtain the authenticity score; the scientificity score is determined according to the number of medical terms and the number of colloquial words in the answer text output by the medical forensic model.

2. The method according to claim 1, characterized in that Generate medical instruction data based on the medical record data and the large language model, including: Based on the medical case record data and the large language model, obtain medical instruction data and diagnostic evidence corresponding to each answer text in the medical instruction data; The diagnostic thought process text corresponding to the answer text is extracted from the diagnostic basis, and the answer text is updated based on the diagnostic thought process text.

3. The method according to claim 2, characterized in that Based on the medical case record data and the large language model, medical instruction data and diagnostic evidence corresponding to each answer text in the medical instruction data are obtained, including: Get the preset model prompt text; Inputting the preset model prompt text and the medical record data into the large language model, so that the large language model generates medical instruction data that meets the preset model prompt text, and outputs the diagnosis basis corresponding to each answer text in the medical instruction data; The preset model prompt text includes that false information cannot be constructed, the number of words in the answer text exceeds the preset number of words, and the format of the answer text meets the preset format.

4. The method according to claim 1, characterized in that: After generating medical instruction data based on the medical record data and the large language model, the method further includes: For each question-answer pair in the medical instruction data, based on a pre-trained medical term extraction model, medical term location is performed on the answer text and the corresponding medical record data in the question-answer pair to obtain generated data corresponding to the medical term in the answer text and real data corresponding to the medical term in the medical record data; Determine whether the generated data is consistent with the real data; if not, remove the question-answer pair.

5. The method according to claim 1, characterized in that: Fine-tuning and training the medical diagnosis model to be iterated based on the medical instruction data includes: Determining the complexity of each question-answer pair in the medical instruction data, and dividing the medical instruction data into a plurality of sample sets according to the complexity of each question-answer pair; Sort all sample sets in order of complexity from low to high, and use the first sample set as the current training set; The medical diagnosis model to be iterated is fine-tuned using the current training set, and after the fine-tuning training is completed, the next sample set of the current training set is used as the new current training set, and the step of fine-tuning the medical diagnosis model to be iterated using the current training set is returned to, until the current training set is the last sample set.

6. The method according to claim 1, characterized in that Determining model performance scores of the medical diagnosis model to be iterated and after iteration based on the large language model includes: Generate model prompt text based on the answer text output by the medical diagnosis model to be iterated and after iteration, and the template text; The model prompt text is input into the large language model to obtain the correctness score, adequacy score, authenticity score and scientificity score of the medical diagnosis model to be iterated and after iteration.

7. The method according to claim 6, characterized in that The step of selecting a target model according to the model performance scores of the medical diagnosis model to be iterated and the iterated model comprises: Determine whether the correctness scores of the medical diagnosis model to be iterated and after iteration are lower than a preset score threshold; If the correctness scores of the medical diagnosis model to be iterated and after the iteration are both higher than the preset score threshold, or the correctness scores of the medical diagnosis model to be iterated and after the iteration are both lower than the preset score threshold, then based on the adequacy score, authenticity score and scientificity score of the medical diagnosis model to be iterated and after the iteration, determine the fusion score of the medical diagnosis model to be iterated and after the iteration; The target model is determined according to the fusion score of the medical diagnosis model to be iterated and the iterated model.

8. The method according to claim 6, characterized in that After selecting the target model based on the model performance scores of the medical diagnosis model to be iterated and the iterated model, the following steps are also included: Extract the model reinforcement test set from all medical record data used in historical iterations; Inputting the model reinforcement test set into the target model, and determining the correctness score, adequacy score, authenticity score and scientificity score of the target model based on the output of the target model and the large language model; If there is at least one score lower than the corresponding set threshold, a feedback data set is constructed based on the model reinforcement test set, wherein the feedback data set is used for the next model iteration update.

9. An automatic iteration system for a medical diagnosis model, characterized in that: The system includes a data processing module, a model fine-tuning module, and a model evaluation module, wherein: The data processing module is used for acquiring medical record data for model iteration update in response to detecting that a preset iteration cycle is met, and generating medical instruction data based on the medical record data and the large language model, wherein the preset iteration cycle is dynamically adjusted based on the data increment of the outpatient information management system; The model fine-tuning module is used to obtain the medical diagnosis model to be iterated, and fine-tune the medical diagnosis model to be iterated based on the medical instruction data to obtain the iterated medical diagnosis model, wherein the medical diagnosis model to be iterated is the target model determined after the last iterative training and evaluation; The model evaluation module is used to determine the model performance scores of the medical diagnosis model to be iterated and after the iteration based on the large language model, and select a target model from the medical diagnosis model to be iterated and after the iteration according to the model performance scores of the medical diagnosis model to be iterated and after the iteration; The target model is used for medical diagnosis and is used as a medical diagnosis model to be iterated in the next iteration process; Among them, the medical forensic model to be iterated and after iteration is evaluated from the perspective of correctness, sufficiency, authenticity and scientificity; the keywords in the answer text output by the medical forensic model are identified, and the correctness score is determined according to the number of keywords contained in the corresponding medical record data; the sufficiency score is determined according to the number of questions answered by the medical forensic model, the number of questions in the corresponding question text and whether the answer data is empty; the difference text that is not included in the medical record data is located in the answer text output by the medical forensic model, the difference text and the medical record data are input into the large language model, and it is determined whether the difference text can be derived from the medical record data to obtain the authenticity score; the scientificity score is determined according to the number of medical terms and the number of colloquial words in the answer text output by the medical forensic model.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, which enables a computer to execute the steps of the automatic iteration method of the medical diagnosis model according to any one of claims 1 to 8.

Images