Medical diagnosis model training method, medical diagnosis method and related device

Abstract

The invention provides a medical diagnosis model training method, a medical diagnosis method and a related device. The medical diagnosis model training method comprises the following steps: acquiring first medical data and second medical data; training LLM by taking the first medical data as a training sample and taking knowledge experience obtained from a medical textbook as priori knowledge to obtain a common knowledge expert diagnosis module; using the second medical data as a training sample, and adopting a retrieval enhancement generation (RAG) technology to train the LLM to obtain a case expert diagnosis module; and taking a first diagnosis result generated by the common sense expert diagnosis module in the training stage and a second diagnosis result generated by the case expert diagnosis module in the training stage as training samples, and training LLM by adopting a thinking chain CoT technology to obtain an unbiased diagnosis module so as to obtain a medical diagnosis model. According to the method, the diagnosis accuracy of the medical diagnosis model on diversified diseases is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of auxiliary medical diagnosis, and in particular to a medical diagnosis model training method, a medical diagnosis method and related devices. Background Art

[0002] In the process of developing a medical multimodal large language model (MLLM), choosing appropriate training data is crucial, as it will directly affect the performance and application effect of the model.

[0003] In related technologies, existing MLLMs usually rely on public medical literature libraries, such as PubMed. Since the medical needs in the real world are more extensive, using only the cases recorded in the medical literature library to assist in the diagnosis and analysis of diseases will introduce specific biases, which can easily cause bias in multimodal language models due to the singleness of the data, resulting in unreliable diagnostic results and inability to meet the management and treatment needs of diverse diseases. Summary of the invention

[0004] The present invention provides a medical diagnosis model training method, a medical diagnosis method and related devices, which are used to solve the defect that the prior art uses MLLM to assist in disease diagnosis, which is prone to generate specific biases and leads to inaccurate diagnosis results, thereby improving the diagnostic accuracy of the medical diagnosis model for a variety of diseases.

[0005] The present invention provides a medical diagnosis model training method, comprising: Acquire first medical data and second medical data; wherein the first medical data is determined based on medical literature data, and the second medical data is determined based on medical clinical data; The first medical data is used as a training sample, and the large language model LLM is trained with the knowledge and experience obtained from medical textbooks as prior knowledge to obtain a common sense expert diagnosis module; the second medical data is used as a training sample, and the retrieval enhancement generation RAG technology is used to train the LLM to obtain a case expert diagnosis module; the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage are used as training samples, and the thinking chain CoT technology is used to train the LLM to obtain an unbiased diagnosis module; based on the common sense expert diagnosis module, the case expert diagnosis module and the unbiased diagnosis module, the medical diagnosis model is obtained.

[0006] According to a medical diagnosis model training method provided by the present invention, after acquiring the first medical data and the second medical data, the method further includes: performing data cleaning on the first medical data and the second medical data in sequence to obtain cleaned data; A generative adversarial network is used to perform sample expansion on the cleaned data to obtain expanded first medical data and expanded second medical data.

[0007] According to a medical diagnosis model training method provided by the present invention, after obtaining the medical diagnosis model, the method further includes: Collecting output data of the medical diagnosis model in a plurality of benchmark data sets respectively; A fairness index is calculated according to the plurality of output data, and the performance of the medical diagnosis model is evaluated according to the fairness index.

[0008] According to a medical diagnosis model training method provided by the present invention, after obtaining the medical diagnosis model, the method further includes: Obtain feedback data from clinicians and patients; The network parameters of each module of the medical diagnosis model are adjusted according to the feedback data to update the medical diagnosis model.

[0009] The present invention also provides a medical diagnosis method, comprising: Acquiring medical image data to be diagnosed; The medical image data is diagnosed based on a medical diagnosis model to obtain a diagnosis result; wherein the medical diagnosis model is trained based on the medical diagnosis model training method.

[0010] The present invention also provides a medical diagnosis model training device, comprising: A first data acquisition module, used to acquire first medical data and second medical data; wherein the first medical data is determined based on medical literature data, and the second medical data is determined based on medical clinical data; A training module is used to use the first medical data as a training sample and the knowledge and experience obtained from medical textbooks as prior knowledge to train the large language model LLM to obtain a common sense expert diagnosis module; use the second medical data as a training sample and use the retrieval enhancement generation RAG technology to train the LLM to obtain a case expert diagnosis module; use the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples and use the chain of thought CoT technology to train the LLM to obtain an unbiased diagnosis module; obtain the medical diagnosis model based on the common sense expert diagnosis module, the case expert diagnosis module and the unbiased diagnosis module.

[0011] The present invention also provides a medical diagnostic device, comprising: A second data acquisition module, used to acquire medical image data to be diagnosed; A diagnosis module is used to diagnose the medical image data based on a medical diagnosis model to obtain a diagnosis result; wherein the medical diagnosis model is trained based on the medical diagnosis model training method.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements any of the medical diagnosis model training methods or medical diagnosis methods described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the medical diagnosis model training methods or medical diagnosis methods described above.

[0014] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the medical diagnosis model training methods or medical diagnosis methods described above.

[0015] The medical diagnosis model training method, medical diagnosis method and related devices provided by the present invention obtain first medical data and second medical data; use the first medical data as training samples, and use knowledge and experience obtained from medical textbooks as prior knowledge to train a large language model LLM to obtain a common sense expert diagnosis module; use the second medical data as training samples, and use the retrieval enhancement generation RAG technology to train the LLM to obtain a case expert diagnosis module; use the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples, and use the thinking chain CoT technology to train the LLM to obtain an unbiased diagnosis module, thereby obtaining a medical diagnosis model, which improves the diagnostic accuracy of the medical diagnosis model for diverse diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is one of the flow charts of the medical diagnosis model training method provided by the present invention.

[0018] Figure 2It is a schematic diagram of the process of training the LLM model using the RAG technology provided by the present invention.

[0019] Figure 3 It is a schematic diagram of the process of training the LLM model using the CoT technology provided by the present invention.

[0020] Figure 4 This is the second flow chart of the medical diagnosis model training method provided by the present invention.

[0021] Figure 5 It is a schematic flow chart of the medical diagnosis method provided by the present invention.

[0022] Figure 6 It is a structural schematic diagram of the medical diagnosis model training device provided by the present invention.

[0023] Figure 7 It is a schematic diagram of the structure of the medical diagnostic device provided by the present invention.

[0024] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Combine the following Figure 1-Figure 7 The medical diagnosis model training method, medical diagnosis method and related devices of the present invention are described.

[0027] Figure 1 This is one of the flow charts of the medical diagnosis model training method provided by the present invention, such as Figure 1 As shown, the medical diagnosis model training method includes the following steps: Step 110: Acquire first medical data and second medical data; wherein the first medical data is determined based on medical literature data, and the second medical data is determined based on medical clinical data.

[0028] In this step, the first medical data includes literature data such as medical books, papers or journals recorded in a public medical database. This type of medical data includes text, image or video data; the first medical data includes a variety of medical common sense data.

[0029] In this step, the second medical data includes the patient's clinical condition, diagnosis, treatment process and result data. This type of medical data may also include text, image or video data; the second medical data includes diagnosis and analysis data of a variety of rare diseases; for example, the second medical data can be obtained from PMC (PubMed Central) data.

[0030] Step 120: Using the first medical data as training samples and the knowledge and experience obtained from medical textbooks as prior knowledge to train the large language model LLM, so as to obtain a common sense expert diagnosis module; using the second medical data as training samples and using the retrieval enhancement generation RAG technology to train the LLM, so as to obtain a case expert diagnosis module; using the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples, and using the chain of thought CoT technology to train the LLM, so as to obtain an unbiased diagnosis module; obtaining a medical diagnosis model based on the common sense expert diagnosis module, the case expert diagnosis module and the unbiased diagnosis module.

[0031] In this step, the initial framework of the medical diagnosis model includes at least three LLMs (Large Language Models), and the network parameter settings of each LLM can be exactly the same, for example, the weight factor ratio corresponding to the two LLMs used for disease diagnosis can be 1:1; the network parameter settings of each LLM can be different, for example, the weight factor ratio corresponding to the two LLMs used for disease diagnosis can be 2:3 or other proportion data.

[0032] In this embodiment, the LLM is trained and fine-tuned with adjustments and instructions from medical textbooks. For example, by adding instructions to the input image data according to medical textbook knowledge, the model can be guided to generate diagnostic results based on medical common sense required for specific tasks.

[0033] In this embodiment, in the initial stage of LLM training, the first medical data is used to train the LLM to learn the statistical laws, semantic information and contextual relationships of the language; pre-training usually adopts a self-supervised learning method, such as training the model by predicting masked words (Masked Language Model) or the next sentence (Next Sentence Prediction), and the LLM can acquire a wide range of language understanding and generation capabilities through pre-training; after pre-training, the LLM will be fine-tuned on the data of specific tasks to adapt to specific medical common sense diagnosis scenarios; during the training process, feedback reinforcement learning can also be used to guide the learning process of the model by comparing the training output results with the actual results, so as to continuously adjust the network parameters until the model prediction performance meets the expected values ​​and a usable common sense expert diagnosis module is obtained.

[0034] Figure 2 It is a flow chart of the LLM model training process using RAG technology provided by the present invention. Figure 2 In the illustrated embodiment, training the LLM using the RAG (Retrieval-Augmented Generation) technique includes the following steps: (1) Create quick queries based on user needs; (2) Retrieval stage: Retrieve relevant information from pre-built external knowledge sources based on the user's quick query; In this embodiment, the external knowledge source can be a structured database, an unstructured text document, or a document collection indexed specifically for this task; the relevance between the query and the documents in the database is measured by the retriever to achieve effective information retrieval.

[0035] In this embodiment, the type of the retriever can be sparse, such as retrieval based on inverted index and word frequency, or dense, such as semantic retrieval based on vector embedding. The granularity of the retrieval can be document level, paragraph level, word level or entity level, depending on the requirements of the task and the design of the system.

[0036] (3) Enhancement stage: The retrieved relevant information is used to enhance the contextual semantics or context of the LLM, for example, by extracting key facts or summaries from the document through natural language processing techniques; (4) Generation phase: LLM uses the enhanced context to generate a response by combining the retrieved information with the original query and then generating a coherent, accurate, and informative answer through the model; In this embodiment, the case expert diagnosis module can also be iterated and optimized, for example, the retrieval strategy and generation model are continuously adjusted according to user feedback and performance evaluation results to improve the overall performance and user experience; or the retriever or generator is fine-tuned to adapt to specific tasks or data sets.

[0037] In this embodiment, the Chain of Thought (CoT) technology simulates the step-by-step reasoning process of humans, allowing the model to gradually derive a series of intermediate steps or sub-goals before generating the final answer. These intermediate steps constitute a "chain of thought" that ultimately guides the model to the correct result.

[0038] Figure 3 It is a flow chart of using CoT technology to train LLM model provided by the present invention. Figure 3 In the illustrated embodiment, training the LLM model using the CoT technology may be achieved by the following steps: (1) Problem identification: clarify the problems that need to be solved in the training task; (2) Decompose them into sub-problems and analyze whether these problems require multi-step reasoning.

[0039] (3) For each word problem, manually design or let the model automatically generate the intermediate reasoning steps required to solve the problem, which are used to solve each sub-problem; for example, these steps can be problem decomposition, logical reasoning, calculation process, etc.

[0040] (4) Integrate the solution and construct a training dataset that includes questions, intermediate reasoning steps, and final answers; this data can be manually annotated (Few-shot CoT) or automatically generated through specific prompts (Zero-shot CoT).

[0041] (5) Train the model to implement CoT reasoning and conclusion. Use the training data constructed in step (3) to train the LLM. The model generates reasonable intermediate steps based on the given question and finally obtains the correct answer.

[0042] In this embodiment, the common sense expert diagnosis module and the case expert diagnosis module can share an input terminal, and the output terminals of the common sense expert diagnosis module and the case expert diagnosis module are respectively connected to the input terminal of the unbiased diagnosis module.

[0043] For example, the test image A is input into the common sense expert diagnosis module and the case expert diagnosis module respectively to obtain the corresponding common sense expert diagnosis results and pathology expert diagnosis results, and then the common sense expert diagnosis results and the pathology expert diagnosis results are input into the unbiased diagnosis module for unbiased reasoning, and the unbiased diagnosis module outputs the corresponding unbiased diagnosis results.

[0044] The medical diagnosis model training method provided by the embodiment of the present invention obtains first medical data and second medical data; uses the first medical data as training samples, and uses knowledge and experience obtained from medical textbooks as prior knowledge to train a large language model LLM to obtain a common sense expert diagnosis module; uses the second medical data as training samples, and uses the retrieval enhancement generation RAG technology to train the LLM to obtain a case expert diagnosis module; uses the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples, and uses the thinking chain CoT technology to train the LLM to obtain an unbiased diagnosis module, thereby obtaining a medical diagnosis model, which improves the diagnostic accuracy of the medical diagnosis model for a variety of diseases.

[0045] Figure 4 This is the second flow chart of the medical diagnosis model training method provided by the present invention. Figure 4In the illustrated embodiment, case data (corresponding to patient cases) are obtained from relevant literature, and the case data are respectively input into a common sense expert diagnosis module (corresponding to LLMA-generalized knowledge model) and a case expert diagnosis module (corresponding to LLMB-rare case model, obtained based on RAG technology training), to obtain two corresponding diagnosis results (the validity of each other cannot be determined), and the two results are respectively input into an unbiased diagnosis module (corresponding to LLMC-unbiased summarizer, obtained based on thought chain technology training), to output the final diagnosis result (corresponding to the decision: antiphospholipid syndrome caused by systemic lupus erythematosus).

[0046] According to a medical diagnosis model training method provided by the present invention, after obtaining the first medical data and the second medical data, the method also includes: performing data cleaning on the first medical data and the second medical data in turn to obtain cleaned data; using a generative adversarial network to perform sample expansion on the cleaned data to obtain expanded first medical data and expanded second medical data.

[0047] In this embodiment, the data cleaning method includes missing value processing, such as processing missing values ​​by deleting records containing missing values, filling with mean or median, or adopting more complex interpolation methods; it also includes duplicate value processing, such as identifying and deleting duplicate records in the data set to ensure that each record is unique; it also includes outlier processing, such as detecting outliers or outliers in the data; it also includes data format standardization, such as converting data into a unified format, such as date format, number format or text format; it also includes data type conversion, data normalization and data dimension reduction operations.

[0048] In this embodiment, sample expansion includes geometric transformation, such as flipping, rotating, scaling, translating and cropping of images. These operations can increase the diversity of samples without changing the semantics of the image; it also includes color transformation, such as adjusting the brightness, contrast, saturation, etc. of the image, simulating images under different lighting conditions, and improving the adaptability of the model to lighting changes; it also includes noise addition, such as adding Gaussian noise, salt and pepper noise, etc. to the image to improve the robustness of the model to noise; it also includes blurring, using generative models such as generative adversarial networks (GAN) or variational autoencoders (VAE) to generate new data samples, defining rules to generate new data samples and synthesizing data, and other operations.

[0049] The present invention provides a medical diagnosis model training method, which improves data quality by performing data cleaning and sample expansion on first medical data and second medical data, thereby improving the prediction performance of LLM.

[0050] In some embodiments, after obtaining the medical diagnostic model, the method further includes: collecting output data of the medical diagnostic model in multiple benchmark data sets respectively; calculating fairness indicators based on the multiple output data, and evaluating the performance of the medical diagnostic model based on the fairness indicators.

[0051] In this embodiment, fairness indicators include perplexity (PPL), accuracy, F1 score, etc.; in addition, a dedicated benchmark test set can also be used to evaluate the performance of the model on a specific task.

[0052] In this embodiment, different data sets are used to test the trained medical diagnosis model to obtain multiple test results, and the corresponding fairness index values ​​are calculated based on the multiple test results. Finally, the calculated data is compared with the preset threshold to obtain the performance of the medical diagnosis model.

[0053] The present invention provides a medical diagnosis model training method, which collects output data of the medical diagnosis model in multiple benchmark data sets respectively; calculates fairness indicators based on the multiple output data; and evaluates the performance of the medical diagnosis model based on the fairness indicators, thereby realizing a scientific evaluation of the performance of the medical diagnosis model, thereby improving the reliability of the prediction results output by the model.

[0054] In some embodiments, after obtaining the medical diagnosis model, the method further includes: obtaining feedback data from clinicians and patients; and adjusting network parameters of each module of the medical diagnosis model according to the feedback data to update the medical diagnosis model.

[0055] In this embodiment, the medical diagnosis model is continuously optimized in the following manner: (1) Establish a feedback mechanism to collect feedback from clinicians and patients and continuously optimize the model.

[0056] Specifically, you can collect user feedback, performance indicators, error logs and other data after the model is deployed; identify model performance problems by analyzing the feedback data, such as low accuracy, low recall, bias, unfairness, etc.; use appropriate evaluation indicators to quantify the performance of the model; deeply analyze the model errors to understand on which types of data the model performs poorly and the reasons for the errors; then adjust the model parameters based on the analysis results; retrain the model with the adjusted parameters; or deploy the retrained model to the production environment to replace the old model; finally, you can continuously monitor the performance of the new model to ensure that the adjusted model achieves the expected results and discover new problems in a timely manner.

[0057] (2) Use new technologies or new large models to continuously iterate and update the model to maintain its advancement and effectiveness.

[0058] The present invention provides a medical diagnosis model training method, which obtains feedback data from clinicians and patients; adjusts the network parameters of each module of the medical diagnosis model according to the feedback data to update the medical diagnosis model, realizes online update of the medical diagnosis model, and ensures that the performance of the medical diagnosis model is continuously reliable.

[0059] The medical diagnosis method provided by the present invention is described below. The medical diagnosis method described below and the medical diagnosis model training method described above can be referenced to each other.

[0060] Figure 5 It is a schematic diagram of the flow chart of the medical diagnosis method provided by the present invention, such as Figure 5 As shown, the medical diagnosis method comprises the following steps: Step 510: Acquire medical image data to be diagnosed; Step 520: diagnose the medical image data based on the medical diagnosis model to obtain a diagnosis result; wherein the medical diagnosis model is trained based on a medical diagnosis model training method.

[0061] In this step, the medical image data to be diagnosed may be literature data such as medical books, papers or journals recorded in a public medical database, or may be the patient's clinical condition, diagnosis, treatment process and result data.

[0062] In this step, the medical diagnosis model includes a common sense expert diagnosis module, a case expert diagnosis module and an unbiased diagnosis module; wherein, the common sense expert diagnosis module uses the first medical data as a training sample, and uses the knowledge and experience obtained from medical textbooks as prior knowledge to train the large language model LLM; the case expert diagnosis module uses the second medical data as a training sample, and uses the retrieval enhancement generation RAG technology to train the LLM; the unbiased diagnosis module uses the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples, and uses the thinking chain CoT technology to train the LLM.

[0063] In this embodiment, the common sense expert diagnosis module, the case expert diagnosis module and the unbiased diagnosis module are as described in the above embodiment and will not be described in detail in this embodiment.

[0064] In this embodiment, the diagnosis result may be the common sense expert diagnosis result output by the common sense expert diagnosis module or the case expert diagnosis result output by the case expert diagnosis module, or it may be a new diagnosis result obtained by integrating and reasoning the common sense expert diagnosis result and the case expert diagnosis result.

[0065] The medical diagnosis method provided by the embodiment of the present invention diagnoses medical image data through a medical diagnosis model to obtain a diagnosis result, thereby improving the diagnostic accuracy and efficiency of various diseases.

[0066] The medical diagnosis model training device provided by the present invention is described below. The medical diagnosis model training device described below and the medical diagnosis model training method described above can be referenced to each other.

[0067] Figure 6 is a schematic diagram of the structure of the medical diagnosis model training device provided by the present invention, such as Figure 6 As shown, the medical diagnosis model training device includes: a first data acquisition module 610 and a training module 620.

[0068] A first data acquisition module 610 is used to acquire first medical data and second medical data; wherein the first medical data is determined based on medical literature data, and the second medical data is determined based on medical clinical data; The training module 620 is used to use the first medical data as a training sample and the knowledge and experience obtained from medical textbooks as prior knowledge to train the large language model LLM to obtain a common sense expert diagnosis module; use the second medical data as a training sample and use the retrieval enhancement generation RAG technology to train the LLM to obtain a case expert diagnosis module; use the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples, and use the thinking chain CoT technology to train the LLM to obtain an unbiased diagnosis module; based on the common sense expert diagnosis module, the case expert diagnosis module and the unbiased diagnosis module, a medical diagnosis model is obtained.

[0069] The medical diagnosis model training device provided by the embodiment of the present invention obtains first medical data and second medical data; uses the first medical data as training samples, and uses knowledge and experience obtained from medical textbooks as prior knowledge to train a large language model LLM to obtain a common sense expert diagnosis module; uses the second medical data as training samples, and uses the retrieval enhancement generation RAG technology to train the LLM to obtain a case expert diagnosis module; uses the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples, and uses the thinking chain CoT technology to train the LLM to obtain an unbiased diagnosis module, thereby obtaining a medical diagnosis model, which improves the diagnostic accuracy of the medical diagnosis model for a variety of diseases.

[0070] The medical diagnostic device provided by the present invention is described below. The medical diagnostic device described below and the medical diagnostic method described above can be referenced to each other.

[0071] Figure 7is a schematic diagram of the structure of the medical diagnostic device provided by the present invention, such as Figure 7 As shown, the medical diagnosis device includes: a second data acquisition module 710 and a diagnosis module 720 .

[0072] The second data acquisition module 710 is used to acquire medical image data to be diagnosed; The diagnosis module 720 is used to diagnose the medical image data based on the medical diagnosis model to obtain a diagnosis result; wherein the medical diagnosis model is trained based on the medical diagnosis model training method.

[0073] The medical diagnosis device provided in the embodiment of the present invention diagnoses medical image data through a medical diagnosis model to obtain a diagnosis result, thereby improving the diagnostic accuracy and efficiency of various diseases.

[0074] Figure 8 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 8 As shown, the electronic device may include: a processor (processor) 810, a communication interface (Communications Interface) 820, a memory (memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logic instructions in the memory 830 to execute the medical diagnosis model training method, which includes: obtaining first medical data and second medical data; wherein the first medical data is determined based on medical literature data, and the second medical data is determined based on medical clinical data; using the first medical data as a training sample, using the first medical data as a training sample, and using the knowledge and experience obtained from medical textbooks as prior knowledge to train the large language model LLM to obtain a common sense expert diagnosis module; using the second medical data as a training sample, and using the retrieval enhancement generation RAG technology to train the LLM to obtain a case expert diagnosis module; using the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples, and using the thinking chain CoT technology to train the LLM to obtain an unbiased diagnosis module; based on the common sense expert diagnosis module, the case expert diagnosis module and the unbiased diagnosis module, a medical diagnosis model is obtained.

[0075] Alternatively, a medical diagnosis method is executed, the method comprising: acquiring medical image data to be diagnosed; diagnosing the medical image data based on a medical diagnosis model to obtain a diagnosis result; wherein the medical diagnosis model is trained based on a medical diagnosis model training method.

[0076] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0077] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the medical diagnosis model training method provided by the above methods, which method includes: obtaining first medical data and second medical data; wherein the first medical data is determined based on medical literature data, and the second medical data is determined based on medical clinical data; using the first medical data as a training sample, and using the knowledge and experience obtained from medical textbooks as prior knowledge to train the large language model LLM to obtain a common sense expert diagnosis module; using the second medical data as a training sample, and using the retrieval enhancement generation RAG technology to train the LLM to obtain a case expert diagnosis module; using the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples, and using the thinking chain CoT technology to train the LLM to obtain an unbiased diagnosis module; based on the common sense expert diagnosis module, the case expert diagnosis module and the unbiased diagnosis module, a medical diagnosis model is obtained.

[0078] Alternatively, a medical diagnosis method is executed, the method comprising: acquiring medical image data to be diagnosed; diagnosing the medical image data based on a medical diagnosis model to obtain a diagnosis result; wherein the medical diagnosis model is trained based on a medical diagnosis model training method.

[0079] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the medical diagnosis model training method provided by the above-mentioned methods, the method comprising: obtaining first medical data and second medical data; wherein the first medical data is determined based on medical literature data, and the second medical data is determined based on medical clinical data; using the first medical data as a training sample, and using the knowledge and experience obtained from medical textbooks as prior knowledge to train the large language model LLM to obtain a common sense expert diagnosis module; using the second medical data as a training sample, and using the retrieval enhancement generation RAG technology to train the LLM to obtain a case expert diagnosis module; using the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples, and using the thinking chain CoT technology to train the LLM to obtain an unbiased diagnosis module; obtaining a medical diagnosis model based on the common sense expert diagnosis module, the case expert diagnosis module and the unbiased diagnosis module.

[0080] Alternatively, a medical diagnosis method is executed, the method comprising: acquiring medical image data to be diagnosed; diagnosing the medical image data based on a medical diagnosis model to obtain a diagnosis result; wherein the medical diagnosis model is trained based on a medical diagnosis model training method.

[0081] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A medical diagnosis model training method, characterized in that: include: Acquire first medical data and second medical data; wherein the first medical data is determined based on medical literature data, and the second medical data is determined based on medical clinical data; The first medical data is used as a training sample, and the large language model LLM is trained with the knowledge and experience obtained from medical textbooks as prior knowledge to obtain a common sense expert diagnosis module; the second medical data is used as a training sample, and the retrieval enhancement generation RAG technology is used to train the LLM to obtain a case expert diagnosis module; the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage are used as training samples, and the thinking chain CoT technology is used to train the LLM to obtain an unbiased diagnosis module; based on the common sense expert diagnosis module, the case expert diagnosis module and the unbiased diagnosis module, the medical diagnosis model is obtained.

2. The medical diagnosis model training method according to claim 1, characterized in that: After acquiring the first medical data and the second medical data, the method further includes: performing data cleaning on the first medical data and the second medical data in sequence to obtain cleaned data; A generative adversarial network is used to perform sample expansion on the cleaned data to obtain expanded first medical data and expanded second medical data.

3. The medical diagnosis model training method according to claim 1, characterized in that: After obtaining the medical diagnosis model, the method further includes: Collecting output data of the medical diagnosis model in a plurality of benchmark data sets respectively; A fairness index is calculated according to the plurality of output data, and the performance of the medical diagnosis model is evaluated according to the fairness index.

4. The medical diagnosis model training method according to claim 1, characterized in that: After obtaining the medical diagnosis model, the method further includes: Obtain feedback data from clinicians and patients; The network parameters of each module of the medical diagnosis model are adjusted according to the feedback data to update the medical diagnosis model.

5. A medical diagnosis method, characterized in that: include: Acquiring medical image data to be diagnosed; The medical image data is diagnosed based on a medical diagnosis model to obtain a diagnosis result; wherein the medical diagnosis model is trained based on the medical diagnosis model training method according to any one of claims 1 to 4.

6. A medical diagnosis model training device, characterized in that: include: A first data acquisition module, used to acquire first medical data and second medical data; wherein the first medical data is determined based on medical literature data, and the second medical data is determined based on medical clinical data; A training module is used to use the first medical data as a training sample and the knowledge and experience obtained from medical textbooks as prior knowledge to train the large language model LLM to obtain a common sense expert diagnosis module; use the second medical data as a training sample and use the retrieval enhancement generation RAG technology to train the LLM to obtain a case expert diagnosis module; use the first diagnosis result generated by the common sense expert diagnosis module in the training stage and the second diagnosis result generated by the case expert diagnosis module in the training stage as training samples and use the chain of thought CoT technology to train the LLM to obtain an unbiased diagnosis module; obtain the medical diagnosis model based on the common sense expert diagnosis module, the case expert diagnosis module and the unbiased diagnosis module.

7. A medical diagnostic device, characterized in that: include: A second data acquisition module, used to acquire medical image data to be diagnosed; A diagnosis module is used to diagnose the medical image data based on a medical diagnosis model to obtain a diagnosis result; wherein the medical diagnosis model is trained based on the medical diagnosis model training method according to any one of claims 1 to 4.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images