Multi-agent medical inquiry system based on improved GRPO algorithm

Abstract

The invention discloses a multi-agent medical inquiry system based on an improved GRPO algorithm, and belongs to the field of intelligent medical treatment. According to the method, firstly, a new medical data set containing illusion answers is constructed through an open source medical data set, clinical correlation of the data set is evaluated through multiple large models, and weights are given to samples; then, an improved GRPO algorithm is adopted to optimize the multi-modal large model, and the model is encouraged to explore different answers by progressively decreasing KL divergence constraints in the optimization process; the optimized model is applied to construction of a medical question and answer system, the system works cooperatively through multiple agents including a user agent, a hospital guide agent, a doctor agent and an expert agent, and automatic illness state analysis, consultation department judgment, medical opinion generation and specialized correction of an inquiry result are achieved. According to the invention, the accuracy and specialty of medical inquiry are effectively improved, the misdiagnosis rate is reduced, the medical efficiency is improved, and more accurate and comprehensive inquiry services are provided for users.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and medicine, and in particular to a multi-agent medical consultation system based on an improved GRPO algorithm and an optimization method thereof. Background Art

[0002] With the rapid development of artificial intelligence technology, intelligent applications in the medical field have gradually received widespread attention, especially in auxiliary diagnosis and medical decision support. The application potential of intelligent medical systems is huge.

[0003] However, existing systems still face numerous challenges in practical application. For one thing, the medical data upon which these systems rely is often incomplete, inaccurate, or outdated, and erroneous answers in the data are not fully exploited, resulting in inaccurate diagnostic results, insufficient hallucination resistance, and insufficient robustness. Furthermore, despite the enormous potential of large multimodal models, existing systems lack targeted optimization algorithms for optimizing these models. For example, key parameters such as the KL divergence constraint in large medical multimodal models are not effectively adjusted, limiting further improvements in model performance. Furthermore, most existing systems utilize a single-agent architecture, making it difficult to leverage the specialized strengths of different agents when faced with complex medical problems, preventing comprehensive optimization of diagnostic results. This is particularly true when dealing with complex cases requiring collaboration among physicians from multiple departments. The ability to coordinate diagnostic opinions across different departments is limited, impacting both the quality and efficiency of diagnostic consultations. Summary of the Invention

[0004] This invention aims to provide a multi-agent medical consultation system based on an improved GRPO algorithm. By constructing a dataset containing incorrect answers to improve hallucination resistance, improving the optimization algorithm to enhance model performance, and leveraging multi-agent collaborative optimization of consultation results, this system effectively improves the accuracy and efficiency of medical consultations, addressing the issues raised in the aforementioned background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] Obtain an open-source medical dataset in the image-question-answer triple format. The images in this dataset are medical images, and the corresponding questions and answers are in text format, all based on images. Based on this open-source medical dataset of image-question-answer triples, after inputting an image and question, a large-scale model is used to generate multiple hallucinated candidate answers by varying its diversity parameters.

[0007] The BERT model based on the transform structure is used to calculate the semantic similarity between each hallucinated answer and the correct answer. The hallucinated answer with the smallest semantic similarity is selected as the final incorrect answer. Finally, a dataset containing: image-question-answer-incorrect answer is constructed.

[0008] Each sample has a different effect on model optimization, so the clinical relevance of the sample is scored as the sample weight; clinical relevance refers to the sample's discriminative value and decision support ability in a real clinical setting. The specific steps are as follows:

[0009] S1. Obtain open source datasets, use a basic large model to generate hallucination answers, and construct a multimodal medical dataset containing hallucination interference samples;

[0010] S2. Score the clinical relevance of the sample as the sample weight; where the clinical relevance refers to the discriminative value and decision-making support ability of the sample in a real clinical setting;

[0011] S3. Design a reward function for optimizing the basic large model from multiple angles and provide direction for model optimization.

[0012] S4. Use the improved GRPO algorithm to optimize and improve the basic large model to obtain the target large model;

[0013] S5. Based on the target large model, a medical consultation system is constructed in which the guidance agent automatically triages the user's condition.

[0014] S6. Users with a single medical condition use a single-discipline doctor agent, while users with multiple departments use a multidisciplinary team agent for consultation.

[0015] S7. Based on the system, medical experts and text expert agents are used to optimize the consultation results to obtain the final consultation results.

[0016] Preferably, S1 includes:

[0017] S11, an open-source medical dataset based on image-question-answer triples, which inputs an image and a question and generates multiple hallucinated candidate answers by changing the diversity parameters of a basic large model;

[0018] S12. Use the BERT model based on the transform structure to calculate the semantic similarity between each hallucinated answer and the correct answer, select the hallucinated answer with the smallest semantic similarity as the final incorrect answer, and finally construct a dataset containing image-question-answer-incorrect answer.

[0019] Preferably, S2 includes:

[0020] S21. Use multiple different large models to score the clinical relevance of the sample based on the same prompt word; select the average of the scores as the final value of the clinical relevance of the sample;

[0021] S22, use the softmax function to normalize as the final weight of the sample , used to weight the training loss and guide the model to preferentially fit highly correlated samples; the normalization formula is:

[0022] ;

[0023] There are n large models, and each large model scores the clinical relevance of an answer as .

[0024] Preferably, S3 includes:

[0025] S31. Correctness Reward , rewards are given based on whether the answer is correct:

[0026] ;

[0027] S32. Simplicity Reward , rewards are given based on the brevity of the answer:

[0028] ;

[0029] S33, Strict Format Reward , rewards are given based on whether the user's answer strictly complies with the format requirements:

[0030] ;

[0031] S34. XML tag counting reward , rewards are given based on the number of specific XML tags in the answer:

[0032] ;

[0033] S35. Total reward function:

[0034] .

[0035] Preferably, S4 includes:

[0036] S41, basic large model Give answers to questions based on the images in the dataset;

[0037] S42. Use the reward function to calculate the reward for the answer, thereby performing advantage calculation and optimizing the model to obtain the optimized model. ;

[0038] S43. Optimization model and the base model Perform KL divergence calculation using the decreasing KL divergence constraint:

[0039] set up parameter:

[0040] ;

[0041] in, are the original parameters of KL.

[0042] Preferably, in step S5, the target large model is used as the base to construct a medical consultation system in which the guidance agent automatically triages the user's condition: the guidance agent in the system receives and processes the multimodal data input by the user. , including text and image information, combined with the large model analysis results to extract disease characteristics and determine whether the user needs a single department or multi-department consultation.

[0043] Preferably, S6 includes:

[0044] S61. After determining the user's consultation needs, the system uses regular expressions to extract the target department name based on the output of the guidance agent and dynamically creates the corresponding doctor agent;

[0045] S62. Each doctor agent automatically injects identity and professional ability prompts based on department information and task requirements to complete specific diagnosis and treatment tasks;

[0046] S63. For single-department consultations, the system creates a doctor agent with professional knowledge of the corresponding department to provide diagnosis and treatment recommendations. For multidisciplinary consultations, the system creates multiple department doctor agents in parallel, each of which is responsible for the diagnosis and treatment of its own department, and finally generates a comprehensive diagnosis result by speaking in turn. .

[0047] Preferably, S7 includes:

[0048] S71. The medical expert agent reviews the consultation results generated by the doctor agent based on professional knowledge in the medical field; the medical expert agent combines the consultation results , user information and preset professional prompt words , the feedback result is generated by the following mapping : ;

[0049] in, It represents the function generated by the medical expert agent based on the target large model;

[0050] S72. The medical expert agent is responsible for optimizing the language expression of the consultation results to improve their clarity, logic, and readability; the medical expert agent combines the consultation results , user information and preset professional prompt words , the feedback result is generated by the following mapping : ;

[0051] in, Represents the function generated by the text expert agent based on the target large model;

[0052] S73, the system collects feedback results from two expert agents and After that, combined with user information Results of the original medical consultation , the final optimized consultation result is generated through the following mapping : ;

[0053] in Represents the optimization function that generates the final diagnosis result by integrating system feedback and user information.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] This invention constructs a high-quality dataset containing hallucination interference data, optimizes it using an improved GRPO algorithm, and leverages the collaborative work of multiple intelligent agents to achieve automated condition analysis, precise departmental judgment, diagnostic correction based on expert feedback, and optimized consultation results. In this way, the invention effectively improves the accuracy and efficiency of medical consultations and reduces misdiagnosis rates, providing users with more intelligent and comprehensive medical consultation services. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0057] Figure 1 This is a flow chart of a multi-agent medical consultation system based on an improved GRPO algorithm of the present invention;

[0058] Figure 2 is a flow chart of the medical question-answering system of the present invention;

[0059] Figure 3 It is a flow chart of the GRPO algorithm of the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0061] See also Figure 1-Figure 3 , the present invention provides a technical solution:

[0062] Example 1:

[0063] The specific steps of a multi-agent medical consultation system based on the improved GRPO algorithm are as follows:

[0064] The present invention comprises the following steps:

[0065] (1) Constructing a hallucination answer dataset:

[0066] Based on an open-source medical dataset of image-question-answer triplets, after inputting an image and a question, the basic large model changes its diversity parameters to generate multiple hallucinated candidate answers. The BERT model based on the transform structure is used to calculate the semantic similarity between each hallucinated answer and the correct answer, and the hallucinated answer with the smallest semantic similarity is selected as the final incorrect answer. Finally, a dataset containing: image-question-answer-incorrect answer is constructed.

[0067] (2) Calculate sample weight:

[0068] Score the clinical relevance of the sample as the sample weight; clinical relevance refers to the sample's discriminative value and decision-making support capabilities in a real clinical setting. The specific steps are as follows:

[0069] S21. Use multiple different large models to score the clinical relevance of the sample based on the same prompt word; select the average of the scores as the final value of the clinical relevance of the sample.

[0070] S22. Use the softmax function to normalize as the final weight of the sample , used to weight the training loss and guide the model to preferentially fit highly correlated samples. The normalization formula is:

[0071] ;

[0072] There are n large models, and each large model scores the clinical relevance of an answer as .

[0073] (3) Reward function construction:

[0074] The GRPO algorithm requires the construction of a reward function, which provides the model optimization direction based on the score of the reward function. The specific function is as follows:

[0075] S31. Correctness Reward , rewards are given based on whether the answer is correct:

[0076] ;

[0077] S32. Simplicity Reward , rewards are given based on the brevity of the answer:

[0078] ;

[0079] S33, Strict Format Reward , rewards are given based on whether the user's answer strictly complies with the format requirements:

[0080] ;

[0081] S34. XML tag counting reward , rewards are given based on the number of specific XML tags in the answer:

[0082] ;

[0083] S35. Total reward function:

[0084] .

[0085] (4) Decreasing KL divergence constraint, GRPO optimization target large model:

[0086] Use the GRPO algorithm with decreasing constraints to constrain the large model so that the model can be better optimized. The specific process is as follows:

[0087] S41, basic large model Give answers to questions based on the images in the dataset;

[0088] S42. Use the reward function to calculate the reward for the answer, thereby performing advantage calculation and optimizing the model to obtain the optimized model. ;

[0089] S43. Optimization model and the base model Perform KL divergence calculation using the decreasing KL divergence constraint:

[0090] set up parameter: ;

[0091] in, are the original parameters of KL.

[0092] (5) Target large model to build a multi-agent system:

[0093] like Figure 2 The following is a flow chart of the system. Based on the target large model, a medical consultation system is constructed in which the guidance agent automatically triages the user's condition. The guidance agent in the system receives and processes the multimodal data input by the user. , including text and image information, combined with the large model analysis results to extract disease characteristics and determine whether the user needs a single department or multi-department consultation.

[0094] (6) Single-discipline and multi-discipline physician agents:

[0095] S61. After determining the user's consultation needs, the system uses regular expressions to extract the target department name based on the output of the guidance agent and dynamically creates the corresponding doctor agent;

[0096] S62. Each doctor agent automatically injects prompts such as identity and professional ability based on department information and task requirements to complete specific diagnosis and treatment tasks;

[0097] S63. For single-department consultations, the system creates a doctor agent with professional knowledge of the corresponding department to provide diagnosis and treatment recommendations. For multidisciplinary consultations, the system creates multiple department doctor agents in parallel, each of which is responsible for the diagnosis and treatment of its own department, and finally generates a comprehensive diagnosis result by speaking in turn. .

[0098] (7) Expert Agent Optimization:

[0099] Use medical experts and text expert agents to optimize the consultation results. The specific steps are as follows:

[0100] S71. The medical expert agent reviews the consultation results generated by the doctor agent based on the professional knowledge in the medical field. , user information and preset professional prompt words , the feedback result is generated by the following mapping .in, Represents the medical expert agent based on the target large model generation function

[0101] ;

[0102] S72. The medical expert agent is responsible for optimizing the language expression of the consultation results to improve their clarity, logic and readability. , user information and preset professional prompt words , the feedback result is generated by the following mapping .in, Represents the text expert agent generating function based on the target large model

[0103] ;

[0104] S73. The system collects feedback from two expert agents and After that, combined with user information Results of the original medical consultation , the final optimized consultation result is generated through the following mapping ,in Represents the optimization function that generates the final diagnosis result by integrating system feedback and user information

[0105] .

[0106] The embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by ordinary technicians in this field without departing from the spirit of the present invention.

[0107] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit and essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description. It is intended that all variations within the meaning and range of equivalents of the claims be embraced herein, and any reference signs in the claims should not be construed as limiting the claims to which they relate.

[0108] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A multi-agent medical consultation system based on an improved GRPO algorithm, characterized by: The system comprises: S1. Obtain open source datasets, use a basic large model to generate hallucination answers, and construct a multimodal medical dataset containing hallucination interference samples; S2. Score the clinical relevance of the sample as the sample weight; where the clinical relevance refers to the discriminative value and decision-making support ability of the sample in a real clinical setting; S3. Design a reward function for optimizing the basic large model from multiple angles and provide direction for model optimization. S4. Use the improved GRPO algorithm to optimize and improve the basic large model to obtain the target large model; S5. Based on the target large model, a medical consultation system is constructed in which the guidance agent automatically triages the user's condition. S6. Users with a single medical condition use a single-discipline doctor agent, while users with multiple departments use a multidisciplinary team agent for consultation. S7. Based on the system, medical experts and text expert agents are used to optimize the consultation results to obtain the final consultation results.

2. A multi-agent medical consultation system based on the improved GRPO algorithm as claimed in claim 1, characterized in that: Said S1 comprises: S11, an open-source medical dataset based on image-question-answer triples, which inputs an image and a question and generates multiple hallucinated candidate answers by changing the diversity parameters of a basic large model; S12. Use the BERT model based on the transform structure to calculate the semantic similarity between each hallucinated answer and the correct answer, select the hallucinated answer with the smallest semantic similarity as the final incorrect answer, and finally construct a dataset containing image-question-answer-incorrect answer.

3. The multi-agent medical consultation system based on the improved GRPO algorithm according to claim 1, characterized in that: Said S2 comprises: S21. Use multiple different large models to score the clinical relevance of the sample based on the same prompt word; select the average of the scores as the final value of the clinical relevance of the sample; S22, use the softmax function to normalize as the final weight of the sample , used to weight the training loss and guide the model to preferentially fit highly correlated samples; the normalization formula is: ； There are n large models, and each large model scores the clinical relevance of an answer as .

4. The multi-agent medical consultation system based on the improved GRPO algorithm according to claim 1, characterized in that: Said S3 comprises: S31. Correctness Reward , rewards are given based on whether the answer is correct: ； S32. Simplicity Reward , rewards are given based on the brevity of the answer: ； S33, Strict Format Reward , rewards are given based on whether the user's answer strictly complies with the format requirements: ； S34. XML tag counting reward , rewards are given based on the number of specific XML tags in the answer: ； S35. Total reward function: 。 5. The multi-agent medical consultation system based on the improved GRPO algorithm as claimed in claim 1, characterized in that: Said S4 comprises: S41, basic large model Give answers to questions based on the images in the dataset; S42. Use the reward function to calculate the reward for the answer, thereby performing advantage calculation and optimizing the model to obtain the optimized model. ; S43. Optimization model and the base model Perform KL divergence calculation using the decreasing KL divergence constraint: set up parameter: ; in, are the original parameters of KL.

6. The multi-agent medical consultation system based on the improved GRPO algorithm according to claim 1, characterized in that: In step S5, the target large model is used as the basis to build a medical consultation system in which the guidance agent automatically triages the user's condition: the guidance agent in the system receives and processes the multimodal data input by the user. , including text and image information, combined with the large model analysis results to extract disease characteristics and determine whether the user needs a single department or multi-department consultation.

7. The multi-agent medical consultation system based on the improved GRPO algorithm as claimed in claim 1, characterized in that: Said S6 comprises: S61. After determining the user's consultation needs, the system uses regular expressions to extract the target department name based on the output of the guidance agent and dynamically creates the corresponding doctor agent; S62. Each doctor agent automatically injects identity and professional ability prompts based on department information and task requirements to complete specific diagnosis and treatment tasks; S63. For single-department consultations, the system creates a doctor agent with professional knowledge of the corresponding department to provide diagnosis and treatment recommendations. For multidisciplinary consultations, the system creates multiple department doctor agents in parallel, each of which is responsible for the diagnosis and treatment of its own department, and finally generates a comprehensive diagnosis result by speaking in turn. .

8. The multi-agent medical consultation system based on the improved GRPO algorithm as claimed in claim 1, characterized in that: The S7 includes: S71. The medical expert agent reviews the consultation results generated by the doctor agent based on professional knowledge in the medical field; the medical expert agent combines the consultation results , user information and preset professional prompt words , the feedback result is generated by the following mapping : ; in, It represents the function generated by the medical expert agent based on the target large model; S72. The medical expert agent is responsible for optimizing the language expression of the consultation results to improve their clarity, logic, and readability; the medical expert agent combines the consultation results , user information and preset professional prompt words , the feedback result is generated by the following mapping : ; in, Represents the function generated by the text expert agent based on the target large model; S73, the system collects feedback results from two expert agents and After that, combined with user information Results of the original medical consultation , the final optimized consultation result is generated through the following mapping : ; in, Represents the optimization function that generates the final diagnosis result by integrating system feedback and user information.

Images