A multi-agent medical auxiliary diagnosis method based on dynamic semantic perception reward

Abstract

The application provides a multi-agent medical auxiliary diagnosis method based on dynamic semantic perception reward, comprising: processing the current object's visit description and historical case memory information to obtain a multi-modal composite state; inputting the multi-modal composite state into a multi-modal large model for forward feature extraction to obtain a final hidden state, and determining the activation probability of each expert in a plurality of expert agent libraries according to the final hidden state by using a linear projection head; according to the activation probability of each expert, selecting an expert routing chain containing a main responsibility expert and a collaborative expert, sequentially calling the specified expert agent in the routing chain, obtaining an initial diagnosis result by the main responsibility expert agent according to the multi-modal composite state, and obtaining a supplementary result by the collaborative expert agent according to the multi-modal composite state and the initial diagnosis result; and generating a predicted auxiliary diagnosis result according to the initial diagnosis result and the supplementary result, which contains the probability of the object suffering from each type of preset disease, an auxiliary diagnosis and treatment report and a comprehensive confidence.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, specifically to the field of medical big data processing and multi-agent system technology, and more specifically, to a multi-agent medical auxiliary diagnosis method based on dynamic semantic perception reward. Background Technology

[0002] With the leaps and bounds in deep learning technology, especially the widespread application of the Transformer architecture and the explosive rise of Large Language Models (LLMs), the landscape of Artificial Intelligence (AI) applications in the medical field is undergoing a profound paradigm shift. Early medical AI was primarily limited to single-modal tasks. However, the complexity of modern medicine far exceeds simple sensory tasks, requiring systems with comprehensive analysis, logical reasoning, and long-term memory capabilities. Therefore, current technological frontiers are no longer satisfied with simple image screening tools but are rapidly evolving towards Clinical Decision Support Systems (CDSS) capable of handling complex clinical workflows. This wave of technological iteration has primarily given rise to two distinct schools of thought: monolithic multimodal large-model solutions and orchestrator-based multi-agent collaborative solutions that emphasize division of labor and cooperation.

[0003] The primary challenge of monolithic models lies in their prohibitively high customization costs and computational burden. The healthcare industry has deep professional barriers, and a general-purpose medical model often struggles to meet the refined needs of specific departments (such as ophthalmology and neurosurgery). To fine-tune all parameters for a specific subfield, the computational resources and time costs required, given that the model's underlying parameters can easily reach hundreds of billions, are prohibitive for most medical institutions. More importantly, clinical diagnosis and treatment span a vast time frame; patient records often extend for months or even years, containing multiple consultations, examination reports, and medication responses. Monolithic models, limited by their context window length, struggle to fully "memorize" and process such massive amounts of longitudinal data through simple text concatenation, leading to "catastrophic amnesia" or loss of crucial information when dealing with complex cases. Furthermore, medical decision-making demands extremely high interpretability, and a giant monolithic model is essentially a huge "black box." Doctors cannot accurately trace the specific reasoning path behind the model's diagnostic recommendations, and this lack of interpretability poses a significant trust barrier and regulatory challenge in the life-or-death medical field.

[0004] To overcome the limitations of the single-agent model, an architecture more aligned with the working patterns of human medical teams—a multi-agent collaborative scheme based on an orchestrator—has emerged. This architecture is heavily influenced by autonomous agent concepts such as AutoGPT. Its core idea is no longer to train an all-around "general practitioner," but rather to build a collaborative system composed of an "expert team." In this architecture, the system is decoupled into a central "orchestrator" and multiple specialized "executors." The orchestrator plays a role similar to an "attending physician" or "team leader." It may not possess the strongest image processing capabilities, but it has extremely strong task decomposition, logical planning, and resource scheduling abilities. When the system receives complex clinical instructions, the orchestrator first parses the user's intent, breaking down the large-scale diagnostic and treatment task into several sub-tasks, and then dynamically calls specialized sub-models to execute them. For example, it might call a specially optimized visual model to analyze fundus images, use a knowledge base retrieval tool to consult the latest clinical guidelines, or call a specialized pharmaceutical model to verify drug interactions.

[0005] However, whether it's a single-agent model or a multi-agent system, pre-training alone is far from sufficient to enable agents to truly understand complex medical logic and strictly follow human instructions. Alignment techniques must be introduced, and reinforcement learning training of large models has become a key technical approach to improving model intelligence. In medical scenarios, reinforcement learning from human feedback (RLHF) aims not only to make the generated text more fluent but also to ensure that the model's decision preferences align with medical ethics and the professional judgment of human doctors.

[0006] Among numerous reinforcement learning algorithms, Proximal Policy Optimization (PPO) is currently the most widely used mainstream algorithm. PPO ensures the stability of the training process by limiting the step size of policy updates, effectively preventing performance collapse during optimization. However, the PPO algorithm also faces significant computational challenges in practical engineering applications. PPO employs an Actor-Critic architecture. To accurately estimate the value of each state and reduce the variance of gradient estimation, it requires maintaining not only a policy model (Actor) that generates responses but also simultaneously running a value model (Critic) of similar size during training. This means that when training a large medical model with 70 parameters, the PPO algorithm actually requires GPU memory to simultaneously support two 70-B models and their optimizer states, which multiplies GPU memory consumption and severely limits the ability to train high-performance medical models with limited resources. Therefore, how to reduce the resource overhead of the value model while maintaining the effectiveness of reinforcement learning, or exploring optimization algorithms such as GRPO that do not require an independent value model, has become one of the key directions for current research on training large medical models.

[0007] Although breakthroughs in large language models and multimodal technologies have provided a strong foundation for intelligent assisted diagnosis and treatment, existing technologies still have significant shortcomings in terms of training resource efficiency, long-term temporal memory fusion, and adaptation to specialized and refined diagnosis and treatment in the process of evolving from single image screening to complex ophthalmic clinical decision support systems and multi-agent collaborative architectures.

[0008] Traditional RLHF-based alignment techniques are limited in training massive multimodal models. They rely on a complex Actor-Critic architecture, requiring the simultaneous maintenance of the policy model, reference model, value model, and reward model during training. However, when dealing with modern large-scale medical models like Qwen-VL and GPT-4V, which have hundreds of billions or even trillions of parameters, this "four-model parallelism" mechanism leads to insurmountable computational bottlenecks and optimization challenges. This architecture presents a severe challenge to existing technologies. It consumes enormous computational resources and results in unstable training: the standard PPO algorithm requires the value model to have a parameter count equivalent to the policy model to accurately evaluate state value. This leads to a significant increase in GPU memory requirements, making it impossible for most medical institutions to perform full parameter fine-tuning on privately deployed hardware.

[0009] While some studies have attempted to solve all problems through generalized large models, existing technologies still suffer from the following core technical deficiencies when dealing with the highly specialized and subspecialized complexities of ophthalmology (i.e., the problems that this invention focuses on addressing):

[0010] A single, general-purpose model struggles to address the contradiction between refined specialty classification and a lack of specialization. General-purpose models attempt to handle all tasks through uniform neural network weights, leading to scattered attention and difficulty in considering the subtle features of different subspecialties. Furthermore, existing multi-agent systems rely solely on current patient information, and knowledge gaps exist between different specialties, resulting in room for improvement in the accuracy of assisted diagnosis.

[0011] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solution of the present invention, and does not imply that the relevant information is necessarily prior art. The relevant information was submitted and disclosed together with the present invention, and should not be considered prior art unless there is evidence that the relevant information was disclosed before the filing date of the present invention. Summary of the Invention

[0012] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a medical auxiliary diagnosis method based on multiple agents.

[0013] The objective of this invention is achieved through the following technical solution:

[0014] According to a first aspect of the present invention, a multi-agent-based medical auxiliary diagnosis method is provided, comprising: performing structured processing on the current object's medical visit description and historical case memory information to obtain a multimodal composite state, wherein the medical visit description includes the object's chief complaint and medical images, and the historical case memory information includes temporal information capturing the evolution of the condition extracted based on the object's previous medical visit descriptions and diagnostic results; inputting the multimodal composite state into an open-source multimodal large model fine-tuned by reinforcement learning for forward feature extraction to obtain a final hidden state; and using a linear projection head to determine multiple [missing information] based on the final hidden state. The scheme calculates the activation probabilities of each expert in the expert agent library; based on these probabilities, it selects an expert routing chain containing the primary expert and collaborating experts, sequentially calling the designated expert agents in the routing chain. The primary expert agent performs an initial diagnosis based on the multimodal composite state, while the collaborating expert agents perform supplementary consultations based on the multimodal composite state and the initial diagnosis, yielding supplementary results. An open-source multimodal large model is then used to generate predicted auxiliary diagnosis results based on the initial and supplementary results. These results include the probability of the subject suffering from various preset diseases, the auxiliary diagnosis report, and the overall confidence level of the prediction. This scheme achieves at least the following beneficial technical effects: First, by structuring the description of medical visits and historical case memory information, a multimodal composite state containing temporal information is constructed, enabling dynamic capture of disease progression and avoiding information fragmentation and key temporal gaps in traditional snapshot-based diagnoses, thus improving diagnostic continuity and accuracy. Second, by sequentially calling the designated expert agents through the expert routing chain provided by the linear projection head, it achieves hierarchical collaboration between general practice scheduling and specialist diagnosis and treatment, isolating interference from specialist knowledge, improving the reliability of auxiliary diagnosis, and outputting more accurate auxiliary diagnosis results.

[0015] Optionally, before inference, the open-source multimodal large model and linear projection head are fine-tuned. A reference model with parameter freezing is used during fine-tuning. The fine-tuning includes multiple operations: acquiring a training set containing multiple samples, each with sample content and labels. The sample content is the patient's medical history description, and the labels are the patient's diagnosis results. The open-source multimodal large model generates a first probability distribution of routing actions based on the final hidden states extracted from the patient's medical history description. The reference model generates a second probability distribution of routing actions based on the final hidden states extracted from the patient's medical history description. The KL divergence between the first and second probability distributions is determined. Using the GRPO algorithm based on a valueless model, multiple expert routing chains are sampled to obtain the training auxiliary diagnosis results for each chain. A composite reward function is used to determine a composite reward containing multiple reward indicators based on the training auxiliary diagnosis results and labels. The relative advantage value of each routing chain is determined based on the composite reward. Based on a preset total loss function, a loss value determined by the relative advantage value and KL divergence of each chain is obtained. The parameters of the open-source multimodal large model and linear projection head are updated with the goal of minimizing the loss value. This scheme achieves at least the following beneficial technical effects: It employs a value-free GRPO algorithm, abandoning the traditional four-model architecture of policy-value models and retaining only the policy model and reference model. It replaces the independent value model by calculating the relative advantage within a group of routing chains through parallel sampling, significantly reducing training memory overhead and enabling fine-tuning of multimodal large models with limited resources. It introduces a parameter-frozen reference model to constrain the policy output distribution, using KL divergence to prevent catastrophic forgetting of language abilities or semantic collapse during reinforcement learning, ensuring the stability of medical terminology generation and diagnostic logic. Based on multi-dimensional evaluation using a composite reward function, it guides the model to learn multiple aspects of knowledge, improving model performance.

[0016] Optionally, the total loss function is:

[0017]

[0018] in, This indicates the number of expert routing chains sampled. This is the cutting factor. The weighting coefficients for the KL divergence are... This indicates the routing chain generated by the current policy model and the policy model before the update. The ratio of the probabilities, Indicates according to expert routing chain The expert routing chains are determined by the composite reward, the average composite reward of each expert routing chain, and the standard deviation of the composite reward. The relative advantage value, Indicates will Limited to [ , The clipping function within the interval. Let KL divergence be the given value. This scheme can achieve at least the following beneficial technical effects: by limiting the policy update magnitude through the PPO pruning mechanism, it prevents performance collapse caused by excessive step size during fine-tuning, thus ensuring the stability of reinforcement learning; by combining KL divergence constraints to form a double insurance, it optimizes the probability of high-reward regions while constraining the policy to not deviate from the pre-training distribution, thus avoiding pattern collapse and output distribution distortion.

[0019] Optionally, the composite reward function is:

[0020]

[0021] in, Indicates expert routing chain Compound rewards, This indicates an expert routing chain used to guide more standardized output formats and keywords. Rewards in the format of This indicates a more accurate expert routing chain used to guide the medical logic output. Content rewards This refers to the expert routing chain used to guide and optimize the expert routing chain to improve the accuracy of scheduling expert agents. Routing rewards Representation of reward coefficient format This represents the content reward coefficient. This represents the routing reward coefficient. This scheme achieves at least the following beneficial technical effects: It designs a composite reward function encompassing format, content, and routing dimensions, comprehensively covering the standardization of clinical reports, the accuracy of medical logic, and the precision of expert scheduling, avoiding optimization blind spots caused by a single scalar reward; through multi-objective decomposition, the model simultaneously learns more rigorous clinical report formats, deeper pathological logical reasoning, and more precise expert routing strategies during training, ensuring that auxiliary diagnostic results achieve superior performance in compliance, accuracy, and logic.

[0022] Optional, the format reward is:

[0023]

[0024] in, Indicates the number of structural segments that should appear. This indicates the actual number of segments hit by the training auxiliary diagnosis results. This represents the total number of keywords necessary to accurately describe the condition. This indicates the number of keywords hit in the training and auxiliary diagnosis results; This indicates whether the current format meets the requirements; a value of 1 indicates yes, and a value of 0 indicates no. , , This represents the weighting coefficient. This scheme can achieve at least the following beneficial technical effects: guiding the model to quickly learn a standardized clinical report structure through rule-matching-based format rewards; strengthening the model's coverage of core disease elements through necessary medical keyword hit detection; and using the format validity indicator function as a hard constraint to make the format more standardized.

[0025] Optional content rewards include:

[0026]

[0027] in, This indicates the degree of consistency between the diagnostic conclusions from the training-assisted diagnosis and the results indicated on the label. This indicates the coverage of lesion descriptions based on the training and auxiliary diagnostic results. This indicates the coverage of treatment or follow-up consultation recommendations based on the results of training and auxiliary diagnosis. , , This represents the weighting coefficient. This scheme can achieve at least the following beneficial technical effects: by comparing diagnostic conclusions with labeled results, it constrains the correctness of the medical logic output of the model, reducing the risk of misdiagnosis and missed diagnosis; by assessing the coverage of lesion descriptions, it ensures the model's complete identification and representation of key pathological features, reducing the omission of key information; and by assessing the coverage of treatment or follow-up recommendations, it guides the model to generate more clinically feasible treatment plans, improving the comprehensiveness of auxiliary diagnostic results.

[0028] Optional, routing rewards are:

[0029]

[0030] in, The characteristic function of the responsible expert is defined when the responsible expert in the expert routing chain... This belongs to the correct set of responsible experts. The value is 1 if the condition is met, and 0 otherwise. The length of the chain. Indicates adjacent experts and Output semantic similarity; This indicates whether the current expert routing chain is deemed capable of performing the auxiliary diagnosis task; a value of 1 indicates yes, and a value of 0 indicates no. , , , This represents the weighting coefficient. This scheme can achieve at least the following beneficial technical effects: By rewarding the primary expert for successful hits, it improves the accuracy of the initial routing decision, laying a correct foundation for subsequent collaborative diagnosis; by evaluating the semantic similarity of adjacent experts, it encourages the professional coherence of the expert chain, avoiding knowledge fragmentation caused by cross-specialty jumps; by determining task coverage, it ensures that the expert chain as a whole can handle auxiliary diagnostic tasks; and by using length penalties to suppress redundant calls, it controls computational resource consumption and response latency, achieving a balance between diagnostic efficiency and resource utilization.

[0031] Optionally, during fine-tuning, a phased dynamic weight adjustment mechanism can be implemented. In the initial stage of fine-tuning, the format reward coefficient is set lower than the content reward coefficient and the routing reward coefficient to suppress over-optimization of the format reward. In the middle stage of fine-tuning, the values ​​of the content reward coefficient and the routing reward coefficient are gradually increased. In the later stage of fine-tuning, the values ​​of the format reward coefficient, content reward coefficient, and routing reward coefficient are dynamically and gradually adjusted to achieve a balance. This scheme can achieve at least the following beneficial technical effects: In the early stages of training, the format weight is significantly reduced, suppressing the model's tendency to over-optimize simple format tasks for speculative scoring, forcing the model to focus on core diagnostic logic and routing capabilities. As training progresses, the content and routing weights are gradually increased, guiding the model to deepen medical reasoning and scheduling strategies after mastering basic formats. In the later stages of training, the three factors are dynamically balanced to ensure the model reaches Nash equilibrium in multi-objective games, effectively eliminating gradient interference between tasks, solving the problem of loss function oscillation in the later stages of training, and enabling the model to converge simultaneously in terms of compliance, accuracy, and logic, thereby improving model performance.

[0032] According to a second aspect of the present invention, a computer device / apparatus / system is provided, comprising a memory, a processor, and a computer program / instructions stored in the memory, characterized in that the processor executes the computer program / instructions to implement the steps of the method described in the first aspect.

[0033] According to a third aspect of the present invention, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the method described in the first aspect. Attached Figure Description

[0034] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0035] Figure 1 This is a schematic diagram illustrating the principle of the multi-agent-based medical auxiliary diagnosis method according to an embodiment of the present invention during fine-tuning. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0037] As mentioned in the background section, existing multi-agent systems rely solely on current patient information, and interference between different specialties leads to limitations in the accuracy of assisted diagnosis. To address this, this invention constructs a multimodal composite state containing temporal information by structuring patient descriptions and historical case memories. This enables dynamic capture of disease progression, avoiding the information fragmentation and missing key temporal sequences inherent in traditional snapshot-based diagnoses, thus improving diagnostic continuity and accuracy. Furthermore, by sequentially invoking designated expert agents through an expert routing chain provided by a linear projection head, it achieves hierarchical collaboration between general practice and specialist treatment, isolating interference from specialist knowledge, improving the reliability of assisted diagnosis, and outputting more accurate diagnostic results.

[0038] The following section introduces the reinforcement learning framework, the construction of composite reward functions, the construction of advantage functions, the fine-tuning process, and application scenarios.

[0039] I. Reinforcement Learning Framework

[0040] When the inventors were building and researching multi-agent collaborative networks for complex medical (such as ophthalmology) diagnosis and treatment scenarios, they found that existing single large models, when acting as "core schedulers" to process multimodal data and select expert models, generally suffer from problems such as poor generalization ability, uncontrollable reasoning logic, and excessively high training resource thresholds.

[0041] To address this pain point, the inventors, while analyzing cases of failure in existing technologies, discovered that the root cause lies in the fact that existing methods confuse the "probabilistic nature of general text generation" with the "certainty of medical decision-making," and ignore the contradiction between the "limited nature of computational resources" and the "completeness of diagnostic and treatment information." Through research, the following conclusions were reached:

[0042] (1) Insight 1 (Relative Advantage of Computational Perspective): In medical assisted diagnosis, especially in multi-method comparison, the quality evaluation of the model output is often relative rather than absolute (for example, assisted diagnosis A is more accurate than assisted diagnosis B, but the absolute score of non-assisted diagnosis A is 95 points). The traditional PPO algorithm forcibly introduces a value model (Critic) to fit the absolute value, which leads to double the memory consumption on multimodal models such as Qwen-VL with a large number of parameters, and the fitting error of the value model (Critic) will introduce serious training oscillations. The inventors believe that the direction of gradient optimization can be obtained simply by within-group comparison, without the need to maintain a huge value model.

[0043] (2) Insight 2 (Spatiotemporal Continuity from a Data Perspective): Ophthalmological diagnosis and treatment is not a single-moment slice observation, but a spatiotemporally continuous process that includes the patient's complaint, objective examination, assessment, and planning. Existing technologies often simplify visual question answering (VQA) to "talking about pictures," ignoring the constraints of long-term medical record text on the understanding of visual features (e.g., the history of diabetes determines the diagnostic weight of fundus microaneurysms). This lack of context leads to "illusions" or misdiagnosis when the model faces difficult and complex cases due to insufficient information dimensions.

[0044] (3) Insight 3: Traditional reinforcement learning adopts a policy model-value model architecture, which leads to an excessive number of parameters.

[0045] To address this, based on the innovative concepts of "relative optimization" and "holographic state," this invention abandons the traditional architecture, removes the value model, and uses a resource-efficient multimodal group reinforcement learning framework (GRPO-Oph) built with an improved GRPO algorithm to fine-tune and optimize open-source multimodal large models. Examples of open-source multimodal large models include Llama or Qwen. The following explanation uses Qwen3-VL as an example.

[0046] According to one embodiment of the present invention, the present invention provides a reinforcement learning model, including an object description module, a memory information module, a policy model, a reference model for frozen parameters, and a library of various expert agents. Wherein:

[0047] The object description module is used to convert the subject of an object and medical images into an intermediate clinical object state. For example, converting them into a standardized JSON object.

[0048] The memory information module is used to maintain historical case memory information for each object, and to perform structured processing on the intermediate clinical object state and historical case memory information of the current object to obtain a multimodal composite state. Historical case memory information includes chronological information capturing the evolution of the condition, extracted from the subject's previous medical visits and diagnoses. Furthermore, when a new medical report is received from the consenting subject, the historical case memory information is updated based on the diagnosis in the new report. The multimodal composite state includes a basic description, detected symptoms or lesions, and imaging modality. The diagnostic results in the medical report are either a diagnosis issued by a human physician or a confirmed auxiliary diagnostic report.

[0049] The strategy model comprises an open-source multimodal large model and a linear projection head. The open-source multimodal large model is used for forward feature extraction of multimodal composite states to obtain the final hidden state. The final hidden state refers to the medical visit features of the object output by the last hidden state layer. The linear projection head is used to determine the activation probabilities of each expert in various expert agent databases based on the final hidden state. During forward propagation, the last hidden state is input into a trainable linear projection head (Action_head). This linear projection head maps high-dimensional semantic vectors to the expert action space, obtaining the scores (logits) of each candidate specialist expert, and then obtaining the activation probabilities of each expert through Softmax or Categorical distribution.

[0050]

[0051] in, This represents the final hidden state of the policy model. and These represent the weights and biases of the linear projection head, respectively. This represents the currently selected expert action. The activation probability characterizes the matching strength between the current composite input state and each candidate expert. express The activation probability is a normalized selection probability output by the policy model for each candidate expert agent. It characterizes the matching strength between the current composite state and each candidate expert, and is used to select a single-step expert action. Multiple consecutive actions can form a complete expert collaboration link for subsequent on-device collaborative diagnosis. For example, the projection head maps a general semantic vector to a specific specialty decision space, outputting activation probabilities for "retinal expert," "glaucoma expert," or "corneal expert," achieving a physical layer mapping from general perception to specialty triage. The policy model can sample actions based on this probability distribution; multiple consecutive actions can form a complete expert collaboration link, i.e., an expert routing chain, such as "retinal expert—OCT recognition expert—treatment suggestion expert—report generation expert."

[0052] The reference model for frozen parameters is a large, parameter-frozen model isomorphic to the policy model. By maintaining a parameter-frozen reference model, which has already been trained, a stable baseline distribution is provided. During inference, the KL divergence between the policy model's output distribution and the reference model's output distribution is calculated in real-time as a regularization term. This prevents catastrophic forgetting of language abilities or semantic collapse during reinforcement learning and prevents the policy model from going astray during fine-tuning.

[0053] For example, multimodal composite states Input policy model and frozen reference model The policy model is used to provide the first probability distribution of expert routing actions, while the reference model provides a stable baseline distribution, i.e., the second probability distribution of expert routing actions. To prevent the policy model from deviating from its original language capabilities and medical semantic boundaries during reinforcement learning, a KL divergence constraint is introduced into the training objective. Let the policy model be in state... Down Output Action The probability distribution is The output distribution of the reference model is The KL divergence between the two can be expressed as:

[0054]

[0055] The KL divergence is added as a regularization penalty term to the final loss function to limit the distributional shift of the policy model relative to the reference model. When the policy model generates overly aggressive outputs that deviate from medical semantics or have abnormal language structure in order to obtain higher rewards, the penalty term corresponding to the KL divergence will increase, thereby suppressing such updates. Through this mechanism, the model can maintain the language generation ability and medical semantic stability of the original large model while exploring better diagnostic and treatment strategies.

[0056] Multiple expert agent libraries are provided to offer a variety of expert libraries for different types of intelligent agents. For example, see [link to relevant documentation]. Figure 1 It provides expert databases such as retinal, glaucoma, and corneal specialists. Additionally, it can offer OCT recognition experts, treatment recommendation experts, and / or report generation experts. Each expert database can also contain multiple different trained expert agents to provide diverse options.

[0057] The reinforcement learning strategy, based on the improved GRPO algorithm, samples multiple expert routing chains for the same input's composite state. The strategy model does not directly output a single result, but instead generates a set of decision trajectories in parallel, i.e., a set of expert routing chains. In this embodiment, for example, the group size is set. This means the model performs eight independent Monte Carlo samplings based on the current probability distribution, generating eight distinct decision paths. Each path contains a complete sequence of actions: the activated expert routing chain and the generated auxiliary diagnostic report. These eight parallel trajectories are then simultaneously input into a simulated medical environment for evaluation. The composite reward of each chain is evaluated based on a constructed composite reward function, thereby assessing the relative advantage value. Furthermore, the open-source multimodal large model and the reference model generate probability distributions for each lexical term based on the final hidden state extracted from the patient's medical description, obtaining the KL divergence between the two probability distributions. The total loss function, based on the loss value determined by the relative advantage value and the KL divergence, guides the parameter updates of the policy model.

[0058] II. Construction of Composite Reward Function

[0059] To reduce the reward variance caused by a single sampling trajectory, this invention employs a group parallel sampling mechanism. For the same input, the composite state... The strategy model does not generate just one expert routing chain (also known as a diagnosis and treatment decision trajectory or path), but rather samples in parallel. G candidate expert routing chains (G candidate trajectories). In one embodiment, let... ,by For example, eight expert routing chains are independently sampled for the same case status. Each expert routing chain... Each chain includes an expert routing action sequence, such as the activated primary expert, collaborating experts, and their invocation order. Each expert routing chain also corresponds to an auxiliary diagnostic report (natural language diagnostic text, such as diagnostic conclusions, pathological descriptions, and treatment or follow-up recommendations) obtained based on that chain. The above 8 expert routing chains, corresponding auxiliary diagnostic reports, and tags are respectively input into the reward evaluation module, where they are evaluated for quality by environmental or expert rules.

[0060] According to one embodiment of the present invention, the reward evaluation module employs a composite reward function. The composite reward function determines sub-rewards for each expert routing chain from multiple dimensions, and then performs a weighted sum of these sub-rewards to obtain the composite reward value for that expert routing chain. For example, for the first... Each expert routing chain calculates its own format reward. Content rewards and routing rewards Preferably, the composite reward function is:

[0061]

[0062] in, Indicates expert routing chain Compound rewards, This indicates an expert routing chain used to guide more standardized output formats and keywords. Rewards in the format of This indicates a more accurate expert routing chain used to guide the medical logic output. Content rewards This refers to the expert routing chain used to guide and optimize the expert routing chain to improve the accuracy of scheduling expert agents. Routing rewards Representation of reward coefficient format This represents the content reward coefficient. This represents the routing reward coefficient. By using this multi-dimensional composite reward function, instead of a single scalar reward, the problem of competition interference in multi-task learning can be solved.

[0063] According to one embodiment of the present invention, format reward Based on rule matching, the system detects whether the output text strictly follows a predefined structure and contains necessary medical keywords. This reward guides the model to quickly learn the standardized report format. Preferably, the format reward is:

[0064]

[0065] in, Indicates the number of structural segments that should appear. This indicates the actual number of segments hit by the training auxiliary diagnosis results. This represents the total number of keywords necessary to accurately describe the condition. This indicates the number of keywords hit in the training and auxiliary diagnosis results; This indicates whether the current format meets the requirements; a value of 1 indicates yes, and a value of 0 indicates no. , , This represents the weighting coefficient.

[0066] According to one embodiment of the present invention, content rewards The reward assesses the accuracy of diagnostic conclusions and the completeness of pathological descriptions based on expert knowledge or tag comparison. This reward reflects the correctness of medical logic. Preferably, the content reward is:

[0067]

[0068] in, This indicates the degree of consistency between the diagnostic conclusions from the training-assisted diagnosis and the results indicated on the label. This indicates the coverage of lesion descriptions based on the training and auxiliary diagnostic results. This indicates the coverage of treatment or follow-up consultation recommendations based on the results of training and auxiliary diagnosis. , , This represents the weighting coefficient. For example, for a patient with moderate non-proliferative diabetic retinopathy and at risk of macular edema, expert standard recommendations include: controlling blood sugar, blood pressure, and blood lipids; repeating fundus photography and OCT within 3 months; considering anti-VEGF treatment if macular edema progresses; and seeking medical attention promptly if sudden vision loss occurs. If the model output is "It is recommended that the patient strengthen blood sugar and blood pressure control, and repeat fundus photography and OCT after 3 months; if subsequent OCT shows worsening macular edema, anti-VEGF treatment can be considered," then this output covers 3 of the 4 standard recommendations, but does not cover the risk warning of "seeking medical attention promptly if sudden vision loss occurs." Therefore, the coverage of treatment or follow-up recommendations is [not specified]. =3 / 4=0.75. This indicator can be used to quantify the extent to which the treatment or follow-up recommendations output by the model cover the expert standard treatment protocol.

[0069] According to one embodiment of the present invention, the routing reward is:

[0070]

[0071] in, The characteristic function of the responsible expert is defined when the responsible expert in the expert routing chain... This belongs to the correct set of responsible experts. The value is 1 if the condition is met, and 0 otherwise. The length of the chain. Indicates adjacent experts and Output semantic similarity; This indicates whether the current expert routing chain is deemed capable of performing the auxiliary diagnosis task; a value of 1 indicates yes, and a value of 0 indicates no. , , , This represents the weighting coefficient.

[0072] III. Construction of the Advantage Function

[0073] In the same state Composite rewards for expert routing chains Subsequently, instead of using an additional value model to estimate state value, relative advantage is calculated based on within-group statistics.

[0074] According to one embodiment of the invention, the invention does not rely on a large value model to predict the baseline, but rather on the statistical characteristics of sampled multiple expert routing chains. For the first... The relative advantage value of each expert routing chain (trajectory) for:

[0075]

[0076] in, Indicates expert routing chain (No. In this fine-tuning of the composite reward system, (the expert routing chain) This indicates that expert routing chains 1 to 10 were included in this fine-tuning. The average compound reward, This indicates that expert routing chains 1 to 10 were included in this fine-tuning. The standard deviation of compound rewards, This represents a stable term that prevents the denominator from being zero. The minimum value is typically taken. This calculation method, based on relative merit within a group, significantly reduces the variance of gradient estimation and eliminates the need to maintain a value model of the same size as the policy model in memory.

[0077] The average compound reward is: ;

[0078] The standard deviation of the compound reward is: ;

[0079] in, Indicates expert routing chain (No. The expert routing chain) is included in the composite reward in this fine-tuning.

[0080] If the composite reward for a certain trajectory is higher than the group average, then This indicates that the trajectory is superior to other candidate trajectories in the same group, and the generation probability of action sequences and text output in this trajectory should be increased during fine-tuning; if the reward of a certain trajectory is lower than the group average, then... This indicates that the trajectory is relatively poor, and its generation probability should be reduced during fine-tuning. Since this advantage calculation only depends on the relative relationships between samples within the same group, it does not require additional loading and updating of the value model, thus saving GPU memory while reducing the instability caused by differences in reward scales among different cases.

[0081] IV. Fine-tuning process

[0082] The fine-tuning process uses PPO trimming updates: leveraging calculated advantages A loss function is constructed by combining the pruning mechanism of the PPO algorithm, and the parameters of the open-source multimodal large model and the linear projection head are updated through backpropagation. This process iterates continuously, increasing the probability density of the trajectory generated by the model in the high-reward region, ultimately achieving a balance between compliance, accuracy, and logic.

[0083] According to one embodiment of the present invention, before inference, an open-source multimodal large model and a linear projection head are fine-tuned. During fine-tuning, a reference model with frozen parameters is provided. The fine-tuning includes multiple operations as follows: acquiring a training set comprising multiple samples, each sample including sample content and labels; the sample content being the medical description of the sample object, and the labels being the diagnostic results of the sample object; generating a first probability distribution of routing actions from the final hidden states extracted from the medical descriptions of the sample objects by the open-source multimodal large model; generating a second probability distribution of routing actions from the final hidden states extracted from the medical descriptions of the sample objects by the reference model; determining the KL divergence between the first and second probability distributions; sampling multiple expert routing chains using the GRPO algorithm based on a valueless model to obtain the training auxiliary diagnosis results corresponding to each chain; determining a composite reward containing multiple reward indicators based on the training auxiliary diagnosis results and labels using a composite reward function; determining the relative advantage value of each routing chain based on the composite reward; and obtaining a loss value determined based on the relative advantage value and KL divergence of each chain using a preset total loss function, updating the parameters of the open-source multimodal large model and the linear projection head with the goal of minimizing the loss value.

[0084] According to one embodiment of the present invention, in order to prevent the policy update from being too large, the probability ratio is pruned, and a KL divergence penalty term is combined to form the final total loss function:

[0085]

[0086] in, This indicates the number of expert routing chains sampled. This is the cutting factor. The weighting coefficients for the KL divergence are... This indicates the routing chain generated by the current policy model and the policy model before the update. The ratio of the probabilities, Indicates according to expert routing chain The expert routing chains are determined by the composite reward, the average composite reward of each expert routing chain, and the standard deviation of the composite reward. The relative advantage value, Indicates will Limited to [ , The clipping function within the interval. Let represent the KL divergence. During the parameter update phase, a proximal policy with pruning is used to optimize the objective. Let the old policy (the policy model before the update) be... The current strategy model is Then the routing chain The corresponding probability ratio is:

[0087]

[0088] in, This indicates that the current policy model generates a routing chain. The probability, This indicates that the policy model generates routing chains before the update. The probability of.

[0089] The above pruning terms are used to limit the update magnitude between the old and new policies, preventing the model from over-updating due to a single high-reward sample. The KL divergence term is used to constrain the distribution difference between the policy model and the reference model, preventing language degradation, medical semantic drift, or unreasonable routing in the policy model during reinforcement learning. Finally, the trainable parameters and linear projector head parameters in the Qwen3-VL policy model are updated simultaneously through backpropagation, gradually increasing the probability of generating high-reward trajectories and gradually decreasing the probability of generating low-reward trajectories.

[0090] According to one embodiment of the present invention, during fine-tuning, the weighting coefficients in the composite reward function can always be set to the same value to balance the learning of knowledge from all aspects. However, further improvements are possible. Optionally, during fine-tuning, a phased dynamic weight adjustment mechanism is set, wherein: in the initial stage of fine-tuning, the format reward coefficient is less than the content reward coefficient and the routing reward coefficient to suppress over-optimization of the format reward; in the middle stage of fine-tuning, the values ​​of the content reward coefficient and the routing reward coefficient are gradually increased; in the later stage of fine-tuning, the values ​​of the format reward coefficient, the content reward coefficient, and the routing reward coefficient are dynamically and gradually adjusted to tend towards balance. Thus, in the early stage of training, the system reduces the weight of the format reward to avoid the model from generating "speculative score-boosting" by prioritizing the learning of simple formats; in the middle stage of training, the weights of the content reward and the routing reward are gradually increased to make the model pay more attention to medical diagnostic logic and expert scheduling accuracy. And in the later stage of training, the three are dynamically balanced to ensure that the model reaches Nash equilibrium in multi-objective games. This mechanism effectively eliminates gradient interference between tasks and solves the problem of loss function oscillation in the later stage of training. Through dynamic weight adjustment, this embodiment can alleviate the competitive interference between the three types of tasks: format generation, medical reasoning, and expert routing.

[0091] Through the above fine-tuning process, this invention forms a closed-loop multi-agent scheduling optimization process: First, the object description module and the memory information module construct a holographic input state, i.e., a multimodal composite state; second, the policy model and the reference model propagate forward in parallel, with the reference model constraining policy drift through KL divergence; third, the linear projection head generates expert activation probabilities based on the final hidden state and samples them to form expert routing links (actions, expert action links); finally, dynamic weighted composite rewards are used to evaluate each expert routing link and its corresponding auxiliary diagnosis and treatment report, and parameters are updated based on intra-group relative advantage and a pruning objective function. This fine-tuning method does not require an independent value model and can achieve reinforcement learning alignment of large multimodal models under limited computing power, while improving the model's expert routing accuracy, auxiliary diagnostic logic integrity, and long-range memory utilization capabilities in complex ophthalmic diagnosis and treatment scenarios.

[0092] According to one example of the present invention, the fine-tuning training process of the policy model (scheduling model) adopts a deep reinforcement learning framework based on Group Relative Policy Optimization (GRPO). This framework uses a Qwen3-VL multimodal large model as the policy model, a parameter-frozen isomorphic model as the reference model, and combines recursive memory of historical case memory information (SOAP), object description modules, linear projection heads, group parallel sampling, dynamic weighted composite rewards, and KL divergence constraints to achieve joint optimization of expert link scheduling and auxiliary diagnostic text generation for complex ophthalmology diagnosis and treatment scenarios. Unlike traditional PPO methods, this example does not additionally train a value model (Critic) of the same scale as the policy model. Instead, it estimates the advantage function through the intra-group relative merit relationships between multiple candidate trajectories in the same state, thereby significantly reducing training memory overhead and improving the stability of multi-agent routing training. Illustratively, in each fine-tuning iteration, the multimodal composite state of the current round is first constructed. This state is composed of three parts: the patient's textual complaint in the current dialogue round, the available ophthalmic images, and the historical treatment context (i.e., historical case memory information). The historical treatment context is maintained by the SOAP memory information module and includes subjective information, objective examination information, previous assessment information, and treatment or follow-up visit plan information. The object description module parses the currently input medical images (such as fundus photography, OCT images, or fluorescein angiography images) and outputs a standardized JSON object. The historical treatment context and the standardized JSON object undergo structuring processing. This processing transforms the original complaint text, images, and historical medical records into a unified holographic state representation, i.e., a multimodal composite state. This allows the policy model to simultaneously perceive current image evidence and long-term treatment context during training, rather than making "snapshot" judgments based solely on a single image input.

[0093] V. Application Scenarios

[0094] According to an embodiment of the present invention, a multi-agent-based medical auxiliary diagnosis method includes: structuring the current patient's medical history description and historical case memory information to obtain a multimodal composite state, wherein the medical history description includes the patient's chief complaint and medical images, and the historical case memory information includes temporal information capturing the evolution of the patient's condition extracted based on the patient's previous medical history descriptions and diagnostic results; inputting the multimodal composite state into an open-source multimodal large model fine-tuned by reinforcement learning for forward feature extraction to obtain a final hidden state; and using a linear projection head to determine multiple specialized agents based on the final hidden state. The activation probabilities of each expert in the expert agent library are calculated. Based on the activation probabilities of each expert, an expert routing chain containing the primary expert and collaborating experts is selected. The expert agents specified in the expert routing chain are called in sequence. The primary expert agent performs an initial diagnosis based on the multimodal composite state to obtain the initial diagnosis result. The collaborating expert agents perform a supplementary consultation based on the multimodal composite state and the initial diagnosis result to obtain the supplementary result. An open-source multimodal large model is used to generate a predicted auxiliary diagnosis result based on the initial diagnosis result and the supplementary result. This prediction result includes the probability that the object has various preset diseases, the auxiliary diagnosis report, and the comprehensive confidence of this prediction.

[0095] According to one embodiment of the present invention, after fine-tuning, the application process of the present invention can be summarized as a complete closed loop of loading the fine-tuned policy model (Router), constructing the current composite state, generating expert routing chains, machine-side collaborative diagnosis, human-side follow-up diagnosis when necessary, outputting results, and writing back to memory. Illustratively, firstly, the system loads the parameters of the trained policy model. Based on the above example, the saved content includes the LoRA adapter of the Qwen3-VL large model and the parameter files of the linear projection head, serving as the inference basis during deployment. The reference model is mainly used for KL constraints during the training phase and no longer participates in joint forward propagation during the inference phase. Secondly, the system receives the patient's current input. This input includes the patient's chief complaint text and optional ophthalmic images. Before entering the policy model, the object description module first standardizes the current chief complaint and images to obtain the standardization result. Then, the standardization result and historical case memory information are structured to obtain the multimodal composite state. The object description module currently supports text input, single or multiple image inputs, and performs format recognition, compression, and encoding processing on the images. Then, the system inputs the composite state into the tokenizer of the policy model, and then forward propagates it through the Qwen3-VL large model to obtain the final hidden state. This hidden state is then mapped to the expert space through a linear projection head to obtain the activation probability of each candidate expert. Based on this activation probability, a primary expert is selected, and subsequent collaborating experts can be selected iteratively to form an expert routing chain. In the fine-tuning phase, Categorical(logits=logits) is used to sample actions from the probability distribution; in the deployment phase, maximum probability or Top-k strategies can be used for expert selection. After forming the expert routing chain, the on-device multi-agent collaborative diagnosis module is invoked. In the current implementation, the first element of the expert routing chain index (agent_index) corresponds to the primary expert, and the remaining elements correspond to collaborating experts. The primary expert performs an initial diagnosis, and the collaborating experts provide supplementary consultations based on the primary expert's context. Finally, the large model integrates the opinions of the primary expert and the collaborating experts, outputting the final auxiliary diagnosis, differential diagnosis analysis, and comprehensive confidence score, such as the probability of the object suffering from various preset diseases, the auxiliary diagnosis report, and the comprehensive confidence score of this prediction. If the overall confidence level is lower than a preset threshold, or if there are obvious conflicting conclusions, the human-side follow-up consultation module can be further triggered. The current working method of the human-side module is as follows: Based on the responsible expert in the machine's output, it maps to the corresponding human-side digital doctor. The patient's chief complaint, complex status, the machine's final auxiliary treatment report, and detailed diagnostic logs are input into the human-side model, which then outputs the final follow-up consultation opinion and next steps suggestions. Finally, the modified final auxiliary treatment report is output to the patient, including auxiliary results, treatment suggestions, and follow-up consultation suggestions. The chief complaint, patient description, machine conclusion, and human-side follow-up consultation results are written back to the memory information module for subsequent follow-up consultations or multiple rounds of dialogue, thus achieving memory write-back and application logic for reuse in future assistance.

[0096] To verify the effectiveness of the method of this invention in ophthalmic multi-agent collaborative diagnosis and treatment networks, a systematic comparative experiment has been conducted around the core technical solution of this invention. In this experiment, the method of this invention is implemented based on the improved GRPO Qwen3-VL policy model. In the current fine-tuning environment, the input state of the policy model consists of historical case memory information, the patient's chief complaint, and medical images, and scheduling is completed by selecting an expert routing chain; the machine end is responsible for collaborative assisted diagnosis, and the human end performs a second consultation when necessary. To comprehensively evaluate the effect of this invention, two main experiments were designed. Among them:

[0097] Experiment 1: Multiple assisted diagnoses and single-round dialogues

[0098] This means that patients may visit the hospital multiple times, but each time they only interact with the strategy model once. This experiment mainly examines the system's assisted diagnosis accuracy, average number of expert calls, key information retention rate, and token overhead in relatively independent case scenarios.

[0099] Experiment 2: Single-round auxiliary diagnosis and multi-turn dialogue

[0100] This involves patients engaging in multiple rounds of dialogue with the strategy model during a single medical visit. The experiment primarily examines the system's long-range memory utilization, scheduling stability, and resource overhead in continuous context scenarios.

[0101] The baselines compared in this invention mainly include five representative schemes: EYE-Llama, as a single-model baseline specifically for the ophthalmology field, reflects the professional knowledge enhancement brought about by domain pre-training and instruction fine-tuning; VisionUnite, as a single-model baseline for ophthalmology vision-language, reflects multimodal understanding capabilities; MoA, as a general multi-agent collaborative baseline, reflects a multi-agent iterative fusion mechanism; SGHR, as a routing baseline based on dense vector semantic retrieval, reflects a strategy of selecting experts based solely on semantic similarity; and DR, as a heuristic retrieval baseline based on static graphs, reflects a strategy of extending links based on predefined expert relationship graphs. By comparing with the above baselines, the comprehensive advantages of this invention in ophthalmology multimodal understanding, dynamic expert scheduling, multi-round memory utilization, and resource overhead control can be comprehensively verified.

[0102] The experimental results of the two groups are as follows:

[0103] Experiment 1: Multiple assisted diagnoses and single-round dialogues

[0104]

[0105] Experiment 2: Single-round auxiliary diagnosis and multi-turn dialogue

[0106]

[0107] Comparative experiments show that this invention outperforms existing single-model assisted diagnosis schemes, vision-language assisted diagnosis schemes, multi-agent hybrid schemes, and static retrieval scheduling schemes in terms of assisted diagnosis accuracy, context preservation capability, and resource utilization efficiency. Especially in multi-round assisted diagnosis scenarios, this invention significantly improves key information retention and final assisted diagnosis accuracy while maintaining a low number of expert calls and low token overhead. These experimental results demonstrate that the method of this invention not only achieves efficient integration of multimodal medical information but also enhances the overall performance of multi-agent collaborative diagnosis networks through dynamic expert routing chain selection.

[0108] Experimental results demonstrate that the collaborative network based on dynamic weighted composite reward and holographic memory enhancement proposed in this invention has achieved significant breakthroughs in complex ophthalmic diagnostic and treatment scenarios. Compared with traditional SFT or standard PPO methods, this invention significantly reduces computational resource consumption while substantially improving the model's coherence in long-term medical record analysis and the stability of multi-task processing. Particularly in complex cases requiring refined triage, such as glaucoma and diabetic retinopathy, the system's triage accuracy and the quality of auxiliary diagnostic reports have both been improved to a certain extent.

[0109] In summary, some embodiments of the present invention can achieve at least one or more of the following beneficial effects:

[0110] (1) To address the problem of target competition interference caused by improper reward mechanism design in the multi-task reinforcement learning training of existing ophthalmic artificial intelligence systems, and the problems of logical breaks and insufficient professionalism in auxiliary diagnosis caused by the lack of long-range SOAP medical record memory and the lack of refined specialty routing mechanism in the inference stage, a medical auxiliary diagnosis method based on multi-agent is proposed. In reinforcement learning, this method successfully eliminates the competition interference between multiple tasks by designing a composite reward function containing "format reward, content reward, and routing reward" and dynamically adjusting the weights using a multi-objective optimization algorithm; at the same time, it realizes full-information diagnosis and treatment reasoning by reconstructing the state space containing time-series medical records, and realizes intelligent diversion from general perception to specialized diagnosis by designing a dedicated linear projection routing head, thereby constructing a logically rigorous, specialized, and stable automated auxiliary diagnosis and treatment system.

[0111] (2) For the fine-tuning architecture, a creative strategy of "group relative policy optimization of valueless models" is proposed. That is, during the training phase, the traditional four-model architecture is abandoned, and only the policy model and the reference model are retained. For each ophthalmology diagnosis and treatment problem, the model samples in parallel to generate a set of outputs. The average reward of this set of outputs is calculated as the baseline, and then the relative advantage of each output is calculated to guide the policy update.

[0112] (3) For input representation: a “SOAP holographic state space” is proposed. That is, at the input of the model, it is required to perform multimodal fusion of the patient’s historical memory text record and the current fundus image features to ensure that the model is based on the complete “medical record + image” when making inference, rather than a single visual fragment.

[0113] (4) For decision output: A "dual-channel" mechanism of "hard routing and soft inference in parallel" is introduced. That is, in addition to the regular text generation channel, a dedicated linear projection head is added inside the model to directly output the activation signal of the expert model, so as to realize the precise guidance and diversion of diagnosis and treatment tasks.

[0114] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0115] This invention can be a system, method, electronic device, computing device, computer program product and / or computer-readable medium.

[0116] Computer program products mainly refer to software products that implement various aspects of the present invention through computer programs, or hardware products that carry software that implements various aspects of the present invention.

[0117] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0118] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A medical auxiliary diagnosis method based on multi-agent intelligence, comprising: The current patient's medical visit description and historical case memory information are structured to obtain a multimodal composite state. The medical visit description includes the patient's chief complaint and medical images, and the historical case memory information contains temporal information that captures the evolution of the disease based on the patient's previous medical visit descriptions and diagnostic results. The multimodal composite state is input into an open-source multimodal large model fine-tuned by reinforcement learning for forward feature extraction to obtain the final hidden state. The activation probability of each expert in various expert agent databases is determined based on the final hidden state using a linear projection head. Based on the activation probability of each expert, an expert routing chain containing the primary expert and collaborating experts is selected. The expert agents specified in the expert routing chain are called in sequence. The primary expert agent performs an initial diagnosis based on the multimodal composite state to obtain the initial diagnosis result. The collaborating expert agents perform a supplementary consultation based on the multimodal composite state and the initial diagnosis result to obtain the supplementary result. Using an open-source multimodal large model, predictive auxiliary diagnostic results are generated based on the initial diagnosis results and supplementary results. These results include the probability of the subject having various preset diseases, auxiliary diagnosis and treatment reports, and the overall confidence level of this prediction.

2. The method according to claim 1, characterized in that, Before inference, the open-source multimodal large model and linear projection head are fine-tuned. A reference model with parameter freezing is set during fine-tuning. The fine-tuning includes the following operations multiple times: Obtain the training set, which includes multiple samples. Each sample includes sample content and labels. The sample content is the medical visit description of the sample object, and the labels are the diagnosis results of the sample object. The first probability distribution of routing actions is generated by the open-source multimodal large model based on the final hidden state extracted from the medical visit description of the sample object; The second probability distribution of routing actions is generated by the reference model based on the final hidden state extracted from the medical visit description of the sample object; Determine the KL divergence between the distributions based on the first probability distribution and the second probability distribution; The GRPO algorithm based on the no-value model samples multiple expert routing chains to obtain the training and diagnosis results corresponding to each chain. It uses a composite reward function to determine a composite reward containing multiple reward indicators based on the training and diagnosis results and labels. Based on the composite reward, it determines the relative advantage value of each routing chain. Based on the preset total loss function, the loss value is obtained according to the relative advantage value and KL divergence of each chain. The parameters of the open-source multimodal large model and the linear projection head are updated with the goal of minimizing the loss value.

3. The method according to claim 2, characterized in that, The total loss function is: in, This indicates the number of expert routing chains sampled. This is the cutting factor. The weighting coefficients for the KL divergence are... This indicates the routing chain generated by the current policy model and the policy model before the update. The ratio of the probabilities, Indicates according to expert routing chain The expert routing chains are determined by the composite reward, the average composite reward of each expert routing chain, and the standard deviation of the composite reward. The relative advantage value, Indicates will Limited to [ , The clipping function within the interval. This represents the KL divergence.

4. The method according to claim 2, characterized in that, The composite reward function is: in, Indicates expert routing chain Compound rewards, This indicates an expert routing chain used to guide more standardized output formats and keywords. Rewards in the format of This indicates a more accurate expert routing chain used to guide the medical logic output. Content rewards This refers to the expert routing chain used to guide and optimize the expert routing chain to improve the accuracy of scheduling expert agents. Routing rewards Representation of reward coefficient format This represents the content reward coefficient. This represents the routing reward coefficient.

5. The method according to claim 4, characterized in that, The format of the reward is as follows: in, Indicates the number of structural segments that should appear. This indicates the actual number of segments hit by the training auxiliary diagnosis results. This represents the total number of keywords necessary to accurately describe the condition. This indicates the number of keywords hit in the training and auxiliary diagnosis results; This indicates whether the current format meets the requirements; a value of 1 indicates yes, and a value of 0 indicates no. , , This represents the weighting coefficient.

6. The method according to claim 4, characterized in that, Content rewards are: in, This indicates the degree of consistency between the diagnostic conclusions from the training-assisted diagnosis and the results indicated on the label. This indicates the coverage of lesion descriptions based on the training and auxiliary diagnostic results. This indicates the coverage of treatment or follow-up consultation recommendations based on the results of training and auxiliary diagnosis. , , This represents the weighting coefficient.

7. The method according to any one of claims 4-6, characterized in that, The routing reward is: in, The characteristic function of the responsible expert is defined when the responsible expert in the expert routing chain... This belongs to the correct set of responsible experts. The value is 1 if the condition is met, and 0 otherwise. The length of the chain. Indicates adjacent experts and Output semantic similarity; This indicates whether the current expert routing chain is deemed capable of performing the auxiliary diagnosis task; a value of 1 indicates yes, and a value of 0 indicates no. , , , This represents the weighting coefficient.

8. The method according to claim 4, characterized in that, During fine-tuning, a phased dynamic weight adjustment mechanism is set up, wherein: In the initial stage of fine-tuning, the format reward coefficient is set to be smaller than the content reward coefficient and the routing reward coefficient to suppress over-optimization of the format reward. During the fine-tuning phase, gradually increase the values ​​of the content reward coefficient and the routing reward coefficient; In the later stages of fine-tuning, the values ​​of the format reward coefficient, content reward coefficient, and routing reward coefficient are dynamically and gradually adjusted to tend towards balance.

9. A computer device / equipment / system, comprising a memory, a processor, and computer programs / instructions stored in the memory, characterized in that, The processor executes the computer program / instructions to implement the steps of the method according to any one of claims 1-8.

10. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-8.

Images