Data processing method and electronic device for medical discipline knowledge service

Abstract

The application provides a data processing method and electronic equipment for medical discipline knowledge service, and belongs to the technical field of medicine. The method comprises the following steps: in response to a medical discipline knowledge question, a multi-agent collaboration team is formed; a plurality of target information dimensions required for answering the medical discipline knowledge question and an importance parameter corresponding to each target information dimension are determined through the multi-agent collaboration team to output a preliminary service result, wherein in the process of performing reasoning, the multi-agent collaboration team retrieves a positive and negative example reference library to obtain a historical case associated with the medical discipline knowledge question, the target information dimension and / or the current reasoning stage as context knowledge; the correctness of the preliminary service result is evaluated, and a final service result is generated and output according to the evaluation result. The application can continuously absorb experience through the positive and negative example reference library by using the multi-agent collaboration team to improve the accuracy of the solution to the medical discipline knowledge question without heavy calculation parameter training.

Description

Technical Field

[0001] This application relates to the field of medical technology, and in particular to a data processing method and electronic device for providing medical knowledge services. Background Technology

[0002] As society ages, the need for advanced knowledge discovery is becoming increasingly urgent for complex diseases such as Alzheimer's and cancer. Understanding these diseases often relies on the comprehensive analysis and reasoning of multi-dimensional and multi-modal information, such as integrating medical literature, neuroimaging (e.g., MRI), laboratory data, and bioinformatics data, to accomplish complex tasks such as disease risk prediction and biological age estimation.

[0003] Existing auxiliary tools based on large language models (LLMs) face two prominent bottlenecks in such complex tasks. First, while general-purpose large language models with a large parameter scale possess some generalization ability, their accuracy for specialized medical problems is insufficient, and they consume significant resources. Conversely, while large language models with a small parameter scale are more efficient, they struggle to effectively model complex semantic relationships between cross-modal data (such as text, images, and structured indicators). Second, to achieve model evolution, existing technologies largely rely on online reinforcement learning or continuous fine-tuning, which essentially involves repeatedly adjusting massive amounts of model parameters. This process not only requires enormous computing power but can also lead to model instability or catastrophic forgetting, making it difficult to achieve efficient and stable performance improvements in resource-constrained real-world scenarios.

[0004] Therefore, how to enable the system to continuously absorb experience and improve the accuracy and reliability of answering complex medical knowledge questions without recalculating parameters during training has become a key technical problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a data processing method and electronic device for medical subject knowledge services, in order to solve the above-mentioned problems.

[0006] To achieve the above objectives, firstly, this application proposes a data processing method for medical subject knowledge services, the method comprising: In response to a user's input of a medical subject knowledge question, a multi-agent collaborative team is formed to serve the medical subject knowledge question. Through the multi-agent collaborative team, multiple target information dimensions and importance parameters corresponding to each target information dimension are determined for answering the medical knowledge question. Based on the data information and importance parameters corresponding to the multiple target information dimensions, a preliminary service result for the medical knowledge question is output. During the inference process, each agent in the multi-agent collaborative team retrieves historical cases associated with the medical knowledge question, the multiple target information dimensions, and / or the current inference stage as contextual knowledge by searching a positive and negative example reference library. The result evaluation agent evaluates the correctness of the preliminary service results and generates and outputs the final service results for the medical subject knowledge question based on the evaluation results.

[0007] In some implementations, the step of determining multiple target information dimensions and corresponding importance parameters for each target information dimension through the multi-agent collaborative team includes: Through the multi-agent collaborative team, the initial information dimensions and initial information data required to answer the questions about medical subject knowledge are determined; Based on the initial information dimension and the initial information data, the question-generating agent generates supplementary prompt questions for obtaining supplementary data. Obtain the user's supplementary answer to the supplementary prompt question, and summarize the initial information dimensions, the initial information data and the supplementary answer through the process summary agent to obtain multiple target information dimensions and corresponding data information required to answer the medical knowledge question; The importance parameters are obtained by ranking and weighting the multiple target information dimensions and corresponding data information by the multi-agent collaborative team.

[0008] In some implementations, determining the initial information dimensions and initial information data required to answer the medical subject knowledge question through the multi-agent collaborative team includes: The intelligent agent generates information dimension questions to determine the information dimensions. By having each agent in the multi-agent collaborative team answer the information dimension question based on the medical subject knowledge question, an information dimension answer set is obtained; The process summary agent summarizes the set of answers for the information dimensions to determine the initial information dimensions and initial information data required to answer the questions about medical knowledge.

[0009] In some implementations, the importance parameter includes an importance order and / or an importance weight. The step of ranking and weighting the multiple target information dimensions and corresponding data information by the multi-agent collaborative team to obtain the importance parameter includes: The problem-generating agent generates a sorting problem to determine the importance order of the multiple target information dimensions and corresponding data information. The importance ranking answer set is obtained by having each agent in the multi-agent collaborative team answer the ranking question based on the medical subject knowledge question. The process summarization agent summarizes the importance ranking answer set to obtain the importance order of the multiple target information dimensions and corresponding data information, as well as the importance score; The remaining information dimensions are defined as those other than the first-ranked information dimension in importance. The importance weights are then calculated based on the importance order and corresponding importance scores of these remaining information dimensions.

[0010] In some implementations, the step of outputting preliminary service results for the medical subject knowledge question based on the data information and importance parameters corresponding to the multiple target information dimensions includes: Based on the data information corresponding to the target information dimension ranked first in importance, the basic results of the medical subject knowledge question are determined. Based on the data information corresponding to the remaining information dimensions and the importance weights, the correction result for the medical subject knowledge question is determined; Based on the basic results and the correction results, the overall answer result is determined; The preliminary service result is obtained by having each agent in the multi-agent collaborative team verify the overall response result and adjust the overall response result based on the verification result.

[0011] In some implementations, the step of having each agent in the multi-agent collaborative team validate the overall response result and adjust the overall response result based on the validation results to obtain the preliminary service result includes: The intelligent agent generates a result argumentation question to validate the overall answer result. The result argumentation question is generated based on the medical subject knowledge question, the basic result, the correction result, the sub-correction result corresponding to each of the remaining information dimensions, the overall answer result, the importance order, the importance weight, and the previous round of argumentation results. The result argumentation answer set is obtained by having each agent in the multi-agent collaborative team answer the result argumentation question; The process summary agent summarizes the result argumentation answer set, and when more than a preset proportion of agents in the multi-agent collaboration team determine to adjust the importance weight, the adjusted importance weight, the adjusted correction result, and the adjusted overall answer result are generated. Repeat the step of generating a result argumentation question through a question-generating agent to argue the overall answer result until the preset argumentation termination condition is met; Based on the adjusted importance weights, the overall response results are adjusted to obtain preliminary service results.

[0012] In some implementations, after the result evaluation agent evaluates the correctness of the preliminary service result and generates and outputs the final service result of the medical subject knowledge question based on the evaluation result, the process includes: The medical subject knowledge questions, the multiple target information dimensions and corresponding data information, reasoning process, preliminary service results, final service results and evaluation results are used as service records, and the service records are stored in a positive and negative example reference library; The process knowledge management intelligent agent summarizes the service records to obtain summary information on the knowledge questions of the medical discipline, and stores the summary information in association with the service records in a positive and negative example reference library. The summary information includes result summary, evaluation reasons and target knowledge information.

[0013] In some embodiments, the method further includes: Based on the content of knowledge fields in the positive and negative example reference library, the target knowledge information is evaluated by a process knowledge management intelligent agent; When the result of the knowledge evaluation reaches the first preset level, the system queries whether there are historical case records in the positive and negative example reference library that have a similarity greater than a preset similarity threshold with the target knowledge information. If there are such records, the target knowledge information is used to cover the historical knowledge information of the historical case records. When the result of the knowledge evaluation reaches the second preset level, the system queries whether there are historical case records in the positive and negative example reference library that have a similarity greater than a preset similarity threshold with the target knowledge information. If there are such records, the system optimizes the historical knowledge information of the historical case records based on the target knowledge information. When the result of the knowledge evaluation reaches the third preset level, the target knowledge information and associated service records are deleted from the positive and negative example reference library.

[0014] In some implementations, the agent applied to the method performs a reasoning process by invoking a pre-built, small-parameter-scale multimodal large language model of medical disciplines. The agent performing the reasoning process generates a function call request based on the question content and maps the function call request to a tool call request by invoking a pre-built basic model. This allows the agent to call the corresponding processing tool from a preset tool library and obtain the processing result to be incorporated into the contextual knowledge. Each processing tool in the preset tool library is labeled with a classification tag and information identifier and is organized according to a graph pattern based on dependencies. The classification tags include topic type, level, modality, and granularity. The information identifiers include tool name, tool function, required input parameters, data type, return result, related attention, and mapping rules.

[0015] Secondly, to achieve the above objectives, this application also proposes an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the data processing method for medical subject knowledge services as described above.

[0016] Compared with the prior art, the beneficial effects of this application include: Firstly, by forming a multi-agent collaborative team to address specific medical knowledge questions, it is possible to solve the problems of high resource consumption and insufficient accuracy of general-purpose large language models with large parameter scales, and poor performance and insufficient generalization ability of large language models with small parameter scales, despite low resource consumption. This improves the overall service effect when multiple sources of information, such as medical images, structured data, and natural language, are required to answer questions, demonstrating higher accuracy and reasoning efficiency.

[0017] Secondly, by determining multiple target information dimensions and the importance parameters corresponding to each target information dimension and outputting preliminary service results, the problem of multiple information dimensions and difficulty in effective integration and prioritization in complex medical disciplines is solved. Information fusion and weighting processing, as well as phased data processing, are realized, thereby improving the scientificity, reliability and interpretability of the preliminary service results.

[0018] Thirdly, by enabling each agent to retrieve a reference library of positive and negative examples during the inference process and using the retrieved historical cases as contextual knowledge to assist in decision-making, this process does not involve updating or fine-tuning the underlying model parameters of the agent. It achieves a reinforcement learning-like effect in an offline learning state, thereby completely avoiding the huge computing power overhead, training instability and catastrophic forgetting problems caused by traditional reinforcement learning or continuous fine-tuning, and achieving low-cost, lightweight, stable and reliable continuous performance improvement.

[0019] Fourthly, by using an independent result evaluation agent to assess the correctness of the preliminary results and generating the final service results based on the evaluation results, the credibility and robustness of the final output service results are enhanced. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0021] Figure 1 This is a flowchart illustrating a data processing method for providing medical subject knowledge services in one embodiment. Figure 2 This is a schematic diagram of the electronic device involved in the data processing method for medical subject knowledge services in the embodiments of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0024] For example, the terms "comprising" or "including" used in this application indicate the presence of features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0025] As mentioned earlier, existing auxiliary tools based on large language models (LLMs) face two prominent bottlenecks in such complex tasks. First, while general-purpose large language models with large parameter scales possess a certain generalization ability, their accuracy for specialized medical questions is insufficient, and they consume significant resources. Conversely, while large language models with small parameter scales are more efficient, they struggle to effectively model complex semantic relationships between cross-modal data (such as text, images, and structured indicators). Second, to achieve model evolution, existing technologies largely rely on online reinforcement learning or continuous fine-tuning, which essentially involves repeatedly adjusting massive amounts of model parameters. This process not only requires enormous computing power but may also lead to model instability or catastrophic forgetting, making it difficult to achieve efficient and stable performance improvements in resource-constrained real-world scenarios. Therefore, how to enable the system to continuously absorb experience and improve the accuracy and reliability of answering complex medical knowledge questions while avoiding computationally intensive parameter training has become a critical technical challenge that urgently needs to be addressed. To this end, this application proposes a data processing method and electronic device for medical subject knowledge services, which can improve the accuracy and reliability of answering complex medical subject knowledge questions by utilizing a multi-agent collaborative team to continuously absorb experience through a positive and negative example reference library, without avoiding heavy computational parameter training.

[0026] like Figure 1 As shown, this application provides a data processing method for medical subject knowledge services. The method involves a multi-agent collaborative system, which includes six types of intelligent agent roles constructed according to the needs of medical subject knowledge services, including user intelligent agents, planning intelligent agents, expert intelligent agents (including neurology expert intelligent agents, psychiatry expert intelligent agents, gerontology expert intelligent agents, nutrition expert intelligent agents, breast cancer and other oncology expert intelligent agents, etc.), data intelligent agents (including MRI image intelligent agents, CT image intelligent agents, laboratory data intelligent agents, bioinformatics intelligent agents, EEG intelligent agents, MRI intelligent agents, voice intelligent agents, video intelligent agents, text intelligent agents, etc.), support intelligent agents (including problem-generating intelligent agents, process summarizing intelligent agents, and result evaluation intelligent agents), and AI intelligent agents (including process knowledge management intelligent agents and reasoning intelligent agents).

[0027] Each type of agent applied to the method performs the inference process by invoking a pre-built medical multimodal large language model with a small parameter scale (no more than 10B parameters). This medical multimodal large language model is obtained by continuously training and fine-tuning a base model (such as deepseek-r1-distill-qwen2.5-7b) across the entire medical field, fine-tuning it in specific subject areas (such as Alzheimer's disease (AD), Mild Cognitive Impairment (MCI), and oncology), and enhancing its multimodal capabilities. Furthermore, it can load more domain knowledge and information through methods such as retrieving positive and negative example reference libraries and invoking tools.

[0028] Regarding the enhancement of multimodal capabilities, considering the specific problem of constructing a large number of cross-modal training datasets for building a medical multimodal large language model with reasoning capabilities, this application proposes a highly efficient construction process for the medical multimodal large language model in order to minimize the workload and ensure the reasoning efficiency of the medical multimodal large language model as much as possible.

[0029] Specifically, a training dataset and an initial multimodal model are obtained. The initial multimodal model includes a visual encoder, a multilayer perceptron (MLP) adapter, and a replacement model for the medical subject-specific large language model with reasoning capabilities (obtained through continuous training and fine-tuning in general medicine and subject-specific fine-tuning). With the parameters of the visual encoder and the replacement model both frozen, the initial multimodal model is trained using the training dataset to update the parameters of the MLP adapter. The replacement model is replaced with the medical subject-specific large language model with reasoning capabilities to obtain a recombined model. An instruction set with reasoning processes is obtained, and the recombined model is trained using the instruction set with reasoning processes to obtain the medical subject-specific multimodal large language model.

[0030] Taking the multimodal capability enhancement process of the AD / MCI multimodal large language model as an example (considering that neuroimaging data includes a large amount of MRI data, the construction of a multimodal large language model based on MRI images is used as an example), the specific construction process is described as follows: (1) Neuroimaging data processing: The collected neuroimaging data is processed. Taking a structural image as an example, the processing work includes format conversion (e.g., converting DICOM format to NIFTI format), skull stripping (removing non-brain structures such as the skull from the image), image registration (using affine transformation and other methods to register the image; available standard template spaces include MNI, Chinese2020, etc.), data stratification (slicing the data into specific numbers of layers according to the transverse, coronal, and sagittal planes (e.g., 60 layers), feature calculation (adding the stratification results of each layer in each dimension, and then averaging according to the number of layers; for other modalities such as functional images, it also includes averaging at each time point to obtain the results x, y, z), and result calculation (applying the results of the three dimensions according to the predetermined weights w). x w y w z The processing result R for this image is obtained.

[0031] (2) Dataset construction: Based on the processed image data results, the corresponding text content and brain MRI image pairs collected during the image collection process are used to construct the dataset. The text content includes relevant text questions (such as asking what the brain age is) and answers (such as brain age numbers). For those that do not have suitable text content, manual annotation is used to obtain it.

[0032] (3) Initial model construction: Select a visual encoder vis (e.g., vit), an AD / MCI large language model reas with reasoning ability (e.g., fine-tuned based on deepseek-r1-distill-qwen2.5-7b), and an alternative model subs with reasoning ability (e.g., fine-tuned based on qwen2.5-7b-instruct). By constructing an MLP adapter theta, the initial model model is constructed. initial = vis * theta * subs. At this point, the parameters of vis and subs are frozen, and only the parameters of theta are optimized. The optimization process is carried out based on the instructions for visual-text (including MRRI-text) alignment that have been collected. After reaching the specified number of training epochs, the training process is stopped and the theta model is obtained.

[0033] (4) Model recombination: Replacing subs with reas yields the model. new =vis*theta*reas, then we get a new, reorganized model.

[0034] (5) Model optimization training: In order to optimize the training of the new model... newIt possesses multimodal reasoning capabilities, and through constructing a small set of instructions with reasoning processes, it can perform model... new Training is then performed. The process of constructing the instruction set is as follows: First, text content and visual image (brain MRI images) pairs are collected (selected from step (2)). The text content includes text questions and answers, forming the initial dataset D. inital The process involves inputting text content and visual images into an existing multimodal model, such as GPT-4O. The task is to describe multiple thought processes leading to an answer based on the images and text, and to score the difficulty of the questions, thus creating multiple samples. After manual review, samples with correct reasoning processes are manually labeled and scored based on the length of their reasoning (shorter reasoning processes receive higher scores). This results in a dataset D comprising (text question, text answer, image, reasoning process, difficulty score, and quality score). new The training process is as follows: First, based on dataset D... initail For model new Fine-tuning training is performed to obtain the model. new1 Secondly, select data set D. new Samples with a quality score greater than a certain threshold constitute D new1 For model new1 Further fine-tuning yields the model new2 Select data set D new1 The samples that were incorrectly inferred during the previous fine-tuning process constitute a new dataset D. new2 For model new2 Further fine-tuning yields the model new3 Select data set D new The sample with the highest quality score constitutes D new3 The GRPO method is used to process the model. new3 Perform augmentation training to obtain the model new4 .

[0035] Thus, by effectively reducing the workload of constructing training datasets, a multimodal large language model for medical disciplines with high reasoning capabilities was obtained through a phased training process.

[0036] The specific method of invoking the multi-agent collaborative system involves calling a multimodal large language model of the medical discipline through prompts, assigning different roles to the corresponding agents, performing specific tasks, and facilitating interactions between them. The prompts consist of: identity definition, role description, key responsibilities, work objectives, problem description, relevant rules, relevant knowledge, and relevant tools.

[0037] The method includes the following steps: Step S10: In response to the medical subject knowledge question input by the user, a multi-agent collaborative team is formed to serve the medical subject knowledge question.

[0038] In this embodiment, medical subject knowledge issues refer to issues related to medical disciplines (such as Alzheimer's disease, tumors, etc.) (such as biological age prediction).

[0039] In some implementations, in response to a medical subject knowledge question input by a user, a comprehensive large language model (LLM) that has been continuously trained and fine-tuned across all medical disciplines and enhanced with multimodal capabilities is used to pre-judge the medical subject knowledge question to determine the most relevant medical subject area and whether the medical subject knowledge question requires an advanced knowledge discovery process (here, advanced knowledge discovery process refers to knowledge discovery that cannot be completed by relying solely on text, but requires the integration of multimodal data such as text and images to answer).

[0040] Specifically, this can be achieved by setting prompts to pre-judge questions related to medical subject knowledge. yorno Based on the language model of the whole university, and combined with a sample dataset consisting of user questions and pre-judgment results collected in advance, the pre-judgment conclusion is obtained by using a few-shot learning method. The judgment process can be expressed as follows: Field, senior yorno = LLM medical (query, prompt) yorno ).

[0041] Where Field represents the medical discipline field of medical knowledge questions, senior yorno This indicates the type of intent to perform advanced knowledge discovery processes for medical subject knowledge questions, according to LLM. medical This represents a pre-trained general language model, where `query` represents a medical subject knowledge question, and `prompt` represents a prompt. yorno The prompt content can include: identity definition (e.g., you are a domain and advanced knowledge discovery and judgment expert), work objectives (e.g., output judgment results and relevant evidence), problem description (clearly listing the user's problem or relevant explanation), and relevant samples (e.g., citing similar problems and judgment results).

[0042] If the preliminary judgment concludes that no advanced knowledge discovery process is required, then based on the determined medical discipline, the method of answering the question is determined according to the pre-constructed pre-judgment model (e.g., based on BERT), and knowledge base retrieval and / or large language model interaction are performed based on the method of answering the question to obtain the final answer corresponding to the medical discipline knowledge question.

[0043] If the preliminary judgment is that an advanced knowledge discovery process needs to be performed, then, in conjunction with the determined medical discipline, a multi-agent collaborative team is formed by planning agents to serve the knowledge problem of the medical discipline. The multi-agent collaborative team includes at least one expert agent and at least one data agent.

[0044] Specifically, this is achieved by setting prompts to identify the expert agents and data agents participating in the formation of a multi-agent collaborative team. expert The data is sent to the planning agent to obtain the relevant expert agent type and data agent type. The determination process can be expressed as the following formula: experts, datas = Agent plan (query, prompt) expert ) Among them, "experts" refers to one or more of the aforementioned expert agents (including neurology expert agents, psychiatry expert agents, gerontology expert agents, nutrition expert agents, oncology expert agents such as breast cancer expert agents, etc.), and "datas" refers to one or more of the aforementioned data agents (including MRI image agents, CT image agents, laboratory data agents, bioinformatics agents, EEG agents, MRI agents, speech agents, video agents, text agents, etc.). plan This represents a planning agent. The query represents a question about medical subject knowledge. expert This includes identity definition (e.g., you are a planning expert for advanced knowledge discovery in a medical discipline), role description (e.g., analyzing user input questions; determining the type of expert agent or data agent needed to answer user questions), key responsibilities (e.g., in-depth analysis of user input information; providing highly relevant judgment criteria; combining your own extensive experience and abilities when making judgments), work objectives (e.g., outputting the results and relevant evidence determined by the expert agent and data agent), problem description (clearly listing the user's question or relevant explanation here), relevant rules (e.g., determining results must be based on a careful analysis process; the analysis process must be combined with highly relevant evidence), relevant knowledge (e.g., listing similar questions, corresponding results, and relevant summary information), and relevant tools (e.g., the corresponding processing tools and corresponding processing results that can be invoked).

[0045] Therefore, based on the output results, a multi-agent collaborative team (Agent) composed of the aforementioned experts and data can be constructed. team (The number of members is N=N) exerts +N datas ).

[0046] Step S20: Through the multi-agent collaborative team, determine the multiple target information dimensions required to answer the medical knowledge question and the importance parameters corresponding to each target information dimension, and output the preliminary service results of the medical knowledge question based on the data information and importance parameters corresponding to the multiple target information dimensions.

[0047] In this embodiment, the target information dimension refers to the data type or information category required to answer the medical knowledge question, such as MRI images, laboratory indicators, medical history texts, bioinformatics data, etc. Importance parameters include importance order (priority) and / or importance weight (e.g., values ​​between 0 and 1), used to quantify the contribution of each target information dimension to the preliminary service result of outputting the medical knowledge question. The preliminary service result refers to the initial generated result for the medical knowledge question determined by the multi-agent collaborative team based on the data information and importance parameters corresponding to multiple target information dimensions, and has not yet undergone final evaluation.

[0048] In some implementations, step S20 involves the multi-agent collaborative team determining multiple target information dimensions required to answer the medical subject knowledge question and the importance parameters corresponding to each target information dimension, including: Step S21: Through the multi-agent collaborative team, determine the initial information dimensions and initial information data required to answer the medical subject knowledge questions.

[0049] In this embodiment, the initial information dimension refers to the data category or information type initially determined after preliminary analysis of medical questions, which is required to answer medical questions. The initial information data refers to the information data that already exists in the determined initial information dimension.

[0050] In some implementations, an information dimension question can be generated by a question-generating agent to determine the information dimension. The question generation process can be represented as: question infoget = Agent ques (query, prompt) ques-infoget Among them, question infoget For information dimension issues; Agent ques To generate an intelligent agent for the problem; prompt ques-infoget This is the prompt content used to generate information dimension questions (the setting method is similar to prompt). expert ).

[0051] By having each agent in the multi-agent collaborative team answer the information dimension question based on the medical subject knowledge question, an information dimension answer set is obtained, which includes the answers of each agent in the multi-agent collaborative team to the information dimension question.

[0052] Specifically, multi-agent collaborative team agents team The various intelligent agents (Agents) i (i<=N) Answer the information dimension questions in sequence. The answer requirements are: no other information needed, other information needed - specific information dimension, multi-agent collaborative team agent. team The various intelligent agents (Agents) i The process of answering a question can be represented as: Answer agent-i-infoget =Agent i (query, question) infoget prompt agent-i-infoget ). Among them, Answer agent-i-infoget multi-agent collaborative team agent team The i-th agent's answer to the information dimension question, prompt agent-i-infoget The prompt content used to enable the i-th agent to answer the information dimension question (the setting method is similar to prompt). expert ).

[0053] The process summary agent summarizes the set of answers for the information dimensions to determine the initial information dimensions and initial information data required to answer the questions about medical knowledge.

[0054] Specifically, a prompt-like approach can be taken for the process summary agent. expert The method is to set the prompt content used to summarize the initial information dimensions and initial information data. sum-infoget The initial information dimensions are required to correspond to one or more of the aforementioned data agent types (MRI images, CT images, laboratory data, bioinformatics, EEG, MRI, speech, video, text, and other preset data categories). The summary process can be expressed as: Data sum-infoget Answer sum-infoget =Agent sum (query, Answer) agent-i-infoget prompt sum-infoget ). Among them, Data sum-infoget Represents initial information data; Answer sum-infoget Indicates the initial information dimension; Agent sum The process is summarized by the intelligent agent.

[0055] Step S22: Based on the initial information dimension and the initial information data, the question-generating agent generates supplementary prompt questions for obtaining supplementary data.

[0056] Based on Datasum-infoget Answer sum-infoget The problem-generating agent produces supplementary prompts to interact with the user and obtain additional data. Specifically, this can be done through a prompt-like mechanism. expert The method is used to set the prompt content for generating supplementary questions. ques-new-infoget The process of generating the supplementary prompt question can be represented as: Question new -infoget = Agent ques (query, Data) sum-infoget Answer sum-infoget prompt ques-new-infoget ).

[0057] Step S23: Obtain the user's supplementary answer to the supplementary prompt question, and summarize the initial information dimensions, the initial information data and the supplementary answer through the process summarization agent to obtain multiple target information dimensions and corresponding data information required to answer the medical knowledge question.

[0058] Considering that some dimensions might be empty or ignored due to reasons such as the user's inability to provide them, a supplementary hint question is generated based on the problem, which is proposed by the agent. new-infoget The user agent interacts with the user to obtain the user's supplementary answers to the supplementary prompts. user-new-infoget The process summary agent then summarizes the initial information dimensions, initial information data, and supplementary answers to obtain multiple target information dimensions and corresponding data information required to answer questions about medical knowledge. The process can be represented as: Data sum-infoget-final Answer sum-infoget-final =Agent sum (query, Data) sum-infoget Answer sum-infoget Answer user-new-infoget prompt sum -infoget-final ). Among them, Answer sum-infoget-final For target information dimensions; Data sum-infoget-final The data information corresponding to the target information dimension; prompt sum-infoget-final To pass through something like a prompt expert The method is set to generate prompts for obtaining the target information dimension and corresponding data information.

[0059] Step S24: The multi-agent collaborative team sorts the importance of the multiple target information dimensions and corresponding data information and assigns weights to obtain importance parameters.

[0060] In some implementations, a problem-generating agent can generate a ranking problem to determine the importance order of the multiple target information dimensions and their corresponding data information.

[0061] Specifically, it can be done through something like a prompt. expert The method is used to set the prompt content for generating sorting problems. ques-infoget-rank The process of generating the sorting problem can be represented as: question infoget-rank = Agent ques (query, Data) sum-infoget-final Answer sum-infoget-final prompt ques-infoget-rank ).

[0062] The importance ranking answer set is obtained by having each agent in the multi-agent collaborative team answer the ranking question based on the medical subject knowledge question.

[0063] Specifically, the ranking question is generated by the agent based on the question. infoget-rank Multi-agent collaborative team team The various intelligent agents (Agents) i Answer in order (specifically through a prompt similar to...) expert The method for setting the prompt content for answering sorting questions agent-i-infoget-rank The requirements for the answers are as follows: Rank the target information dimensions according to their importance in answering medical subject knowledge questions, from highest to lowest, and score their importance on a 10-point scale. (Multi-agent collaborative team agent) team The various intelligent agents (Agents) i The process of answering a sorting question can be represented as: Answer agent-i-infoget-rank = Agent i (query, question) infoget-rank Data sum -infoget-final Answer sum-infoget-final prompt agent-i-infoget-rank ).

[0064] The process summary agent summarizes the importance ranking answer set to obtain the importance order of the multiple target information dimensions and corresponding data information, as well as the importance score.

[0065] Specifically, it can be done through something like a prompt. expertThe method involves setting the order of importance of multiple target information dimensions and their corresponding data information, as well as prompts indicating the importance scores. sum-infoget-rank The result requires an answer. sum-infoget-final The process of ranking multiple target information dimensions according to the average importance scores given by all agents in the multi-agent collaborative team for each target information dimension can be represented as: Data sum-infoget-rank Answer sum-infoget-rank = Agent sum (query, Answer) agent-i-infoget-rank prompt sum -infoget-rank ). Among them, Data sum-infoget-rank This represents the data information corresponding to the target information dimension; Answer sum -infoget-rank This represents the order of importance of the target information dimensions and their respective importance scores. info-j (The importance score of the target information dimension is the average of the above importance scores), j represents the ordinal number of the target information dimension.

[0066] The remaining information dimensions are defined as those other than the first-ranked information dimension in importance. The importance weights are then calculated based on the importance order and corresponding importance scores of these remaining information dimensions.

[0067] Specifically, based on the obtained Data sum-infoget-rank Answer sum-infoget-rank It can determine the importance order and importance score of the target information dimension that ranks first in importance, as well as the other remaining information dimensions (second in importance and so on).

[0068] Based on the data types and importance scores (info) of each remaining information dimension obtained above j score info-j The relative weight of the j-th target information dimension is calculated: Weight info-j = score info-j / ∑(score) info-j The relative weights of the remaining information dimensions together constitute the importance weight. info .

[0069] In some implementations, step S20 outputs preliminary service results for the medical subject knowledge question based on the data information and importance parameters corresponding to the multiple target information dimensions, including: Step S25: Determine the basic results of the medical subject knowledge problem based on the data information corresponding to the target information dimension ranked first in importance.

[0070] Based on Data sum-infoget-rank Data information ranked first in importance first and its corresponding user-provided data content DataValue first Through data first The data agent, corresponding to the data type, determines the basic result for responding to the user's question. For example, for numerical calculation questions such as biological age estimation, the basic result should be an age.

[0071] As a feasible and practical way to determine the underlying outcomes, a problem-generating agent generates a fundamental question for determining the underlying outcomes, which can be specifically achieved through a prompt-like mechanism. expert The method for setting the prompt content that generates basic questions ques-basic-answer The process of generating the question can be represented as: Question ques-basic-answer = Agent ques (query, Data) sum-infoget-rank Answer sum-infoget-rank prompt ques-basic-answer ).

[0072] Based on the fundamental question raised by the intelligent agent. ques-basic-answer The corresponding data agent provides the response (specifically through a prompt-like mechanism). expert The method for setting the prompt content that generates the basic results. basic -answer The answer should be specified according to the question type (e.g., for numerical calculation questions such as brain age estimation, the basic result should be in numerical format; for non-numerical calculation questions, the answer should be in the required format). The basic result is then obtained. basic-answer The process (for neuroimaging data, before the problem is submitted to the data agent, the data undergoes the neuroimaging data processing steps described above; others are similar) can be represented as: Answer basic-answer =Agent first (query, Question) ques-basic-answer Data sum-infoget-rank Answer sum-infoget-rank prompt basic-answer Among them, Agent first It refers to the data agent corresponding to the target information dimension that ranks first in importance.

[0073] Step S26: Determine the correction result of the medical subject knowledge problem based on the data information corresponding to the remaining information dimension and the importance weight.

[0074] If Answer sum-infoget-rank If the target information dimension determined in the code is greater than or equal to 2, then a data agent corresponding to the data type of the remaining information dimension needs to correct the result for the remaining information dimension (answer). added-answer The calculation of (e.g., for numerical calculation problems such as biological age estimation, the corrected result should be an incremental age number based on the above basic result), specifically includes the answer. 2th-answer , answer 3th-answer Equal correction results.

[0075] Taking the calculation of the sub-correction result of the target information dimension, which ranks second in importance among the remaining information dimensions, as an example, the problem-generating agent generates a rule question to determine the rules for the correction result, specifically through a prompt-like mechanism. expert The prompt content for setting the rules that generate correction results. ques-2th-answer The process of generating the correction problem can be represented as: Question ques-2th-answer = Agent ques (query, Answer) basic-answer Data sum-infoget-rank Answer sum-infoget-rank prompt ques-2th-answer ).

[0076] Based on the correction question ques-2th-answer The answer is provided through a corresponding data intelligence agent (specifically through a prompt-like mechanism). expert The method for setting the prompt content for generating correction results rules. 2th-answer-rule The requirements for the answer are based on the type of medical subject knowledge question (for example, for numerical calculation questions such as biological age estimation, the required calculation rules for the corrected result are: the result is an incremental number based on the base result, the input and rule body of the rule are determined based on the data provided by the user, and the rule is expressed as a formula, etc.). The process of obtaining the second-degree corrected result rule can be represented as: Answer 2th-answer-rule = Agent 2th (query, Answer) basic-answer Question ques -2th-answer Data sum-infoget-rank Answer sum-infoget-rank prompt2th-answer-rule ).

[0077] According to Answer 2th-answer-rule Defined rules 2th and Data sum-infoget-rank Answer sum -infoget-rank Calculate the answer 2th-answer answer 2th-answer = rule 2th (Data) sum-infoget-rank (For non-numerical computation problems, the corresponding data agent is required to follow the rule.) 2th (Obtain non-numerical correction results).

[0078] Finally, summarize the answers 2th-answer , answer 3th-answer The answer is calculated from the sub-correction results. added -answer , answer added-answer = ∑Weight info-j *answer j-th-answer (A - symbol has been added here to distinguish j and th.) (For non-numerical computation problems, the reasoning agent is required to summarize and form a unified format of correction results based on the sub-correction results of each remaining information dimension and the corresponding weight information.)

[0079] Step S27: Determine the overall answer result based on the basic results and the correction results.

[0080] Regarding the aforementioned numerical calculation questions, the overall answer is as follows: all-answer =answer basic-answer +answer added-answer .

[0081] It should be noted that for questions that are not related to numerical calculations, the answer will be used as the basis for the question. basic-answer , answer added -answer weight info The reasoning agent obtains the overall answer to the medical subject knowledge question.

[0082] Step S28: The overall response result is verified by each agent in the multi-agent collaborative team, and the overall response result is adjusted according to the verification result to obtain the preliminary service result.

[0083] To ensure the reasonableness of the preliminary service results, the overall response obtained by combining the basic results and the correction results is evaluated by each agent in the multi-agent collaborative team in various dimensions. If there are different opinions, the relative weights between the target information dimensions are adjusted. Then, the evaluation can be carried out again until the preset evaluation termination conditions are met, and the preliminary service results are obtained.

[0084] In some implementations, a question-generating agent generates a result-based argumentation question to justify the overall answer. result-argu-k The results demonstrate the problem Question result-argu-k Based on the aforementioned medical subject knowledge query and the aforementioned basic result answer basic-answer The correction result is answer. added-answer The sub-correction results corresponding to each of the remaining information dimensions. j-th-answer The overall answer result all-answer The order of importance (Data) sum-infoget-rank and Answer sum-infoget-rank The importance weight info and the answer to the previous round of arguments sum- result-argu-k-1 generate.

[0085] Specifically, the question of result argumentation result-argu-k The generation process can be represented as: Question result-argu-k = Agent ques (query, answer) all-answer , answer basic-answer , answer added-answer Data sum-infoget-rank Answer sum-infoget-rank Weight info , answer j-th-answer , answer sum-result-argu-k-1 prompt ques -result-argu-k ).

[0086] The result argumentation answer set is obtained by having each agent in the multi-agent collaborative team answer the result argumentation question.

[0087] Specifically, through something like a prompt expert The prompt content for answering the argument question is set in a certain way. agent-i-result-argu-kThe answer requirements are: no other adjustments are needed, only adjustments to the information dimension weights – specific information dimensions, and new weights. The process of each agent in a multi-agent collaborative team answering questions can be represented as: Answer agent-i-result-argu-k = Agent i (query, question) result-argu-k , answer all-answer , answer basic -answer , answer added-answer Data sum-infoget-rank Answer sum-infoget-rank Weight info , answer j -th-answer , answer sum-result-argu-k-1 prompt agent-i-result-argu-k ).

[0088] The process summary agent summarizes the result argumentation answer set, and when more than a preset proportion of agents in the multi-agent collaborative team determine to adjust the importance weight, the adjusted importance weight, the adjusted correction result, and the adjusted overall answer result are generated.

[0089] Specifically, the result-argument answer set includes the result-argument answers from all agents in the multi-agent collaborative team. agent-i-result-argu-k The process involves a process-summarizing agent summarizing the set of responses to the results. When more than a preset proportion of agents in the multi-agent collaborative team agree to adjust the importance weights, the adjusted importance weights for this round (round k) are generated (the adjustment rule is the average of the weights provided by all agents who agree to adjust the importance weights). The preset proportion can be 2 / 3, representing a professional-consensus balanced agent communication and coordination mechanism that can obtain accurate results with maximum efficiency.

[0090] via something like prompt expert The method of setting up a prompt to summarize the set of results-based arguments and responses sum-result-argu-k The process of generating the adjusted importance weights can be represented as: Answer sum-result-argu-k =Agent sum (query, question) result-argu-k , answer all-answer , answer basic-answer , answer added-answer Data sum-infoget-rank Answer sum-infoget-rank Weightinfo , answer j-th-answer , answer sum-result-argu-k-1 Answer agent-i-result-argu-k prompt sum-result-argu-k ).

[0091] The step of generating a result argumentation question through a question-generating agent to evaluate the overall answer result is repeated until a preset argumentation termination condition is met. The overall answer result is then adjusted according to the adjusted importance weights to obtain a preliminary service result.

[0092] Specifically, the pre-set termination condition for the argumentation process could be that the importance weight no longer updates or the maximum number of rounds is reached, thus terminating the argumentation process and obtaining the final adjusted importance weight. info-final The final adjusted correction result (answer) added-answer-final and preliminary service results answer all-answer-final .

[0093] It should be noted that for non-numerical calculation problems, the reasoning agent obtains the overall answer to the medical knowledge question. The effect of adjusting the weights here is to influence the degree of reference to information from each dimension during the final result generation process. It is worth noting that the result argumentation logic here implies several key points: firstly, it uses the basic results obtained by the data agents for each information dimension as a foundation; secondly, it combines considerations of the relative importance of the target information dimensions by the data agents and expert agents for each target information dimension, maximizing the combination of professionalism and team consensus (a balance between professionalism and consensus), ensuring the effectiveness of the argumentation process (reducing unnecessary discussion rounds); thirdly, by limiting the depth of each agent's access to historical argumentation information (one layer), excessive interference is reduced, enhancing the effectiveness of the discussion and reducing the cognitive burden (unnecessary discussion rounds) during the result argumentation process.

[0094] Furthermore, in this embodiment, each agent in the multi-agent collaborative team, during the inference process, retrieves historical cases related to the medical subject knowledge question, the multiple target information dimensions, and / or the current inference stage as contextual knowledge (corresponding to relevant knowledge in the prompt content) by searching the positive and negative example reference library. The positive and negative example reference library is a structured, dynamically updated medical knowledge case library. It stores medical subject questions processed by the system in the past and their complete service records, clearly marking the correctness (positive examples) or error (negative examples) of the results, as well as deeper knowledge summaries.

[0095] It should be noted that the positive and negative example reference library in this embodiment can provide a shared knowledge retrieval service for the entire multi-agent collaborative system. That is, each agent in the multi-agent collaborative system can retrieve historical cases related to the medical subject knowledge problem, the multiple target information dimensions, and / or the current reasoning stage by searching the positive and negative example reference library during its respective reasoning process. These cases can be used as contextual knowledge (corresponding to relevant knowledge in the prompt content) and incorporated into its decision-making process, thereby improving the accuracy and reliability of reasoning.

[0096] Step S30: The result evaluation agent evaluates the correctness of the preliminary service result, and generates and outputs the final service result of the medical subject knowledge question based on the evaluation result.

[0097] Regarding the preliminary service results obtained all-answer-final The result evaluation agent performs a correctness assessment before providing feedback on the results. Its main purpose is to further verify the correctness of the results, identify any inaccuracies, and, based on this, format and generate the final, evaluated answer. all-answer-final-eval Then, the user's intelligent agent will reason about the key processes of medical knowledge questions, such as answering. all-answer-final-eval The basic results, the final adjusted importance weights, and the final adjusted correction results are all fed back to the user as the final service result to ensure the interpretability of the results.

[0098] If the preliminary service results fail the correctness evaluation of the evaluation agent, the feedback that the user cannot answer will be provided. This process is specifically implemented through a prompt-like mechanism. expert The method of setting up prompts to generate results evaluation and feedback eval , prompt user (Based on answer) all-answer-final-eval The process involves (basic results, final adjusted importance weights, final adjusted correction results, etc.) followed by calling the corresponding intelligent agent to complete the task. This embodiment will not elaborate further.

[0099] It is understandable that the result evaluation agent, as one of the agents in the multi-agent collaborative system, can also evaluate the correctness of the preliminary service results by searching the positive and negative example reference library, and generate and output the final service results of the medical subject knowledge problem based on the evaluation results, thereby enhancing the reasoning and evaluation capabilities of the evaluation agent.

[0100] In the data processing method for medical subject knowledge services proposed in this application, in the first aspect, by forming a multi-agent collaborative team for specific medical subject knowledge problems, it is possible to solve the problems of large resource consumption and insufficient accuracy of general large language models with large parameter scale, and small parameter scale large language models with low resource consumption but poor performance and insufficient generalization ability. This improves the overall service effect when multiple sources of information such as medical images, structured data and natural language need to be processed to answer questions, and shows higher accuracy and reasoning efficiency.

[0101] Secondly, in the multi-agent collaborative system, by decomposing the process links of complex medical knowledge services, multi-dimensional agent roles including user agent, planning agent, expert agent, data agent, support agent, and AI agent are designed, making full use of the professional working capabilities and collaborative advantages of the language model.

[0102] Thirdly, by determining multiple target information dimensions and the importance parameters corresponding to each target information dimension and outputting preliminary service results, the problem of multiple information dimensions and difficulty in effective integration and prioritization in complex medical disciplines is solved. Information fusion and weighting processing, as well as phased data processing, are realized, thereby improving the scientificity, reliability and interpretability of the preliminary service results.

[0103] Fourthly, by enabling each agent to retrieve a reference library of positive and negative examples during the inference process and using the retrieved historical cases as contextual knowledge to assist in decision-making, this process does not involve updating or fine-tuning the underlying model parameters of the agent. It achieves a reinforcement learning-like effect in an offline learning state, thereby completely avoiding the huge computing power overhead, training instability and catastrophic forgetting problems caused by traditional reinforcement learning or continuous fine-tuning, and achieving low-cost, lightweight, stable and reliable continuous performance improvement.

[0104] Fifthly, by using an independent result evaluation agent to assess the correctness of the preliminary results and generating the final service results based on the evaluation results, the credibility and robustness of the final output service results are enhanced.

[0105] Sixthly, to ensure the interpretability of the results of the medical knowledge service process, a neural symbolic reasoning approach was adopted. This approach involves acquiring reasoning rules through specialized intelligent agents, ranking the importance of target information dimensions through multi-agent collaboration, and determining the importance weights between information dimensions. Finally, based on the basic results, a correction result is obtained through weighted summation (for numerical calculation problems) or weighted synthesis (for non-numerical calculation problems), thus obtaining the preliminary service result and ensuring the interpretability of the knowledge service result.

[0106] Seventhly, in order to improve the multimodal reasoning ability of the multimodal large language model in the medical discipline, methods such as model building, model reorganization and model optimization were adopted to ensure that a reasoning model with improved multimodal reasoning ability was obtained on the basis of minimal workload in building training datasets.

[0107] In one embodiment, after step S30, the method further includes: Step S40: The medical subject knowledge questions, the multiple target information dimensions and corresponding data information, reasoning process, preliminary service results, final service results and evaluation results are recorded as service records, and the service records are stored in the positive and negative example reference library.

[0108] Specifically, for each user query and its response, if the evaluation agent evaluates the answer... all -answer-final If the evaluation result is correct, then the information related to the user query and the response result will be processed according to the user query and the final answer result. all-answer-final-eval Preliminary service results all-answer-final Overall answer result all-answer Basic Result (answer) basic-answer Correction result answer added-answer Data information corresponding to the target information dimension sum-infoget-rank Order of importance Answer sum-infoget-rank Importance weight info The sub-correction results for each remaining information dimension. j-th-answer Final adjusted importance weights info-final The final adjusted correction result (answer) added-answer-final Fields such as the correctness evaluation result (true or false) are stored as service records in the positive example reference library (the result correctness flag is true).

[0109] Conversely, if the agent is evaluated on the answer... all-answer-final If the evaluation result is incorrect, the information related to the user query and response result will be saved to the negative example reference library according to the data information of the user query and other fields (the result correctness flag is false).

[0110] The specific implementation of step S40 is completed by the process knowledge management intelligent agent, specifically through a prompt-like mechanism. expert The method for setting the prompt content for positive and negative example reference libraries. positive , prompt negative The process is then completed by calling the intelligent agent, which will not be described in detail in this embodiment.

[0111] Step S50: The process knowledge management intelligent agent summarizes the service records to obtain summary information for the medical discipline knowledge problem, and stores the summary information in association with the service records in the positive and negative example reference library. The summary information includes result summary, evaluation reasons and target knowledge information.

[0112] As a way to obtain a result summary answer summary A feasible implementation method is to take the user query and relevant information stored in the reference library as input, and require the process knowledge management agent to generate the corresponding record process and result summary.

[0113] As a reason for obtaining evaluation answers caus A feasible implementation method is to take the user query and relevant information already stored in the reference database as input—including the answer. summary The process knowledge management agent is required to generate the corresponding record and explain whether the result is correct or incorrect.

[0114] As a way to obtain target knowledge information answer know A feasible implementation method is to take the user query and relevant information already stored in the reference database as input—including the answer. caus The process knowledge management agent is required to generate corresponding records to determine whether the reasoning is correct or incorrect.

[0115] The specific implementation of step S50 is completed by the process knowledge management intelligent agent, specifically through a prompt-like mechanism. expert The method for setting the prompt content for generating and saving positive and negative example reference library summaries summary , prompt caus , prompt know The process is then completed by calling the intelligent agent, which will not be described in detail in this embodiment.

[0116] In some embodiments, the method further includes: Based on the content of knowledge fields in the positive and negative example reference library, the target knowledge information is evaluated by a process knowledge management intelligent agent.

[0117] Specifically, in answer know Using the knowledge fields of other records in the database as input, the process knowledge management agent is required to generate a corresponding answer. know Evaluation information answer know-eval Specifically, this includes a first preset level (high), a second preset level (medium), and a third preset level (low). For each corresponding answer... knowThe generated evaluation information is saved to the corresponding field of the corresponding service record in the database.

[0118] Knowledge evaluation is accomplished by a process knowledge management intelligent agent, specifically through a mechanism similar to a prompt. expert The method for setting the prompt content for generating and saving positive and negative example reference libraries for evaluation know-eval The process is then completed by calling the intelligent agent, which will not be described in detail in this embodiment.

[0119] When the knowledge evaluation result reaches a first preset level, the system queries the positive and negative example reference library to see if there are any historical case records in the reference library whose similarity to the target knowledge information is greater than a preset similarity threshold. If such a record exists, the target knowledge information is used to overwrite the historical knowledge information in the historical case record. If the record does not exist, it remains unchanged.

[0120] When the knowledge evaluation result reaches the second preset level, the system queries the positive and negative example reference library to see if there are any historical case records in the reference library whose similarity to the target knowledge information is greater than a preset similarity threshold. If so, the historical knowledge information of the historical case records is optimized based on the target knowledge information. If not, the information remains unchanged.

[0121] When the knowledge evaluation result reaches the third preset level, the target knowledge information and associated service records are deleted from the positive and negative example reference library. If they do not exist, they remain unchanged.

[0122] Knowledge updates are accomplished by a process knowledge management intelligent agent, specifically through a mechanism similar to a prompt. expert The method for setting the prompt content for updating the positive and negative example reference libraries. update The process is then completed by calling the intelligent agent, which will not be described in detail in this embodiment.

[0123] In the data processing method for medical subject knowledge services proposed in this application embodiment, in the first aspect, each service process is transformed into a structured service record and stored in a positive and negative example reference library. At the same time, the process knowledge management intelligent agent extracts summary information such as content summary, evaluation reasons and target knowledge information, thereby realizing the systematic accumulation and archiving of experience knowledge and providing traceable and reusable learning materials for each intelligent agent.

[0124] Secondly, the process knowledge management intelligent agent performs hierarchical evaluation based on knowledge content and intelligently performs knowledge coverage, optimization or deletion operations according to different levels, thereby ensuring the quality, timeliness and conciseness of knowledge in the positive and negative example reference library.

[0125] Thirdly, through process knowledge management, the system can continuously absorb successful experiences and avoid historical errors in offline learning states. This not only achieves low-cost and lightweight performance evolution, avoiding the computational burden and catastrophic forgetting risks brought about by traditional fine-tuning, but also significantly improves the accuracy and reliability of subsequent intelligent agents in reasoning and decision-making through continuous refinement and updating of the knowledge base, ultimately forming a stable, interpretable, and self-improving medical discipline knowledge service ecosystem.

[0126] In one embodiment, the agent performing the reasoning process generates a function call request based on the question content and maps the function call request to a tool call request by calling a pre-built basic model, so as to call the corresponding processing tool from a preset tool library and obtain the processing result to be incorporated into the context knowledge. Each processing tool in the preset tool library is labeled with a classification label and information identifier, and is organized according to a graph pattern based on dependencies. The classification label includes topic type, level, modality and granularity, and the information identifier includes tool name, tool function, required input parameters, data type, return result, related attention and mapping rules.

[0127] Specifically, in order to ensure the high reasoning ability of the medical multimodal large language model, it is necessary to guarantee its basic capabilities through training, while also ensuring its continuous growth. To this end, this application proposes a tool invocation mechanism for the medical multimodal large language model: (1) Tool Construction. The processing tools in this embodiment include literature database access (such as PubMed) and brain imaging database access (such as ADNI) to improve knowledge discovery capabilities (considering issues such as data transmission efficiency, the tool calls here mainly refer to tools that only transmit a small amount of data). In order to effectively manage tools and accurately call tools, the processing tools are classified and tagged, and organized according to: topic (such as AD, MCI, etc.), type (such as clinical trials, randomized controlled trials, etc.), level (such as high recommendation, low recommendation, etc.), modality (such as MRI, CT, etc.), and granularity (such as document, entity, etc.). Each processing tool is identified and provided with an interface through information such as tool name, tool function, required input parameters, data type, return result, related notes, and mapping rules (conversion rules from function calls to tool APIs, etc.).

[0128] (2) Tool Organization. Considering the inter-calling relationships among numerous processing tools in the medical field, the processing tools are organized according to a graph or similar pattern to ensure successful invocation. Nodes represent processing tools; the relationships between nodes represent the dependencies between processing tools (including four types: direct dependency - indicating that one tool is completely dependent on another tool; optional dependency - indicating that one tool can benefit from the execution of another tool but is not required to; parameter direct dependency - the parameter needs to be obtained from another tool; and parameter optional dependency - the parameter can be more clearly obtained from the execution of another tool but is not required to).

[0129] (3) Tool Invocation. To ensure efficient and accurate tool invocation, the process from problem content to tool invocation is decomposed into two stages: function invocation and tool invocation. The former maps the problem content to the function invocation, while the latter maps the function invocation to the tool invocation and completes the tool invocation process (in JSON format, where the JSON request contains the name of the tool to be invoked and its corresponding parameter information, based on which the corresponding tool is invoked). Function invocation is initiated by the agent that performs the reasoning process in the aforementioned medical subject knowledge problem service process, while tool invocation is initiated by the pre-built basic model (an SLM that has been fine-tuned by instructions). Combining the aforementioned tool organization graph and tags, for tool invocations with dependencies, the tools are executed sequentially according to the constructed tool graph to obtain the final processing result.

[0130] In the process of solving each user query-related problem, such as in the aforementioned stages of determining information dimensions and verifying results, each agent can obtain the processing results by calling relevant tools and incorporate them into the contextual knowledge (corresponding to the relevant tools in the prompt content) to make more accurate reasoning decisions.

[0131] (4) Model training. The above-mentioned function calls and tool calls have different base models (the base model of the former is the aforementioned AD / MCI multimodal large language model, and the base model of the latter is such as the finely tuned qwen2.5-1.5b, etc.). The training process of the latter is as follows: after constructing the instruction set (including input, reasoning process including tool calls, answers, etc., the construction process includes task construction - generating user questions, reasoning process construction - thinking process, answer construction - results, result verification - including format and content verification, etc.), it is obtained by fine-tuning the original model (such as qwen2.5-1.5b).

[0132] The data processing method for medical subject knowledge services proposed in this application improves the basic capabilities of the agent executing the reasoning process during medical subject knowledge problem services by constructing a unified set of basic tools and a tool invocation model. Simultaneously, it ensures the continuous improvement of the entire system's capabilities based on continuous updates to the data source. Furthermore, specific interface design and model training avoid the inefficiencies of general protocols such as MCP in calling tools for Alzheimer's disease, oncology, and other medical subjects.

[0133] In one embodiment, an electronic device is also provided, including one or more processors; and a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the steps in the above method embodiments.

[0134] In one embodiment, such as Figure 2 The diagram illustrates the structure of an electronic device used to implement an embodiment of this application. The electronic device includes a central processing unit (CPU) 101, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 102 or a program loaded from a storage portion 108 into a random access memory (RAM) 103. The RAM 103 also stores various programs and data required for the operation of the electronic device 100. The CPU 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0135] The following components are connected to I / O interface 105: an input section 106 including a keyboard, mouse, etc.; an output section 107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 108 including a hard disk, etc.; and a communication section 109 including a network interface card such as a LAN card, modem, etc. The communication section 109 performs communication processing via a network such as the Internet. A drive 110 is also connected to I / O interface 105 as needed. A removable medium 111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 110 as needed so that computer programs read from it can be installed into storage section 108 as needed.

[0136] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer-readable medium carrying instructions that, in such embodiments, can be downloaded and installed from a network via communication section 109, and / or installed from removable medium 111. When the instructions are executed by central processing unit (CPU) 101, the various method steps described in this application are performed.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0138] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, any of the embodiments or implementations claimed above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A data processing method for medical discipline knowledge services, characterized in that, The method includes: In response to a user's input of a medical subject knowledge question, a multi-agent collaborative team is formed to serve the medical subject knowledge question. The multi-agent collaborative team determines the initial information dimensions and initial information data required to answer the medical knowledge question. The question-generating agent generates supplementary prompt questions based on these initial information dimensions and data to obtain additional information. The user provides supplementary answers to these prompts. A process-summarizing agent summarizes the initial information dimensions, initial information data, and supplementary answers to obtain multiple target information dimensions and corresponding data required to answer the medical knowledge question. The multi-agent collaborative team then prioritizes and assigns weights to these target information dimensions and corresponding data to obtain importance parameters. Based on the data and importance parameters, a preliminary service result for the medical knowledge question is output. During the inference process, each agent in the multi-agent collaborative team retrieves historical cases related to the medical knowledge question, the multiple target information dimensions, and / or the current inference stage from a positive and negative example reference library as contextual knowledge. The result evaluation agent evaluates the correctness of the preliminary service results and generates and outputs the final service results for the medical subject knowledge question based on the evaluation results.

2. The data processing method for medical subject knowledge services according to claim 1, characterized in that, The process of determining the initial information dimensions and initial information data required to answer the medical subject knowledge questions through the multi-agent collaborative team includes: The intelligent agent generates information dimension questions to determine the information dimensions. By having each agent in the multi-agent collaborative team answer the information dimension question based on the medical subject knowledge question, an information dimension answer set is obtained; The process summary agent summarizes the set of answers for the information dimensions to determine the initial information dimensions and initial information data required to answer the questions about medical knowledge.

3. The data processing method for medical subject knowledge services according to claim 1, characterized in that, The importance parameters include importance order and / or importance weight. The importance parameters are obtained by the multi-agent collaborative team ranking and weighting the multiple target information dimensions and corresponding data information according to their importance, including: The problem-generating agent generates a sorting problem to determine the importance order of the multiple target information dimensions and corresponding data information. The importance ranking answer set is obtained by having each agent in the multi-agent collaborative team answer the ranking question based on the medical subject knowledge question. The process summary agent summarizes the importance ranking answer set to obtain the importance order of the multiple target information dimensions and corresponding data information, as well as the importance score; The remaining information dimensions are defined as those other than the first-ranked information dimension in importance. The importance weights are then calculated based on the importance order and corresponding importance scores of these remaining information dimensions.

4. The data processing method for medical subject knowledge services according to claim 3, characterized in that, Based on the data information and importance parameters corresponding to the multiple target information dimensions, the system outputs preliminary service results for the medical subject knowledge question, including: Based on the data information corresponding to the target information dimension ranked first in importance, the basic results of the medical subject knowledge question are determined. Based on the data information corresponding to the remaining information dimensions and the importance weights, the correction result for the medical subject knowledge question is determined; Based on the basic results and the correction results, the overall answer result is determined; The preliminary service result is obtained by having each agent in the multi-agent collaborative team verify the overall response result and adjust the overall response result based on the verification result.

5. The data processing method for medical subject knowledge services according to claim 4, characterized in that, The process of having each agent in the multi-agent collaborative team validate the overall response result and adjust the overall response result based on the validation results to obtain the preliminary service result includes: The intelligent agent generates a result argumentation question to validate the overall answer result. The result argumentation question is generated based on the medical subject knowledge question, the basic result, the correction result, the sub-correction result corresponding to each of the remaining information dimensions, the overall answer result, the importance order, the importance weight, and the previous round of argumentation results. The result argumentation answer set is obtained by having each agent in the multi-agent collaborative team answer the result argumentation question; The process summary agent summarizes the result argumentation answer set, and when more than a preset proportion of agents in the multi-agent collaboration team determine to adjust the importance weight, the adjusted importance weight, the adjusted correction result, and the adjusted overall answer result are generated. Repeat the step of generating a result argumentation question through a question-generating agent to argue the overall answer result until the preset argumentation termination condition is met; Based on the adjusted importance weights, the overall response results are adjusted to obtain preliminary service results.

6. The data processing method for medical subject knowledge services according to claim 1, characterized in that, After the result evaluation agent evaluates the correctness of the preliminary service result and generates and outputs the final service result of the medical subject knowledge question based on the evaluation result, the process includes: The medical subject knowledge questions, the multiple target information dimensions and corresponding data information, reasoning process, preliminary service results, final service results and evaluation results are used as service records, and the service records are stored in a positive and negative example reference library; The process knowledge management intelligent agent summarizes the service records to obtain summary information on the knowledge questions of the medical discipline, and stores the summary information in association with the service records in a positive and negative example reference library. The summary information includes result summary, evaluation reasons and target knowledge information.

7. The data processing method for medical subject knowledge services according to claim 6, characterized in that, The method further includes: Based on the content of knowledge fields in the positive and negative example reference library, the target knowledge information is evaluated by a process knowledge management intelligent agent; When the result of the knowledge evaluation reaches the first preset level, the system queries whether there are historical case records in the positive and negative example reference library that have a similarity greater than a preset similarity threshold with the target knowledge information. If there are such records, the target knowledge information is used to cover the historical knowledge information of the historical case records. When the result of the knowledge evaluation reaches the second preset level, the system queries whether there are historical case records in the positive and negative example reference library that have a similarity greater than a preset similarity threshold with the target knowledge information. If there are such records, the system optimizes the historical knowledge information of the historical case records based on the target knowledge information. When the result of the knowledge evaluation reaches the third preset level, the target knowledge information and associated service records are deleted from the positive and negative example reference library.

8. The data processing method for medical subject knowledge services according to claim 1, characterized in that, The agent applied to the method performs the reasoning process by invoking a pre-built, small-parameter-scale multimodal large language model of medical disciplines. The agent performing the reasoning process generates a function call request based on the question content and maps the function call request to a tool call request by invoking a pre-built basic model. This allows the agent to call the corresponding processing tool from a preset tool library and obtain the processing result to be incorporated into the context knowledge. Each processing tool in the preset tool library is labeled with a classification tag and information identifier and is organized according to a graph pattern based on dependencies. The classification tags include topic type, level, modality, and granularity. The information identifiers include tool name, tool function, required input parameters, data type, return result, related attention, and mapping rules.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the data processing method for medical subject knowledge services as described in any one of claims 1 to 8.

Images