Medical data analysis method driven by large-model multi-agent framework

Abstract

The invention discloses a large-model multi-agent framework-driven medical data analysis method, which comprises the following steps of: S1, based on collected electronic medical record data of a patient, constructing a medical knowledge graph in an auxiliary manner by virtue of a large language model, and storing the medical knowledge graph in a graph database; s2, constructing a graph retrieval enhancement generation agent, combining the graph retrieval enhancement generation agent with a medical knowledge graph, and performing preliminary analysis on the input data; s3, the assessment agent assesses the data complexity according to the original medical data and the analysis result in S2, and the recruitment agent creates a corresponding number of medical expert agents according to the assessment result; and S4, designing a multi-agent cooperative analysis algorithm, calculating a weighted similarity analysis matrix, judging the consistency of analysis results of the medical expert agent group, and finally obtaining a reliable analysis conclusion. According to the method, the illusion problem of a large model is effectively relieved, and the quality, accuracy and credibility of medical data analysis are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of AI technology application and medical data analysis, and specifically refers to a medical data analysis method driven by a large-model multi-agent framework. Background Art

[0002] The rapid development of large language models (LLMs) has not only driven advancements in natural language processing and artificial intelligence (AI) but has also spawned innovative applications across multiple industries, particularly in healthcare. LLMs accumulate a wealth of medical knowledge by learning from vast amounts of diverse text data, including medical literature, medical records, medical textbooks, and authoritative guidelines.

[0003] The application of LLMs in medical data analysis has become a growing research hotspot. Leveraging their powerful text understanding and generation capabilities, LLMs can effectively process and analyze the complex characteristics of medical data. For example, LLMs can use natural language processing techniques to analyze patient test results, drug response data, and other data, identifying patterns and trends, providing strong support for data-driven medical research. Furthermore, LLMs can efficiently extract valuable insights from both structured and unstructured data, promoting the development of personalized and precision medicine, and providing new insights for academic research and clinical practice.

[0004] Although LLMs have significant advantages in medical data analysis, they also face some challenges. First, LLMs are prone to the "hallucination" problem, that is, the model may generate answers or inferences that do not match the actual situation. Especially when dealing with complex medical data, this problem may lead to erroneous analytical conclusions. Second, LLMs have low interpretability and transparency. Especially when faced with multi-source and multi-disciplinary data, the models often lack sufficient interpretability, which makes its analysis process and results difficult for medical experts to clearly understand and verify. In addition, medical data usually involves the intersection and integration of different disciplines, such as clinical data, imaging data, and laboratory test data. These diverse data sources increase the complexity of data analysis. LLMs may find it difficult to fully integrate and process this heterogeneous information, thereby affecting the accuracy and comprehensiveness of the analysis. Summary of the Invention

[0005] Given these challenges, relying solely on large models or single agents based on them for medical data analysis is insufficient to fully meet practical needs. Therefore, this paper proposes a medical data analysis method based on a large-model, multi-agent framework. By constructing a medical knowledge graph and combining graph retrieval-enhanced generation techniques with a multi-agent collaborative analysis algorithm, this method effectively mitigates the "hallucination" problem of large models while improving the quality, accuracy, and reliability of medical data analysis.

[0006] The medical data analysis method driven by a large-scale multi-agent framework includes the following four steps:

[0007] S1. By collecting multidisciplinary patient medical records data, we build a comprehensive medical knowledge graph to provide data support for the graph retrieval enhancement generation process;

[0008] S2. Create a graph retrieval enhanced generative agent to perform preliminary analysis on the patient's medical data to be analyzed;

[0009] S3, performing adaptive evaluation based on the patient's medical data to be analyzed and the preliminary analysis results obtained in step S2, and building an expert multi-agent group;

[0010] S4. The expert multi-agent group uses a collaborative method based on a weighted similarity matrix to conduct in-depth analysis of the medical data to be analyzed, obtain the final analysis results and output them;

[0011] The present invention constructs a medical knowledge graph comprising four sub-steps. First, based on the collected electronic medical records of patients, entities and relationships are extracted using a large language model prompt word project. Compared with traditional methods, the extraction of entities and relationships based on a large model reduces data dependence and cost, and the construction process is more intelligent and dynamic. Secondly, from the formatted information obtained in the previous step, entities and relationships are parsed out using regular matching rules and stored in the Neo4j database. Next, in order to enhance the generated vector retrieval process for subsequent graph retrieval, a word embedding model is used to add a vector index to the entity. Finally, the Neo4jGDS algorithm library is used to identify similar entities, and the entities with the same semantics among the potentially similar entities are merged with the help of a large language model to complete the entity merging verification work and obtain a medical knowledge graph.

[0012] The present invention creates a graph retrieval enhanced generation agent, which is used to preliminarily analyze the input medical data such as patient description information, test indicators, etc. The analysis content includes information such as correlation analysis between symptoms and diseases and risk assessment screening. First, symptoms, examination items and examination result information are extracted from the medical data, and these entities are word embedded. Secondly, the extracted entities are matched with the entities in the medical knowledge graph through vector indexing to ensure high-precision association at the semantic level. Next, a Cypher query statement is generated based on the matching results to retrieve disease entities related to symptoms and examinations, and construct disease-centered knowledge triples. In order to improve the intuitiveness of semantic expression, the triples are converted into natural language sentences to optimize the information transmission effect. Finally, the graph retrieval enhanced generation agent uses natural language sentences as an auxiliary basis and uses a large language model prompt word project to achieve a preliminary analysis of medical data.

[0013] Medical data is usually highly complex and diverse, and often involves knowledge and technology from different disciplines. Therefore, in order to effectively solve these problems, experts from multiple fields are usually required to collaborate across disciplines. In the present invention, we simulate this behavior by creating multiple expert agents for collaboration. The evaluation agent performs adaptive evaluation based on the medical data to be analyzed and the parsed content obtained by S2 based on the large language model prompt word engineering. The number of expert agents that need to be created for the subsequent recruitment agent is determined based on the complexity of the problem. The recruitment agent inputs the original medical data and outputs the medical expertise of each expert agent to be created through the prompt word engineering, and assigns different weights according to the degree of relevance to the medical data to be analyzed. Then, the corresponding expert agent is created based on the results output by the recruitment agent, thereby constructing an expert multi-agent group.

[0014] The expert multi-agent group collaborative analysis algorithm of the present invention includes five sub-steps. First, because the expert multi-agent group adopts a centralized communication structure, a central agent is created for global control. In this stage, the central agent is responsible for formulating a task specification based on the medical data to be analyzed, the analysis results obtained in step S2, and the medical expertise of each expert agent. Second, each expert agent conducts problem analysis and provides the initial analysis results based on its own medical expertise and the task specification provided by the central agent. Third, the central agent collects and summarizes the initial analysis results of each expert agent, calculates semantic similarity and generates a similarity matrix. Then, a weighted similarity analysis matrix is generated based on the weights assigned by the recruiting agents. Next, the weighted mean similarity between each expert agent's analysis results and the other agents' analysis results is calculated, and the consistency of the multi-agent collaborative analysis results is determined in combination with a threshold. Finally, an adjustment mechanism is provided if a consistent analysis result cannot be obtained within the maximum number of iterations, until a consistent analysis result is obtained and output.

[0015] The present invention has the following characteristics and beneficial effects

[0016] This paper designs a medical data analysis method driven by a large-scale multi-agent framework. Compared to existing multi-agent frameworks, this paper focuses more on the medical field. By combining enhanced generation technology based on graph retrieval and designing an expert multi-agent collaborative analysis algorithm, it effectively mitigates the "hallucination" problem that may occur with large models, while significantly improving the accuracy, reliability, and insight of medical data analysis.

[0017] This invention also constructs a comprehensive medical knowledge graph by collecting multidisciplinary patient medical records. Based on this, combined with graph retrieval and enhanced generation technology, it enables preliminary analysis of input medical data, including patient descriptions and test indicators. This analysis includes symptom-disease correlation analysis and risk assessment screening.

[0018] This paper proposes an expert multi-agent collaborative analysis algorithm based on a weighted similarity matrix. By introducing multiple expert agents, this algorithm simulates interdisciplinary collaboration among experts from multiple fields to conduct in-depth analysis of medical data. By using a weighted similarity matrix, it achieves consistency control for expert multi-agent group collaborative analysis and quantifies consistency determination during the analysis process, making the process more objective and accurate. This method not only improves the efficiency of collaboration between expert agents but also significantly enhances the quality and accuracy of medical data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The overall flow chart of the medical data analysis method driven by a large-scale multi-agent framework;

[0020] Figure 2 Build process flow charts for medical knowledge graphs;

[0021] Figure 3 Provides a flowchart for the algorithm of expert multi-agent collaborative analysis process; DETAILED DESCRIPTION

[0022] The overall flow chart is as follows Figure 1 As shown in the figure, the medical data analysis method driven by the large-model multi-agent framework includes four steps: constructing a medical knowledge graph, graph retrieval enhancement to generate preliminary analysis by intelligent agents, adaptive evaluation and construction of expert agent groups, and designing an expert multi-agent collaborative analysis algorithm to draw analysis conclusions.

[0023] 1. Medical knowledge graph construction module

[0024] The present invention is based on the collected electronic medical records of patients, uses a large language model to assist in building a knowledge graph, and finally stores the graph in the Neo4j graph database. Figure 2 As shown, the medical knowledge graph construction process of the present invention includes the following four steps:

[0025] Step 1: Extraction of entities and relationships:

[0026] Compared with traditional knowledge graph construction methods, the knowledge graph construction method based on large models can extract entities and relationships from unstructured data more efficiently and accurately, reduce data dependence and development costs, and provide stronger versatility and scalability, which makes the knowledge graph construction process more intelligent, dynamic and flexible. The prompt words used in this invention to extract entities and relationships are as follows:

[0027]

[0028]

[0029] Step 2: Write the entity relationship diagram to Neo4j

[0030] Based on the formatted information obtained in step 1, the nodes and relationships are parsed using regular matching rules, and a GraphDocument object is created. Finally, the entity relationship graph is written to the Neo4j database for storage using the graph.add_graph_documents method.

[0031] Step 3: Create vector index

[0032] The BAAI / bge-m3 model is used to create an index on the ['id', 'description'] attributes of entity nodes. This is because the description attribute contains the summary descriptive information summarized by the LLM during the entity relationship extraction phase, which is much richer than the id attribute. By combining the two, a vector is embedded, which makes subsequent vector searches much more accurate. This paper uses the Neo4jVector.from_existing_graph method in the Langchain framework to construct a vector index.

[0033] Step 4: Entity alignment optimization and coreference resolution

[0034] The entity alignment optimization and coreference resolution of the present invention includes two parts: using the Neo4j GDS library to identify potential similar entities and using a large model to perform entity merging verification.

[0035] First, we use the Neo4j GDS library to identify similar entities and analyze potential similar entities. All algorithms in the Neo4j GDS library operate on projected graphs, so we load the projected graph into memory using gds.graph.project(). We then use the K-nearest neighbor algorithm gds.knn.mutate() to find similar entities. In this paper, the similarity threshold for the K-nearest neighbor algorithm is set to 90%.

[0036] Secondly, because natural language descriptions have thousands of subtle variations, we ultimately use the large model's powerful natural language processing capabilities and the knowledge accumulated from ultra-large-scale parameters to determine which entities can ultimately be merged. The prompt words used by this invention for entity verification using the large model are as follows:

[0037]

[0038]

[0039] 2. Graph Retrieval Enhanced Generative Agent

[0040] First, using the large language model as the basic architecture, create a graph retrieval enhancement generation agent by setting the value of the message_role field in the calling model API:

[0041] message_role="If you are a medical information analysis expert, you need to complete the search enhancement generation task and provide preliminary analysis results for the patient's medical data to be analyzed."

[0042] The Graph Retrieval Enhanced Generative Agent first extracts symptom descriptions, examination items, and results from the input medical data. The next step is to match these entities with entities in the medical knowledge graph using semantic similarity. The extracted entities are embedded using the BAAI / bge-m3 model and combined with the vector indexes created for the entities in the medical knowledge graph building module to ultimately find the most relevant entity mappings in the graph.

[0043] After generating a Cypher query, the system queries the graph database to retrieve disease entities related to the symptom description, as well as disease entities related to examination items and test results that meet these characteristics. Through these query operations, a series of disease-centric knowledge triples are constructed, namely multiple one-hop subgraphs.

[0044] Initially acquired triples (including subject, predicate, and object) can pose challenges for LLM in complex semantic understanding and reasoning. Therefore, by reconstructing the subject, predicate, and object in the triples into grammatically correct natural language sentences, the intuitiveness and coherence of information expression are enhanced.

[0045] Ultimately, the graph retrieval enhanced generative agent will use the information retrieved from the knowledge graph as an auxiliary basis to conduct preliminary analysis of the complex medical data to be analyzed, including correlation analysis between symptoms and diseases and risk assessment screening.

[0046] The prompt words for the graph retrieval enhanced generation agent are as follows:

[0047]

[0048] 3. Adaptive Evaluation and Construction of Expert Agent Groups

[0049] Based on the large language model architecture, create evaluation agents and recruitment agents by setting the value of the message_role field in the calling model API:

[0050] message_role="If you are a medical assessment expert, you need to perform problem assessment tasks based on the patient's medical data to be analyzed."

[0051] message_role="If you are a hospital guide, you need to analyze the specific medical experts that need to be recruited based on the patient data to be analyzed."

[0052] The main function of the evaluation agent is to perform adaptive evaluation based on the original medical data and the parsing content of the graph retrieval enhancement agent. It also determines the number of expert agents needed for subsequent recruitment based on the complexity of the problem. The prompts for the evaluation agent to implement the evaluation process are as follows:

[0053]

[0054] The main function of the Recruitment Agent is to output the medical expertise of each expert agent to be created based on the number of expert agents output by the Evaluation Agent and the content of the original medical data, and assign different weights based on the degree of relevance to the medical data to be analyzed. Then, using Python to perform string data processing, it parses the value of the message_role field from the results output by the Recruitment Agent and creates the corresponding expert agent, thus constructing an expert multi-agent group. The prompt words used by the Recruitment Agent to implement the recruitment process are as follows:

[0055]

[0056] The creation process of each expert agent is the same as that of the other agents mentioned above, but the message_role is parsed from the output of the recruiting agent. The message_role of the expert agent is as follows:

[0057] message_role="You are a pediatrician specializing in the medical care of infants, children, and adolescents"

[0058] 4. Expert Multi-Agent Collaborative Analysis Algorithm

[0059] The core of the present invention is the analysis of medical data by a collaborative method of expert multi-agent groups based on a weighted similarity matrix.

[0060] The pseudo code of the expert multi-agent collaborative analysis algorithm is as follows:

[0061]

[0062]

[0063] like Figure 3 As shown in FIG, the expert multi-agent collaborative analysis algorithm designed by the present invention is roughly divided into the following five steps:

[0064] Step 1: The central agent formulates the task description:

[0065] First, using the large language model as the infrastructure, create a central agent by setting the value of the message_role field in the calling model API:

[0066] message_role="If you are the leader of a medical team, you need to coordinate the collaboration of various experts within the medical team. For example, you need to complete tasks such as task decomposition, feedback collection, result integration, and process assertion."

[0067] The central agent formulates a task description based on the medical data to be analyzed, the analysis results of the graph retrieval enhanced generation agent, and the medical expertise of each expert agent;

[0068] The prompt words used for the central agent's task decomposition are as follows:

[0069]

[0070] Step 2: Preliminary analysis and answer generation by the agent:

[0071] In the first round of multi-agent collaborative analysis, the expert agent analyzes the problem and gives the first round of analysis results based on its own medical expertise and the task instructions provided by the central agent.

[0072] The prompt words for each expert agent's first round of speeches are as follows:

[0073]

[0074] In the multi-agent collaboration framework, each expert agent's speech is stored in the investigation report as a key-value pair in the format [name, speech content]. It is important to emphasize that this framework utilizes a centralized communication structure, and communication between expert agents is isolated. During the speech phase, the output of a single expert agent is completely independent and unaffected by the speech of other agents. Each agent's speech is collected and aggregated by a central agent.

[0075] The core purpose of this design is to address the "hallucination" problem inherent in large models, where the model can generate inaccurate or even false information. In a decentralized communication model, an erroneous output from one agent can have a cascading impact on the judgments of other agents, amplifying the risk of erroneous conclusions spreading. A centralized, isolated communication structure effectively mitigates this impact, ensuring that each expert agent independently provides analysis results within its domain, thereby enhancing the robustness of the system and the reliability of its conclusions.

[0076] Step 3: Construct a weighted similarity analysis matrix

[0077] The central agent extracts and analyzes the speeches of each expert agent, formatting them. It then uses its Function Calling capability to calculate semantic similarity between each pair. This method implements semantic similarity queries through word embedding, using the BAAI / bge-m3 embedding model. The cosine similarity between two vectors is calculated to generate an expert answer similarity matrix. Combined with the recruiting agent's assessment of the relevance of each expert agent, the similarity matrix is weighted to produce a weighted similarity analysis matrix.

[0078] The prompt words used by the central agent to integrate information are as follows:

[0079]

[0080] Step 4: Determine the consistency of analysis results

[0081] For each expert agent, calculate the weighted average similarity between its answers and those of other expert agents. Set a similarity threshold. If the average similarity of any expert agent exceeds the threshold, its conclusion is output as the final analysis result. If the average similarity of multiple expert agents exceeds the threshold, the conclusion of the agent with the highest average similarity is selected as the final analysis result and output.

[0082] If the average similarity of all agents is below the threshold, an agent with a high weight but low similarity is selected to speak again. In this invention, the maximum number of iterations is set to 3 for optimal performance, so this process allows a maximum of 3 iterations. After re-speaking, the relevant content in the investigation must be updated and replaced.

[0083] The prompt words for the expert agent to re-speak are as follows:

[0084]

[0085]

[0086] Step 5: Dynamic Adjustment Mechanism

[0087] When the consistency analysis result cannot be obtained within the maximum number of iterations, the system will start the adjustment mechanism to re-evaluate the number of required expert agents or adjust the threshold until the consistency analysis result is obtained and output.

[0088] Comparative experimental results:

[0089] GLM4 GLM4+ framework GPT3.5 GPT3.5+this framework GPT-4 GPT-4+this framework MedQA 67.11％ 81.37％ 55.05％ 69.28％ 76.38％ 93.24％ MedMCQA 85.00％ 96.55％ 67.33％ 78.00％ 75.00％ 98.25％ PubMedQA 55.35％ 73.50％ 67.26％ 67.28％ 68.33％ 89.76％ DDxPlus 74.83％ 76.06％ 55.62％ 64.10％ 80.24％ 88.79％

[0090] Comparative experiments on multiple datasets such as MedQA and MedMCQA show that the large language model based on the framework proposed in this invention has a significantly improved accuracy compared to the original large language model, thus verifying the quality and accuracy of the present invention in analyzing complex medical data.

Claims

1. A medical data analysis method driven by a large-scale multi-agent framework, characterized by: The following steps are involved: S1. By collecting multidisciplinary patient medical records data, we build a comprehensive medical knowledge graph to provide data support for the graph retrieval enhancement generation process; S2. Create a graph retrieval enhanced generative agent to perform preliminary analysis on the patient's medical data to be analyzed; S3, performing adaptive evaluation based on the patient's medical data to be analyzed and the preliminary analysis results obtained in step S2, and building an expert multi-agent group; S4. The expert multi-agent group uses a collaborative method based on a weighted similarity matrix to conduct in-depth analysis of the medical data to be analyzed, obtain the final analysis results and output them.

2. The large-model multi-agent framework driven medical data analysis method according to claim 1, characterized in that: In step S1, constructing a medical knowledge graph includes four sub-steps: S11. Based on the collected multidisciplinary patient medical record data, use the large language model prompt word project to extract entities and relationships, and generate formatted information according to the prompt word requirements; S12. Based on the formatted information obtained above, entities and relationships are parsed using regular matching rules and stored in a Neo4j graph database; S13. Use the BAAI / bge-m3 word embedding model to add vector indexes to entities in the Neo4j graph database. S14. Use the Neo4j GDS algorithm library to identify similar entities, and calculate vector similarity through vector indexing to obtain potential similar entities. Then, use the large language model to merge entities with the same semantics among the potential similar entities, complete the entity merging and verification work, and obtain the medical knowledge graph.

3. The large-model multi-agent framework driven medical data analysis method according to claim 1, characterized in that: In step S2, the method for performing preliminary analysis by generating an agent through graph retrieval enhancement is: S21. The graph retrieval enhancement generation agent extracts symptoms, examination items, and examination result feature information from the patient's medical data to be analyzed, and performs word embedding on the extracted entities; S22. Match the extracted entities with entities in the medical knowledge graph through vector indexing to ensure high-precision association at the semantic level; S23. Generate Cypher query statements based on the matching results through the prompt word engineering of the large language model, retrieve disease entities related to symptoms and examinations, and construct knowledge triples centered on the disease entities. The triples include subject, predicate, and object. To improve the intuitiveness of semantic expression, the triples are converted into natural language sentences to optimize information transmission. S24. Graph retrieval enhancement generative agent uses natural language sentences as auxiliary basis and applies large language model prompt word engineering to achieve preliminary analysis of medical data.

4. The large-model multi-agent framework driven medical data analysis method according to claim 1, characterized in that: The preliminary analysis includes correlation analysis between symptoms and diseases and risk assessment screening.

5. The large-model multi-agent framework driven medical data analysis method according to claim 1, characterized in that: In the step S3, two agents are created, namely the evaluation agent and the recruitment agent. The creation method is: based on the large language model as the basic architecture, the evaluation agent and the recruitment agent are obtained respectively by setting different message_role fields.

6. The large-model multi-agent framework driven medical data analysis method according to claim 5, characterized in that: The method for adaptively evaluating the evaluation agent is: based on the large language model prompt word engineering, adaptively evaluate the medical data to be analyzed and the parsed content obtained by S2, so as to determine the number of expert agents that need to be created for subsequent recruitment of agents based on the complexity of the problem.

7. The large-model multi-agent framework driven medical data analysis method according to claim 6, characterized in that: The method for obtaining an expert multi-agent group by recruiting an agent is as follows: the recruiting agent inputs the original medical data and outputs the medical expertise of each expert agent to be created through prompt word engineering and assigns different weights according to the degree of relevance to the medical data to be analyzed, and then parses the value of the message_role field from the result output by the recruiting agent and creates the corresponding expert agent, thereby constructing an expert multi-agent group.

8. The large-model multi-agent framework driven medical data analysis method according to claim 7, characterized in that: The expert agent is based on a large language model and is obtained by recruiting the message_role field parsed by the agent. The number of the expert agents is determined by the output of the evaluation agent.

9. The large-model multi-agent framework driven medical data analysis method according to claim 1, characterized in that: In step S4, the expert multi-agent group collaborative analysis algorithm includes the following steps: S41. The expert multi-agent group adopts a centralized communication structure and creates a central agent for global control. The central agent is created by setting the message_role field based on the large language model as the basic architecture. The central agent decomposes tasks based on the medical data to be analyzed, the preliminary analysis results obtained in step S2, and the medical expertise of each expert agent. S42. In the first round of collaboration, each expert agent analyzes the problem and provides the first round of analysis results based on their own medical expertise and the task instructions provided by the central agent through prompt word engineering; S43. The central agent collects and summarizes the first round of analysis results of each expert agent, calculates the semantic similarity between the two agents using the cosine similarity calculation formula and generates a similarity matrix, and then generates a weighted similarity analysis matrix based on the weights assigned by the recruited agents; S44. Calculate the weighted similarity mean of each expert agent's analysis results with those of other expert agents based on the weighted similarity analysis matrix, and determine the consistency of the multi-agent collaborative analysis results in combination with the threshold. If there is an expert agent whose weighted similarity mean is greater than or equal to the threshold, the central agent considers that the judgment is passed and gives the final analysis result as output; if there are at least two expert agents whose average similarities exceed the threshold, the conclusion given by the expert agent with the largest average similarity is selected as the final analysis result and output; if all similarity means are lower than the threshold, the central agent will select the expert agent with higher weight but lower similarity to speak again and iterate.

10. The large-model multi-agent framework driven medical data analysis method according to claim 9, characterized in that: In step S44, when all similarity means are lower than the threshold, the speech is repeated and the performance is optimal when the maximum number of iterations is set to 3.

11. The large-model multi-agent framework driven medical data analysis method according to claim 10, characterized in that: When the consistency analysis result cannot be obtained within the maximum number of iterations, the system will activate the adjustment mechanism, re-evaluate the number of recruited expert agents or adjust the threshold until the consistency analysis result is obtained and output.

Images