A multi-modal medical guide system based on multi-agent cooperation

Abstract

This invention discloses a multimodal medical triage system based on multi-agent collaboration, comprising a multimodal information interaction module, a preprocessing module, a dynamic medical knowledge base module, a multi-agent collaboration module, and a department identification module. The multimodal information interaction module receives symptom descriptions provided by patients; the preprocessing module standardizes and processes symptom descriptions and information; the dynamic medical knowledge base module stores structured medical knowledge and updates it in real time; the multi-agent collaboration module determines the activation order of each agent according to a preset workflow and dynamically adjusts the process; and the department identification module recommends outpatient departments and doctors based on symptom retrieval results and simultaneously pushes a guide map of the corresponding department. This invention solves the problems of rigid interaction and poor adaptability in traditional medical triage systems through the synergistic effect of the multimodal information interaction module, preprocessing module, dynamic medical knowledge base module, multi-agent collaboration module, and department identification module.

Description

Technical Field

[0001] This invention relates to the field of medical triage system technology, and in particular to a multimodal medical triage system based on multi-agent collaboration. Background Technology

[0002] Medical guidance is a professional service established by medical institutions to facilitate patients' access to medical care. Like a "compass" within the hospital, it is provided by specially trained guides or intelligent systems. Its core objectives are to guide patients, answer their questions, and optimize the medical process, thereby improving the overall efficiency of medical services and the patient experience. Specifically, guidance services include directing patients to departments, answering their questions, assisting patients with special needs, and maintaining order in the hospital environment, ensuring that patients can complete their treatment efficiently and smoothly. In addition, guidance services also undertake pre-screening and triage functions, initially assessing the urgency of patients' conditions, classifying them according to standards, and rationally diverting patients to the appropriate departments to avoid delays in treatment for critically ill patients, while also balancing the allocation of medical resources. With the development of technology, the medical triage service has transformed from the traditional manual mode to intelligent mode. Traditional medical triage systems mostly use linear decision trees or rule engines, which have limitations such as rigid interaction, poor adaptability, and inability to handle complex symptom descriptions. Therefore, this invention proposes a multimodal medical triage system based on multi-agent collaboration to solve the problems existing in the prior art. Summary of the Invention

[0003] To address the aforementioned problems, the present invention aims to propose a multimodal medical triage system based on multi-agent collaboration. This system supports multiple input methods, including text, voice, and images, through a multimodal information interaction module. It also uses image-assisted methods to confirm the patient's discomfort area with a diagram, thereby accurately obtaining the patient's condition. A preprocessing module standardizes information through voice recognition and image feature extraction. A dynamic medical knowledge base module provides real-time updated structured medical knowledge support. A multi-agent collaboration module simulates an expert consultation process, dynamically scheduling agents for consultation, knowledge retrieval, decision reasoning, and emergency response through a workflow controller, achieving accurate symptom mapping, disease retrieval, and emergency response. A department identification module recommends departments and doctors based on the reasoning results. Through the synergistic effect of the multimodal information interaction module, preprocessing module, dynamic medical knowledge base module, multi-agent collaboration module, and department identification module, the system solves the problems of rigid interaction, poor adaptability, and inability to handle complex symptom descriptions in traditional medical triage systems.

[0004] To achieve the objectives of this invention, the invention is implemented through the following technical solution: a multimodal medical triage system based on multi-agent collaboration, comprising a multimodal information interaction module, a preprocessing module, a dynamic medical knowledge base module, a multi-agent collaboration module, and a department identification module. The multimodal information interaction module is used to receive symptom description information provided by patients. The preprocessing module is used to standardize the symptom descriptions and symptom information. The dynamic medical knowledge base module is used to store structured medical knowledge and update it in real time. The multi-agent collaboration module is used to determine the activation order of each agent according to a preset workflow and dynamically adjust the process. The multi-agent collaboration module includes a workflow controller, a consultation agent unit, a knowledge retrieval agent unit, a decision reasoning agent unit, and an emergency treatment agent unit. The department identification module is used to recommend outpatient departments and doctors based on the symptom retrieval results, and simultaneously push a guide map of the corresponding department.

[0005] Further improvements include: the workflow controller dynamically determines and schedules intelligent agents based on preset logic and real-time interaction status; the consultation intelligent agent unit effectively communicates with patients, understands the text and image features of the symptoms described by the patients, and maps the symptom descriptions into standard medical terms and standardized codes; the knowledge retrieval intelligent agent unit accesses and queries the medical knowledge base to retrieve relevant diseases; the decision reasoning intelligent agent unit determines the condition and type of the disease based on the retrieval results of the knowledge retrieval intelligent agent unit; and the emergency treatment intelligent agent unit determines whether the disease is in an emergency state and opens a green channel for rapid medical treatment when an emergency state occurs.

[0006] A further improvement is that the natural language to medical terminology mapping process executed by the consultation intelligent agent unit adopts a semantic matching algorithm based on medical knowledge graph embedding, calculates the similarity between the symptom phrases described by the patient and the standard terms in the knowledge base, and selects the standard term with the highest similarity as the mapping result.

[0007] Further improvements include: the emergency response intelligent agent unit has a built-in emergency symptom keyword library and a list of high-risk diseases.

[0008] Further improvements include: the multimodal information interaction module comprises a human-computer interaction interface unit, a text input unit, a voice acquisition unit, an image upload unit, and an image assistance unit. The human-computer interaction interface unit provides a graphical interface for uploading multimodal information, displaying intelligent follow-up questions, and showing triage results. The text input unit is used to directly input text descriptions. The voice acquisition unit is used to acquire voice information describing symptoms from patients. The image upload unit is used to receive medical images of the affected area uploaded by patients. The image assistance unit is used to interact with patients to obtain schematic diagrams of the uncomfortable areas, thereby accurately obtaining the patient's condition.

[0009] A further improvement is that the preprocessing module includes a speech recognition unit and an image preprocessing unit. The speech recognition unit is used to convert speech into text in real time, and the image preprocessing unit is used to extract features from uploaded images and generate structured feature vectors.

[0010] Further improvements are made in that the dynamic medical knowledge base module includes a data acquisition and processing unit, a knowledge integration and storage unit, and a standardized interface unit. The data acquisition and processing unit is used to acquire information from medical literature and historical medical records and convert unstructured text into structured data. The knowledge integration and storage unit is used to import knowledge from different data sources into the hard drive for easy and quick access, and at the same time, upload and back up the knowledge data to the cloud drive to prevent data loss. The standardized interface unit is used for knowledge query and access, and provides a unified data interface for external devices.

[0011] Further improvements are made in that the department identification module generates a recommendation list based on the disease type and probability output by the decision reasoning intelligent agent unit, combined with the real-time scheduling information of hospital departments and the doctor's area of ​​expertise.

[0012] The beneficial effects of this invention are as follows: This invention supports multiple input methods such as text, voice, and images through a multimodal information interaction module, and uses image-assisted methods to confirm the diagram of the uncomfortable area with the patient, thereby accurately obtaining the patient's condition; the preprocessing module achieves information standardization through voice recognition and image feature extraction; the dynamic medical knowledge base module provides real-time updated structured medical knowledge support; the multi-agent collaboration module simulates the expert consultation process, dynamically scheduling the consultation, knowledge retrieval, decision reasoning, and emergency response agents through a workflow controller to achieve accurate symptom mapping, disease retrieval, and emergency response; the department identification module recommends departments and doctors based on the reasoning results. Through the synergistic effect of the multimodal information interaction module, preprocessing module, dynamic medical knowledge base module, multi-agent collaboration module, and department identification module, this invention solves the problems of rigid interaction, poor adaptability, and inability to handle complex symptom descriptions in traditional medical triage systems. Attached Figure Description

[0013] Figure 1 This is a system architecture diagram of the present invention; Figure 2 This is a flowchart of the process of the present invention. Detailed Implementation

[0014] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0015] according to Figure 1 , 2As shown, this embodiment provides a multimodal medical triage system based on multi-agent collaboration, including a multimodal information interaction module, a preprocessing module, a dynamic medical knowledge base module, a multi-agent collaboration module, and a department identification module. The multi-agent collaboration module determines the activation order of each agent according to a preset workflow and dynamically adjusts the process. The multi-agent collaboration module includes a workflow controller, a consultation agent unit, a knowledge retrieval agent unit, a decision-making and reasoning agent unit, and an emergency handling agent unit. The workflow controller dynamically determines and schedules agents based on preset logic and real-time interaction status. The consultation agent unit effectively communicates with patients, understands the text and image features of the symptoms described by the patient, and maps the symptom descriptions to standard medical terms and standardized codes. The natural language to medical terminology mapping process executed by the consultation agent unit uses a semantic matching algorithm based on medical knowledge graph embedding to calculate the similarity between the symptom phrases described by the patient and the standard terms in the knowledge base, and selects the standard term with the highest similarity as the mapping result. The knowledge retrieval agent unit is used for... The system accesses and queries a medical knowledge base to retrieve relevant diseases. A decision-making reasoning agent determines the disease's condition and type based on the retrieval results. An emergency response agent determines if the disease is in an emergency state, activating a fast-track system for rapid medical attention in such cases. The emergency response agent has a built-in emergency symptom keyword database and a list of high-risk diseases. A multi-agent collaboration module simulates the reasoning process of expert consultation, dynamically scheduled by a workflow controller. First, the consultation agent is activated, engaging in multiple rounds of interaction with the patient and mapping the patient's described symptoms to standard medical terminology. Next, the knowledge retrieval agent searches the knowledge base for relevant disease information using these standard terms. Then, the decision-making reasoning agent performs comprehensive analysis and probability judgment based on the retrieval results. If information is insufficient, the workflow controller instructs the consultation agent to ask follow-up questions. The emergency response agent monitors the entire process, immediately interrupting the routine process and initiating an emergency response upon detecting critical keywords or diseases. Through the specialized division of labor and collaborative cooperation among multiple agents, accurate, reliable, and secure diagnostic reasoning is achieved, ensuring priority treatment for emergency patients.

[0016] The multimodal information interaction module receives symptom descriptions from patients. It includes a human-computer interaction interface unit, a text input unit, a voice acquisition unit, an image upload unit, and an image assistance unit. The human-computer interaction interface unit provides a graphical interface for uploading multimodal information, displaying intelligent follow-up questions, and showing triage results. The text input unit allows direct input of text descriptions. The voice acquisition unit collects voice information describing symptoms from patients. The image upload unit receives medical images of the affected area uploaded by patients. The image assistance unit interacts with patients to display diagrams of the affected areas, thereby accurately understanding the patient's condition. As the front-end entry point for system-patient interaction, the multimodal information interaction module receives symptom descriptions submitted by patients through various channels. Users can directly type descriptions in the text input box, use the microphone for voice narration, upload photos of the affected area or medical report images from the device's album, add or click on image assistance, select the uncomfortable area, and provide a unified input interface that supports raw information in three modalities: text, voice, and image. The image assistance method allows for interaction with the patient to confirm the uncomfortable area, laying the foundation for subsequent in-depth processing, effectively reducing the user threshold for patients, and ensuring the comprehensiveness and convenience of information acquisition through multiple input methods.

[0017] The preprocessing module standardizes symptom descriptions and information. It includes a speech recognition unit and an image preprocessing unit. The speech recognition unit converts speech into text in real time, while the image preprocessing unit extracts features from uploaded images to generate structured feature vectors. By transforming the unstructured raw information collected by the multimodal input module into standardized text data that can be uniformly processed and analyzed by the system backend, the preprocessing module achieves information standardization. It unifies heterogeneous multimodal data into structured text descriptions, thus removing obstacles for subsequent accurate analysis and understanding by the intelligent agent.

[0018] The dynamic medical knowledge base module stores and updates structured medical knowledge in real time. It comprises a data acquisition and processing unit, a knowledge integration and storage unit, and a standardized interface unit. The data acquisition and processing unit extracts information from medical literature and historical medical records, converting unstructured text into structured data. The knowledge integration and storage unit integrates structured knowledge from different data sources into a medical knowledge graph, storing it on the local hard drive for quick retrieval and simultaneously backing it up to a cloud storage server. The standardized interface unit handles knowledge querying and provides a unified data interface for external devices. As the core data hub of the system, the dynamic medical knowledge base module automatically crawls and integrates new knowledge from authoritative medical literature, clinical guidelines, and anonymized medical records through the data acquisition unit, converting this unstructured text into structured data using natural language processing. Then, the knowledge integration unit constructs a vast medical knowledge graph from this data and stores it locally and in the cloud. Finally, the standardized interface unit responds to internal system queries, providing a continuously updated, clearly structured, and quickly searchable massive medical knowledge database, offering reliable data support for the decision-making of all intelligent agents.

[0019] The department identification module recommends outpatient departments and doctors based on the symptom search results, and pushes a guide map of the corresponding departments. The department identification module generates a recommendation list based on the disease type and probability output by the decision reasoning intelligent agent unit, combined with the real-time scheduling information of hospital departments and the doctor's area of ​​expertise.

[0020] This multimodal medical triage system, based on multi-agent collaboration, allows patients to provide symptom descriptions through a multimodal information interaction module, including text, voice, or image input. The preprocessing module then converts the voice into text in real time and extracts features from the images, generating standardized structured text data. Next, the workflow controller of the multi-agent collaboration module dynamically schedules agents. First, the consultation agent is activated to interact with the patient in multiple rounds, mapping spoken symptoms to standard medical terminology. Then, the knowledge retrieval agent is launched to query relevant disease information from a dynamic medical knowledge base module. The decision-making reasoning agent performs comprehensive analysis and probability judgment based on the retrieval results. If the information is insufficient, the workflow controller instructs the consultation agent to ask follow-up questions. Simultaneously, the emergency handling agent monitors the input content; if it detects critical keywords or diseases, it immediately interrupts the regular process and initiates an emergency response. Finally, the department identification module, based on the disease type and probability output by the decision-making reasoning, combined with the hospital's real-time schedule and doctors' specialties, generates a recommended list of outpatient departments and doctors and pushes a navigation map, completing the entire diagnosis and triage process.

[0021] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multimodal medical triage system based on multi-agent collaboration, characterized in that: The system includes a multimodal information interaction module, a preprocessing module, a dynamic medical knowledge base module, a multi-agent collaboration module, and a department identification module. The multimodal information interaction module receives symptom descriptions from patients. The preprocessing module standardizes and processes symptom descriptions and information. The dynamic medical knowledge base module stores and updates structured medical knowledge in real time. The multi-agent collaboration module determines the activation order of each agent based on a preset workflow and dynamically adjusts the process. The multi-agent collaboration module includes a workflow controller, a consultation agent unit, a knowledge retrieval agent unit, a decision-making and reasoning agent unit, and an emergency treatment agent unit. The department identification module recommends outpatient departments and doctors based on symptom retrieval results and simultaneously pushes a guide map of the corresponding department.

2. The multimodal medical triage system based on multi-agent collaboration according to claim 1, characterized in that: The workflow controller dynamically determines and schedules intelligent agents based on preset logic and real-time interaction status. The consultation intelligent agent unit effectively communicates with patients, understands the text and image features of the symptoms described by the patients, and maps the symptom descriptions into standard medical terms and standardized codes. The knowledge retrieval intelligent agent unit accesses and queries the medical knowledge base to retrieve relevant diseases. The decision reasoning intelligent agent unit determines the condition and type of the disease based on the retrieval results of the knowledge retrieval intelligent agent unit. The emergency treatment intelligent agent unit determines whether the disease is in an emergency state and opens a green channel for rapid medical treatment when an emergency state occurs.

3. A multimodal medical triage system based on multi-agent collaboration according to claim 2, characterized in that: The natural language to medical terminology mapping process executed by the consultation intelligent agent unit adopts a semantic matching algorithm based on medical knowledge graph embedding. It calculates the similarity between the symptom phrases described by the patient and the standard terms in the knowledge base, and selects the standard term with the highest similarity as the mapping result.

4. A multimodal medical triage system based on multi-agent collaboration according to claim 2, characterized in that: The emergency response intelligent agent unit has a built-in emergency symptom keyword library and a list of high-risk diseases.

5. A multimodal medical triage system based on multi-agent collaboration according to claim 1, characterized in that: The multimodal information interaction module includes a human-computer interaction interface unit, a text input unit, a voice acquisition unit, an image upload unit, and an image assistance unit. The human-computer interaction interface unit provides a graphical interface for uploading multimodal information, displaying intelligent follow-up questions, and showing triage results. The text input unit is used to directly input text descriptions. The voice acquisition unit is used to acquire voice information describing symptoms from patients. The image upload unit is used to receive medical images of the affected area uploaded by patients. The image assistance unit is used to interact with patients to obtain schematic diagrams of the uncomfortable areas, thereby accurately obtaining the patient's condition.

6. A multimodal medical triage system based on multi-agent collaboration according to claim 1, characterized in that: The preprocessing module includes a speech recognition unit and an image preprocessing unit. The speech recognition unit is used to convert speech into text in real time, and the image preprocessing unit is used to extract features from uploaded images and generate structured feature vectors.

7. A multimodal medical triage system based on multi-agent collaboration according to claim 1, characterized in that: The dynamic medical knowledge base module includes a data acquisition and processing unit, a knowledge integration and storage unit, and a standardized interface unit. The data acquisition and processing unit is used to acquire information from medical literature and historical medical records, and to convert unstructured text into structured data. The knowledge integration and storage unit is used to integrate structured knowledge from different data sources into a medical knowledge graph, store it on the local hard drive for quick access, and simultaneously back it up to the cloud storage server. The standardized interface unit is used for knowledge query and access, and provides a unified data interface for external devices.

8. A multimodal medical triage system based on multi-agent collaboration according to claim 1, characterized in that: The department identification module generates a recommendation list based on the disease type and probability output by the decision reasoning intelligent agent unit, combined with the real-time scheduling information of hospital departments and doctors' areas of expertise.

Images