Aplexy patient management method and system based on multi-mode RAG and LLM model

Abstract

The invention discloses an apoplexy patient management method and system based on a multi-mode RAG and LLM model, and the method comprises the steps: obtaining the standardized electronic medical record data of a patient based on an FHIR standard electronic medical record; performing feature extraction on the electronic medical record data to obtain multi-modal data; integrating and representing a time-space relationship among the multi-modal data, and obtaining a dynamically updated dynamic knowledge graph; analyzing and integrating the electronic medical record data to obtain integrated information; constructing a stroke risk causal network based on an LLM model in combination with a causal reasoning mechanism; obtaining a personalized health report based on the dynamic knowledge graph and a stroke risk causal network; and based on a personalized report result and feedback data of the patient, a treatment scheme and a management strategy of the patient are optimized in real time. According to the application, the causal reasoning mechanism is embedded into the Transform architecture of the LLM, so that the modeling and prediction capabilities of the stroke risk causal relationship are enhanced.

Description

Technical Field

[0001] This application belongs to the technical field of medical data management, and specifically relates to a stroke patient management method and system based on multi-modal RAG and LLM models. Background Art

[0002] As one of the main causes of disability and death globally, early prevention and personalized management of stroke are of particular importance. Traditional stroke management systems mostly rely on a single data source, lacking the integration and analysis of comprehensive patient information, and it is difficult to implement personalized intervention measures. In recent years, the information management technologies for stroke mainly include: Electronic Health Record (EHR) system: The EHR system can integrate information such as the patient's medical history, examination results, treatment plans, etc., provide real-time data access and sharing, and help doctors make decisions. The clinical information of stroke patients can be effectively managed through the EHR system to ensure the accuracy and timeliness of information; Clinical Decision Support System (CDSS): The CDSS uses algorithms and data analysis tools to provide treatment suggestions and warnings based on the patient's historical data and current symptoms. This is of great significance for the early identification and intervention of stroke patients; Telemedicine system: Through video conferencing and real-time monitoring technologies, doctors can remotely evaluate and manage stroke patients. Especially in remote areas, this technology can significantly improve the accessibility of medical services; Data analysis and artificial intelligence system: Using big data analysis and artificial intelligence technologies, risk assessment and personalized management of stroke patients can be carried out. For example, machine learning algorithms (such as an ensemble model combining gradient boosting trees and Cox models) can analyze multiple factors (such as genetics, lifestyle, etc.) to predict stroke risk and help formulate personalized prevention and treatment plans; Speech recognition and Natural Language Processing (NLP) can develop a speech interaction system designed specifically for stroke patients to facilitate recording symptoms and rehabilitation progress; Virtual Reality (VR) and Augmented Reality (AR) technologies can simulate real environments to help patients with limb function training and cognitive recovery, enhancing the sense of participation and rehabilitation effect; Mobile Health (mHealth) system: Mobile applications can help patients self-manage their health, providing functions such as stroke risk assessment tools, rehabilitation guidance, and medication reminders, enhancing patient participation and compliance.

[0003] However, stroke management systems face numerous challenges in the current medical environment, which limit their level of intelligence and the effectiveness of patient management. Many systems are unable to effectively integrate data from multiple sources (such as electronic health records, imaging materials, and laboratory results), resulting in isolated information and an inability to comprehensively assess the patient's health status. In addition, most existing stroke management systems rely on standardized treatment plans and lack personalized management for individual patient differences, unable to meet the specific needs of different patients. Many systems also lack real-time monitoring of patients' physiological data and fail to detect potential health risks in a timely manner, leading to insufficient intervention. Current decision support functions often rely on static clinical guidelines and lack dynamic intelligent decision support based on real-time data and machine learning. At the same time, many stroke management systems have limited interaction with patients and lack effective communication channels, resulting in low patient participation in their own health management. Existing systems usually lack sufficient data analysis capabilities and are difficult to extract valuable information from large amounts of data, affecting the scientific nature of clinical decision-making. Summary of the Invention

[0004] This application provides a method and system for managing stroke patients based on multi-modal RAG and LLM models to solve the above technical problems.

[0005] To solve the above technical problems, a technical solution adopted in this application is: A method for managing stroke patients based on multi-modal RAG and LLM models, including the following steps:

[0006] Based on the FHIR standard electronic medical record, obtain the standardized electronic medical record data of the patient;

[0007] Based on NLP and the electronic medical record data, extract features from the electronic medical record data to obtain multi-modal data;

[0008] Based on the spatio-temporal hypergraph modeling method, integrate and characterize the spatio-temporal relationships between multi-modal data to obtain a dynamically updated dynamic knowledge graph;

[0009] Based on the FHIR data resource, parse and integrate the electronic medical record data to obtain integrated information;

[0010] Based on the LLM model combined with the causal reasoning mechanism, construct a stroke risk causal network;

[0011] Based on the dynamic knowledge graph and the stroke risk causal network, enhance retrieval and generation through the RAG model to obtain a personalized health report;

[0012] Based on the personalized report results and the patient's feedback data, optimize the patient's treatment plan and management strategy in real time.

[0013] Further, the electronic medical record data includes the patient's basic demographic information, medical history, medication records, retinal images, microcirculation imaging data, lifestyle information, hospitalization and surgical records.

[0014] Further, the spatio-temporal hypergraph modeling method includes:

[0015] Expand the time series data in the multimodal data to construct a composite hyperedge structure with spatio-temporal constraints. Herein, a hyperedge is defined as a set of multimodal nodes containing timestamps, which is used to describe the clinical event chain and its spatio-temporal relationship.

[0016] Further, the method for constructing a stroke risk causal network includes:

[0017] Construct an initial stroke risk causal network based on the integrated information and causal graph;

[0018] Based on the initial stroke risk causal network and the causal reasoning mechanism embedded in the LLM model of the Transformer architecture, use SEM to quantify the strength of causal relationships and perform intervention simulation on the initial stroke risk causal network;

[0019] Based on the results of the intervention simulation, obtain counterfactual reasoning data, strengthen the prediction accuracy of the LLM model for stroke-related causal relationships, and obtain a stroke risk causal network.

[0020] Another technical solution adopted by this application is: a stroke patient management system based on multimodal RAG and LLM model, including:

[0021] The personal information management module is used to obtain the patient's electronic medical record data;

[0022] The data processing module is used to extract features from the electronic medical record data to obtain multimodal data;

[0023] The spatio-temporal hypergraph construction module is used to integrate and characterize the spatio-temporal relationship between multimodal data to form a dynamically updated dynamic knowledge graph;

[0024] The FHIR interface integrated data module is used to integrate the electronic medical record data;

[0025] The causal network construction module embeds the causal reasoning mechanism into the LLM model to obtain a stroke risk causal network;

[0026] The report generation module generates a personalized health report through the RAG module combined with real-time retrieval technology and a generation model;

[0027] The update module uses the newly obtained clinical data and user feedback to dynamically update the parameters of the dynamic knowledge graph and the Transformer model, and realizes the continuous optimization of the stroke patient management plan.

[0028] Further, the data extraction module obtains the electronic medical record data of patients through different medical systems; and converts the electronic medical record data into a structured format through the API.

[0029] Further, the data processing module performs natural language processing on the text data in the electronic medical record data, image recognition on the image data in the electronic medical record data, and feature extraction on the time series data in the electronic medical record data; and encodes it into a three-dimensional hyperedge structure through a spatio-temporal alignment algorithm.

[0030] The beneficial effects of this application are as follows: The causal reasoning mechanism is embedded in the Transformer architecture of the large language model (LLM), thereby enhancing the modeling and prediction capabilities of the causal relationship of stroke risk. Combining with the retrieval-augmented generation (RAG) technology, the system can generate personalized health reports, medication reminders, and psychological support suggestions. At the same time, through the real-time feedback and closed-loop management mechanism, the system continuously optimizes the patient's risk assessment, treatment plan, and rehabilitation guidance, thereby enhancing the patient's compliance and improving the health management effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a schematic flowchart of an embodiment of the stroke patient management method based on the multi-modal RAG and LLM models of this application;

[0032] Figure 2 in Figure 1 is a schematic flowchart of an embodiment of step S5 in

[0033] Figure 3 is a structural block diagram of an embodiment of the stroke patient management system based on the multi-modal RAG and LLM models of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] To make the objectives, technical solutions, and advantages of the present invention clearer, the following further describes the present invention in detail with reference to specific embodiments.

[0035] Many specific details are set forth in the following description in order to provide a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Therefore, the present invention is not limited by the limitations of the specific embodiments disclosed in the following specification.

[0036] Refer to Figure 1 , Figure 1 is a schematic flowchart of an embodiment of the stroke patient management method based on the multi-modal RAG and LLM models of this application. The method includes:

[0037] Step S1. Based on the FHIR standard electronic medical record, obtain the standardized electronic medical record data of the patient.

[0038] Specifically, the patient's standard electronic medical records are automatically obtained from different medical systems through the FHIR standard interface. The electronic medical record data includes the patient's basic demographic information, medical history, medication records, retinal images, microcirculation imaging data, lifestyle information, hospitalization and surgical records, and other data.

[0039] Step S2. Based on NLP and electronic medical record data, feature extraction is performed on the electronic medical record data to obtain multimodal data.

[0040] Specifically, the data is parsed through natural language processing (NLP) technology, and the API is used to convert the data into a structured format for subsequent processing. Then, advanced data preprocessing technology is used to clean the data, convert the format, and extract features to obtain multimodal data including physiological indicators such as blood pressure, heart rate, and blood sugar, as well as the patient's lifestyle, eating habits, and exercise status.

[0041] Step S3. Based on the spatiotemporal hypergraph modeling method, the spatiotemporal relationship between multimodal data is integrated and represented to obtain a dynamically updated dynamic knowledge graph.

[0042] Specifically, the spatiotemporal hypergraph modeling method is used to expand the traditional time series data and construct a composite hyperedge structure with spatiotemporal constraints. The hyperedge definition covers multimodal information such as images, physiological indicators, and medication records to describe complex clinical event chains (for example: MRI abnormality → sudden increase in blood pressure → medication adjustment).

[0043] The spatiotemporal hypergraph modeling method encodes cross-modal data such as abnormal areas of imaging examination (such as the coordinates of infarct lesions in MRI), physiological monitoring data in the same period (blood pressure curve characteristics), and related medication records into three-dimensional hyperedges (spatial coordinates + time window + modal characteristics) through a spatiotemporal alignment algorithm; the hyperedge attenuation function is designed as follows:

[0044] f(t)=e∧(-λΔt);

[0045] Among them, λ is the attenuation coefficient, Δt is the time interval, and the hyperedge weight is dynamically adjusted to reflect the characteristic that the influence of clinical events decays over time. A counterfactual comparison network is constructed, and the masking mechanism is used to simulate the changes in the hypergraph structure after deleting specific treatment events. The difference between P(Y|do(X)) and P(Y|X) is calculated to evaluate the intervention effect.

[0046] The image features are extracted into 128-dimensional embeddings by 3D ResNet; the physiological time-series data are encoded into 64-dimensional vectors by LSTM; the medication records are mapped to 300-dimensional knowledge embeddings through an ontology. A composite representation is generated through a gated fusion layer h = σ(W_g · [v_img||v_phy||v_drug]), and finally the hyperedge feature e = LayerNorm(h + FFN(h)).

[0047] Secondly, entities and their relationships in the knowledge graph are identified through data mining techniques. Concepts and their hierarchical structures in the stroke domain are defined to form an ontology model for unified representation in the knowledge graph.

[0048] Meanwhile, by selecting suitable graph databases (such as Neo4j, GraphDB, etc.) to store and manage the knowledge graph, entity alignment and conflict resolution are implemented for multi-source data fusion, ensuring the integrity and consistency of the knowledge graph. The technology focuses on multi-source data alignment and conflict resolution. The entity alignment problem can also be solved through rule-based and deep learning-based semantic alignment. For data conflicts among them, automatic decision-making is carried out through credibility weighting methods, voting methods, or statistical learning algorithms.

[0049] The knowledge graph should be updated regularly. Through a streaming data processing framework and graph version control, combined with automated error detection techniques such as consistency verification or anomaly detection-based correction, to maintain its timeliness and accuracy; feedback is collected through clinical applications to optimize the construction and use of the knowledge graph and improve its practical application effect. Traditional retrieval models such as BM25 and TF-IDF, or deep learning-based retrieval models (such as BERT, DPR) are selected for information retrieval.

[0050] A generative model (such as GPT, T5, etc.) is selected to combine the retrieved information to generate the interaction relationships between data sources, thereby preprocessing the queries input by users to ensure the normalization and standardization of the input; with the help of tools such as D3.js and Gephi, the graph structure is visually presented to facilitate user understanding and application.

[0051] Step S4. Based on the FHIR data resource, parse and integrate the electronic medical record data to obtain integrated information.

[0052] Specifically, using the FHIR data resource, intelligent integration analysis is carried out on the standardized electronic medical record data and information to obtain integrated information.

[0053] Among them, the FHIR standard defines a series of resources, each representing an aspect of medical information, such as Patient, Observation, Encounter, etc. Each type of resource has a standard structure and fields. The system can automatically parse the standardized electronic medical record data to extract the patient's historical medical records and examinations. The intelligent integration and analysis of information are achieved using FHIR data resources. The system usually uses HTTP requests to obtain FHIR resources. Different FHIR resources have different structures, so the parsing logic needs to be adjusted according to the specific resource type during parsing. For example, the Observation resource may contain different fields and nested structures.

[0054] Following the SMART ON FHIR standard, it can be quickly deployed in existing medical information systems, support interoperability with other medical applications, and improve the applicability and promotion value of the system. The architecture of SMART on FHIR includes: a server that constructs the FHIR API to provide structured access to medical data, such as patient information, medication lists, laboratory results, etc., and stores medical resources (such as patients, observations, etc.); through OAuth 2.0 to ensure secure user login; programming languages and frameworks: select appropriate programming languages (such as JavaScript, Python, Java, etc.) and frameworks (such as React, Angular, Flask, etc.); existing FHIR servers can be used, such as HAPI FHIR, Microsoft Azure API for FHIR, or build a FHIR server by yourself; register your application in the FHIR server or medical system to obtain a client ID and a client secret. These are used for OAuth2.0 authentication (client ID: uniquely identifies the application; client secret: used to verify the identity of the application). Use the obtained access token to call the FHIR API to obtain medical data. Ensure that the access token is included in the request header.

[0055]

[0056] Display the obtained FHIR data, such as patient information, observation records, etc.

[0057]

[0058] After development is completed, deploy the application to an appropriate environment, which can be a cloud platform (such as AWS, Azure, etc.) or a local server. By integrating multi-modal data sources, the system can achieve personalized management of stroke patients, including risk assessment, treatment plan recommendation, and health guidance.

[0059] Step S5. Based on the LLM model combined with the causal reasoning mechanism, construct a causal network for stroke risk.

[0060] Specifically, referring to Figure 2 , Step S5 includes:

[0061] Step S51. Based on the integrated information combined with the causal graph, construct an initial causal network for stroke risk.

[0062] Step S52. Based on the initial causal network for stroke risk and the causal reasoning mechanism embedded in the LLM model with the Transformer architecture, use SEM to quantify the strength of causal relationships and perform intervention simulations on the initial causal network for stroke risk;

[0063] Specifically, based on clinical expert annotations and causal discovery algorithms (such as the PC algorithm, LiNGAM), determine the causal direction between biomarkers (such as blood pressure, blood lipid levels) and stroke risk from multi-modal data; use structural equation modeling (SEM) to quantify the strength of causal effects between variables, and the model expression is:

[0064] Y = β1X1 + β1X1 + ε;

[0065] Among them, Y is the stroke risk score, X1 and X2 are biomarker variables determined in the causal graph, and ε is the error term.

[0066] Storage and update: Store the causal network topology and SEM parameters in the Neo4j graph database to support real-time causal reasoning and dynamic update.

[0067] Step S53. Based on the results of the intervention simulation, obtain counterfactual reasoning data, strengthen the prediction accuracy of the LLM model for stroke-related causal relationships, and obtain a causal network for stroke risk.

[0068] Specifically, the LLM model uses the Transformer architecture, and the embedded causal reasoning mechanism uses the Do-Calculus and counterfactual reasoning methods to obtain counterfactual reasoning data and quantify the causal effects of clinical intervention measures on stroke risk, thereby improving the interpretability and accuracy of the LLM model.

[0069] Among them, the Do-Calculus intervention calculation process includes: defining a clinical intervention operation (such as do(systolic blood pressure ≤ 120 mmHg)) and verifying the identifiability condition; calculating the post-intervention risk distribution based on the causal graph: P(Y|do(X)) = ∑ZP(Y|X,Z)P(Z), where Z is the set of confounding variables; inputting the counterfactual prediction results into the Transformer decoder to generate personalized intervention recommendations.

[0070] The implementation of the counterfactual reasoning engine includes: Treatment event intervention sampling: Generate an intervention sample set for each treatment event \(x_i\). Retain all features except \(x_i\); Hypergraph structure reconstruction: Generate the post-intervention representation through gated graph diffusion convolution: where is the adjacency matrix after deleting the node corresponding to \(x_i\), and \(\theta\) is the pre-trained graph network parameter; Causal effect quantification: \(CE(x_i)=MLP(z)-MLP(z_{cf})\), where MLP is a multi-layer perceptron risk prediction model; Significant factor screening: Set the threshold \(\theta = 0.1\), and screen the set of intervention factors \(F = \{x_i||CE|>\theta\}\) that satisfy \(|CE(x_i)|>\theta\). Input the screened significant factors \(F\) into the Transformer decoder to generate personalized intervention suggestions.

[0071] Function causal_effect_analysis(graph G, treatment_node \(x_i\)):

[0072] Step 1: Create a counterfactual graph structure

[0073] \(G_{cf}=G.mask_nodes(x_i)\)

[0074] Step 2: Hypergraph neural network representation learning

[0075] \(z = GNN.forward(G, feat\_matrix)\)

[0076] \(z_{cf}=GNN.forward(G_{cf}, feat\_matrix)\)

[0077] Step 3: Causal effect quantification

[0078] \(y_{orig}=MLP(z)\)

[0079] \(y_{cf}=MLP(z_{cf})\)

[0080] \(ce = y_{orig}-y_{cf}\)

[0081] Step 4: Significance filtering

[0082] if \(abs(ce)>0.1\):

[0083] return \(\{"treatment":x_i, "effect":ce, "significant":True\}\)

[0084] else:

[0085] return {"treatment": x_i, "effect": ce, "significant": False}

[0086] def batch_counterfactual(graph, treatment_list):

[0087] results = []

[0088] for node in treatment_list:

[0089] res = causal_effect_analysis(graph, node)

[0090] results.append(res)

[0091] return filter(lambda x: x["significant"], results)。

[0092] Step S6. Based on the dynamic knowledge graph and the stroke risk causal network, enhance retrieval and generation through the RAG model to obtain a personalized health report.

[0093] Specifically, based on the constructed dynamic knowledge graph, use Elasticsearch indexing and BM25 / BERT models for fast and accurate retrieval to obtain the latest imaging, monitoring, medication and other data.

[0094] Embed causal reasoning modules (such as Do-Calculus and counterfactual reasoning) into the LLM model based on the Transformer architecture to enhance its ability to model and predict stroke-related causal relationships. Input the retrieved data into the stroke risk causal network constructed based on the causal reasoning mechanism for prediction and intervention simulation to quantify the patient's stroke risk. At the same time, combine the personalized management system of retrieval-augmented generation (RAG) technology, integrate retrieval and generation technologies to improve the intelligent level of the system and the patient management effect.

[0095] Step S7. Based on the personalized report results and the patient's feedback data, optimize the patient's treatment plan and management strategy in real time, and dynamically update the Transformer model parameters.

[0096] Specifically, based on the combination of LLM and causal reasoning, the system automatically generates psychological support and treatment suggestions according to the patient's personalized data and emotional state, provides intelligent consultation and feedback, enhances treatment compliance and the effectiveness of health management, and optimizes the plan through real-time interaction and data analysis. In addition, through real-time feedback and a closed-loop management mechanism, the system continuously updates the knowledge graph and the LLM model parameters, thereby realizing the dynamic optimization of the stroke management strategy.

[0097] To further enhance the adaptive ability of the model, the specific measures for updating the Transformer model parameters in the update module are as follows: The system continuously collects the latest information from multi-modal data sources such as clinical monitoring, imaging examinations, and medication records, as well as the patient's real-time feedback. These data will be used as the training samples for the online update of the model. The update module uses an online learning method to fine-tune the Transformer model. Using the gradient descent method, the system calculates the prediction error through backpropagation and automatically adjusts the learning rate in combination with the feedback of dynamic evaluation (such as performance metrics on the validation set). To prevent overfitting, a model state preservation and parameter freezing strategy is also adopted to ensure the stability of the model during the update process. The updated Transformer model will be immediately applied to the next round of personalized report generation, combined with the retrieval results of the dynamic knowledge graph and the prediction output of the stroke risk causal network, to form a continuous feedback and closed-loop optimization mechanism, continuously improving the accuracy and reliability of the overall risk assessment and management strategy.

[0098] Refer to Figure 3 , Figure 3 FIG. is the structural block diagram of an embodiment of the stroke patient management system based on the multi-modal RAG and LLM models of the present application. The system includes:

[0099] The personal information management module 1 is used to obtain the patient's electronic medical record data;

[0100] The data processing module 2 is used to extract features from the electronic medical record data to obtain multi-modal data;

[0101] The spatio-temporal hypergraph construction module 3 is used to integrate and characterize the spatio-temporal relationship between multi-modal data to form a dynamically updated dynamic knowledge graph;

[0102] The FHIR interface integrated data module 4 is used to integrate the electronic medical record data;

[0103] The causal network construction module 5 embeds the causal reasoning mechanism into the LLM model to obtain the stroke risk causal network;

[0104] The report generation module 6 generates a personalized health report through the RAG module combined with real-time retrieval technology and a generation model;

[0105] Update module 7 utilizes newly obtained clinical data and user feedback to dynamically update the parameters of the dynamic knowledge graph and the Transformer model, achieving continuous optimization of the stroke patient management plan.

[0106] Among them, the data extraction module 2 obtains the electronic medical record data of patients through different medical systems; and converts the electronic medical record data into a structured format through the API. The data processing module performs natural language processing on the text data in the electronic medical record data, image recognition on the image data in the electronic medical record data, and feature extraction on the time series data in the electronic medical record data; and encodes it into a three-dimensional hyperedge structure through a spatio-temporal alignment algorithm.

[0107] This application utilizes large language model (LLM) technology and retrieval-augmented generation (RAG) technology. The system can retrieve information based on the patient's real-time health data (such as blood pressure, heart rate, weight, etc.), evaluate whether the drug plan needs to be adjusted, and automatically generate targeted health management suggestions according to the patient's personalized medication plan, health data, and physiological changes. The suggestions not only include reminding the patient to take medicine on time, but also cover the patient's overall health management needs, such as providing diet suggestions, exercise guidance, arrangements for regular health checks, and lifestyle adjustments. The system converts the patient's query into a structured question, analyzes it through the LLM, and generates relevant answers to provide customized health management suggestions. The system can identify the patient's health needs by analyzing the patient's personalized data and provide corresponding support and suggestions. Based on the evaluation results and the patient's specific situation, the system can recommend a personalized treatment plan. In addition, the system uses charts and dashboards to show the patient the changes in their health data to enhance their sense of participation in their own health management. The patient can also feedback their treatment experience and health changes, and the system will continuously optimize the suggestions and plans according to the feedback. The system needs to regularly evaluate its effectiveness, collect the patient's feedback and treatment results to continuously optimize the risk assessment model, treatment plan, and health guidance content.

[0108] The above are only the embodiments of this application, and do not limit the patent scope of this application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of this application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of this application.

Claims

1. A method for managing stroke patients based on multi-modal RAG and LLM models, characterized in that, Including: Based on the FHIR standard electronic medical record, obtain the standardized electronic medical record data of the patient; Based on NLP and the electronic medical record data, extract features from the electronic medical record data to obtain multimodal data; Based on the spatio-temporal hypergraph modeling method, integrate and characterize the spatio-temporal relationships between the multimodal data to obtain a dynamically updated dynamic knowledge graph; Based on the FHIR data resource, parse and integrate the electronic medical record data to obtain integrated information; Based on the LLM model combined with the causal reasoning mechanism, construct a stroke risk causal network; Based on the dynamic knowledge graph and the stroke risk causal network, enhance retrieval and generation through the RAG model to obtain a personalized health report; Based on the personalized report result and the patient's feedback data, optimize the patient's treatment plan and management strategy in real time.

2. The method according to claim 1, characterized in that, The electronic medical record data includes the patient's basic demographic information, medical history, medication records, retinal images, microcirculation imaging data, lifestyle information, hospitalization and surgical records.

3. The method according to claim 1, wherein The spatio-temporal hypergraph modeling method includes: Expand the time series data in the multimodal data to construct a composite hyperedge structure with spatio-temporal constraints, where the hyperedge is defined as a set of multimodal nodes containing timestamps, used to describe the clinical event chain and its spatio-temporal relationship.

4. The method according to claim 1, characterized in that, The method for constructing a stroke risk causal network includes: Based on the integrated information and combined with a causal graph, construct an initial stroke risk causal network; Based on the initial stroke risk causal network and the causal reasoning mechanism embedded in the LLM model of the Transformer architecture, use SEM to quantify the causal relationship strength and perform intervention simulation on the initial stroke risk causal network; Based on the results of the intervention simulation, obtain counterfactual reasoning data, strengthen the prediction accuracy of the LLM model for stroke-related causal relationships, and obtain the stroke risk causal network.

5. A stroke patient management system based on multi-modal RAG and LLM models, characterized in that, Including: Personal information management module, used to obtain the electronic medical record data of the patient; Data processing module, used to extract features from the electronic medical record data to obtain multimodal data; Spatio-temporal hypergraph construction module, used to integrate and characterize the spatio-temporal relationships between multimodal data to form a dynamically updated dynamic knowledge graph; FHIR interface integrated data module, used to integrate the electronic medical record data; Causal network construction module, embed the causal reasoning mechanism into the LLM model to obtain a stroke risk causal network; Report generation module, generate a personalized health report through the RAG module combined with real-time retrieval technology and generation model; Update module, use the newly obtained clinical data and user feedback to dynamically update the parameters of the dynamic knowledge graph and the Transformer model, and realize the continuous optimization of the stroke patient management plan.

6. The system according to claim 5, wherein The data extraction module obtains the electronic medical record data of the patient through different medical systems; and converts the electronic medical record data into a structured format through the API.

7. The system according to claim 5, wherein The data processing module performs natural language processing on the text data in the electronic medical record data, image recognition on the image data in the electronic medical record data, and feature extraction on the time-series data in the electronic medical record data; and encodes it into a three-dimensional hyperedge structure through a spatio-temporal alignment algorithm.

Images