Multi-modal personalized health management scheme generation method and device based on large model

Abstract

The invention relates to a multi-modal personalized health management scheme generation method and device based on a large model. The method comprises the steps of collecting multi-source health data of a user, collecting and transmitting the multi-source health data in real time through various wearable devices and medical monitoring devices, preprocessing the collected multi-source health data to obtain a standardized physical sign feature matrix, inputting the standardized physical sign feature matrix into a large model to obtain a preliminary health assessment result, performing bidirectional association matching with a pre-constructed deep health management knowledge graph to obtain a knowledge graph verification result, dynamically loading medical entity nodes by adopting a graph attention mechanism, and performing enhancement analysis on the knowledge graph verification result through a graph attention network and a differentiable reasoning engine to obtain a pathological association enhancement vector; and performing multi-scale fusion with the standardized physical sign feature matrix, and generating a personalized health management scheme by using a multi-objective optimization algorithm. By adopting the method, the personalized degree of outputting the personalized health management scheme can be improved.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular, to a method and device for generating a multimodal personalized health management solution based on a large model. Background Art

[0002] With the improvement of people's health awareness, the technology in the field of health management has been continuously developing. Traditional health consultations rely on professionals to provide one-on-one services, which, although personalized, are limited by the limited expert resources and high costs. General health monitoring devices such as smart bracelets can monitor basic physiological indicators, but lack in-depth analysis and personalized suggestions. Health applications collect user data, but the management solutions are simple and lack scientificity. Currently, there are serious deficiencies in personalized services. Although traditional health consultations can provide personalized suggestions, the expert resources are extremely limited, resulting in a very low proportion of people who can obtain professional and in-depth personalized services. Summary of the Invention

[0003] Based on this, in order to solve the above technical problems, it is necessary to provide a method and device for generating a multimodal personalized health management solution based on a large model.

[0004] A method for generating a multimodal personalized health management solution based on a large model, the method comprising: Collect multi-source health data of a user, and collect and transmit it in real time through a variety of wearable devices and medical monitoring devices; the multi-source health data includes personal basic information, lifestyle data, medical record data, and real-time physiological monitoring data; Preprocess the collected multi-source health data to obtain a standardized physical sign feature matrix; Input the standardized physical sign feature matrix into a large model to obtain a preliminary health assessment result; Perform bidirectional association matching on the preliminary health assessment result and a pre-constructed in-depth health management knowledge graph to obtain a knowledge graph verification result; the in-depth health management knowledge graph includes three-layer topological structures of a disease ontology tree, a pharmacological action network, and a physiological mechanism chain; Dynamically load medical entity nodes using a graph attention mechanism, and perform enhanced analysis on the knowledge graph verification result through a graph attention network and a differentiable inference engine to obtain a pathologically associated enhanced vector; Perform multi-scale fusion on the pathologically associated enhanced vector and the standardized physical sign feature matrix, and use a multi-objective optimization algorithm to generate a personalized health management solution.

[0005] In one embodiment, the mobile device is communicatively connected to the server, and the mobile device is wirelessly connected to the wearable device and the medical monitoring device. The wearable device and the medical monitoring device collect multi-dimensional health data of the user. The multi-dimensional health data includes: blood pressure, blood glucose, heart rate, blood oxygen saturation, body temperature, blood lipid, cholesterol, uric acid, electrocardiogram, body temperature matrix, body composition, and respiratory rate.

[0006] In one embodiment, it further includes: processing the multi-source health data through data cleaning, data integration, data transformation, and data reduction to obtain health data that meets the input requirements of the general large model and the data format specification.

[0007] In one embodiment, it further includes: processing the physiological signals in the standardized physical sign feature matrix by using a frequency-divided band convolutional kernel group to obtain time-frequency domain hybrid features; pre-labeling medical entities for the somatosensory text in the standardized physical sign feature matrix, and constructing a symptom-sign joint embedding space through contrastive learning; using a cross-attention mechanism to perform causal association modeling on the motion posture sequence and environmental parameters in the standardized physical sign feature matrix to obtain motion-environment spatio-temporal association features; fusing the time-frequency domain hybrid features, the symptom-sign joint embedding space, and the motion-environment spatio-temporal association features and outputting them as a preliminary health assessment result.

[0008] In one embodiment, it further includes: inputting the standardized physical sign feature matrix into the large model and also outputting abnormal feature markers.

[0009] In one embodiment, it further includes: performing reverse retrospective retrieval on the abnormal feature markers through the disease ontology tree to quickly locate abnormal information; performing drug-side effect conflict detection on the preliminary health assessment result through the pharmacological action network to obtain a risk probability; performing personalized correction on the historical health data of the user through the physiological mechanism chain. In one embodiment, it further includes: dynamically loading medical entity nodes by using a graph attention mechanism, performing probability propagation on the knowledge graph verification result through the disease-symptom-treatment triple path of the graph attention network, and converting medical guideline rules into a constraint loss function through a differentiable inference engine to obtain a pathological association enhanced vector.

[0010] A multi-modal personalized health management solution generation device based on a large model, the method includes: A data acquisition module, configured to acquire multi-source health data of the user, and collect and transmit it in real time through a variety of wearable devices and medical monitoring devices. The multi-source health data includes personal basic information, lifestyle data, medical record data, and real-time physiological monitoring data. A preprocessing module for preprocessing the collected multi-source health data to obtain a standardized physical sign feature matrix; A preliminary evaluation module for inputting the standardized physical sign feature matrix into a large model to obtain a preliminary health assessment result; A verification module for performing two-way association matching between the preliminary health assessment result and a pre-constructed deep health management knowledge graph to obtain a knowledge graph verification result; the deep health management knowledge graph includes a three-layer topological structure of a disease ontology tree, a pharmacological action network, and a physiological mechanism chain; A feature enhancement module for dynamically loading medical entity nodes using a graph attention mechanism and performing enhanced analysis on the knowledge graph verification result through a graph attention network and a differentiable inference engine to obtain a pathological association enhanced vector; An inference module for performing multi-scale fusion of the pathological association enhanced vector and the standardized physical sign feature matrix and generating a personalized health management plan using a multi-objective optimization algorithm.

[0011] A computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented: Collect multi-source health data of a user, which is collected and transmitted in real time through a variety of wearable devices and medical monitoring devices; the multi-source health data includes personal basic information, lifestyle data, medical record data, and real-time physiological monitoring data; Preprocess the collected multi-source health data to obtain a standardized physical sign feature matrix; Input the standardized physical sign feature matrix into a large model to obtain a preliminary health assessment result; Perform two-way association matching between the preliminary health assessment result and a pre-constructed deep health management knowledge graph to obtain a knowledge graph verification result; the deep health management knowledge graph includes a three-layer topological structure of a disease ontology tree, a pharmacological action network, and a physiological mechanism chain; Dynamically load medical entity nodes using a graph attention mechanism and perform enhanced analysis on the knowledge graph verification result through a graph attention network and a differentiable inference engine to obtain a pathological association enhanced vector; Perform multi-scale fusion of the pathological association enhanced vector and the standardized physical sign feature matrix and generate a personalized health management plan using a multi-objective optimization algorithm.

[0012] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Collect multi-source health data of a user, which is collected and transmitted in real time through a variety of wearable devices and medical monitoring devices; the multi-source health data includes personal basic information, lifestyle data, medical record data, and real-time physiological monitoring data; Preprocess the collected multi-source health data to obtain a standardized physical sign feature matrix; Input the standardized physical sign feature matrix into a large model to obtain a preliminary health assessment result; Perform bidirectional association matching between the preliminary health assessment result and a pre-constructed deep health management knowledge graph to obtain a knowledge graph verification result; the deep health management knowledge graph includes a three-layer topological structure of a disease ontology tree, a pharmacological action network, and a physiological mechanism chain; Dynamically load medical entity nodes using a graph attention mechanism, and perform enhanced analysis on the knowledge graph verification result through a graph attention network and a differentiable inference engine to obtain a pathologically associated enhanced vector; Perform multi-scale fusion on the pathologically associated enhanced vector and the standardized physical sign feature matrix, and use a multi-objective optimization algorithm to generate a personalized health management plan.

[0013] The above method and device for generating a multi-modal personalized health management plan based on a large model collect multi-dimensional physiological data through sensors, and then through the inference ability of the large model, on the one hand, make a preliminary prediction of the user's health, and on the other hand, learn the correlation information between multi-dimensional physiological data to obtain a preliminary health assessment result. Then, a deep health management knowledge graph with a three-layer topological structure is established, and the preliminary health assessment result is bidirectionally associated and matched with the result of the deep health management knowledge graph to obtain a knowledge graph verification result, which further ensures the accuracy of the health assessment. Then, based on a graph attention network and a differentiable inference engine, enhanced analysis is performed to obtain a pathologically associated enhanced vector, so as to fuse and enhance the original standardized physical sign feature matrix, and a personalized health management plan is generated using a multi-objective optimization algorithm. The above solution can accurately identify an individual's health status and potential risks, and formulate a highly adaptable exclusive plan for the user. Brief Description of the Drawings

[0014] Figure 1 It is a schematic flowchart of a method for generating a multi-modal personalized health management plan based on a large model in an embodiment; Figure 2 It is a structural block diagram of a device for generating a multi-modal personalized health management plan based on a large model in an embodiment; Figure 3 It is an internal structural diagram of a computer device in an embodiment. Detailed Description of the Embodiment

[0015] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0016] In one embodiment, as Figure 1 shown, a method for generating a multimodal personalized health management solution based on a large model is provided, including the following steps: Step 102, collect multi-source health data of the user, and collect and transmit it in real time through a variety of wearable devices and medical monitoring devices.

[0017] Specifically, long-term health data can be collected, such as: 3 days, 7 days, etc. The mobile device can be a mobile phone, a tablet, etc. The sensor has the ability to collect user health data with high precision. The multi-source health data includes personal basic information, lifestyle data, medical record data, and real-time physiological monitoring data, such as collecting blood pressure, blood sugar, heart rate, blood oxygen saturation, body temperature, and respiratory rate, etc.

[0018] Step 104, preprocess the collected multi-source health data to obtain a standardized physical sign feature matrix.

[0019] The preprocessing includes data cleaning, data integration, data transformation, data reduction, constructing a physical sign feature fusion tensor, and performing spatio-temporal alignment processing on the original data using an adaptive noise suppression algorithm to generate a standardized physical sign feature matrix.

[0020] Step 106, input the standardized physical sign feature matrix into the large model to obtain a preliminary health assessment result.

[0021] The large model can extract cross-modal correlation features through a multi-layer Transformer architecture and obtain a preliminary health assessment result.

[0022] Step 108, perform bidirectional association matching between the preliminary health assessment result and a pre-constructed deep health management knowledge graph to obtain a knowledge graph verification result.

[0023] The deep health management knowledge graph includes three-layer topological structures: a disease ontology tree, a pharmacological action network, and a physiological mechanism chain.

[0024] Specifically, the disease ontology tree is a tree-like hierarchical structure extended based on the SNOMED CT standard, including: Root node: major disease categories Middle layer: disease subcategories, symptoms Leaf nodes: specific diagnostic criteria and treatment guideline entries.

[0025] Among them, the major disease categories represent specific disease types, such as cardiovascular diseases, metabolic diseases, etc. The disease subcategories represent specific diseases, such as coronary heart disease, diabetes, etc. The symptoms refer to the corresponding symptoms of the disease. The specific diagnostic criteria and treatment guideline entries refer to the diagnostic standard conditions corresponding to the disease and the corresponding norms.

[0026] For the disease ontology tree, its associated attributes include: Disease - symptom association weight: The co - occurrence probability statistically calculated from clinical data; Disease - treatment timeliness: Mark the recommended time window for treatment means.

[0027] The dynamic network construction of the pharmacological action network includes: Nodes: Drugs (chemical names), metabolites, target proteins, side effects; Edge relationships: Drug - target action (activation / inhibition, mark the IC50 value); Drug - side effect association.

[0028] The conflict detection rules of the pharmacological action network are as follows: Drug interaction rule base: Construct an antagonistic action logical expression based on the DrugBank database. For example: # Example: Conflict detection logic between warfarin and vitamin K if the current medication list contains "HuaXX" and the new drug contains "vitamin K": return risk level = high - risk, conflict type = pharmacodynamic antagonism The physiological mechanism chain uses a causal graph model to express the cascading effects between physiological parameters. For example: Hyperglycemia → Insulin resistance → Elevated inflammatory factors → Vascular endothelial damage And each chain is accompanied by an evidence level (for example, Grade A: Supported by RCT studies; Grade B: Supported by observational studies).

[0029] In this step, the in - depth health management knowledge graph covers multiple key areas such as the medical knowledge graph, the nutrition knowledge graph, and the pathology knowledge graph. They are interconnected and work synergistically to jointly provide comprehensive and in - depth background knowledge support for personalized health management.

[0030] The medical knowledge graph contains rich information such as medical concepts, disease classifications, symptom manifestations, diagnostic methods, and treatment means. For example, it details the names, causes, pathogenesis, diversity of clinical manifestations of various diseases, as well as the corresponding diagnostic criteria, commonly used treatment drugs, and treatment processes. Taking cardiovascular diseases as an example, the medical knowledge graph will clarify the characteristics of different types of cardiovascular diseases such as coronary heart disease and hypertensive heart disease, including the performance differences of their typical symptoms such as chest pain, palpitations, and dyspnea in different diseases, as well as the specific applications and index interpretations of diagnostic methods such as electrocardiogram and echocardiogram, and at the same time covers various treatment approaches such as drug treatment, interventional treatment, and surgical treatment, as well as the corresponding treatment timing and precautions.

[0031] The nutrition knowledge graph focuses on aspects such as the relationship between the nutritional components of foods, the functions of nutrients, dietary structures, and health. It details the specific contents of nutritional components such as proteins, carbohydrates, fats, vitamins, and minerals contained in various foods, as well as the important roles of these nutrients in physiological processes such as human growth and development, metabolism, and immune regulation. For example, it clarifies the promoting effects of vitamin C on collagen synthesis and antioxidant functions, as well as the possible health problems caused by deficiency or excessive intake. At the same time, the nutrition knowledge graph also deeply studies the reasonable dietary structures of different populations (such as children, adolescents, adults, the elderly, as well as people with different occupations and different health conditions), such as the special needs of athletes for proteins and carbohydrates, and the strict control of carbohydrate intake for diabetic patients, providing a scientific basis for personalized diet recommendations.

[0032] The pathology knowledge graph focuses on the occurrence and development processes of diseases, the mechanisms of pathological changes, as well as the interrelationships and transformations between diseases. It deeply analyzes the complete path of diseases from the action of the cause on the body, through a series of pathophysiological processes, and finally leading to changes in the morphology and function of tissues and organs. Taking cancer as an example, the pathology knowledge graph will elaborate in detail the origin, proliferation, invasion, and metastasis mechanisms of cancer cells, as well as the differences in cell morphology, gene expression, histopathological characteristics, etc. of different types of cancer, providing key information for the early diagnosis, pathological typing, and prognosis assessment of cancer. At the same time, the pathology knowledge graph also reveals the potential connections between diseases, such as the association between chronic inflammation and an increased risk of certain cancers, providing an important reference basis for disease prevention and comprehensive treatment.

[0033] Step 110, adopt the graph attention mechanism to dynamically load medical entity nodes, and perform enhanced analysis on the knowledge graph verification results through the graph attention network and the differentiable inference engine to obtain the pathology-associated enhanced vector.

[0034] In this step, the design rules of the constraint loss function of the differentiable inference engine are as follows: First, convert the clinical guidelines into differentiable mathematical constraints. Taking the example that the systolic blood pressure of hypertensive patients should be < 140 mmHg, the loss function is designed as:

[0035] Among them, SBP i is the systolic blood pressure of the i-th user, and I hypertension is the loss function for hypertensive patients. The above is only the design of blood pressure-related constraints, and the design for other disease-related ones can be referred to, which will not be listed one by one here.

[0036] The generation process of the pathology-associated enhanced vector is as follows: Graph attention network propagation: Calculate the attention weights along the disease-symptom-treatment path:

[0037] Among them, \(h_i\) is the embedding representation of node \(i\), and \(N_i\) is the set of neighbor nodes. represents the learnable weight matrix, and the output path probability is: , where \(u\) represents the starting node of the path and \(v\) represents the target node of the path. represents the attention weight corresponding to the edge \((u, v)\), reflecting the connection from node u to node v The association strength or importance of this connection in the "disease - symptom - treatment" path.

[0038] Constraint loss backpropagation: Joint optimization objective: , where represents the loss function of the graph attention network, is the weight coefficient.

[0039] Update the knowledge graph node embedding through gradient descent:

[0040] Finally, according to the updated knowledge graph, a pathological association enhanced vector is obtained.

[0041] Step 112: Perform multi-scale fusion on the pathological association enhanced vector and the standardized physical sign feature matrix, and use a multi-objective optimization algorithm to generate a personalized health management plan.

[0042] In this embodiment, multi-scale fusion is performed on the pathological association enhanced vector and the standardized physical sign feature matrix at different scales. Specifically, it includes time scale, space scale, and feature scale. For the time scale, a recurrent neural network (RNN) or Transformer is used to capture the long-term change trend of physical sign data. For example, the development of chronic diseases, while a convolutional neural network (CNN) is used to extract short-term fluctuation features and splice them with the pathological vector. For the space scale, the body system is divided, and graph neural network (GNN) modeling is performed on different sub-matrices respectively to capture the inter-system association, and then cross-system fusion is performed with the pathological vector. Finally, based on the pathological association enhanced vector and the standardized physical sign feature matrix, an attention mechanism is used for fusion, so that the information contained in the two is fused and superimposed, which simultaneously covers rich time information and space information. Finally, a personalized health management plan is generated through a multi-objective optimization algorithm. Specifically, the multi-objective optimization algorithm can be the NSGA – II algorithm or a multi-objective optimization evolutionary algorithm.

[0043] In the above method for generating a multi-modal personalized health management solution based on a large model, multi-dimensional physiological data is collected through sensors, and then through the inference ability of the large model, on the one hand, a preliminary prediction of the user's health is made, and on the other hand, the correlation information between multi-dimensional physiological data is learned to obtain a preliminary health assessment result. Then, a deep health management knowledge graph with a three-layer topological structure is established, and the preliminary health assessment result is bidirectionally associated and matched with the result of the deep health management knowledge graph to obtain a knowledge graph verification result, which further ensures the accuracy of the health assessment. Then, enhanced analysis is performed based on a graph attention network and a differentiable inference engine to obtain a pathological correlation enhanced vector, which is used to fuse and enhance the original standardized physical sign feature matrix, and a personalized health management solution is generated using a multi-objective optimization algorithm. The above solution can accurately identify an individual's health status and potential risks, and formulate a highly adaptable exclusive solution for the user.

[0044] In one embodiment, the mobile device is communicatively connected to the server, and the mobile device is wirelessly connected to a wearable device and a medical monitoring device. The wearable device and the medical monitoring device collect multi-dimensional health data of the user; the multi-dimensional health data includes: blood pressure, blood sugar, heart rate, blood oxygen saturation, body temperature, blood lipids, cholesterol, uric acid, electrocardiogram, human body temperature matrix, body composition, and respiratory rate.

[0045] In one embodiment, the multi-source health data is processed through data cleaning, data integration, data transformation, and data reduction to obtain health data that meets the input requirements of the general large model and the data format specification.

[0046] In another embodiment, a group of frequency-divided convolutional kernels is used to process the physiological signals in the standardized physical sign feature matrix to obtain time-frequency domain mixed features; medical entity pre-annotation is performed on the somatosensory text in the standardized physical sign feature matrix, and a symptom-sign joint embedding space is constructed through contrastive learning; a cross-attention mechanism is used to perform causal association modeling on the motion posture sequence and environmental parameters in the standardized physical sign feature matrix to obtain motion-environment spatio-temporal association features; the time-frequency domain mixed features, the symptom-sign joint embedding space, and the motion-environment spatio-temporal association features are feature-fused and then output as a preliminary health assessment result.

[0047] Specifically, the steps for causal association modeling of the motion posture sequence and environmental parameters are as follows: First, the environmental parameters are defined, including temperature (°C), humidity (%), PM2.5 concentration (μg / m³), noise decibel (dB), etc., and then a structural equation model (SEM) is used to quantify the impact of the environment on motion, specifically:

[0048] Among them, , is the fitting coefficient, represents the error term. In this embodiment, an example of using the cross-attention mechanism to perform causal association modeling on the motion posture sequence and environmental parameters in the standardized physical sign feature matrix is as follows: Input: query = the embedding matrix of the motion posture sequence; key = value = the embedding matrix of the environmental parameters.

[0049] Calculation: , where Partially calculate the correlation matrix between the motion posture and the environmental parameters, Perform scale scaling.

[0050] Causal feature extraction: causal_features = attention_weights @ value, where attention_weights represents the attention weights.

[0051] For the above multi-modal fusion, a hierarchical fusion method can be adopted, specifically: Early fusion: Concatenation of time-frequency domain hybrid features and motion-environment features.

[0052] Late fusion: Dynamically weight the symptom-sign embedding and causal features through a gating mechanism:

[0053]

[0054] where σ is the sigmoid function, Wg is the learnable parameter matrix, || represents the concatenation operation, represents the embedding representation of symptoms - signs, represents the causal feature, represents the fused vector.

[0055] In one embodiment, the standardized physical sign feature matrix is input into the large model, and abnormal feature markers are also output. In summary, based on the input of the standardized physical sign feature matrix, the large model outputs a preliminary health assessment result and abnormal feature markers. Among them, the abnormal feature markers are used for quick positioning during two-way association matching, and the preliminary health assessment result is the feedback of the large model for the input, which is reflected by multi-dimensional fusion features.

[0056] In this embodiment, taking the example that the user's nighttime heart rate > 100 beats per minute for 3 consecutive days, the steps of reverse backtracking retrieval of the abnormal feature marker include: 1. Retrieve the node containing "tachycardia" in the disease ontology tree 2. Backtrack along the path of "tachycardia → hyperthyroidism → elevated FT4" 3. Suggestions for triggering thyroid function tests (TSH and FT4 tests) In one embodiment, reverse backtracking retrieval is performed on abnormal feature markers through the disease ontology tree to quickly locate abnormal information; drug-side effect conflict detection is performed on the preliminary health assessment results through the pharmacological action network to obtain a risk probability; personalized correction is performed on the user's historical health data through the physiological mechanism chain.

[0057] In this embodiment, the accuracy of feature extraction can be further improved through bidirectional association matching.

[0058] In a specific example, the input is: preliminary evaluation suggestion "take Hua XX", and the user's current medications include "vitamin K" Detection process: 1. Retrieve the "Hua XX - vitamin K" edge in the pharmacological action network 2. Calculate the conflict risk probability: P = 0.89 (from FAERS data) 3. Output a warning: "The combination of Hua XX and vitamin K will reduce the anticoagulant effect (risk level: high risk)" In one embodiment, a graph attention mechanism is used to dynamically load medical entity nodes, probability propagation is performed on the knowledge graph verification results through the disease-symptom-treatment triple path of the graph attention network, and medical guideline rules are transformed into a constraint loss function through a differentiable inference engine to obtain a pathological association enhancement vector.

[0059] In one embodiment, the personalized health management plan is sent to the user's mobile device to continuously collect the user's health data; the continuously collected health data is used as input data to optimize the deep learning model and update the personalized health management plan.

[0060] Specifically, after the personalized health management plan is formulated, it will be sent to the user's mobile device (such as a smartphone application) in a timely manner for the user to view and follow at any time. At the same time, the mobile device is connected to external health monitoring devices (such as smart bracelets, smart blood pressure monitors, etc.) to continuously collect the user's health data. These data include but are not limited to real-time physiological indicators (such as heart rate, blood pressure, blood oxygen saturation, etc.), exercise data (such as steps, exercise type, exercise duration, etc.), sleep data (such as sleep time, sleep quality, etc.), and the user's diet records (either manually input or through data sharing with diet tracking applications). The data collection process follows strict privacy protection and secure transmission protocols to ensure the confidentiality and integrity of the user's personal health information. The continuously collected health data is sent back to the system as input data for optimizing the deep learning model. The system first preprocesses the newly collected data, including operations such as data cleaning and feature engineering, to ensure that the data quality is suitable for model training. Then, the processed data is input into the deep learning model. By integrating with the previous training data and using incremental learning or online learning algorithms, the model can continuously learn new data patterns and changes in the user's health status. For example, over time, the user's exercise habits may change, and the deep learning model can adjust the assessment and prediction of the user's health condition in a timely manner based on the new exercise data, improving the model's adaptability to the individual health changes of the user. According to the optimization results of the deep learning model, the personalized health management plan will be updated accordingly. If the model detects an improvement in the user's health condition, such as a gradual decrease in weight and stable physiological indicators, the plan may appropriately adjust the diet and exercise recommendations to encourage the user to maintain good habits and gradually increase the exercise intensity or adjust the diet structure to further improve the health level. Conversely, if the model finds an increase in the user's health risks, such as a continuous increase in blood pressure or a serious decline in sleep quality, the plan will promptly adjust the intervention measures, which may include increasing the monitoring frequency, recommending stricter diet control, or adjusting the medication treatment plan, and reminding the user to seek medical attention for further examination in a timely manner. The updated plan will be pushed to the user's mobile device again, and at the same time, detailed explanations and suggestions for the plan adjustment will be provided to the user to help the user understand the reasons for the changes and actively cooperate with the new health management plan. Through this continuous cycle of data collection, model optimization, and plan update, the health management system can continuously adapt to the user's health changes and provide more accurate, personalized, and effective health management services.

[0061] In one embodiment, the health management knowledge graph is updated according to the continuously collected health data.

[0062] Specifically, the system deeply integrates the newly collected health data with the existing historical data and conducts comprehensive processing using data analysis techniques. Through data mining algorithms, potential patterns and regularities in health data are discovered, such as the correlation between different exercise patterns and physiological index changes, and the relationship between dietary intake and blood glucose fluctuations. Additionally, machine learning models are used to classify and predict the data, identifying key features and trends that may indicate health risks or changes in the state, such as the potential connection between consecutive days of insufficient sleep and increased fatigue and reduced work efficiency, and the correlation between specific food combinations and changes in digestive function.

[0063] Then, based on the data analysis results, the nodes and relationships in the health management knowledge graph are dynamically updated. In terms of node updates, if new physiological index change patterns or disease risk factors are discovered, the system will add them as new nodes to the knowledge graph. For example, if a new deficiency of a certain trace element is found to be related to a specific health problem, corresponding trace element nodes and related health problem nodes will be created in the knowledge graph. At the same time, for existing nodes, such as disease nodes, their attribute information will be updated according to the latest medical research results and user data feedback, such as the latest diagnostic criteria for diseases and improvements in treatment methods. In terms of relationship updates, based on the feature associations and causal relationships revealed by data analysis, the relationships in the knowledge graph are adjusted or added. For example, if a significant association is found between long-term high-intensity exercise and an increased risk of joint injury, the system will establish a new relationship from the high-intensity exercise node to the joint injury node in the knowledge graph and mark the association strength and related influencing factors. Another example is when it is determined that a new nutritional intervention measure has a positive impact on a specific chronic disease, the relationship between nutrition and the disease is updated to clarify the specific mechanism of action and scope of application of this nutritional intervention.

[0064] The following is illustrated with a specific embodiment: Input data: User A: Fasting blood glucose 9.2 mmol / L, BMI 28, exercise frequency 2 times / week, environmental data: average daily temperature 25 °C, PM2.5 at the place of residence = 65 μg / m³ Processing flow: 1. Preliminary evaluation by the large model: Diabetes risk probability 87%, exercise deficiency marked 2. Verification by the knowledge graph: Match the "type 2 diabetes" node in the disease ontology tree Detect no conflict in current medications in the pharmacological action network Infer the "hyperglycemia → insulin resistance" path in the physiological mechanism chain 3. Generation of solutions: Dietary advice: Carbohydrate intake < 150 g / day Exercise advice: Indoor aerobic exercise (to avoid the impact of PM2.5) Medication plan: Increase XX dimethyl by 500 mg bid It should be understood that although Figure 1 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 at least a part of the steps in can include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential either, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0065] In one embodiment, as Figure 2 shown, a multi-modal personalized health management plan generation device based on a large model is provided, including: a data collection module 202, a preprocessing module 204, a preliminary evaluation module 206, a verification module 208, a feature enhancement module 210, and an inference module 212, where: The data collection module is used to collect multi-source health data of users, and collect and transmit it in real time through a variety of wearable devices and medical monitoring devices; the multi-source health data includes personal basic information, lifestyle data, medical record data, and real-time physiological monitoring data; The preprocessing module is used to preprocess the collected multi-source health data to obtain a standardized physical sign feature matrix; The preliminary evaluation module is used to input the standardized physical sign feature matrix into the large model to obtain a preliminary health evaluation result; The verification module is used to perform two-way association matching between the preliminary health evaluation result and a pre-constructed in-depth health management knowledge graph to obtain a knowledge graph verification result; the in-depth health management knowledge graph includes a three-layer topological structure of a disease ontology tree, a pharmacological action network, and a physiological mechanism chain; The feature enhancement module is used to dynamically load medical entity nodes by using a graph attention mechanism, and perform enhanced analysis on the knowledge graph verification result through a graph attention network and a differentiable inference engine to obtain a pathologically associated enhanced vector; The inference module is used to perform multi-scale fusion on the pathologically associated enhanced vector and the standardized physical sign feature matrix, and use a multi-objective optimization algorithm to generate a personalized health management plan.

[0066] For the specific limitations of the device for generating a multimodal personalized health management solution based on a large model, reference can be made to the limitations of the method for generating a multimodal personalized health management solution based on a large model in the foregoing text, which will not be elaborated here. Each module in the above-mentioned device for generating a multimodal personalized health management solution based on a large model can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0067] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structural diagram can be as Figure 3 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for generating a multimodal personalized health management solution based on a large model. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the outer shell of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0068] Those skilled in the art can understand that Figure 3 the structure shown in

[0069] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0070] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method in the above embodiment.

[0071] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0072] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0073] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for generating a multi-modal personalized health management solution based on a large model, characterized in that, The method includes: Collecting multi-source health data of the user, which is collected and transmitted in real time through a variety of wearable devices and medical monitoring devices; the multi-source health data includes personal basic information, lifestyle data, medical record data, and real-time physiological monitoring data; Preprocessing the collected multi-source health data to obtain a standardized physical sign feature matrix; Inputting the standardized physical sign feature matrix into a large model to obtain a preliminary health assessment result; Performing bidirectional association matching between the preliminary health assessment result and a pre-constructed deep health management knowledge graph to obtain a knowledge graph verification result; the deep health management knowledge graph includes a three-layer topological structure of a disease ontology tree, a pharmacological action network, and a physiological mechanism chain; Dynamically loading medical entity nodes using a graph attention mechanism, and performing enhanced analysis on the knowledge graph verification result through a graph attention network and a differentiable inference engine to obtain a pathology-associated enhanced vector; Performing multi-scale fusion on the pathology-associated enhanced vector and the standardized physical sign feature matrix, and using a multi-objective optimization algorithm to generate a personalized health management plan.

2. The method according to claim 1, wherein The mobile device is communicatively connected to the server, the mobile device is wirelessly connected to the wearable device and the medical monitoring device, and the wearable device and the medical monitoring device collect multi-dimensional health data of the user; The multi-dimensional health data includes: blood pressure, blood glucose, heart rate, blood oxygen saturation, body temperature, blood lipid, cholesterol, uric acid, electrocardiogram, human body temperature matrix, body composition, and respiratory rate.

3. The method according to claim 1, characterized in that Preprocessing the collected multi-source health data includes: Processing the multi-source health data through data cleaning, data integration, data transformation, and data reduction to obtain health data that meets the input requirements of the general large model and the data format specification.

4. The method according to claim 1, characterized in that, Inputting the standardized physical sign feature matrix into the large model to obtain a preliminary health assessment result, including: Processing the physiological signals in the standardized physical sign feature matrix using a frequency-divided convolutional kernel group to obtain time-frequency domain hybrid features; Performing pre-annotation of medical entities on the somatosensory text in the standardized physical sign feature matrix, and constructing a symptom-sign joint embedding space through contrastive learning; Using a cross-attention mechanism to perform causal association modeling on the motion posture sequence and environmental parameters in the standardized physical sign feature matrix to obtain motion-environment spatio-temporal association features; Performing feature fusion on the time-frequency domain hybrid features, the symptom-sign joint embedding space, and the motion-environment spatio-temporal association features and outputting them as a preliminary health assessment result.

5. The method according to claim 4, characterized in that, The method further includes: Inputting the standardized physical sign feature matrix into the large model, and also outputting an abnormal feature mark.

6. The method according to claim 5, characterized in that Performing bidirectional association matching between the preliminary health assessment result and the pre-constructed deep health management knowledge graph includes: Performing reverse retrospective retrieval on the abnormal feature mark through the disease ontology tree to quickly locate abnormal information; Performing drug-side effect conflict detection on the preliminary health assessment result through the pharmacological action network to obtain a risk probability; Performing personalized correction on the user's historical health data through the physiological mechanism chain.

7. The method according to claim 6, wherein Dynamically load medical entity nodes using the graph attention mechanism, and perform enhanced analysis on the knowledge graph verification results through the graph attention network and the differentiable inference engine to obtain a pathologically associated enhanced vector, including: Dynamically load medical entity nodes using the graph attention mechanism, perform probability propagation on the knowledge graph verification results through the disease-symptom-treatment triple path of the graph attention network, and convert medical guideline rules into constraint loss functions through the differentiable inference engine to obtain a pathologically associated enhanced vector.

8. An apparatus for generating a multimodal personalized health management solution based on a large model, characterized in that, The device includes: A data acquisition module for acquiring multi-source health data of users, which is collected and transmitted in real time through a variety of wearable devices and medical monitoring devices; the multi-source health data includes personal basic information, lifestyle data, medical record data, and real-time physiological monitoring data; A preprocessing module for preprocessing the acquired multi-source health data to obtain a standardized physical sign feature matrix; A preliminary evaluation module for inputting the standardized physical sign feature matrix into a large model to obtain a preliminary health evaluation result; A verification module for performing two-way association matching between the preliminary health evaluation result and a pre-constructed deep health management knowledge graph to obtain a knowledge graph verification result; the deep health management knowledge graph includes a three-layer topological structure of a disease ontology tree, a pharmacological action network, and a physiological mechanism chain; A feature enhancement module for dynamically loading medical entity nodes using the graph attention mechanism, and performing enhanced analysis on the knowledge graph verification results through the graph attention network and the differentiable inference engine to obtain a pathologically associated enhanced vector; An inference module for performing multi-scale fusion of the pathologically associated enhanced vector and the standardized physical sign feature matrix, and generating a personalized health management plan using a multi-objective optimization algorithm.

Images