A method and system for generating health management plans based on multimodal knowledge graph

Abstract

The present invention relates to a method and system for generating a health management plan based on a multimodal knowledge graph, and relates to the field of health management technology, wherein the system comprises: a graph establishment module, used to establish a multimodal knowledge graph; an information collection module, used to collect multimodal user health data of users to be managed; a risk assessment module, used to determine health risks based on the multimodal knowledge graph and the multimodal user health data of users to be managed; a health management module, used to generate a health management plan based on health risks, which has the advantage of improving the quality of health management.

Description

Technical Field

[0001] The present invention relates to the field of health management technology, and in particular to a method and system for generating a health management plan based on a multimodal knowledge graph. Background Art

[0002] With the rapid economic development and the acceleration of industrialization, social culture and living habits have changed greatly. Problems such as unreasonable dietary structure and insufficient physical activity have become increasingly prominent. In fact, lifestyle and environmental factors are the main factors affecting the progression, remission and even onset of diseases. Whether it is a chronic disease or a major disease, early prevention will greatly reduce the incidence of the disease, reduce the pain of patients after treatment, and effectively relieve medical pressure.

[0003] Health is the most important thing for everyone. However, when people are not sick, they rarely pay attention to their health. On the one hand, there is a lack of awareness of health management, and few people take the initiative to manage themselves or urge others to manage their health. On the other hand, it is unclear how to manage health, and there is a lack of a health management system and health management methods to manage health. At present, many people will go to medical institutions for physical examinations regularly to understand their physical condition, but only some people can insist on regular physical examinations. At the same time, offline physical examinations also have problems such as limited medical resources and lack of long-term dynamic management and tracking. With the improvement of network applications and transmission speeds, the upgrading of intelligent sensing technology, and the improvement of the information infrastructure of medical institutions, mobile medical care has developed rapidly, and medical and health products have emerged. However, there are also some problems with current medical and health management products. Current medical and health products only do simple data collection, with a single data source, and cannot establish a complete health file for users. Although data collection has been done, it is impossible to provide users with real-time feedback and health management suggestions based on their health files. It just blindly adds insurance, physical examinations and other businesses, and does not use these data, and does not provide users with substantial services.

[0004] Therefore, there is a need to provide a method and system for generating a health management plan based on a multimodal knowledge graph to improve the quality of health management. Summary of the invention

[0005] The present invention provides a health management program generation system based on a multimodal knowledge graph, comprising: a graph establishment module, used to establish a multimodal knowledge graph; an information collection module, used to collect multimodal user health data of users to be managed; a risk assessment module, used to determine health risks based on the multimodal knowledge graph and the multimodal user health data of the users to be managed; and a health management module, used to generate a health management program based on the health risks.

[0006] Furthermore, the graph establishment module establishes a multimodal knowledge graph, including: obtaining sample data corresponding to each health risk, wherein the sample data includes questionnaire data, non-medical images, medical test data and physiological test data of multiple sample users; for each of the health risks, based on the sample data corresponding to the health risk, extracting the questionnaire features, non-medical image features, medical test features and physiological test features corresponding to the health risk; based on the questionnaire features, non-medical image features, medical test features and physiological test features corresponding to each of the health risks, establishing the multimodal knowledge graph.

[0007] Furthermore, the information collection module collects multimodal user health data of the user to be managed, including: obtaining real-time physiological detection data of the user to be managed; determining a first risk value of the user to be managed for at least one of the health risks based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph; judging whether to perform multimodal collection based on the first risk value of the user to be managed for at least one of the health risks; when it is determined to perform multimodal collection, generating a medical questionnaire, non-medical image prompt information and a medical detection plan based on the first risk value of the user to be managed for at least one of the health risks; and obtaining non-medical images, medical detection data and physiological detection data of the user to be managed based on the medical questionnaire, non-medical image prompt information and medical detection plan.

[0008] Furthermore, the information acquisition module determines the first risk value of the user to be managed in at least one of the health risks based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph, including: obtaining the background information of each of the sample users; for the background information of any two of the sample users, determining the first background similarity between the two sample users; clustering a plurality of sample users based on the first background similarity between the two sample users by a K-means clustering algorithm to determine a plurality of sample user clusters; for each of the sample user clusters, grouping a plurality of sample users included in the sample user cluster based on the health risk corresponding to the sample user, determining a sample user group corresponding to the sample user cluster, and determining a sample user group corresponding to the sample user cluster based on the health risk corresponding to each of the sample user groups. health risks of the user to be managed; obtaining the background information of the user to be managed; determining a second background similarity between the background information of the user to be managed and the background information of the cluster center of each of the sample user clusters; determining similar sample user clusters based on the second background similarity between the background information of the user to be managed and the background information of the cluster center of each of the sample user clusters; determining a first relevant parameter of each health risk corresponding to the user to be managed based on the health risks corresponding to each of the similar sample user clusters; sorting multiple health risks according to the first relevant parameter of each health risk corresponding to the user to be managed to obtain a sorting result; according to the sorting result, based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph, determining a first risk value of the user to be managed in at least one of the health risks.

[0009] Furthermore, the information acquisition module obtains the real-time physiological detection data of the user to be managed, including: determining the second correlation parameter between any two physiological characteristics based on the physiological detection data of multiple sample users included in the similar sample user cluster; for each of the physiological characteristics, determining the current abnormal parameter corresponding to the physiological characteristic based on the physiological detection data corresponding to the physiological characteristic in the current detection period; judging whether there is at least one abnormal physiological characteristic based on the current abnormal parameter corresponding to each of the physiological characteristics; and performing data correction on the physiological detection data corresponding to the abnormal physiological characteristic in the current detection period based on the second correlation parameter between any two physiological characteristics and the physiological detection data corresponding to each non-abnormal physiological characteristic in the current detection period through a data correction model.

[0010] Furthermore, the information collection module generates a medical questionnaire, non-medical image prompt information and a medical detection plan based on the first risk value of at least one of the health risks of the user to be managed, including: taking the health risk whose first risk value is greater than the preset first risk value threshold as a candidate health risk; based on the sample medical questionnaire, non-medical image acquisition location and medical detection item corresponding to each of the candidate health risks, generating the medical questionnaire, non-medical image prompt information and medical detection plan for the user to be managed.

[0011] Furthermore, the risk assessment module determines the health risk based on the multimodal knowledge graph and the multimodal user health data of the user to be managed, including: determining the questionnaire characteristics, non-medical image characteristics, medical detection characteristics and physiological detection characteristics corresponding to the user to be managed based on the multimodal user health data of the user to be managed; determining the health risk through a risk assessment model based on the questionnaire characteristics, non-medical image characteristics, medical detection characteristics and physiological detection characteristics corresponding to the user to be managed and the multimodal knowledge graph.

[0012] Furthermore, the health management module generates a health management plan based on the health risks, including: obtaining the dietary preference information of the user to be managed; generating a dietary management plan based on the dietary preference information of the user to be managed and the health risks through a first plan generation model, wherein the health management plan includes the dietary management plan.

[0013] Furthermore, the health management module generates a health management plan based on the health risks, including: obtaining the exercise preference information of the user to be managed; generating an exercise management plan based on the exercise preference information of the user to be managed and the health risks through a second plan generation model, wherein the health management plan includes the exercise management plan.

[0014] The present invention provides a method for generating a health management plan based on a multimodal knowledge graph, comprising: establishing a multimodal knowledge graph; collecting multimodal user health data of users to be managed; determining health risks based on the multimodal knowledge graph and the multimodal user health data of the users to be managed; and generating a health management plan based on the health risks.

[0015] Compared with the prior art, the method and system for generating a health management plan based on a multimodal knowledge graph provided in this specification has at least the following beneficial effects:

[0016] 1. By collecting multimodal user health data, more comprehensive data support is provided for determining the health risks of users to be managed. By establishing a multimodal knowledge graph, data structuring is achieved, which improves the efficiency of determining the health risks of users to be managed. On this basis, the most suitable health management plan can be selected from the preset plan library according to the health risks and needs of users to be managed. Personalized plans can be generated according to the user's physical condition, preferences and living habits, thereby improving the quality of health management;

[0017] 2. Sort the first relevant parameters of each health risk of the managed users from large to small according to the first relevant parameters, and first determine the first risk value of the health risk of the managed users with the highest ranking based on the real-time physiological detection data of the managed users and the multimodal knowledge graph, so as to improve the efficiency of health risk screening;

[0018] 3. The first solution generation model and the second solution generation model can achieve automatic evolution and optimization, continuously improve the intelligence level of the first solution generation model and the second solution generation model, and improve the quality of generated diet management plans and health management plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0020] Figure 1 It is a module diagram of a health management program generation system based on a multimodal knowledge graph shown in an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of a multimodal knowledge graph shown in an embodiment of the present application;

[0022] Figure 3 is a flow chart of determining a first risk value of at least one health risk of a user to be managed shown in an embodiment of the present application;

[0023] Figure 4 This is a flowchart of a method for generating a health management plan based on a multimodal knowledge graph shown in one embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly introduces the drawings required for describing the embodiments.

[0025] Figure 1 is a module diagram of a health management program generation system based on a multimodal knowledge graph shown in an embodiment of the present application, such as Figure 1As shown, a health management program generation system based on a multimodal knowledge graph includes a graph building module, an information collection module, a risk assessment module and a health management module.

[0026] The graph building module can be used to build a multimodal knowledge graph.

[0027] include:

[0028] Obtain sample data corresponding to each health risk, wherein the sample data includes questionnaire data, non-medical images, medical test data and physiological test data (for example, blood pressure value, blood sugar value, heart rate value, blood oxygen saturation, body temperature value and respiratory rate, etc.) of multiple sample users. Non-medical images may be images acquired using non-medical image acquisition equipment. Medical test data may include medical image test data (for example, nuclear magnetic resonance images, computer tomography images, X-ray images and ultrasound imaging, etc.) and non-image test data (for example, blood test data, urine test data, stool test data, endocrine test data and immunology test data, etc.);

[0029] For each health risk, based on the sample data corresponding to the health risk, the questionnaire features, non-medical image features, medical detection features and physiological detection features corresponding to the health risk are extracted, wherein the questionnaire features may include answer keyword features for multiple questionnaire questions, the non-medical image features may include color features, morphological features, etc., the medical detection features may include medical image features (for example, the grayscale distribution and statistical characteristics of pixels or voxels in medical images, parameters reflecting the fine structure and characteristics of the internal tissue, such as texture roughness, directionality, contrast, etc.) and non-image detection features (for example, blood detection features, urine detection features, feces detection features, endocrine detection features and immunological detection features, etc.), and the physiological detection features may include blood pressure features, blood sugar features, heart rate features, blood oxygen saturation features, body temperature features and respiratory rate features, etc.;

[0030] A multimodal knowledge graph is established based on the questionnaire characteristics, non-medical image features, medical test characteristics and physiological test characteristics corresponding to each health risk.

[0031] For example only, Figure 2 is a schematic diagram of a multimodal knowledge graph shown in an embodiment of the present application, such as Figure 2As shown, the multimodal knowledge graph may include nodes representing health risks, questionnaire features corresponding to health risks, non-medical image features, medical detection features, and physiological detection features. Taking coronary heart disease as an example, the questionnaire features corresponding to coronary heart disease include affirmative answers to questionnaire questions 1 and 2, no non-medical image features, medical detection features including pixel density or brightness of coronary arteries and their surroundings in X-ray computed tomography (CT) or magnetic resonance imaging (MRI), medical detection features may also include typical waveforms of myocardial ischemia or myocardial infarction that may appear in electrocardiograms (ECGs), elevated myocardial enzyme spectra in blood tests, and physiological detection features including abnormal respiratory rate and abnormal heartbeat.

[0032] The information collection module can be used to collect multimodal user health data of the users to be managed.

[0033] Specifically include:

[0034] Acquire real-time physiological detection data of the user to be managed. For example, the real-time physiological detection data of the user to be managed may be acquired through various means such as mobile devices and sensors;

[0035] Determine a first risk value of at least one health risk of the user to be managed based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph;

[0036] Determining whether to perform multimodal collection based on a first risk value of at least one health risk of the user to be managed;

[0037] When it is determined that multimodal acquisition is to be performed, a medical questionnaire, non-medical image prompt information and a medical detection plan are generated based on a first risk value of at least one health risk of the user to be managed;

[0038] Based on the medical questionnaire, non-medical image prompt information and medical detection scheme, non-medical images, medical detection data and physiological detection data of the user to be managed are obtained.

[0039] Specifically, the risk prediction model can be used to determine the first risk value of at least one health risk of the user to be managed based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph. The larger the first risk value, the greater the possibility that the user to be managed has the health risk. The risk prediction model can be a Long Short-Term Memory (LSTM) model.

[0040] Figure 3 is a flowchart of determining a first risk value of at least one health risk of a user to be managed shown in an embodiment of the present application, such as Figure 3As shown, in some embodiments, the information acquisition module determines a first risk value of at least one health risk of the user to be managed based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph, including:

[0041] Obtain background information for each sample user;

[0042] For the background information of any two sample users, determining a first background similarity between the two sample users;

[0043] Clustering the multiple sample users based on the first background similarity between two sample users by using a K-means clustering algorithm to determine multiple sample user clusters;

[0044] For each sample user cluster, based on the health risks corresponding to the sample users, a plurality of sample users included in the sample user cluster are grouped to determine a sample user group corresponding to the sample user cluster, and based on the health risks corresponding to each sample user group, a health risk corresponding to the sample user cluster is determined;

[0045] Obtain background information of the user to be managed;

[0046] Determine a second background similarity between the background information of the user to be managed and the background information of the cluster center of each sample user cluster;

[0047] Determine similar sample user clusters based on a second background similarity between background information of the user to be managed and background information of the cluster center of each sample user cluster;

[0048] Based on the health risks corresponding to each similar sample user cluster, determine a first relevant parameter corresponding to each health risk of the user to be managed;

[0049] Sorting the multiple health risks according to the first relevant parameter of each health risk of the user to be managed, and obtaining the sorting result;

[0050] According to the sorting result, based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph, a first risk value of at least one health risk of the user to be managed is determined.

[0051] Specifically, a first background similarity between two sample users can be determined based on the background information of the two sample users through a similarity determination model, and a second background similarity between the background information of the user to be managed and the background information of the cluster center of each sample user cluster can be determined based on the similarity determination model, wherein the similarity determination model can be a Convolutional Neural Network (CNN) model.

[0052] The sample user clusters whose second background similarity is greater than a preset second background similarity threshold may be used as similar sample user clusters.

[0053] For example, the first correlation parameter may be calculated based on the following formula:

[0054]

[0055] in, is the first relevant parameter of the hth health risk, is the maximum value of the second background similarity corresponding to multiple sample user clusters corresponding to the h-th health risk, is the total number of health risks, is the maximum value of the second background similarities corresponding to multiple sample user clusters corresponding to the kth health risk.

[0056] The first relevant parameters of each health risk of the users to be managed are sorted from large to small according to the first relevant parameters. First, based on the real-time physiological detection data of the users to be managed and the multimodal knowledge graph, the first risk value of the health risk of the users to be managed that is ranked high is determined to improve the efficiency of health risk screening.

[0057] In some embodiments, the information collection module obtains real-time physiological detection data of the user to be managed, including:

[0058] Determine a second correlation parameter between any two physiological characteristics based on the physiological detection data of a plurality of sample users included in the similar sample user cluster;

[0059] For each physiological characteristic, based on the physiological detection data corresponding to the physiological characteristic in the current detection period, determine the current abnormal parameter corresponding to the physiological characteristic;

[0060] Based on the current abnormal parameters corresponding to each physiological characteristic, determining whether there is at least one abnormal physiological characteristic;

[0061] The data correction model is used to correct the physiological detection data corresponding to the abnormal physiological characteristics in the current detection period based on the second correlation parameter between any two physiological characteristics and the physiological detection data corresponding to each non-abnormal physiological characteristic in the current detection period, wherein the data correction model can be a convolutional neural network (CNN) model.

[0062] For example, the current anomaly parameter can be calculated based on the following formula:

[0063]

[0064] in, is the current abnormal parameter corresponding to the i-th physiological feature, is the ith physiological feature in the current detection cycle. The value at a time point, The length of the current detection cycle.

[0065] When the current abnormal parameter corresponding to a certain physiological feature is greater than a preset current abnormal parameter threshold, the physiological feature is an abnormal physiological feature.

[0066] For example, the second correlation parameter may be calculated based on the following formula:

[0067]

[0068]

[0069] in, is the second correlation parameter between the ith physiological characteristic and the jth physiological characteristic, is the second correlation parameter between the ith physiological feature and the jth physiological feature corresponding to the nth sample user in the similar sample user cluster, is the total number of sample users in the similar sample user cluster, is the ith physiological feature of the nth sample user in the similar sample user cluster in the historical detection period The value at a time point, is the jth physiological feature of the nth sample user in the similar sample user cluster in the historical detection period The value at a time point, is the length of the historical detection cycle.

[0070] When there is a user to be managed whose first risk value of at least one health risk is greater than a preset first risk value threshold, it is determined to perform multimodal acquisition.

[0071] In some embodiments, the information collection module generates a medical questionnaire, non-medical image prompt information and a medical detection plan based on a first risk value of at least one health risk of the user to be managed, including:

[0072] Taking the health risks whose first risk value is greater than the preset first risk value threshold as candidate health risks;

[0073] Based on the sample medical questionnaire, non-medical image acquisition location and medical test items corresponding to each candidate health risk, a medical questionnaire, non-medical image prompt information and medical test plan for the user to be managed are generated.

[0074] The risk assessment module can be used to determine health risks based on the multimodal knowledge graph and the multimodal user health data of the users to be managed.

[0075] Specifically include:

[0076] Based on the multimodal user health data of the user to be managed, determine the questionnaire characteristics, non-medical image characteristics, medical detection characteristics and physiological detection characteristics corresponding to the user to be managed;

[0077] Health risks are determined through the risk assessment model based on the questionnaire characteristics, non-medical image features, medical test characteristics, physiological test characteristics and multimodal knowledge graph corresponding to the managed users.

[0078] The risk assessment model can achieve automatic evolution and optimization, continuously improving the intelligence level of the risk assessment model and improving the accuracy and efficiency of determining health risks.

[0079] The health management module can be used to generate health management plans based on health risks.

[0080] Specifically include:

[0081] Acquire dietary preference information of the user to be managed, wherein the dietary preference information may include ingredients that the user prefers to eat;

[0082] Generate a diet management plan based on the diet preference information and health risks of the user to be managed through a first plan generation model, wherein the health management plan includes a diet management plan;

[0083] Acquire exercise preference information of the user to be managed, wherein the diet preference information may include the type of exercise preferred by the user;

[0084] A sports management plan is generated based on the sports preference information and health risks of the user to be managed through the second plan generation model, wherein the health management plan includes the sports management plan, and the first plan generation model and the second plan generation model can both be Generative Adversarial Networks (GAN) models.

[0085] The first solution generation model and the second solution generation model can achieve automatic evolution and optimization, continuously improve the intelligence level of the first solution generation model and the second solution generation model, and improve the quality of generated diet management plans and health management plans.

[0086] Figure 4 is a flowchart of a method for generating a health management plan based on a multimodal knowledge graph shown in an embodiment of the present application, such as Figure 4 As shown, a method for generating a health management plan based on a multimodal knowledge graph may include the following steps.

[0087] Step 410, establishing a multimodal knowledge graph;

[0088] Step 420, collecting multimodal user health data of the user to be managed;

[0089] Step 430, determining health risks based on the multimodal knowledge graph and the multimodal user health data of the user to be managed;

[0090] Step 440, generating a health management plan based on the health risks.

[0091] A method for generating a health management plan based on a multimodal knowledge graph can be executed by a health management plan generation system based on a multimodal knowledge graph. For more descriptions of a method for generating a health management plan based on a multimodal knowledge graph, please refer to the relevant description of a health management plan generation system based on a multimodal knowledge graph, which will not be repeated here.

[0092] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A health management plan generation system based on multimodal knowledge graph, characterized in that: include: Graph building module, used to build multimodal knowledge graph; An information collection module, used to collect multimodal user health data of the user to be managed; A risk assessment module, used to determine health risks based on the multimodal knowledge graph and the multimodal user health data of the user to be managed; Health management module, used to generate health management plans based on health risks; The graph building module builds a multimodal knowledge graph, including: Obtaining sample data corresponding to each health risk, wherein the sample data includes questionnaire data, non-medical images, medical test data, and physiological test data of multiple sample users; For each health risk, based on the sample data corresponding to the health risk, the questionnaire characteristics, non-medical image characteristics, medical detection characteristics and physiological detection characteristics corresponding to the health risk are extracted; Establish a multimodal knowledge graph based on the questionnaire characteristics, non-medical image characteristics, medical test characteristics and physiological test characteristics corresponding to each health risk; The information collection module collects multimodal user health data of the user to be managed, including: Based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph, determine the first risk value of at least one health risk of the user to be managed, and judge whether to perform multimodal collection; If yes, based on the first risk value of at least one health risk of the user to be managed, generate a medical questionnaire, non-medical image prompt information and a medical test plan, and obtain non-medical images, medical test data and physiological test data of the user to be managed; The information collection module determines a first risk value of at least one health risk of the user to be managed based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph, including: For the background information of any two sample users, determining a first background similarity between the two sample users; Clustering the multiple sample users based on the first background similarity between two sample users by using a K-means clustering algorithm to determine multiple sample user clusters; For each sample user cluster, based on the health risks corresponding to the sample users, a plurality of sample users included in the sample user cluster are grouped to determine a sample user group corresponding to the sample user cluster, and based on the health risks corresponding to each sample user group, a health risk corresponding to the sample user cluster is determined; Determine a second background similarity between the background information of the user to be managed and the background information of the cluster center of each sample user cluster; Determine similar sample user clusters based on a second background similarity between background information of the user to be managed and background information of the cluster center of each sample user cluster; Based on the health risks corresponding to each similar sample user cluster, determine a first relevant parameter corresponding to each health risk of the user to be managed; Sorting the multiple health risks according to the first relevant parameter of each health risk of the user to be managed, and obtaining the sorting result; According to the sorting result, based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph, a first risk value of at least one health risk of the user to be managed is determined.

2. According to claim 1, a health management program generation system based on multimodal knowledge graph is characterized in that: The information collection module obtains the real-time physiological detection data of the user to be managed, including: Determining a second correlation parameter between any two physiological characteristics based on the physiological detection data of a plurality of sample users included in the similar sample user cluster; For each of the physiological characteristics, based on the physiological detection data corresponding to the physiological characteristic in the current detection period, determine the current abnormal parameter corresponding to the physiological characteristic; Based on the current abnormal parameters corresponding to each of the physiological characteristics, determining whether there is at least one abnormal physiological characteristic; The data correction model is used to correct the physiological detection data corresponding to the abnormal physiological characteristics in the current detection period based on the second correlation parameter between any two physiological characteristics and the physiological detection data corresponding to each non-abnormal physiological characteristic in the current detection period.

3. According to claim 2, a health management program generation system based on multimodal knowledge graph is characterized in that: The information collection module generates a medical questionnaire, non-medical image prompt information and a medical detection plan based on the first risk value of the user to be managed in at least one of the health risks, including: Taking the health risks whose first risk value is greater than the preset first risk value threshold as candidate health risks; Based on the sample medical questionnaire, non-medical image acquisition location and medical test items corresponding to each candidate health risk, a medical questionnaire, non-medical image prompt information and medical test plan for the user to be managed are generated.

4. A health management program generation system based on a multimodal knowledge graph according to any one of claims 1 to 3, characterized in that: The risk assessment module determines the health risk based on the multimodal knowledge graph and the multimodal user health data of the user to be managed, including: Based on the multimodal user health data of the user to be managed, determining the questionnaire characteristics, non-medical image characteristics, medical detection characteristics and physiological detection characteristics corresponding to the user to be managed; The health risk is determined through a risk assessment model based on the questionnaire characteristics, non-medical image characteristics, medical detection characteristics and physiological detection characteristics corresponding to the user to be managed and the multimodal knowledge graph.

5. A health management program generation system based on a multimodal knowledge graph according to any one of claims 1 to 3, characterized in that: The health management module generates a health management plan based on the health risk, including: Acquire dietary preference information of the user to be managed; A diet management plan is generated based on the diet preference information of the user to be managed and the health risk through a first plan generation model, wherein the health management plan includes the diet management plan.

6. A health management program generation system based on a multimodal knowledge graph according to any one of claims 1 to 3, characterized in that: The health management module generates a health management plan based on the health risk, including: Acquire the sports preference information of the user to be managed; A sports management plan is generated based on the sports preference information of the user to be managed and the health risk through a second plan generation model, wherein the health management plan includes the sports management plan.

7. A method for generating a health management plan based on a multimodal knowledge graph, characterized in that: include: Build a multimodal knowledge graph; Collect multimodal user health data of the user to be managed; Determining health risks based on the multimodal knowledge graph and the multimodal user health data of the user to be managed; Based on the health risks, generate a health management plan; Build a multimodal knowledge graph, including: Obtaining sample data corresponding to each health risk, wherein the sample data includes questionnaire data, non-medical images, medical test data, and physiological test data of multiple sample users; For each of the health risks, based on the sample data corresponding to the health risk, extract the questionnaire characteristics, non-medical image characteristics, medical detection characteristics and physiological detection characteristics corresponding to the health risk; Establishing the multimodal knowledge graph based on the questionnaire characteristics, non-medical image characteristics, medical detection characteristics and physiological detection characteristics corresponding to each of the health risks; Collect multimodal user health data of the user to be managed, including: Acquiring real-time physiological detection data of the user to be managed; Determine a first risk value of the user to be managed in at least one of the health risks based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph; Based on a first risk value of at least one of the health risks of the user to be managed, determining whether to perform multimodal acquisition; When it is determined to perform multimodal acquisition, a medical questionnaire, non-medical image prompt information and a medical detection plan are generated based on a first risk value of at least one of the health risks of the user to be managed; Based on the medical questionnaire, non-medical image prompt information and medical detection scheme, obtaining non-medical images, medical detection data and physiological detection data of the user to be managed; Determining a first risk value of the user to be managed in at least one of the health risks based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph includes: Obtaining background information of each of the sample users; For the background information of any two of the sample users, determining a first background similarity between the two sample users; Clustering the plurality of sample users based on the first background similarity between two of the sample users by using a K-means clustering algorithm to determine a plurality of sample user clusters; For each of the sample user clusters, based on the health risks corresponding to the sample users, a plurality of sample users included in the sample user cluster are grouped to determine a sample user group corresponding to the sample user cluster, and based on the health risks corresponding to each of the sample user groups, a health risk corresponding to the sample user cluster is determined; Obtaining background information of the user to be managed; Determining a second background similarity between the background information of the user to be managed and the background information of the cluster center of each of the sample user clusters; Determining similar sample user clusters based on a second background similarity between the background information of the user to be managed and the background information of the cluster center of each of the sample user clusters; Based on the health risk corresponding to each of the similar sample user clusters, determining a first related parameter of each of the health risks corresponding to the user to be managed; Sorting the multiple health risks according to the first relevant parameter of each health risk of the user to be managed, and obtaining a sorting result; According to the ranking result, based on the real-time physiological detection data of the user to be managed and the multimodal knowledge graph, a first risk value of the user to be managed in at least one of the health risks is determined.

Images