A proactive health stratification management method and electronic devices based on multimodal fusion

Abstract

This application, in the field of intelligent medical technology, provides a proactive health stratification management method and electronic device based on multimodal fusion. The method includes: acquiring dynamic and static data of the person to be assessed; inputting the physiological time-series data and the behavioral time-series data into a spatiotemporal feature extraction module to extract the temporal dependencies in the dynamic data, and using a self-attention mechanism to identify abnormal feature sequences based on the temporal dependencies to obtain dynamic risk features; inputting the genetic data and the environmental exposure data into a static feature extraction module to extract static risk features based on a multi-head attention mechanism; fusing the obtained dynamic risk features and the static risk features using a dynamic gating mechanism to obtain fused features; passing the fused features to a fully connected layer to obtain health risk level results at different time scales, and outputting a health stratification management file based on the health risk level results.

Description

Technical Field

[0001] This application relates to the field of intelligent medical technology, and in particular to a proactive health hierarchical management method and electronic device based on multimodal fusion. Background Technology

[0002] "Proactive health" differs from the previous "passive medical care." It is a new health management model that uses new technologies such as informatics and omics to promote healthy lifestyles, effectively monitor and intervene in health risk factors, and promote the health of all people.

[0003] With increasing health demands, a growing burden of chronic diseases, and insufficient public health awareness, current medical development faces multiple challenges. In the field of scientific research, accurately assessing individual health risks and effectively managing them in a stratified manner is a pressing issue. Existing health risk assessment models often neglect individual differences and dynamic factors, failing to effectively classify and stratify risk levels, and lacking personalized health management records. This results in individuals undergoing health checkups being unaware of their own health status each year and unsure how to choose appropriate health checkup items.

[0004] Therefore, how to develop targeted health stratification management records based on individual differences has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a proactive health stratification management method and electronic device based on multimodal fusion to solve the following technical problem in the prior art: how to develop targeted health stratification management files for individual differences.

[0006] According to a first aspect, this application provides a proactive health stratification management method based on multimodal fusion. The proactive health stratification management method utilizes a pre-trained dynamic risk identification model to perform proactive health stratification management based on the predicted risk level of the person to be assessed. The dynamic risk identification model includes a spatiotemporal feature extraction module, a static feature extraction module, and a fully connected layer. The proactive health stratification management method includes:

[0007] Acquire dynamic and static data of the person to be evaluated, wherein the dynamic data includes physiological time-series data and behavioral time-series data, and the static data includes genetic data and environmental exposure data;

[0008] The physiological time-series data and the behavioral time-series data are input into the spatiotemporal feature extraction module to extract the temporal dependencies in the dynamic data, and the abnormal feature sequence is identified based on the temporal dependencies using a self-attention mechanism to obtain dynamic risk features.

[0009] The genetic data and the environmental exposure data are input into the static feature extraction module, and static risk features are extracted based on the multi-head attention mechanism.

[0010] The obtained dynamic risk features and static risk features are fused using a dynamic gating mechanism to obtain fused features;

[0011] The fusion features are passed to the fully connected layer to obtain health risk level results at different time scales, and a health stratification management file is output based on the health risk level results.

[0012] Optionally, after passing the fusion features to the fully connected layer to obtain the health risk level result, the process includes:

[0013] The health risk level results and the corresponding dynamic risk features are time-series aligned.

[0014] Based on the aligned dynamic and static risk features, driving features are constructed, and the health risk level results are used as current state features to input the pre-constructed risk transition probability model to obtain the risk transition probability of the next time step.

[0015] Based on the health risk level results and the risk transfer probability of the next time step corresponding to the health risk level results, output health guidance recommendations are determined.

[0016] Optionally, the risk transfer probability model is constructed as follows:

[0017] Obtain multiple historical health risk level result sequences and the dynamic and static risk characteristics corresponding to each historical health risk level result;

[0018] Using the dynamic risk features and the static risk features as driving features, and the historical health risk results as state features as inputs to the initialized risk transfer probability model, a risk transfer probability matrix is ​​constructed.

[0019] Optionally, the spatiotemporal feature extraction module includes a hierarchical recurrent neural network;

[0020] The step of inputting the physiological time-series data and the behavioral time-series data into the spatiotemporal feature extraction module includes:

[0021] Extract time series data at at least two time scales from the physiological time series data and the behavioral time series data;

[0022] The time series data at different time scales are input into the recurrent neural network at the corresponding level of the time scale. The time series cumulative features are captured layer by layer and recursively calculated layer by layer from the bottom network to the top network of the hierarchical recurrent neural network to obtain the dependency features of the time series.

[0023] Optionally, the extraction of time series data at at least two time scales from the physiological time series data and the behavioral time series data includes:

[0024] Extract the first dynamic time series data at the first time scale from the physiological time series data and the behavioral time series data;

[0025] The first dynamic time series data is decomposed to extract physiological rhythm data;

[0026] The physiological rhythm data is quantized over time to obtain second dynamic time-series data at a second time scale.

[0027] Optionally, the hierarchical recurrent neural network includes at least a bottom-level network and a top-level network;

[0028] The process involves inputting the time series data at different time scales into a recurrent neural network at a level corresponding to that time scale. The process then involves capturing cumulative temporal features layer by layer and performing recursive calculations from the bottom to the top layer of the hierarchical recurrent neural network to obtain the dependency features of the time series.

[0029] The first dynamic time series data is input into the underlying network, and the first time series cumulative feature of the first dynamic time series feature under the first time scale is captured in the hidden layer of the underlying network.

[0030] Align the first time-series cumulative feature and the second dynamic time-series data at the second time scale;

[0031] The fusion weights of the first time-cumulative feature and the second dynamic time-series data are calculated using a cross-attention mechanism, and the first time-cumulative feature and the second dynamic time-series data are fused based on the fusion weights to obtain multi-scale fusion features;

[0032] The multi-scale fusion features are input into the top-level network to capture the second time-series cumulative features at the second time scale as the dynamic risk features.

[0033] Optionally, the fusion of the obtained dynamic risk features and the static risk features using a dynamic gating mechanism to obtain fused features includes:

[0034] The importance of the abnormal feature sequence of the dynamic risk characteristics is obtained in real time;

[0035] The gating weight is determined based on the importance level, wherein the importance level is positively correlated with the gating weight;

[0036] The dynamic risk characteristics and the static risk characteristics are fused based on the gating weight.

[0037] Optionally, the method of fusing the obtained dynamic risk features and the static risk features using a dynamic gating mechanism to obtain fused features further includes:

[0038] Obtain the medical examination results and / or medical records of the assessors;

[0039] Extract entity features from the physical examination results and / or medical records;

[0040] The entity features are interactively analyzed with the static risk features and the dynamic risk features respectively to determine the interaction relationship between the entity features and the static risk features and the dynamic risk features respectively;

[0041] The gating weights are adjusted based on the said interaction relationship.

[0042] Optionally, the genetic data and the environmental exposure data are input into the static feature extraction module to extract static risk features based on a multi-head attention mechanism;

[0043] The genetic data were analyzed to obtain a polygenic risk score;

[0044] The spatiotemporal trajectory of the person to be evaluated within a preset time period is obtained, and the spatiotemporal trajectory is matched with the environmental exposure data of the corresponding area to obtain the cumulative environmental exposure.

[0045] The multi-gene risk score and the cumulative environmental exposure are input into a static feature extraction module based on a Transformer structure, and a multi-head attention mechanism is used to extract key risk factors as the static risk features.

[0046] According to a second aspect, embodiments of this application provide a proactive health stratification management device based on multimodal fusion. The device utilizes a pre-trained dynamic risk identification model to perform proactive health stratification management based on the predicted risk level of the person to be assessed. The dynamic risk identification model includes a spatiotemporal feature extraction module, a static feature extraction module, and a fully connected layer. The proactive health stratification management device includes:

[0047] The acquisition module is used to acquire dynamic and static data of the person to be evaluated. The dynamic data includes physiological time-series data and behavioral time-series data, and the static data includes genetic data and environmental exposure data.

[0048] The first feature extraction module is used to input the physiological time series data and the behavioral time series data into the spatiotemporal feature extraction module, extract the temporal dependencies in the dynamic data, and use a self-attention mechanism to identify abnormal feature sequences based on the temporal dependencies to obtain dynamic risk features.

[0049] The second feature extraction module is used to input the genetic data and the environmental exposure data into the static feature extraction module and extract static risk features based on the multi-head attention mechanism.

[0050] The feature fusion module is used to fuse the obtained dynamic risk features and the static risk features using a dynamic gating mechanism to obtain fused features;

[0051] The prediction module is used to pass the fused features to the fully connected layer to obtain the health risk level results at different time scales, and output a health stratification management file based on the health risk level results.

[0052] According to a third aspect, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the proactive health hierarchical management method based on multimodal fusion as described in any one of the first aspects above.

[0053] According to a fourth aspect, this application provides an electronic device including one or more processors, one or more memories, and one or more computer program instructions, which, when executed by the processor, implement the proactive health hierarchical management method based on multimodal fusion as described in any of the first aspects above.

[0054] This invention proposes a proactive health stratification management method based on multimodal fusion. This method utilizes a pre-trained dynamic risk identification model to proactively stratify health based on the predicted risk levels of individuals to be assessed. The dynamic risk identification model includes a spatiotemporal feature extraction module, a static feature extraction module, and a fully connected layer. Dynamic data, such as physiological and behavioral time-series data, are input into the spatiotemporal feature extraction module to extract temporal dependencies within the dynamic data. An abnormal feature sequence is identified based on these temporal dependencies using a self-attention mechanism. The dynamic data from different modalities, including physiological and behavioral data, are initially fused to obtain dynamic risk features. These dynamic risk features are designed to assess acute risks and their corresponding risk levels. Static data, such as genetic and environmental exposure data, are input into the static feature extraction module. Static risk features are extracted based on a multi-head attention mechanism. The static data from different modalities, including genetic and environmental exposure data, are then initially fused to obtain dynamic risk features. This fusion approach, which integrates static risk features to assess long-term cumulative / chronic risks and their corresponding risk levels, utilizes a dynamic gating mechanism to dynamically assign gating weights to static and dynamic risk features based on their relative importance. This gating mechanism further integrates these two different modalities to adjust the contribution of a particular risk feature, adapting to the individual's current situation and adjusting feature weights in real time. It balances the different levels of health risks arising from long-term risk factors and short-term physiological dynamic changes, enabling real-time dynamic and tiered health risk assessment and management for individuals. It comprehensively considers multi-dimensional data such as an individual's genetic information, lifestyle habits, and environmental factors, achieving accurate assessment and tiered management of individual health risks from multiple dimensions, including cardiovascular disease, tumors, functional health, and immunity. This results in a comprehensive health risk management profile covering both long-term and short-term risks throughout the entire life cycle. Attached Figure Description

[0055] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0056] Figure 1 Schematic diagrams of the structure of the dynamic risk identification model provided for some embodiments of this application;

[0057] Figure 2 A flowchart illustrating a proactive health hierarchical management method based on multimodal fusion provided for some embodiments of this application;

[0058] Figure 3 Schematic diagram of a proactive health hierarchical management device based on multimodal fusion provided for some embodiments of this application

[0059] Figure 4 Schematic diagrams of electronic devices provided for some embodiments of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] Based on this, this application proposes a proactive health stratification management method based on multimodal fusion. This method utilizes a pre-trained dynamic risk identification model to proactively stratify the health risk of individuals being assessed, taking into account multi-dimensional data such as genetic information, lifestyle habits, and environmental factors. It achieves accurate assessment and stratified management of individual health risks from multiple dimensions, including cardiovascular disease, tumors, functional health, and immunity. Furthermore, it develops health stratification management files covering both short-term and long-term life-cycle aspects, providing individuals with complete health stratification management records.

[0062] like Figure 1 As shown, the dynamic risk identification model includes a spatiotemporal feature extraction module, a static feature extraction module, and a fully connected layer, wherein, as... Figure 2 As shown, the proactive health stratification management method may include the following steps:

[0063] S10. Acquire dynamic and static data of the person to be evaluated. In this embodiment, the dynamic data includes physiological time-series data and behavioral time-series data, and the static data includes genetic data and environmental exposure data. In this embodiment, the dynamic data of the person to be evaluated can be data generated by the person in daily life, such as physiological data collected through wearable devices, including electrocardiogram data, blood oxygen data, blood pressure data, respiratory rate, etc., and behavioral data of the person to be evaluated, such as exercise data, exercise type, sleep data, diet score, and other daily life data. In this embodiment, dynamic data can be collected and recorded through wearable devices, and the dynamic data can be divided into physiological time-series data and behavioral time-series data by the collection frequency and recording frequency.

[0064] Static data can include genetic data and environmental exposure data. Genetic data can be obtained by processing gene sequencing data. Specifically, GATK (Genome Analysis Toolkit) can be used to parse VCF files. When parsing VCF files, quality control conditions QUAL ≥ 20 and DP ≥ 10 are set. QUAL represents the quality value of the variant site; a higher value indicates a higher reliability of the variant site. Setting QUAL ≥ 20 can filter out low-quality variant sites. DP represents sequencing depth, i.e., the number of times the site is covered by sequencing. DP ≥ 10 ensures sufficient sequencing depth for the site, avoiding the inclusion of erroneous variants due to sequencing errors in the analysis. Association analysis is performed on single nucleotide polymorphisms (SNPs) in the VCF files. Significant gene loci are screened; these significant gene loci have a strong association with the trait or disease under study. A multi-gene risk score is constructed based on these significant gene loci to comprehensively consider the effects of multiple gene loci and to assess an individual's genetic risk.

[0065] S20. The physiological time-series data and the behavioral time-series data are input into the spatiotemporal feature extraction module to extract the temporal dependencies in the dynamic data. A self-attention mechanism is then used to identify abnormal feature sequences based on these temporal dependencies to obtain dynamic risk features. In this embodiment, physiological and behavioral time-series data often have short-term effects on health. For example, the ratio of low-frequency to high-frequency components in an electrocardiogram (ECG) signal, or a sudden increase or decrease in heart rate, are examples of short-term physiological time-series data that often indicate cardiac risk. Behavioral data also influences physiological data; for example, an increase in heart rate during exercise is normal, while an increase during sleep may indicate an abnormality. Therefore, in this embodiment, a recurrent neural network can be used to construct the spatiotemporal feature extraction module to capture the temporal dependencies in the dynamic data. A self-attention mechanism is then used to identify abnormal feature sequences based on these temporal dependencies. Dynamic data from different modalities, such as physiological and behavioral data, are initially fused by extracting abnormal feature sequences to obtain dynamic risk features. For example, the recurrent neural network can be any one of an RNN model, a GRU model, or an LSTM model. In this embodiment, dynamic data is diverse and has interrelated influences. For example, physiological data includes electrocardiogram (ECG) data, blood oxygen saturation data, blood pressure data, and respiratory rate; behavioral data includes exercise data, exercise type, sleep data, and diet scores; and behavioral data such as exercise data, exercise type, and sleep data have short-term effects on physiological data like ECG, blood oxygen saturation data, blood pressure data, and respiratory rate, or dietary scores, exercise frequency, and exercise intensity have long-term effects on these physiological data. Therefore, in this embodiment, a bidirectional LSTM model can be used to process time-series data with short-term and long-term dependencies. Bidirectional LSTM combines forward and backward LSTM, enabling it to simultaneously capture the sequential dependencies of time-series data. In health data, for example, changes in heart rate may be influenced by factors such as exercise and rest over a previous period; bidirectional LSTM can effectively capture this temporal correlation. The self-attention mechanism helps the model focus on important segments within the time series. In dynamic data, there may be some abnormal physiological data changes, such as a sudden increase or decrease in heart rate. Self-attention mechanisms can highlight these abnormal segments, making the model pay more attention to these key information, thereby improving the accuracy of prediction.

[0066] S30. The genetic data and environmental exposure data are input into the static feature extraction module, and static risk features are extracted based on a multi-head attention mechanism. In this embodiment, the genetic data is analyzed to obtain a multi-gene risk score; the spatiotemporal trajectory of the person to be assessed within a preset time period is obtained, and the spatiotemporal trajectory is matched with the environmental exposure data of the corresponding area to obtain the cumulative environmental exposure; the multi-gene risk score and the cumulative environmental exposure are input into the static feature extraction module based on the Transformer structure, and key risk factors are extracted as the static risk features using a multi-head attention mechanism.

[0067] For example, environmental exposure data may include air or water pollutants, such as PM2.5 levels in the air or TDS values ​​in water, and may also include specific individuals being tested, such as workers in dusty workplaces who need to be monitored for their environmental exposure to dusty environments. In this embodiment, the spatiotemporal trajectory of an individual is obtained through mobile phone GPS positioning, and this trajectory is matched with gridded environmental exposure data to calculate the individual's residence time ti and the environmental concentration Ci in different grids. Then, the individual's cumulative exposure D is calculated according to the formula D = ∑(ti × Ci).

[0068] The static feature extraction module can employ a Transformer-based feature extraction network. Multigene risk scores and the cumulative environmental exposure are input into this module, and a multi-head attention mechanism extracts key risk factors. Static data from different modalities, such as genetic data and environmental exposure data, are initially fused using this multi-head attention mechanism to extract corresponding key risk factors (attention scores), which serve as the static risk features. Genetic data and environmental exposure data are relatively stable and can reflect an individual's genetic background and long-term environmental exposure, having a long-term impact on their health risks. For example, multigene risk scores can reflect an individual's genetic susceptibility and can predict, to some extent, the potential risk of an individual developing a certain disease. For instance, in the prediction of cardiovascular disease, individuals carrying specific cardiovascular-related gene variants may have higher PRS scores, meaning they have a relatively higher risk of cardiovascular disease. Long-term environmental exposure has a subtle but profound impact on an individual's health; certain environmental factors may increase the risk of specific diseases. For example, long-term exposure to high concentrations of PM2.5 increases the risk of respiratory and cardiovascular diseases.

[0069] In addition, some environmental pollutants may affect gene expression or mutation. For example, long-term exposure to paint, furniture making, and even smoking and alcoholism may lead to the expression of carcinogenic genes and increase the risk of cancer. Therefore, the average accumulation of specific environmental pollutants in an individual can be statistically analyzed to obtain the exposure data as the static environmental characteristics of the individual.

[0070] A multi-head attention mechanism is employed, which learns features in parallel from different representation subspaces to identify key risk factors. For example, the APOEε4 gene is closely associated with the risk of Alzheimer's disease; long-term exposure to high concentrations of PM2.5 increases the risk of respiratory and cardiovascular diseases. The multi-head attention mechanism can highlight the role of these key risk factors in risk identification. As a core component of the Transformer architecture, the multi-head attention mechanism allows the model to learn features in parallel across different representation subspaces. Specifically, it projects the input feature vector into multiple low-dimensional subspaces, each called a "head." Within each head, the model determines the correlation between different positions in the input sequence by calculating attention scores between the query, key, and value. In the static feature extraction module, the use of multi-head attention means that the model learns features simultaneously in multiple different subspaces. Each head has its own independent query, key, and value projection matrices, which transform the input static risk feature vector into different representations. Then, an attention score is calculated for each head, and the values ​​are weighted and summed based on the attention scores to obtain the output of each head. Finally, the outputs of multiple heads are concatenated and a linear transformation is applied to obtain the final output. By learning features in parallel across multiple subspaces, the multi-head attention mechanism can capture richer information from the input data. Different heads can focus on different aspects of the data, thereby improving the model's ability to identify key risk factors. For example, one head may focus more on certain specific gene loci in genetic data, while another head may focus more on certain specific pollutants in environmental exposure data. In the static feature extraction module, the multi-head attention mechanism can highlight key risk factors closely related to disease risk by calculating attention scores.

[0071] S40. The obtained dynamic risk features and static risk features are fused using a dynamic gating mechanism to obtain fused features.

[0072] S50. The fusion features are passed to the fully connected layer to obtain the health risk level results at different time scales, and a health stratification management file is output based on the health risk level results.

[0073] In this embodiment, the dynamic gating mechanism is implemented through the following formula:

[0074] gt =σ(W g [h static h temporal ])

[0075] Where σ is the sigmoid function, W g Let h be the weight matrix. static and h temporal These are hidden representations of static and dynamic risk features, respectively. A gating mechanism is used to fuse these two different modalities of risk features, dynamically adjusting their contribution weights (i.e., fusion weights). The importance of static and dynamic risk features for risk identification may vary depending on health status and data conditions. For example, in the early stages of some diseases, dynamic risk features (such as recent physiological data changes) may be more important; while in assessing the risk of certain hereditary diseases, static risk features (such as gene scores) may play a dominant role. The dynamic gating mechanism can adaptively adjust the weights of the two features according to specific circumstances, making the model more flexible and accurate.

[0076] In one embodiment, since static risk characteristics are often relatively stable, the fusion feature between dynamic and static risk characteristics can be adjusted by varying the importance of dynamic risk characteristics. Therefore, the dynamic gating mechanism can be calculated using the following formula:

[0077] X t =α×S + (1﹣α)×T

[0078] Among them, X t ' represents the fusion feature, S represents the dynamic risk feature, T represents the static risk feature, and α represents the gating weight.

[0079] In this embodiment, the importance of the abnormal feature sequence of the dynamic risk feature is acquired in real time; a gating weight is determined based on the importance, wherein the importance is positively correlated with the gating weight; and the dynamic risk feature and the static risk feature are fused based on the gating weight. For example, when a patient's heart rate suddenly increases, the RR interval becomes abnormal, or blood oxygen saturation decreases, the importance of the abnormal feature sequence increases, and the value of α is increased so that the dynamic risk identification model outputs a high short-term risk level or an acute risk, indicating that immediate medical attention is required.

[0080] Furthermore, a dynamic gating mechanism can also be used to achieve risk classification management in different scenarios. Specifically, the dynamic gating mechanism is used to fuse the obtained dynamic risk features and the static risk features to obtain fused features. This fused feature further includes: obtaining the physical examination results and / or medical records of the assessors; extracting entity features from the physical examination results and / or medical records; performing interactive analysis on the entity features with the static risk features and the dynamic risk features respectively to determine the interaction relationship between the entity features and the static risk features and the dynamic risk features; and adjusting the gating weight based on the interaction relationship. In this embodiment, entity features may include diseases, examination / test results, etc. In this embodiment, electronic medical records are used as an example for illustration.

[0081] BioBERT, a pre-trained language model in the biomedical field, can be used to identify disease entities in electronic medical records and map them to International Classification of Diseases (ICD) codes, such as mapping "Type II diabetes" to ICD code E11. Constructing a medical knowledge graph can link diagnostic records with laboratory indicators, such as associating "taking metformin" with trends in glycated hemoglobin (HbA1c), thereby better understanding the diagnosis and treatment process of diseases. Temporal features such as disease stages (new onset / progression / stability) and complication networks can be extracted from electronic medical records. Determining disease stages helps understand the progression of the disease, while complication networks can demonstrate the interrelationships between diseases, providing a basis for comprehensive disease management and treatment.

[0082] The entity features are interactively analyzed with both the static and dynamic risk features. In this embodiment, LASSO (Least Absolute Shrinkage and Selection Operator) can be used to perform regression screening on the entity features with the static and dynamic risk features, respectively, to identify significant interaction terms. These significant interaction terms determine the degree of interaction between the entity features in the medical records or physical examination reports of the person being assessed and the static or other entity features, and the gating weights are adjusted based on this degree of interaction. For example, if there is a significant interaction between the entity features in the medical records or physical examination reports and the static risk features, it indicates a high probability of long-term risk for the person being assessed, such as genetic diseases, tumors, or the early stages of certain diseases, requiring continuous monitoring. Therefore, the gating weight of the static risk features is increased to allow the dynamic risk prediction model to focus more on high-risk diseases that may develop further in the long term. If there is a significant interaction between the entity features in the medical records or physical examination reports and the dynamic risk features, there may be a risk of myocardial infarction, heart failure, coronary heart disease, etc., which may occur in the short term. Therefore, the gating weight of the dynamic risk features is increased. This allows dynamic risk prediction models to focus more on diseases with high short-term risk levels. It enables dynamic adjustments to health stratification management strategies based on individual real-time conditions, achieving early detection, early diagnosis, and early treatment, thus providing individuals with more precise and comprehensive health services.

[0083] In addition, dynamic monitoring can be implemented according to individual needs. For example, in acute event monitoring (such as myocardial infarction early warning), the dynamic gating mechanism automatically increases the weight of dynamic risk characteristics (such as blood pressure changes per minute). In chronic disease management (such as atrial fibrillation), the weight of static risk characteristics (such as genetic susceptibility) can be maintained at a greater than the preset weight.

[0084] Furthermore, through a dynamic gating mechanism, both dynamic and static risk features can be used simultaneously to perform comprehensive short-term and long-term detection. For example, in atrial fibrillation risk management, static risk feature T can include age, gender, and atrial fibrillation-related gene markers; dynamic risk feature S can include heart rate variability per minute, RR interval, and blood oxygen saturation. When the dynamic risk feature shows an abnormal RR interval (>500ms), α = 0.8 (dynamic risk feature weight is 80%); when the static risk feature contains atrial fibrillation susceptibility genes (such as KCNQ1 mutation), α = 0.4 (static risk feature weight is 60%). The fused feature is then input into the dynamic risk identification model, triggering an early warning rule. By adjusting feature weights in real time, the technical problem of traditional models being unable to balance long-term risk factors and short-term physiological changes is solved.

[0085] Dynamic data, such as physiological and behavioral time-series data, are input into the spatiotemporal feature extraction module. Temporal dependencies in the dynamic data are extracted, and anomaly sequences are identified based on these dependencies using a self-attention mechanism. This initial fusion of dynamic data from different modalities, including physiological and behavioral data, yields dynamic risk features. These dynamic risk features are designed to assess acute risk and its corresponding risk level. Static data, such as genetic and environmental exposure data, are input into the static feature extraction module. Static risk features are extracted using a multi-head attention mechanism. This initial fusion of static data from different modalities, including genetic and environmental exposure data, yields static risk features designed to assess long-term cumulative / chronic risk and its corresponding risk level. Finally, a dynamic gating mechanism is used to combine dynamic and static risk features. The relative importance of risk features is used to dynamically allocate gating weights to static and dynamic risk features. These two different modalities are then merged again through a gating mechanism to adjust the contribution of a certain risk feature. This allows for real-time adjustment of feature weights to adapt to the individual's current situation and balance different levels of health risks brought about by long-term risk factors and short-term physiological dynamic changes. Consequently, it enables real-time dynamic and hierarchical health risk assessment and management for individuals to be evaluated. It comprehensively considers multi-dimensional data such as an individual's genetic information, lifestyle habits, and environmental factors to accurately assess and hierarchically manage individual health risks from multiple dimensions, including cardiovascular and cerebrovascular diseases, tumors, functional and immune systems, achieving a full life-cycle health risk management file that includes both long-term and short-term risks.

[0086] Dynamic risk identification models often assess the current acute health risk probability and long-term risk probability of individuals based on their current dynamic and static data. However, for individuals in the early stages of a disease, a medium timescale may be needed to monitor disease progression. Alternatively, regular check-up or health examination strategies may be developed based on current dynamic and static data (e.g., environmental exposure). Therefore, it may be necessary to predict future health risk levels. To capture changes in the risk status of individuals or systems at different points in time, such as a shift from low risk to high risk, or vice versa, and to better establish health stratification management files and the timing and recommendations for diagnosis / examination, this embodiment can use a risk transfer model to predict future risk evolution patterns based on the current output health risk level results. Simultaneously, the input features of the dynamic risk identification model, including dynamic and static risk features, are closely related to changes in risk status and can influence external variables affecting risk status changes, thus providing sufficient data support. Therefore, in order to accurately predict future risk evolution patterns, this embodiment uses both the input and output features of the dynamic risk identification model as driving variables for risk transfer to predict the transfer probability at the next time step. By deeply coupling the input and output features of the dynamic risk identification model, it overcomes the limitation of traditional Markov models that rely solely on historical states; and by fusing multimodal data, it reveals the causal relationship of risk evolution. Specifically:

[0087] The health risk level result (output of the dynamic risk identification model) and the corresponding dynamic risk features (input features of the dynamic risk identification model) are time-series aligned; driving features are constructed based on the aligned dynamic and static risk features, and the health risk level result is used as the current state input to the risk transition probability model to obtain the risk transition probability of the next time step; health guidance suggestions are determined based on the health risk level result and the corresponding risk transition probability of the next time step.

[0088] When aligning health risk level results with corresponding dynamic risk features over time, a unified timestamp is used for both dynamic risk features and corresponding health risk level results. A sliding window technique, such as using a 1-minute time window, is employed to ensure temporal consistency. Static risk features are relatively stable and can be temporarily assumed to remain unchanged across one or more time steps. Therefore, directly binding static and dynamic risk features yields the driving features. The method for constructing the risk transfer probability model is as follows:

[0089] Obtain multiple historical health risk level result sequences and the dynamic and static risk features corresponding to each historical health risk level result; use the dynamic and static risk features as driving features and the historical health risk results as state features to input into the initialized risk transition probability model to construct a risk transition probability matrix.

[0090] For example, a risk transition probability model can be constructed using Markov chains or Hidden Markov Models (HMMs). Historical health risk level results and the corresponding input features for each result are used as model inputs. The matrix parameters in the risk transition probability matrix of the risk transition probability model are solved, thus obtaining the risk transition probability model. The risk transition probability is jointly determined by the health risk level results and the corresponding dynamic and static risk features. In this embodiment, the dynamic and static risk features are used as driving features, and the health risk level results are used as the current state input. The timestamps of the driving features and the current state input are strictly aligned; for example, a sliding time window can be used to ensure consistent data timeliness. Static risk features tend to remain unchanged throughout the entire time range and are bound to the driving features and the current state features. Driving features are constructed based on the aligned dynamic and static risk features, and the health risk level results are used as the current state input to the risk transition probability model to obtain the risk transition probability for the next time step. Based on the health risk level results and the risk transition probability for the next time step corresponding to the health risk level results, output health guidance recommendations are determined.

[0091] In this embodiment, the risk transfer matrix is ​​constructed using the driving features in the following manner:

[0092] ;

[0093] Among them, A ij Let represent the probability of transitioning from risk state i to state j, where (i, j ∈ {1, 2, …, K}) (K is the total number of risk levels); σ is the sigmoid function, i.e. Ensure the output value is in the range of 0 to 1; S t－1 For historical health risk outcomes at time t-1; F t Let be the driving feature at time t; W be the learnable parameter matrix; and b be the bias term.

[0094] In one embodiment, the risk transfer probability of the next time step can also be fed back to the dynamic risk identification model to drive the incremental learning of the dynamic risk identification model to form a complete closed loop. This realizes a complete closed loop of risk state transfer and dynamic risk identification model formation, time-series prediction, real-time intervention, and model optimization, solving the technical problem that traditional health management systems cannot capture the evolution pattern of risk.

[0095] In one embodiment, the spatiotemporal feature extraction network can employ a recurrent neural network. Since the time scales of physiological time-series data and behavioral time-series data are often inconsistent—for example, physiological time-series data is often minute-level, while behavioral time-series data is often 24-hour time-series—a hierarchical recurrent neural network can be used in this embodiment to extract time-series data at at least two time scales from the physiological and behavioral time-series data when considering dynamic risk features. In this embodiment, at least two time scales exist in the physiological and behavioral time-series data, such as minute-level physiological time-series data, minute-level exercise time-series data, and 24-hour, weekly, or monthly physiological and exercise-quantified time-series data. Therefore, in this embodiment, time-series data at at least two time scales can be extracted from the physiological and behavioral time-series data.

[0096] For example, first dynamic time-series data at a first time scale is extracted from the physiological time-series data and the behavioral time-series data; minute-level time-series data is extracted from the physiological time-series data and the behavioral time-series data as the first dynamic time-series data. Examples include heart rate, steps, and exercise type.

[0097] The first dynamic time-series data is decomposed to extract physiological rhythm data. In this embodiment, the STL decomposition method can be used to decompose the time series into three parts: trend, seasonality, and residual. In wearable device data, STL decomposition can extract physiological rhythm features such as diurnal rhythm amplitude and the LF / HF ratio of heart rate variability. Diurnal rhythm amplitude reflects the fluctuation range of physiological indicators throughout the day, while the LF / HF ratio of heart rate variability is an indicator of sympathetic-parasympathetic balance and is of great significance for assessing an individual's autonomic nervous system function and health status. Physiological rhythm data also includes daily behavioral rhythm data. Since behaviors such as daily activities and exercise are difficult to use as time-series data at a fixed time scale, in this embodiment, it is necessary to quantize the behavioral rhythm data on a time scale to obtain second dynamic time-series data at a second time scale.

[0098] Specifically, behavioral data can be quantified using exercise metabolic equivalents. In this embodiment, based on different activity types (such as walking, running, and cycling) and activity durations, and combined with metabolic equivalent coefficients, an individual's exercise metabolic equivalent is calculated. For example, the metabolic equivalent coefficient for walking is 3.0, for running it is 8.0, and for cycling it is 6.5. Through exercise metabolic equivalents, physiological rhythm data is quantified into a second dynamic time-series data on a weekly time scale.

[0099] The second dynamic time series data may also include physiological time series data at a second time scale, such as weekly sleep status statistics.

[0100] The dynamic risk features at different time scales are input into a hierarchical recurrent neural network corresponding to the time scale. The network captures cumulative time-series features and performs recursive calculations layer by layer from the bottom to the top of the hierarchical recurrent neural network to obtain the dependency features of the time series. Since the first and second dynamic time-series data have different time scales, a hierarchical recurrent neural network is used to capture the combined impact of features at different time scales on health status.

[0101] For example, a hierarchical recurrent neural network includes at least a bottom-level network and a top-level network.

[0102] The first dynamic time-series data is input into the underlying network, and the first cumulative temporal features of the first dynamic time-series features at the first time scale are captured in the hidden layer of the underlying network. In this embodiment, the first dynamic time-series data is processed according to the first time scale by the underlying network, and the hidden state of the last time step is used as the input feature representation of the top-level network to achieve feature compression. By processing the time-series data at different time scales in layers, the first dynamic time-series data is downsampled to the second time scale, aligning the temporal granularity with the second dynamic time-series data. The last hidden state encodes the cumulative changes in physiological states within the second time scale, such as the trend of heart rate fluctuations.

[0103] The first time-series cumulative feature and the second dynamic time-series data are aligned and fused at the second time scale. In this embodiment, the fusion of the first time-series cumulative feature and the second dynamic time-series data can be performed by feature concatenation. The concatenation operation preserves the original feature information and is suitable for scenarios with strong complementarity between features.

[0104] The first time-series cumulative feature and the second dynamic time-series data can also be calculated using a cross-attention mechanism to calculate the fusion weight of the first time-series cumulative feature and the second dynamic time-series data, and the first time-series cumulative feature and the second dynamic time-series data can be fused based on the fusion weight to obtain a multi-scale fusion feature.

[0105] The fusion weight can be expressed as:

[0106] α = σ(Wa × [h1 || x1] + ba)

[0107] hf = α × h1 + (1 - α) × x1

[0108] Where σ is the Sigmoid function (output value is between [0,1]);

[0109] It is the weight matrix, and ba is the bias term;

[0110] α is a scalar weight (not a vector) that represents the importance of the first-time cumulative feature;

[0111] h1 is the first time-series cumulative feature, x1 is the second dynamic time-series number, and hf is the multi-scale fusion feature.

[0112] The multi-scale fusion features are input into the top-level network to capture the second temporal cumulative features at the second time scale as the dynamic risk features. The bottom-level network extracts local patterns at the first time scale (such as a sudden increase in heart rate during exercise), while the top-level network captures global trends at the second time scale (such as the cumulative effect of long-term sleep deprivation, and the cumulative effect of long-term exercise or inactivity). This approach can retain more individual physiological and behavioral characteristics across more time scales, improving the accuracy of risk identification and the scope of health stratification management.

[0113] By leveraging artificial intelligence technology, personalized health stratification management plans are customized based on an individual's health history, genetic information, and lifestyle, providing real-time health status analysis and prediction to achieve precise health stratification management. Simultaneously, based on an individual's health status and needs, personalized health promotion plans are developed, including dietary guidance, exercise recommendations, and disease prevention measures, to help improve lifestyle and enhance health levels.

[0114] like Figure 3 As shown in the illustration, this application also provides a proactive health stratification management device based on multimodal fusion. This proactive health stratification management device utilizes a pre-trained dynamic risk identification model to perform proactive health stratification management based on the predicted risk level of the person to be assessed. The dynamic risk identification model includes a spatiotemporal feature extraction module, a static feature extraction module, and a fully connected layer. The proactive health stratification management device includes:

[0115] The acquisition module 301 is used to acquire dynamic and static data of the person to be evaluated, wherein the dynamic data includes physiological time-series data and behavioral time-series data, and the static data includes genetic data and environmental exposure data.

[0116] The first feature extraction module 302 is used to input the physiological time series data and the behavioral time series data into the spatiotemporal feature extraction module, extract the time series dependencies in the dynamic data, and use a self-attention mechanism to identify abnormal feature sequences based on the time series dependencies to obtain dynamic risk features.

[0117] The second feature extraction module 303 is used to input the genetic data and the environmental exposure data into the static feature extraction module and extract static risk features based on the multi-head attention mechanism.

[0118] The feature fusion module 304 is used to fuse the obtained dynamic risk features and the static risk features using a dynamic gating mechanism to obtain fused features;

[0119] The prediction module 305 is used to pass the fused features to the fully connected layer to obtain the health risk level results at different time scales, and output a health stratification management file based on the health risk level results.

[0120] Figure 4 This is a structural block diagram of an optional electronic device according to an embodiment of this application, such as... Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404. The processor 401, communication interface 402, and memory 403 communicate with each other via the communication bus 404.

[0121] Memory 403 is used to store computer programs;

[0122] When processor 401 executes a computer program stored in memory 403, it performs the following steps:

[0123] Acquire dynamic and static data of the person to be evaluated, wherein the dynamic data includes physiological time-series data and behavioral time-series data, and the static data includes genetic data and environmental exposure data;

[0124] The physiological time-series data and the behavioral time-series data are input into the spatiotemporal feature extraction module to extract the temporal dependencies in the dynamic data, and the abnormal feature sequence is identified based on the temporal dependencies using a self-attention mechanism to obtain dynamic risk features.

[0125] The genetic data and the environmental exposure data are input into the static feature extraction module, and static risk features are extracted based on the multi-head attention mechanism.

[0126] The obtained dynamic risk features and static risk features are fused using a dynamic gating mechanism to obtain fused features;

[0127] The fusion features are passed to the fully connected layer to obtain the health risk level results at different time scales, and a health stratification management file is output based on the health risk level results.

[0128] Optionally, in this embodiment, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0129] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0130] The memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0131] The processor mentioned above can be a general-purpose processor, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0132] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0133] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. The device implementing the above-mentioned proactive health hierarchical management method based on multimodal fusion can be a terminal device, such as a smartphone (e.g., Android phone, iOS phone), tablet computer, handheld computer, mobile internet device (MID), PAD, etc. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, the terminal device may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0134] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0135] According to another aspect of the embodiments of this application, a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to execute program code for a proactive health hierarchical management method based on multimodal fusion.

[0136] Optionally, in this embodiment, the storage medium may be located on at least one of the multiple network devices in the network shown in the above embodiment.

[0137] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

[0138] Acquire dynamic and static data of the person to be evaluated, wherein the dynamic data includes physiological time-series data and behavioral time-series data, and the static data includes genetic data and environmental exposure data;

[0139] The physiological time-series data and the behavioral time-series data are input into the spatiotemporal feature extraction module to extract the temporal dependencies in the dynamic data, and the abnormal feature sequence is identified based on the temporal dependencies using a self-attention mechanism to obtain dynamic risk features.

[0140] The genetic data and the environmental exposure data are input into the static feature extraction module, and static risk features are extracted based on the multi-head attention mechanism.

[0141] The obtained dynamic risk features and static risk features are fused using a dynamic gating mechanism to obtain fused features;

[0142] The fusion features are passed to the fully connected layer to obtain the health risk level results at different time scales, and a health stratification management file is output based on the health risk level results.

[0143] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated in this embodiment.

[0144] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0145] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0146] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0147] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0148] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0149] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution provided in this embodiment, depending on actual needs.

[0150] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0151] For any parts not mentioned in this application, existing technologies may be used or referenced.

[0152] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0153] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A multi-modal fusion-based active health stratified management method, characterized in that, The active health stratified management method utilizes a prediction result of a risk level of an evaluated person by a pre-trained dynamic risk identification model for active health stratified management, the dynamic risk identification model comprising a space-time feature extraction module, a static feature extraction module and a full connection layer, wherein the active health stratified management method comprises: acquiring dynamic data and static data of the evaluated person, wherein the dynamic data comprises physiological time series data and behavior time series data, and the static data comprises genetic data and environmental exposure data; inputting the physiological time series data and the behavior time series data into the space-time feature extraction module, extracting time series dependence in the dynamic data, and identifying abnormal feature sequences based on the time series dependence by using a self-attention mechanism to obtain dynamic risk features, the dynamic risk features being used for evaluating acute risk and corresponding risk levels; inputting the genetic data and the environmental exposure data into the static feature extraction module, extracting static risk features based on a multi-head attention mechanism, comprising: analyzing the genetic data to obtain a multi-gene risk score; acquiring a space-time trajectory of the evaluated person within a preset time period, and matching the space-time trajectory with environmental exposure data of a corresponding region to obtain an environmental cumulative exposure amount; inputting the multi-gene risk score and the environmental cumulative exposure amount into a static feature extraction module based on a Transformer structure, and extracting key risk factors as the static risk features by a multi-head attention mechanism; the static risk features being used for evaluating long-term cumulative risk / chronic risk and corresponding risk levels; fusing the obtained dynamic risk features and the static risk features by using a dynamic gating mechanism to obtain fusion features, comprising real-time acquiring an importance degree of the abnormal feature sequences of the dynamic risk features; determining a gating weight based on the importance degree, wherein the importance degree is positively correlated with the gating weight; acquiring a physical examination result and / or medical record of the evaluated person; extracting entity features in the physical examination result and / or medical record; interacting the entity features with the static risk features and the dynamic risk features respectively to determine interaction relationships between the entity features and the static risk features and the dynamic risk features respectively; adjusting the gating weight based on the interaction relationships, specifically: performing regression screening of significant interaction terms of entity features and the static risk features and entity features and the dynamic risk features respectively by LASSO, determining the interaction degree between the entity features in the medical record or the physical examination report of the current evaluated person and the static risk features or the entity features based on the significant interaction terms, and adjusting the gating weight based on the interaction degree, wherein the gating weight of the static risk features is increased, and the corresponding dynamic risk prediction model can pay more attention to long-term further development of high-risk-level diseases; the gating weight of the dynamic risk features is increased, and the corresponding dynamic risk prediction model can pay more attention to short-term high-risk-level diseases; fusing the dynamic risk features and the static risk features based on the gating weight; The fusion features are transmitted to the full connection layer to obtain health risk level results of different time scales, and a health stratified management archive of the whole life cycle including long-term and short-term is output based on the health risk level results.

2. The multi-modal fusion-based active health stratification management method according to claim 1, wherein, After the fusion features are transmitted to the full connection layer to obtain health risk level results of different time scales, the following steps are included: The health risk level results and dynamic risk features corresponding to the health risk level results are time-aligned; Based on the aligned dynamic risk features and static risk features, driving features are constructed, and the health risk level results are input as current state features into a pre-constructed risk transfer probability model to obtain risk transfer probabilities of the next time step; Based on the health risk level results and risk transfer probabilities of the next time step corresponding to the health risk level results, health guidance suggestions are determined and output.

3. The multi-modal fusion-based active health stratification management method of claim 2, wherein, The risk transfer probability model is constructed as follows: A plurality of historical health risk level result sequences and dynamic risk features and static risk features corresponding to each historical health risk level result are obtained; The dynamic risk features and the static risk features are input as driving features, and the historical health risk results are input as state features into an initialized risk transfer probability model to construct a risk transfer probability matrix.

4. The multi-modal fusion-based active health stratification management method of claim 1, wherein, The spatio-temporal feature extraction module includes a hierarchical recurrent neural network; The input of the physiological time series data and the behavior time series data into the spatio-temporal feature extraction module includes: Extracting time series data in at least two time scales in the physiological time series data and the behavior time series data; Inputting the time series data in different time scales into the recurrent neural network of the corresponding level of the hierarchical recurrent neural network, respectively, and performing layer-by-layer capture of time series cumulative features and layer-by-layer recursive calculation from the bottom network to the top network of the hierarchical recurrent neural network to obtain the time series dependency.

5. The active health stratified management method based on multi-modal fusion according to claim 4, wherein The extraction of time series data in at least two time scales in the physiological time series data and the behavior time series data includes: Extracting first dynamic time series data in a first time scale in the physiological time series data and the behavior time series data; Decomposing the first dynamic time series data to extract physiological rhythm data; Performing time series quantization on the physiological rhythm data to obtain second dynamic time series data in a second time scale.

6. The multi-modal fusion-based active health stratification management method of claim 5, wherein, The hierarchical recurrent neural network includes at least a bottom network and a top network; The input of the time series data in different time scales into the recurrent neural network of the corresponding level of the hierarchical recurrent neural network, respectively, and the performance of layer-by-layer capture of time series cumulative features and layer-by-layer recursive calculation from the bottom network to the top network of the hierarchical recurrent neural network to obtain the time series dependency include: The first dynamic time series data is input into the bottom network to capture first time series cumulative features of the first dynamic time series data in the first time scale in the hidden layer of the bottom network; The first time series cumulative features and the second dynamic time series data are aligned in the second time scale; The cross-attention mechanism is used to calculate a fusion weight of the first time-series cumulative feature and the second dynamic time-series number, and the first time-series cumulative feature and the second dynamic time-series data are fused based on the fusion weight to obtain a multi-scale fusion feature; The multi-scale fusion feature is input into the top-layer network to capture a second time-series cumulative feature under the second time scale as the dynamic risk feature.

7. An electronic device, comprising: The method comprises one or more processors, one or more memories, and one or more computer program instructions, and the computer program instructions are executed by the processor to implement the active health stratification management method based on multi-modal fusion according to any one of claims 1 to 6.

Images