Physical and psychological health state evaluation system based on multi-modal data and electronic equipment

Abstract

The embodiment of the invention provides a physical and psychological health state evaluation system and electronic equipment based on multi-modal data, and the system comprises a data collection module which is used for collecting the multi-modal data of an individual; the index library module is used for constructing a multi-level standard index library based on a physiology-psychology-society theory; the data processing module is used for screening feature indexes from the index library module based on feature engineering; the intelligent evaluation module is used for continuously tracking and evaluating the physical and psychological health state of the patient by utilizing a machine learning algorithm according to the characteristic indexes; the intervention solution module is used for providing an intervention scheme according to the physical and psychological health state of the patient; and the feedback adjustment module is used for optimizing the evaluation and intervention process. The technical scheme provided by the invention is used for solving the problem of low accuracy of physical and psychological health state assessment caused by analyzing psychological problems from a single dimension in the prior art.

Description

Technical Field

[0001] The present invention relates to the fields of health management and mental health assessment, and in particular to a physical and mental health status assessment system and electronic equipment based on multimodal data. Background Art

[0002] With rapid socioeconomic development, the incidence of psychological problems has increased significantly. According to incomplete statistics, the national depression prevalence rate is between 6-10%. At the same time, with the increasing popularity of psychology, more and more people are eager to explore and improve themselves through psychological means.

[0003] At the academic and application levels, the most popular method currently is to use psychological scales to identify or diagnose a disease or trait.

[0004] However, psychological scales also have their limitations: they analyze psychological problems from a single dimension, which greatly reduces the accuracy of the assessment of physical and mental health status. Summary of the Invention

[0005] In view of the above scheme, this application aims to propose a physical and mental health status assessment system and electronic device based on multimodal data to improve the accuracy of physical and mental health status assessment.

[0006] In a first aspect, one or more embodiments of the present application provide a physical and mental health status assessment system based on multimodal data, including: a data acquisition module for collecting multimodal data of an individual; an indicator library module for constructing a multi-level standard indicator library based on physiological-psychological-social theory; a data processing module for screening feature indicators from the indicator library module based on feature engineering; an intelligent assessment module for continuously tracking and evaluating the patient's physical and mental health status using a machine learning algorithm based on the feature indicators; an intervention solution module for providing an intervention plan based on the patient's physical and mental health status; and a feedback adjustment module for optimizing the assessment and intervention process.

[0007] Furthermore, the data acquisition module includes a heart rate monitor and a blood pressure monitor, and collects physiological data using biomedical data acquisition methods to ensure the quality and validity of the data.

[0008] Furthermore, the data acquisition module is a removable device.

[0009] Furthermore, the psychological level indicators in the indicator library module define evaluation indicators such as emotions, behaviors and social functions, as well as corresponding measurement standards based on the PERMA theory and the two-factor model of mental health.

[0010] Furthermore, the data processing module uses data type processing, dimensionality reduction and importance analysis techniques in the data preprocessing stage to improve the efficiency and accuracy of data processing.

[0011] Furthermore, the machine learning algorithm is specifically a Bayesian network algorithm;

[0012] The intelligent evaluation module is used to construct a Bayesian network structure and conditional probability distribution based on the characteristic indicators.

[0013] Furthermore, the intelligent assessment module adopts a Bayesian network algorithm to continuously track and evaluate the individual's physical and mental health status based on changes in individual physiological information, major event records, behavioral data records and intervention record feedback.

[0014] Furthermore, the intelligent evaluation module selects a corresponding classification algorithm according to the characteristic index.

[0015] Furthermore, the intervention solution module provides personalized pre-prevention, current situation improvement and future enhancement intervention plans based on the evaluation results, which are adapted to individual and group characteristics.

[0016] In a second aspect, an embodiment of the present application provides a processor for performing data processing and intelligent evaluation tasks in the system described in any one of the first aspects; a memory for storing collected multimodal data and evaluation results; and an input / output interface for data collection and receiving feedback information.

[0017] Compared with the existing technology, this application can at least achieve the following technical effects:

[0018] By integrating multimodal physiological, psychological, and social data, it can comprehensively and accurately assess an individual's physical and mental health and provide personalized intervention plans. In practical applications, this system can effectively monitor and warn of potential health issues, promote early intervention, and improve individual quality of life. For businesses and organizations, it can help manage team health, improve work efficiency, and reduce losses caused by health issues, with significant social and economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate one or more embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1A schematic structural diagram of a physical and mental health status assessment system based on multimodal data is provided in one or more embodiments of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present application, the technical solutions in one or more embodiments of the present application will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on one or more embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of this document.

[0022] A physical and mental health status assessment system based on multimodal data, such as Figure 1 As shown, including:

[0023] Data Collection Module 101: This module is used to collect multimodal data from individuals, including physiological, psychological, and social data. Physiological data can be collected using biomedical data collection methods such as heart rate monitors and blood pressure monitors to ensure data quality and validity. Psychological data can be collected through questionnaires and psychological tests, reflecting the individual's emotions, behavior, and social functioning. Social data can include information about an individual's occupation, family, and interpersonal relationships, and is used to assess the impact of an individual's social environment on their physical and mental health.

[0024] In an embodiment of the present application, the data acquisition module 101 may also include portable devices such as smart bracelets, smart watches and other portable devices. These devices can monitor the individual's physiological parameters and behavioral data in real time and transmit the data to the system for processing and analysis.

[0025] In the embodiments of the present application, the data collection process ensures that data collection follows a unified process and standard to improve data quality. At the same time, encryption technology is used to ensure the security and privacy of individual data.

[0026] Indicator Library Module 102: Constructs a multi-level standard indicator library based on physiological-psychological-social theory. Physiological indicators include heart rate, blood pressure, blood oxygen saturation, and other physiological parameters. Psychological indicators, based on the PERMA theory and the two-factor model of mental health, define assessment indicators for affect, behavior, and social functioning, along with corresponding measurement standards. These indicators cover cognition, mood, personality traits, coping mechanisms, stress perception, and history of mental illness. Social indicators include career satisfaction, family relationships, interpersonal relationships, educational background, work situation, economic status, legal issues, and cultural and ethnic background. PERMA theory refers to positive emotions, engagement, relationships, meaning, and achievement, all of which are crucial components of mental health. The two-factor model emphasizes the balance between positive and negative aspects of mental health, comprehensively reflecting an individual's mental health by assessing both positive psychological qualities and negative psychological symptoms.

[0027] In the examples of this application, the weights and relevance of each indicator were determined based on literature review and expert opinion. Standardized collection tools and scales were designed to ensure the reliability and validity of the data. This approach continuously updates and optimizes the indicator library to adapt to new research findings and technological developments.

[0028] Data processing module 103: This module is used to select characteristic indicators from the indicator library module based on feature engineering and perform data preprocessing. The data preprocessing stage includes data type processing, dimensionality reduction, and importance analysis to improve the efficiency and accuracy of data processing.

[0029] Specifically, feature engineering is an important part of machine learning, which improves the performance of the model by selecting representative and discriminative features. In the data processing stage, the collected multimodal data first needs to be cleaned and organized to remove data quality issues such as outliers and missing values. Then, the data is pre-processed using techniques such as data type processing, dimensionality reduction, and importance analysis. Data type processing technology can convert different types of data into a unified format and scale for subsequent analysis and processing. Dimensionality reduction technology can reduce the dimension of the data through methods such as principal component analysis (PCA) and linear discriminant analysis (LDA), thereby improving the efficiency and accuracy of data processing. Importance analysis technology can evaluate the impact of each feature on model performance based on feature selection methods (such as chi-square test and mutual information), thereby screening out feature indicators that have an important impact on the assessment of physical and mental health status.

[0030] In the examples of this application, the SHAP (Shapley Additive Explanations) algorithm is used to quantify the contribution of each feature to the evaluation results. For example, in the assessment of depression risk, it is revealed that "decreased frequency of social activities (social layer)" has a higher explanatory weight than "decreased heart rate variability (physiological layer)." Alternatively, a causal network based on a structural equation model is constructed to clarify the causal relationship between physical and mental health status and multimodal indicators, avoiding the black box problem of traditional machine learning models.

[0031] Intelligent Assessment Module 104: Based on characteristic indicators, a machine learning algorithm is used to continuously track and assess the patient's physical and mental health status. Specifically, the machine learning algorithm can employ a Bayesian network algorithm to construct a Bayesian network structure and conditional probability distribution. Based on changes in individual physiological information, major event records, behavioral data records, and intervention record feedback, the module continuously tracks and assesses the individual's physical and mental health status. Furthermore, the intelligent assessment module can also select a corresponding classification algorithm based on characteristic indicators to improve the accuracy and reliability of the assessment. For example, when the amount of physiological data is significantly greater than the amount of psychological and social data, a structure tree algorithm should be used to classify physical and mental health status. When the physiological, psychological, and social data are relatively balanced, a K-nearest neighbor classification algorithm can be used.

[0032] In an embodiment of the present application, the patient's physical and mental health status can also be continuously tracked and evaluated through a machine learning algorithm. For example, the attention mechanism is used to assign weights to different modal data (such as physiological signals, text emotions, and social behaviors) to achieve cross-modal feature fusion. For example, when a user has an abnormal heart rate (physiological) and negative language on social media (psychological) at the same time, the system automatically increases the health risk weight of that time period. LSTM (Long Short-Term Memory, long short-term memory network) is used to capture the dynamic changes in physiological and psychological indicators. For example, the temporal association between a decrease in sleep quality for three consecutive days (physiological) and an increase in the frequency of keywords in a work stress diary (psychological) can predict the risk of anxiety attacks.

[0033] Intervention solution module 105: Provide intervention plans based on the patient's physical and mental health status. Specifically, based on the assessment results, provide personalized intervention plans for pre-emptive prevention, current situation improvement, and future enhancement. Intervention plans may include psychological counseling, drug therapy, lifestyle adjustments, and other means to adapt to the different characteristics of individuals and groups. Specifically, for high-risk groups, preventive measures are recommended, such as regular physical examinations, meditation training, etc. For individuals with existing problems, provide real-time intervention tools, such as cognitive behavioral therapy (CBT) exercise libraries, physiological feedback training, etc. For healthy people, provide personalized suggestions, such as optimizing work and rest schedules, nutritional matching plans, etc.

[0034] Feedback Adjustment Module 106: This module is used to optimize the assessment and intervention process. By collecting subject feedback and intervention effectiveness data, the assessment model and intervention plan are adjusted and optimized to improve the accuracy and effectiveness of the system. Specifically, intervention effectiveness data is collected through user feedback, physiological indicator monitoring, and psychological questionnaires. Cross-validation and parameter adjustment techniques are used to optimize the assessment model and intervention strategy.

[0035] Through the linkage of these modules, the individual's mental state is monitored. This linkage forms a closed loop of information collection - intelligent analysis - proposal - new feedback - re-collection. This closed loop operation enables real-time monitoring and real-time adjustment of treatment plans and intervention measures.

[0036] To illustrate the feasibility of the above solution, the following example is given:

[0037] Example 1

[0038] The patient works more than 12 hours a day and has experienced insomnia and decreased concentration for the past three months. The system uses the following process to assess and intervene:

[0039] Data collection phase:

[0040] Physiological data: The smart bracelet monitored that the average heart rate increased from 72bpm to 85bpm, and the deep sleep time decreased by 40% (Apple Watch data synchronization).

[0041] Psychological data: The frequency of the keywords "tired" and "anxiety" in daily mood logs increased by 300% compared to the baseline.

[0042] Social data: The company's OA system shows that its overtime frequency ranks in the top 5%, and conflict incidents in team communication records have increased by 2 times.

[0043] Data processing and feature extraction:

[0044] Using time series alignment techniques, work calendar events (such as project launches) are matched with fluctuations in physiological indicators (such as elevated cortisol levels).

[0045] Graph neural network (GNN) was used to model the association path of "working hours → heart rate variability → emotional keywords", and the nonlinear effect of work stress on the function of the autonomic nervous system was found.

[0046] Intelligent assessment results:

[0047] The system determines that the user is at "high risk of burnout" (92% confidence level), specifically:

[0048] Physiological layer: HRV (heart rate variability) SDNN value <50ms (normal range >100ms)

[0049] Psychological level: The emotional exhaustion dimension score in the MBI Burnout Inventory was 28 points (the critical value was 27 points).

[0050] Social level: Team support rating 4.2 / 10 (corporate norm 7.5 / 10)

[0051] Intervention plan generation:

[0052] Short-term intervention:

[0053] Push progressive muscle relaxation training (PMR) tutorials and synchronize with smart bracelets for real-time physiological feedback.

[0054] Adjusting the scheduling algorithm suggests: Mandatory insertion of a 5-minute "tech isolation" break every 90 minutes.

[0055] Long-term intervention:

[0056] Access the company's EAP system and recommend psychological counselors to conduct cognitive reconstruction interventions twice a month.

[0057] Through reinforcement learning, the work allocation model is optimized to reduce the number of concurrent tasks in the project.

[0058] Feedback Adjustment:

[0059] Monitoring after 2 weeks showed that HRV recovered to 78ms and the frequency of negative words in the mood log decreased by 45%.

[0060] The system automatically reduces the priority of drug intervention and increases the frequency of cognitive behavioral therapy (CBT) module push.

[0061] An embodiment of the present application provides an electronic device for storing computer-executable instructions, a processor for data processing and intelligent evaluation tasks in the system described in any of the aforementioned embodiments; a memory for storing collected multimodal data and evaluation results; and an input / output interface for data acquisition and receiving feedback information.

[0062] It should be noted that the embodiment of the electronic device in this application and the embodiment of the blockchain-based service provision method in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the corresponding blockchain-based service provision method mentioned above, and the repetitions will not be repeated.

[0063] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0064] One or more embodiments of the present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. One or more embodiments of the present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer electronic devices, including storage devices.

[0065] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment.

[0066] The foregoing description is merely an example of the present invention and is not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims herein.

Claims

1. A physical and mental health status assessment system based on multimodal data, characterized in that: include: Data acquisition module, used to collect multimodal data of individuals; The indicator library module builds a multi-level standard indicator library based on physiological-psychological-social theory; A data processing module is used to screen characteristic indicators from the indicator library module based on feature engineering; an intelligent assessment module uses a machine learning algorithm to continuously track and evaluate the patient's physical and mental health status based on the characteristic indicators; and an intervention solution module provides an intervention plan based on the patient's physical and mental health status; Feedback adjustment module to optimize the assessment and intervention process.

2. The system according to claim 1, wherein: The data acquisition module includes a heart rate monitor and a blood pressure monitor, and collects physiological data using biomedical data acquisition methods to ensure the quality and validity of the data.

3. The system according to claim 1, wherein: The data acquisition module is a removable device.

4. The system according to claim 1, wherein: The psychological level indicators in the indicator library module define evaluation indicators such as emotions, behaviors and social functions, as well as corresponding measurement standards based on the PERMA theory and the two-factor model of mental health.

5. The system according to claim 1, wherein: During the data preprocessing phase, the data processing module utilizes data type processing, dimensionality reduction, and importance analysis techniques to improve the efficiency and accuracy of data processing.

6. The system according to claim 1, wherein: The machine learning algorithm is specifically a Bayesian network algorithm; The intelligent evaluation module is used to construct a Bayesian network structure and conditional probability distribution based on the characteristic indicators.

7. The system according to claim 6, characterized in that The intelligent assessment module adopts the Bayesian network algorithm to continuously track and evaluate the individual's physical and mental health status based on changes in individual physiological information, major event records, behavioral data records and intervention record feedback.

8. The system according to claim 1, wherein: The intelligent evaluation module selects a corresponding classification algorithm according to the characteristic index.

9. The system according to claim 1, wherein: The intervention solution module provides personalized pre-emptive prevention, current situation improvement and future enhancement intervention plans based on the assessment results, adapting to individual and group characteristics.

10. An electronic device, characterized in that: include: A processor for performing data processing and intelligent evaluation tasks in the system according to any one of claims 1 to 9; A memory for storing collected multimodal data and evaluation results; Input / output interface, used for data collection and receiving feedback information.

Images