Gait and balance self-management system based on multi-mode sensing technology

Abstract

The invention discloses a gait and balance self-management system based on a multi-modal sensing technology. The gait and balance self-management system comprises a multi-modal gait data acquisition module, a data fusion processing module and a gait balance information management module, the multi-modal gait data acquisition module is used for acquiring multi-dimensional gait information; the data fusion processing module is used for fusing and predicting the gait information; and the gait balance information management module is used for monitoring and managing the gait information in real time. According to the invention, data of the optical sensor and the inertial sensor are fused, comprehensive gait information and motion evaluation are provided, and multi-dimensional motion information of human motion postures can be collected by overcoming environmental limitation.

Description

Technical Field

[0001] The present invention belongs to the fields of medical information technology and biomedical technology, and in particular relates to a gait and balance self-management system based on multimodal sensing technology. Background Art

[0002] Currently, many existing gait and balance management devices rely on a single sensor system. For example, many devices use accelerometers to collect motion data. Although this system can provide basic gait information, because it only uses a single data source, its information limitations are significant and cannot fully reflect the user's athletic ability in different situations. Specifically, a single sensor cannot capture changes in motion characteristics caused by environmental changes (such as uneven ground and changes in light). In addition, in complex or changing environments, the monitoring effect of such systems may decrease and cannot meet the comprehensive assessment needs of the elderly for athletic ability.

[0003] In addition, some studies use optical sensing technology for gait analysis, using cameras and image processing algorithms to capture the user's motion status. However, this type of optical system is greatly affected by ambient lighting conditions, and insufficient light or strong reflections may lead to inaccurate data collection. In addition, the user's range of activities is limited, making it difficult to effectively monitor in real life scenarios. This type of system has high requirements for environmental conditions when in use, and is often subject to technical limitations in actual applications, resulting in low usage rates.

[0004] In recent years, some gait balance management systems have attempted to combine multiple sensors (such as accelerometers, gyroscopes, and optical sensors) to provide more comprehensive gait analysis, but such integrated systems often fail to fully consider the user experience of elderly users when designing. Complex user interfaces and operating steps make it difficult for the elderly to use, thus reducing the actual application effect of the system. In addition, the data fusion technology of different sensors is still not mature enough, which may lead to inconsistent data or parsing errors, affecting the reliability of the final results.

[0005] In summary, the limitations of existing technologies in gait and balance monitoring are mainly reflected in the limitations of a single data source, environmental dependence, the complexity of system design and the imperfection of data fusion technology. These shortcomings directly affect the elderly's acceptance and use of related technologies. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a gait and balance self-management system based on multimodal sensing technology, which is used for gait balance ability assessment and health management of the elderly population, aiming to reduce the risk of falling and improve the quality of life.

[0007] The present invention provides a gait and balance self-management system based on multimodal sensing technology, which is characterized by: a multimodal gait data acquisition module, a data fusion processing module and a gait balance information management module;

[0008] The multimodal gait data acquisition module is used to collect multi-dimensional gait information;

[0009] The data fusion processing module is used to fuse and predict the gait information;

[0010] The gait balance information management module is used to monitor and manage the gait information in real time.

[0011] Optionally, the multimodal gait data acquisition module includes: an optical sensor, an inertial sensor and a data transmission unit;

[0012] The optical sensor is used to capture the user's skeleton and human body posture data in real time through a high-definition camera and infrared technology;

[0013] The inertial sensor is used to capture the user's motion state data;

[0014] The data transmission unit is used to transmit the posture data and motion state data to a data fusion processing module.

[0015] Optionally, the data fusion processing module includes: a data receiving unit, a data processing unit, a data fusion unit, a gait balance ability prediction unit and a data storage unit;

[0016] The data receiving unit is used to receive the posture data and motion state data;

[0017] The data processing unit is used to process the posture data and motion state data;

[0018] The data fusion unit is used to fuse the posture data and the motion state data through a multimodal data fusion algorithm to obtain fused data;

[0019] The gait balance ability prediction unit is used to input the fusion data into the prediction model to predict the gait balance ability;

[0020] The data storage unit is used to store posture data and motion state data.

[0021] Optionally, the data processing unit includes: a data preprocessing subunit and a feature extraction subunit;

[0022] The data preprocessing unit is used to perform denoising and outlier removal processing on the posture data and motion state data, and fill in missing values ​​using an interpolation method;

[0023] The feature extraction subunit is used to extract features from the preprocessed data and normalize the features.

[0024] Optionally, the fused data is input into a prediction model to predict gait balance ability, including:

[0025] The prediction model includes an elastic network, and the prediction model is trained by a training set, and the training set is normalized feature data.

[0026] Optionally, the gait balance information management module includes: a real-time monitoring unit and an information management unit;

[0027] The real-time monitoring unit is used to monitor the user's gait and balance ability in real time;

[0028] The information management unit is used for health analysis of the monitored data.

[0029] Optionally, the information management unit includes: a health education subunit and a health warning subunit;

[0030] The health education subunit is used to improve the user's cognition and understanding of gait and balance ability;

[0031] The health warning subunit is used to issue a warning for abnormal monitoring data.

[0032] Compared with the prior art, the present invention has the following advantages and technical effects:

[0033] The present invention uses data from fused optical sensors and inertial sensors to provide comprehensive gait information and motion evaluation, and can overcome environmental limitations to collect multi-dimensional motion information of human motion posture.

[0034] By investigating the opinions of the elderly and physical therapists, the present invention clarifies the principles of aging-friendly design, ensures that the system interface is friendly and easy to operate, and improves the user experience of elderly users.

[0035] The present invention utilizes multimodal sensing technology combined with clinical functional test items to conduct real-time monitoring and evaluation of the gait and balance ability of the elderly in a home environment, thereby reducing the workload of medical staff and the pressure on medical resources.

[0036] According to the user's health information and test results, the system of the present invention automatically generates personalized exercise plans, dietary guidance and fall prevention suggestions to improve the gait and balance management ability of the elderly.

[0037] The present invention includes a multimodal gait information acquisition device, a data processing server and an information management platform, which supports users to track gait balance assessment results, view improvement suggestions and manage personal health records. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 It is a structural diagram of a gait and balance self-management system based on multimodal sensing technology according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0041] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] The present invention proposes a gait and balance self-management system based on multimodal sensing technology, such as Figure 1 As shown, it specifically includes: a multimodal gait data acquisition module, a data fusion processing module and a gait balance information management module;

[0043] Multimodal gait data acquisition module, used to collect multi-dimensional gait information;

[0044] Data fusion processing module, used to fuse and predict gait information;

[0045] The gait balance information management module is used to monitor and manage gait information in real time.

[0046] Further, the multimodal gait data acquisition module includes: an optical sensor, an inertial sensor and a data transmission unit;

[0047] Optical sensors are used to capture the user's skeleton and human posture data in real time through high-definition cameras and infrared technology;

[0048] Inertial sensors, used to capture the user's motion status data;

[0049] The data transmission unit is used to transmit the posture data and motion state data to the data fusion processing module.

[0050] Specifically, optical sensors: Through high-definition cameras and infrared technology, they are placed in front of the user to capture the user's skeleton and human posture information in real time, and provide motion coordinates in three-dimensional space for data fusion and feature extraction.

[0051] Inertial sensor: Integrates accelerometer, gyroscope and magnetometer, and is worn on the user's pelvis through a strap to capture dynamic information such as the user's movement speed, acceleration and angular velocity, and provide body movement status data.

[0052] Data transmission unit: Uses low-power Bluetooth to transmit raw sensor data in real time to the local gait balance information management platform for subsequent upload to the cloud server for data processing and fusion, thereby supporting local caching when the network connection is unstable.

[0053] In this embodiment, data from optical sensors and inertial sensors are fused to provide comprehensive gait information and motion evaluation, which can overcome environmental limitations to collect motion information of multiple dimensions of human motion posture.

[0054] Further, the data fusion processing module includes: a data receiving unit, a data processing unit, a data fusion unit, a gait balance ability prediction unit and a data storage unit;

[0055] A data receiving unit, used for receiving posture data and motion state data;

[0056] A data processing unit, used for processing posture data and motion state data;

[0057] A data fusion unit, used for fusing the posture data and the motion state data through a multimodal data fusion algorithm to obtain fused data;

[0058] A gait balance ability prediction unit, used for inputting the fused data into a prediction model to predict gait balance ability;

[0059] The data storage unit is used to store posture data and motion state data.

[0060] Further, the data processing unit includes: a data preprocessing subunit and a feature extraction subunit;

[0061] A data preprocessing unit is used to denoise and remove outliers from the posture data and motion state data, and fill in missing values ​​using interpolation;

[0062] The feature extraction subunit is used to extract features from the preprocessed data and normalize the features.

[0063] Further, the fused data is input into the prediction model to predict the gait balance ability including:

[0064] The prediction model includes an elastic network, and the prediction model is trained by a training set, which is normalized feature data.

[0065] Specifically, data receiving and storage module: The cloud server is equipped with a high-performance CPU and large-capacity memory to ensure the rapid processing and storage of massive data and protect the security and privacy of the data.

[0066] Data cleaning and preprocessing: Detect outliers on sensor data, identify and remove outliers, and use interpolation to fill missing values. Use low-pass filtering algorithm to denoise signal data to eliminate sensor noise and improve data smoothness and reliability.

[0067] In this step, as an example that can be added:

[0068] Data fusion: The data collected from different sensors (optical sensors and inertial sensors) are processed comprehensively, and the data of each sensor is integrated through a multimodal data fusion algorithm to generate a comprehensive gait and motion ability assessment result. This process uses a deep learning algorithm to improve the accuracy and reliability of data fusion. Specifically, the original collected data includes the user's demographic information, the results of the physical health test scale (entered in the form of self-report), and the motion information collected by the sensor (including the acceleration, angular velocity and angle signals collected at the pelvic position and the motion coordinates of the body bone points in three-dimensional space collected by the optical sensor). In this step, as an embodiment that can be added, this embodiment develops a feature extraction scheme for data fusion to extract descriptive statistical features related to gait (such as the maximum value, minimum value, range, root mean square, maximum acceleration, etc. of the signal), nonlinear features (such as approximate entropy, sample entropy, permutation entropy, fuzzy entropy, maximum Lyapunov exponent, fractal dimension, etc.), gait spatiotemporal features (such as gait transfer time, step length, etc.) and spectral features (such as discrete Fourier transform, fast Fourier transform, etc.). Based on the extracted feature data set, the present invention further integrates four different feature selection methods (Fisher score, variance threshold method, sequential forward selection method, sequential backward selection method) to ensure that the feature set is fully analyzed and the best feature subset is retained. The best feature subset after data fusion is normalized to ensure that different features are on the same scale, which is convenient for subsequent data analysis and model training.

[0069] Gait balance ability prediction: Based on the best feature subset after fusion, the elastic network is applied to predict the user's gait and balance ability. In this step, as an embodiment that can be added, first, the basic classifier is constructed using guided sampling, and the weight of each basic classifier is calculated using the Youden index and F score to reduce the bias effect of the basic classifier with performance deviation on the prediction result. By reducing the weight method, the weight of each basic classifier is controlled between 0 and 1. If the weight of the basic classifier is lower than the set critical value, it will be regarded as invalid and discarded in the subsequent integrated classifier. In the proposed integrated classifier, the fusion of the basic classifier is completed by a weighted voting strategy. Finally, the integrated classifier summarizes the result output of the basic classifier and expresses it as the probability of the gait balance prediction result. By training historical data and continuously and automatically iterating, the model can accurately evaluate the user's gait balance ability and automatically identify the user's risk of falling, etc., while providing personalized health advice and intervention measures to help users improve their gait and balance ability.

[0070] Further, the gait balance information management module includes: a real-time monitoring unit and an information management unit;

[0071] A real-time monitoring unit, used to monitor the user's gait and balance ability in real time;

[0072] The information management unit is used to perform health analysis on the monitored data.

[0073] Further, the information management unit includes: a health education subunit and a health warning subunit;

[0074] a health education subunit to improve the user’s knowledge and understanding of gait and balance;

[0075] The health warning subunit is used to issue warnings for abnormal monitoring data.

[0076] Specifically, real-time monitoring and early warning: based on the set threshold, the user's gait and balance ability are monitored in real time. Once an abnormality is detected (such as unstable gait, sudden drop in speed), an alarm and early warning message will be immediately sent to the user and his / her guardian.

[0077] Information management: Elderly users and health care workers can access user data through the management platform, track and manage the health status of the elderly in the community, and provide personalized medical support. The information management platform can also generate community health reports to help health managers analyze the health trends of the elderly population and formulate targeted public health policies.

[0078] User interface: It is divided into a medical staff management interface and an elderly user management interface. The medical staff interface is mainly used to manage and track the gait balance health changes of elderly users, including the gait balance test scores and historical records of elderly users. The elderly user interface provides a simple, intuitive and aging-friendly graphical interface design, supports touch screen operation, and is convenient for elderly users to view their exercise data, evaluation results and health recommendations through smartphones or tablets.

[0079] In this embodiment, the health education module is designed to improve the user's cognition and understanding of gait and balance management. This module provides basic knowledge about gait and balance, including educational resources on common problems, preventive measures, and improvement techniques. For example, it displays exercises and rehabilitation exercises suitable for the elderly to help users perform physical exercises correctly. Based on the user's monitoring data, personalized health tips and suggestions are regularly pushed to promote the user's health management awareness.

[0080] Specifically, based on the user's health information and test results, the system will automatically generate personalized exercise plans, dietary guidance, and fall prevention recommendations to improve the elderly's gait and balance management abilities.

[0081] Real-time monitoring module: The real-time monitoring module is designed to continuously monitor and analyze the user's gait and balance ability. By collecting and analyzing the user's gait data in real time, such as stride length, cadence, center of gravity position, etc., once an abnormal situation (fall risk and gait abnormality, etc.) is detected, an alarm is issued in time to notify the elderly user and his guardian so that necessary intervention measures can be taken. In this module, the user's real-time motion data and evaluation results are displayed through a visual interface to help users instantly understand their gait balance status.

[0082] History module: The history module is designed to store and manage the gait and balance data and health records of elderly users. The system automatically records each gait and balance monitoring data of the user and generates historical data charts for easy viewing and analysis by users. Based on historical data, the system automatically analyzes the changing trends of the user's gait and balance ability to help users understand their long-term health status. Regularly generate detailed health reports and provide them to elderly users and medical care teams. Combined with the expertise of physical therapists, it provides users with personalized exercise plans, dietary recommendations, and guidance on the use of assistive devices.

[0083] Personal Center Module: The Personal Center Module is designed to provide a personalized experience and management interface. Users can view and update their personal information, including basic health information, exercise habits, etc., so that the system can provide personalized health intervention suggestions. Users can also customize the system's notification settings and choose the types of health tips and alerts they want to receive.

[0084] In this embodiment, step 1: device connection

[0085] This embodiment provides a gait and balance self-management system based on multimodal sensing technology, including: an optical sensor, an inertial sensor, a sensor signal data processor, and a gait balance information management platform (taking a tablet as an example).

[0086] This implementation example uses a homemade strap to place the inertial sensor in the strap box, which can be worn on the pelvis of elderly users. Connect the optical sensor to the gait balance information management platform via a data cable and turn on the inertial sensor Bluetooth switch. Click Search Device on the information platform to ensure that the gait balance information management platform is connected to both the inertial sensor and the optical sensor.

[0087] The gait balance information management platform includes four modules: health education, real-time monitoring, history recording and personal center. It can evaluate the gait balance ability of elderly users in real time, such as fall risk and gait abnormality, by combining optical sensors and inertial sensors. By tracking changes in personal gait balance ability and combining the advice of medical staff, the system can provide comprehensive health intervention suggestions such as diet, exercise, and assistive devices to promote the gait balance ability and physical health of the elderly.

[0088] Step 2: Initial Setup

[0089] When a user first uses the gait and balance management system, he or she logs in through the user interface and registers basic personal information and health information, etc. This information will be used for subsequent gait and balance performance analysis and health personal profile tracking.

[0090] Step 3: Gait and balance assessment

[0091] In the real-time monitoring module interface of the gait and balance information management platform, users can select a functional test item to evaluate gait and balance ability. These test items include but are not limited to the 3-meter stand-and-walk test and the five sit-ups test.

[0092] Users can select different functional test items on the interface. Each test item will provide relevant test instructions and applicable scenarios. For example, the 3-meter stand-and-walk test is suitable for evaluating gait and mobility in daily life, while the five sit-ups test focuses on the evaluation of lower limb strength and function. After selecting the test item, the system will provide a detailed test action demonstration, including video guidance and text instructions, to ensure that users can accurately understand and perform the test steps. The video demonstration will be presented by a professional physical therapist or a verified standard animation to ensure that users can imitate the correct action posture.

[0093] After determining the functional test items, the user wears the inertial sensor on the pelvis behind the back through the strap, and places the optical sensor and gait balance information management platform about 1 meter in front. The optical sensor is placed 1.5 meters above the ground using a bracket to ensure that the platform displays the real-time three-axis acceleration, angular velocity and angle information of the inertial sensor, as well as the displacement data of 25 human bone points in three-dimensional space captured by the optical sensor.

[0094] According to the functional test action demonstration, the user performs gait and balance tests. The multimodal sensor will collect the user's posture information in real time and upload the data to the cloud server in real time. The user's gait and balance evaluation results are generated in real time through data fusion algorithms and artificial intelligence algorithms. All collected data and evaluation results will be securely stored in the cloud, and users can view the fall risk, gait cycle and other evaluation results in real time through the information management platform interface.

[0095] Step 4: Personalized health education information recommendation

[0096] Based on the user's gait balance assessment results, the health education module will provide users with personalized health education information. This information is mainly aimed at improving users' health knowledge level and helping them better understand and manage their gait and balance abilities. Specifically, it includes health knowledge popularization (such as how to maintain a good posture, correct walking techniques, how to improve lower limb strength, etc.), preventive health education (such as removing tripping objects in the home, using anti-slip mats, etc.), exercise and rehabilitation knowledge (such as gait training, balance exercises, and flexibility training), diet and lifestyle recommendations (such as how to enhance bone and muscle strength through reasonable nutritional intake) and the use of assistive tools (such as crutches or walkers). Based on this personalized health education information, users can learn in depth how to improve their gait and balance abilities through a scientific lifestyle, thereby achieving self-management.

[0097] Step 5: Tracking changes in personal gait and balance ability and health management

[0098] In the history module of the information management platform, each test data of the user will be recorded and stored in chronological order. Users can view past gait and balance test results through charts, including key parameters such as cadence, stride, stand-up speed, balance score, etc., to understand how their abilities change over time. The system will also automatically analyze the user's historical data and display the trend of the user's gait and balance ability changes in different time periods through visual charts. Users can view past improvements or degradations to better understand their health status. Based on the changes in the user's gait and balance ability, the system will recommend appropriate health interventions. These interventions may include adjusting training plans, strengthening specific balance exercises, or recommending further consultation with medical staff. If the system detects a significant decline in gait ability, it may prompt the user to take additional safety measures, such as increasing the frequency of use of assistive tools. Through the history module, users can obtain long-term health management support, so that they can gradually improve their gait and balance abilities in a data-driven way.

[0099] Step 6: Personalize the experience and manage

[0100] In the Personal Center module, users can view and update their personal information. This includes basic health information, such as age, weight, height, medical history, etc., as well as exercise habits, such as the type, frequency and duration of regular exercise. The system will automatically analyze and provide more accurate health intervention suggestions based on the information updated by the user. In addition, in the custom notification settings section, users can flexibly configure the system's notification function and choose the gait balance prompts, alarm types and frequencies they want to receive. Through these personalized notification settings, users can better manage their gait balance health and ensure that personalized health intervention measures are followed.

[0101] This embodiment uses advanced multimodal sensing technology and intelligent algorithms to provide users with personalized gait balance assessment and health intervention measures through aging-friendly design strategies, providing an effective home health management solution for the elderly, helping them improve their quality of life and continue their ability to live independently, while reducing the burden on the medical system.

[0102] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A gait and balance self-management system based on multimodal sensing technology, characterized by: Multimodal gait data acquisition module, data fusion processing module and gait balance information management module; The multimodal gait data acquisition module is used to collect multi-dimensional gait information; The data fusion processing module is used to fuse and predict the gait information; The gait balance information management module is used to monitor and manage the gait information in real time.

2. The gait and balance self-management system based on multimodal sensing technology according to claim 1, characterized in that: The multimodal gait data acquisition module includes: an optical sensor, an inertial sensor and a data transmission unit; The optical sensor is used to capture the user's skeleton and human body posture data in real time through a camera and infrared technology; The inertial sensor is used to capture the user's motion state data; The data transmission unit is used to transmit the posture data and motion state data to a data fusion processing module.

3. The gait and balance self-management system based on multimodal sensing technology according to claim 2, characterized in that: The data fusion processing module includes: a data receiving unit, a data processing unit, a data fusion unit, a gait balance ability prediction unit and a data storage unit; The data receiving unit is used to receive the posture data and motion state data; The data processing unit is used to process the posture data and motion state data; The data fusion unit is used to fuse the posture data and the motion state data through a multimodal data fusion algorithm to obtain fused data; The gait balance ability prediction unit is used to input the fusion data into the prediction model to predict the gait balance ability; The data storage unit is used to store posture data and motion state data.

4. The gait and balance self-management system based on multimodal sensing technology according to claim 3 is characterized in that: The data processing unit includes: a data preprocessing subunit and a feature extraction subunit; The data preprocessing subunit is used to perform denoising and outlier removal processing on the posture data and motion state data, and fill in missing values ​​using an interpolation method; The feature extraction subunit is used to extract features from the preprocessed data and normalize the features.

5. The gait and balance self-management system based on multimodal sensing technology according to claim 4, characterized in that: The fused data is input into a prediction model to predict gait balance ability including: The prediction model includes an elastic network, and the prediction model is trained by a training set, and the training set is normalized feature data.

6. The gait and balance self-management system based on multimodal sensing technology according to claim 1, characterized in that: The gait balance information management module includes: a real-time monitoring unit and an information management unit; The real-time monitoring unit is used to monitor the user's gait and balance ability in real time; The information management unit is used for health analysis of the monitored data.

7. The gait and balance self-management system based on multimodal sensing technology according to claim 6, characterized in that: The information management unit includes: a health education subunit and a health warning subunit; The health education subunit is used to improve the user's cognition and understanding of gait and balance ability; The health warning subunit is used to issue a warning for abnormal monitoring data.

Images