Personalized rehabilitation plan generation method and system based on multi-modal data

Abstract

The invention discloses a personalized rehabilitation plan generation method and system based on multi-modal data, and relates to the technical field of medical rehabilitation, and the method comprises the steps: obtaining physiological data, motion data and emotion and cognition data of a patient; preprocessing the physiological data, the motion data and the emotion and cognition data to obtain multi-modal fusion data; analyzing the multi-modal fusion data by adopting a pre-trained machine learning model, and determining the rehabilitation potential and demand of the patient to obtain an evaluation result; a personalized rehabilitation plan of a rehabilitation training type is automatically generated according to an evaluation result, and training parameters, psychological counseling and nutrition suggestions are generated according to the conditions of the patient. According to the personalized rehabilitation plan generation method and system based on the multi-modal data, accurate evaluation can be carried out according to the conditions of the patient, the personalized rehabilitation plan is generated, the rehabilitation efficiency of the patient can be improved, and it is ensured that the rehabilitation process is more scientific and efficient.

Description

Technical Field

[0001] The present application relates to the field of medical rehabilitation technology, and in particular to a method and system for generating a personalized rehabilitation plan based on multimodal data. Background Art

[0002] With the continuous advancement of medical technology and the increasing aging of the global population, rehabilitation medicine plays an increasingly important role in modern society. It not only plays a key role in improving the quality of life of the elderly and helping rehabilitation patients return to society, but also shows great potential in many fields such as chronic disease management, sports injury recovery, and neurological disease rehabilitation. Through the combination of personalized treatment plans and modern technology, rehabilitation medicine helps patients restore physical functions, improve their independence in life, and reduce the burden on families and society, becoming an important force in promoting the development of a healthy society.

[0003] However, traditional rehabilitation treatment methods often rely on the doctor's clinical experience and the patient's subjective feedback, which is prone to inaccurate evaluation and difficulty in quantifying treatment effects. Many patients also face difficulties in the rehabilitation process, such as single treatment methods, boring rehabilitation process, lack of personalized and targeted treatment plans, etc. These factors often lead to inefficient rehabilitation effects and poor patient compliance. Summary of the invention

[0004] To this end, the present application provides a method and system for generating a personalized rehabilitation plan based on multimodal data to solve the problem that the existing rehabilitation treatment methods lack personalized and targeted treatment plans.

[0005] In order to achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, a method for generating a personalized rehabilitation plan based on multimodal data comprises:

[0007] Obtain patient physiological data, movement data, and emotional and cognitive data;

[0008] Preprocessing the physiological data, the motion data, and the emotion and cognitive data to obtain multimodal fusion data;

[0009] Using a pre-trained machine learning model to analyze the multimodal fusion data, and determine the patient's rehabilitation potential and needs to obtain an assessment result;

[0010] A personalized rehabilitation plan of the rehabilitation training type is automatically generated based on the evaluation results, and training parameters, psychological counseling and nutritional suggestions are generated based on the patient's own situation.

[0011] Preferably, the physiological data includes muscle activity data, heart rate change data, and plantar or sitting pressure distribution data.

[0012] Preferably, the emotion and cognitive data include facial expression recognition data, speech recognition data and psychological test questionnaire data.

[0013] Preferably, the machine learning model is a cluster analysis model, a classification algorithm model or a regression model.

[0014] Preferably, the method further includes: continuously acquiring the patient's rehabilitation data and adjusting the rehabilitation plan according to the rehabilitation data.

[0015] Preferably, the method further includes: generating rehabilitation training scenes using AR or VR technology.

[0016] Preferably, the method further includes: receiving training effect feedback from patients, and generating corrective suggestions based on the training effect feedback from patients.

[0017] In a second aspect, a personalized rehabilitation plan generation system based on multimodal data includes:

[0018] Multimodal data acquisition module, used to obtain patient physiological data, movement data, and emotional and cognitive data;

[0019] A data preprocessing module, used for preprocessing the physiological data, the motion data, and the emotion and cognitive data to obtain multimodal fusion data;

[0020] An evaluation module, used to analyze the multimodal fusion data using a pre-trained machine learning model, and determine the patient's rehabilitation potential and needs to obtain an evaluation result;

[0021] The rehabilitation plan generation module is used to automatically generate a personalized rehabilitation plan of the rehabilitation training type according to the evaluation results, and generate training parameters, psychological counseling and nutritional suggestions according to the patient's own situation.

[0022] Preferably, the system further comprises: a dynamic adjustment module for continuously acquiring the patient's rehabilitation data and adjusting the rehabilitation plan according to the rehabilitation data.

[0023] Preferably, it also includes: a multimodal rehabilitation experience module for generating rehabilitation training scenes using AR technology.

[0024] Compared with the prior art, this application has at least the following beneficial effects:

[0025] 1. The present application provides a method and system for generating a personalized rehabilitation plan based on multimodal data, including: obtaining the patient's physiological data, motor data, and emotional and cognitive data; preprocessing the physiological data, motor data, and emotional and cognitive data to obtain multimodal fusion data; using a pre-trained machine learning model to analyze the multimodal fusion data, and determine the patient's rehabilitation potential and needs to obtain an evaluation result; automatically generating a personalized rehabilitation plan of the rehabilitation training type based on the evaluation result, and generating training parameters, psychological counseling, and nutritional recommendations based on the patient's own situation. The present application provides a method and system for generating a personalized rehabilitation plan based on multimodal data that can accurately evaluate the patient's own situation and generate a personalized rehabilitation plan, which can improve the patient's rehabilitation efficiency and ensure that the rehabilitation process is more scientific and efficient.

[0026] 2. Using AR or VR technology to generate rehabilitation training scenarios increases the fun and interactivity of the patient's rehabilitation process, and improves patient participation and satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more intuitively illustrate the prior art and the present application, exemplary drawings are given below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing the present application; for example, those skilled in the art are capable of easily making conventional adjustments or further optimizations to the addition / reduction / attribution division, specific shapes, positional relationships, connection methods, dimensional ratios, etc. of certain units (components) based on the technical concepts and exemplary drawings disclosed in the present application.

[0028] Figure 1 A flow chart of a method for generating a personalized rehabilitation plan based on multimodal data provided in Example 1 of the present application. DETAILED DESCRIPTION

[0029] The present application is further described below in detail through specific embodiments in conjunction with the accompanying drawings.

[0030] In the description of this application: unless otherwise specified, the meaning of "plurality" is two or more. The terms "first", "second", "third", etc. in this application are intended to distinguish the objects referred to, and do not have special meanings in terms of technical connotations (for example, they should not be understood as emphasizing the importance or order, etc.). Expressions such as "including", "comprising", "having", etc. also mean "not limited to" (certain units, components, materials, steps, etc.).

[0031] The terms such as "upper", "lower", "left", "right", "middle", etc. cited in this application are usually used to indicate the general relative position relationship for the purpose of intuitive understanding by referring to the drawings, and are not absolute limitations on the position relationship in the actual product.

[0032] Embodiment 1

[0033] See also Figure 1 This embodiment provides a method for generating a personalized rehabilitation plan based on multimodal data, including:

[0034] S1: Obtain patient physiological data, movement data, and emotional and cognitive data;

[0035] Specifically, this step uses integrated electromyographic sensors, electrocardiographic sensors, and pressure distribution sensors to monitor the patient's muscle activity, heart rate changes, and sole or sitting pressure distribution in real time, thereby obtaining physiological data such as muscle activity data, heart rate change data, and sole or sitting pressure distribution data. The collected physiological data needs to be pre-processed by filtering, denoising, and other technologies to ensure the accuracy and reliability of the data.

[0036] Motion data includes the patient's movement trajectory and posture captured by high-frame rate, high-resolution cameras, combined with deep learning algorithms for motion recognition and analysis; wearable devices equipped with sensors such as accelerometers and gyroscopes can also be used to more accurately capture the patient's body movement data.

[0037] Emotional and cognitive data include facial expression recognition data, speech recognition data and psychological test questionnaire data; among them, facial expression recognition data uses image recognition technology to analyze the patient's facial expression in real time and evaluate his or her emotional state; speech recognition data uses speech recognition technology to analyze the patient's voice and intonation to assist in evaluating his or her psychological state; psychological test questionnaire data regularly pushes psychological test questionnaires to patients to collect data on changes in their psychological state and cognitive abilities.

[0038] It should be noted that in this step, the physiological data acquisition sensor selects high-precision, low-power, and easy-to-wear physiological sensors to ensure the accuracy and reliability of the data; the use of deep learning algorithms for motion recognition and analysis can improve the accuracy and stability of motion capture.

[0039] S2: Preprocess physiological data, motion data, emotion and cognitive data to obtain multimodal fusion data;

[0040] Specifically, this step preprocesses the physiological data, motion data, and emotional and cognitive data, including cleaning and integration, to form a unified data format and obtain multimodal fusion data.

[0041] S3: Use pre-trained machine learning models to analyze multimodal fusion data and determine the patient's rehabilitation potential and needs to obtain assessment results;

[0042] Specifically, this step uses trained machine learning algorithms such as cluster analysis models, classification algorithm models, and regression models to conduct in-depth analysis of the fused multimodal fusion data, thereby identifying the rehabilitation potential and needs of rehabilitation patients and obtaining evaluation results.

[0043] It should be noted that during the evaluation, the user is also required to input the patient's age, gender, disease type, physical condition and other personal information through the user interface.

[0044] S4: Automatically generate a personalized rehabilitation plan based on the assessment results, and generate training parameters, psychological counseling and nutritional recommendations based on the patient's own situation.

[0045] Specifically, this step automatically generates a personalized rehabilitation plan for various types of rehabilitation training, including strength training, balance training, and flexibility training, based on the evaluation results, and automatically generates training parameters such as training intensity, frequency, and duration based on the patient's specific situation, as well as personalized psychological counseling and nutritional advice based on the patient's mental and physical condition.

[0046] This embodiment provides a method for generating a personalized rehabilitation plan based on multimodal data, and also includes dynamic adjustment: continuously acquiring the patient's rehabilitation data and adjusting the rehabilitation plan according to the rehabilitation data. Specifically, by continuously monitoring the patient's rehabilitation progress, including changes in physiological parameters, movement recovery, psychological state, etc., and using AI algorithms to automatically adjust the rehabilitation plan, the efficiency and safety of the rehabilitation process are ensured.

[0047] This embodiment provides a method for generating a personalized rehabilitation plan based on multimodal data, and further includes: generating rehabilitation training scenes using AR or VR technology. Specifically, VR technology is used to simulate daily life scenes, such as walking, going up and down stairs, etc., to improve the patient's rehabilitation experience and participation; AR technology is used to design gamified rehabilitation training scenes, such as virtual shooting, virtual fishing, etc., to increase the fun of rehabilitation training.

[0048] This embodiment provides a method for generating a personalized rehabilitation plan based on multimodal data, and further includes: receiving training effects fed back by patients, and generating correction suggestions based on the training effects fed back by patients. Specifically, the training effects of patients are fed back in real time through screen display or wearable devices, such as correctness of movements, power output, etc., and immediate encouragement and correction guidance are provided based on the feedback results to help patients better complete rehabilitation training.

[0049] This embodiment provides a method for generating a personalized rehabilitation plan based on multimodal data, and also includes: remote rehabilitation support. Specifically, this embodiment supports remote video communication between doctors and patients, which facilitates doctors to remotely monitor and immediately guide patients' rehabilitation training. Through remote rehabilitation support, the number of times patients go back and forth to the hospital is reduced, the burden on medical staff is reduced, and the allocation of medical resources is optimized; advanced video communication technology and data encryption technology are used during video to ensure smooth and secure remote communication.

[0050] The method for generating a personalized rehabilitation plan based on multimodal data provided in this embodiment has the following advantages:

[0051] (1) Improve rehabilitation efficiency: Ensure that the patient's rehabilitation process is more scientific and efficient through accurate assessment and personalized rehabilitation plans;

[0052] (2) Enhanced patient experience: Using AR or VR technology to generate rehabilitation training scenarios increases the fun and interactivity of patients during the rehabilitation process, improving patient participation and satisfaction;

[0053] (3) Optimizing resource allocation: Telerehabilitation supports reducing the number of times patients travel to and from the hospital, alleviating the burden on medical staff and optimizing the allocation of medical resources;

[0054] (4) Promote scientific research: The large amount of rehabilitation data collected provides valuable information for research in the field of rehabilitation medicine and promotes the development of the discipline.

[0055] In summary, the personalized rehabilitation plan generation method based on multimodal data provided in this embodiment deeply integrates advanced artificial intelligence technology, biosensor technology, big data analysis, and virtual reality (VR) / augmented reality (AR) technology into the field of rehabilitation medicine, providing patients with a comprehensive, accurate, and personalized rehabilitation experience, which is expected to significantly improve the rehabilitation effect and improve the quality of life of patients.

[0056] Embodiment 2

[0057] This embodiment provides a system for generating a personalized rehabilitation plan based on multimodal data, including:

[0058] Multimodal data acquisition module, used to obtain patient physiological data, movement data, and emotional and cognitive data;

[0059] A data preprocessing module, used for preprocessing the physiological data, the motion data, and the emotion and cognitive data to obtain multimodal fusion data;

[0060] An evaluation module, used to analyze the multimodal fusion data using a pre-trained machine learning model, and determine the patient's rehabilitation potential and needs to obtain an evaluation result;

[0061] The rehabilitation plan generation module is used to automatically generate a personalized rehabilitation plan of the rehabilitation training type according to the evaluation results, and generate training parameters, psychological counseling and nutritional suggestions according to the patient's own situation.

[0062] The embodiment provides a system for generating a personalized rehabilitation plan based on multimodal data, which further includes: a dynamic adjustment module for continuously acquiring rehabilitation data of the patient and adjusting the rehabilitation plan according to the rehabilitation data.

[0063] The present embodiment provides a system for generating a personalized rehabilitation plan based on multimodal data, which also includes: a multimodal rehabilitation experience module for generating rehabilitation training scenarios using AR technology.

[0064] The present embodiment provides a system for generating a personalized rehabilitation plan based on multimodal data, further comprising: a feedback module for receiving training effect feedback from patients, and generating correction suggestions according to the training effect feedback from patients.

[0065] The embodiment provides a personalized rehabilitation plan generation system based on multimodal data, which also includes: a remote video module for remote rehabilitation support, which supports remote video communication between doctors and patients, and facilitates doctors to remotely monitor and instantly guide patients' rehabilitation training.

[0066] For the specific implementation content of each module in a personalized rehabilitation plan generation system based on multimodal data, please refer to the above definition of a personalized rehabilitation plan generation method based on multimodal data, which will not be repeated here.

[0067] The technical features of the above embodiments may be arbitrarily combined (as long as there is no contradiction in the combination of these technical features). To make the description concise, not all possible combinations of the technical features in the above embodiments are described; these embodiments that are not explicitly written should also be considered to be within the scope of this specification.

Claims

1. A method for generating a personalized rehabilitation plan based on multimodal data, characterized in that: include: Obtain patient physiological data, movement data, and emotional and cognitive data; Preprocessing the physiological data, the motion data, and the emotion and cognitive data to obtain multimodal fusion data; Using a pre-trained machine learning model to analyze the multimodal fusion data, and determine the patient's rehabilitation potential and needs to obtain an assessment result; A personalized rehabilitation plan of the rehabilitation training type is automatically generated based on the evaluation results, and training parameters, psychological counseling and nutritional suggestions are generated based on the patient's own situation.

2. The method for generating a personalized rehabilitation plan based on multimodal data according to claim 1, characterized in that: The physiological data includes muscle activity data, heart rate change data, and sole or sitting pressure distribution data.

3. The method for generating a personalized rehabilitation plan based on multimodal data according to claim 1, characterized in that: The emotion and cognitive data include facial expression recognition data, speech recognition data and psychological test questionnaire data.

4. The method for generating a personalized rehabilitation plan based on multimodal data according to claim 1, characterized in that: The machine learning model is a cluster analysis model, a classification algorithm model or a regression model.

5. The method for generating a personalized rehabilitation plan based on multimodal data according to claim 1, characterized in that: Also includes: Continuously obtain the patient's rehabilitation data and adjust the rehabilitation plan based on the rehabilitation data.

6. The method for generating a personalized rehabilitation plan based on multimodal data according to claim 1, characterized in that: Also includes: Use AR or VR technology to generate rehabilitation training scenarios.

7. The method for generating a personalized rehabilitation plan based on multimodal data according to claim 1, characterized in that: Also includes: Receive training effect feedback from patients, and generate corrective suggestions based on the training effect feedback from patients.

8. A personalized rehabilitation plan generation system based on multimodal data, characterized in that: include: Multimodal data acquisition module, used to obtain patient physiological data, movement data, and emotional and cognitive data; A data preprocessing module, used for preprocessing the physiological data, the motion data, and the emotion and cognitive data to obtain multimodal fusion data; An evaluation module, used to analyze the multimodal fusion data using a pre-trained machine learning model, and determine the patient's rehabilitation potential and needs to obtain an evaluation result; The rehabilitation plan generation module is used to automatically generate a personalized rehabilitation plan of the rehabilitation training type according to the evaluation results, and generate training parameters, psychological counseling and nutritional suggestions according to the patient's own situation.

9. The personalized rehabilitation plan generation system based on multimodal data according to claim 8, characterized in that: Also includes: The dynamic adjustment module is used to continuously obtain the patient's rehabilitation data and adjust the rehabilitation plan based on the rehabilitation data.

10. The personalized rehabilitation plan generation system based on multimodal data according to claim 8, characterized in that: Also includes: Multimodal rehabilitation experience module, used to generate rehabilitation training scenarios using AR technology.

Images