Lower extremity exoskeleton robot system based on virtual reality feedback

Abstract

The application provides a lower limb exoskeleton robot system based on virtual reality feedback, which can be applied to the technical field of medical treatment and the technical field of artificial intelligence. The system comprises: an exoskeleton robot device, which is used for collecting electromyographic signals and electrocardiographic signals of a user wearing the exoskeleton robot in a rehabilitation training process, and performing feature extraction on the electromyographic signals and electrocardiographic signals to obtain physiological characteristics, wherein the physiological characteristics comprise the following information: heart rate variability, electromyographic amplitude and electromyographic average power; a communication server, which is used for transmitting the physiological characteristics to a virtual reality device; and the virtual reality device, which is used for inputting the physiological characteristics into a fitting model to obtain a psychological state of the user, wherein a psychological function of the fitting model is obtained by fitting a preset psychological grade set, a heart rate variability set, an electromyographic amplitude set and an electromyographic average power set; and a rehabilitation training mode of the virtual reality device is updated according to the psychological state.

Description

Technical Field

[0001] This invention relates to the fields of medical technology and artificial intelligence technology, and more specifically to a lower limb exoskeleton robot system with virtual reality feedback. Background Technology

[0002] Stroke is a neurological disease with a high rate of disability and is difficult to cure. Motor impairments, in particular, are hard to recover from in a short period, causing immense psychological distress and a heavy economic burden on the individual, their family, and society. Rehabilitation training can promote and induce plasticity in the central nervous system, thereby altering the functional connections and structure of brain regions to adapt to motor demands.

[0003] Because patients are in relatively poor physical condition and have limited access to a variety of rehabilitation training environments, they lack vivid and engaging training scenarios during the rehabilitation process. Summary of the Invention

[0004] In view of the above problems, the present invention provides a lower limb exoskeleton robot system based on virtual reality feedback.

[0005] According to a first aspect of the present invention, a lower limb exoskeleton robot system based on virtual reality feedback is provided, comprising: an exoskeleton robot device for collecting electromyographic (EMG) and electrocardiogram (ECG) signals of a user wearing the exoskeleton robot during rehabilitation training, and extracting features from the EMG and ECG signals to obtain physiological features, the physiological features including the following information: heart rate variability, EMG amplitude, and EMG average power; a communication server for transmitting the physiological features to a virtual reality device; a virtual reality device for inputting the physiological features into a fitting model to obtain the user's psychological state, wherein the psychological function of the fitting model is obtained by fitting a set of preset psychological levels, a set of heart rate variability, a set of EMG amplitude, and a set of EMG average power, the psychological function representing the relationship between the preset psychological levels and the physiological features; and updating the rehabilitation training mode of the virtual reality device according to the psychological state.

[0006] According to an embodiment of the present invention, the exoskeleton device includes: an electrophysiological module for collecting electromyographic and electrocardiographic signals of the user during rehabilitation training; and an exoskeleton robot module for collecting motion information of the user during rehabilitation training.

[0007] According to an embodiment of the present invention, the electrophysiology module includes: an electromyography differential conditioning circuit for acquiring electromyography signals of the user during rehabilitation training; an electrocardiogram differential conditioning circuit for acquiring electrocardiogram signals of the user during rehabilitation training; and an electrophysiology main control unit for extracting features from the electromyography signals and electrocardiogram signals to obtain physiological features.

[0008] According to an embodiment of the present invention, the exoskeleton robot module includes: a sensor unit for collecting motion information of the user during rehabilitation training, the motion information being used to assess the balance function of the exoskeleton robot.

[0009] According to an embodiment of the present invention, the sensor unit includes: a joint module encoder for acquiring joint angles of a user wearing an exoskeleton robot during rehabilitation training, the motion information including joint angles; a distance sensor for measuring the limb length of the user wearing the exoskeleton robot; and a posture sensor for measuring the posture and gait cycle of the user wearing the exoskeleton robot during rehabilitation training, the motion information also including posture and gait cycle.

[0010] According to an embodiment of the present invention, the exoskeleton robot device further includes an execution module, which includes: a joint module unit for driving the user's limb joints to rotate; and an electric push-pull unit for adjusting the limb length of the exoskeleton robot worn by the user.

[0011] According to an embodiment of the present invention, the virtual reality device includes: a wearable device for presenting a virtual scene in a rehabilitation training mode to a user; a locator for locating the user's movement position during the rehabilitation training process; and a mixed reality processor for projecting the movement position onto the virtual scene and / or evaluating motion information during the rehabilitation training process.

[0012] According to an embodiment of the present invention, the rehabilitation training mode includes training tasks and virtual scenes. The rehabilitation training mode of updating the virtual reality device according to the psychological state includes: updating the virtual scene of the virtual reality device in response to a preset psychological state, and obtaining the updated virtual scene. In response to the user being in the preset psychological state in the updated virtual scene, updating the training task of the virtual reality device.

[0013] According to an embodiment of the present invention, the formula between the preset psychological level and the physiological characteristic prediction is:

[0014]

[0015] Y t X is a set of predefined psychological levels. HRV Let X be the set of heart rate variability. EMGF X is the collection of electromyographic amplitudes. EMGV Let C be the set of average electromyographic power, and let C be the psychological function predicted using the fitting model.

[0016] According to an embodiment of the present invention, the mixed reality processor is further configured to: input a preset set of muscle fatigue levels, a set of electromyographic amplitudes, and a set of average electromyographic power into a fitting model to obtain a muscle fatigue function for the user; and use the muscle fatigue function to predict the user's muscle fatigue level, wherein the muscle fatigue function is as follows:

[0017]

[0018] B is the muscle fatigue function, F t This is a set of preset muscle fatigue levels. for The inverse matrix.

[0019] According to an embodiment of the present invention, an exoskeleton robot device is used to collect electromyographic (EMG) and electrocardiogram (ECG) signals of a user wearing the exoskeleton robot during rehabilitation training. Feature extraction is performed on the EMG and ECG signals during rehabilitation training to obtain physiological characteristics. These physiological characteristics are then transmitted to a virtual reality (VR) device using a communication server. The VR device inputs the physiological characteristics into a fitting model to obtain the user's psychological state. The psychological function of the fitting model is obtained by fitting a set of preset psychological levels, a set of heart rate variability, a set of EMG amplitudes, and a set of average EMG power. Therefore, the psychological function can accurately predict the patient's psychological state. The rehabilitation training mode of the VR device is updated based on the psychological state to allow the patient to undergo personalized rehabilitation training in an immersive manner. Attached Figure Description

[0020] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0021] Figure 1 A rehabilitation training scenario combining virtual reality and a lower limb exoskeleton robot according to an embodiment of the present invention is shown.

[0022] Figure 2 A structural block diagram of a lower limb exoskeleton robot system based on virtual reality feedback according to an embodiment of the present invention is shown;

[0023] Figure 3 A structural block diagram of an exoskeleton machine device according to an embodiment of the present invention is shown;

[0024] Figure 4 A structural block diagram of an electrophysiological module according to an embodiment of the present invention is shown;

[0025] Figure 5 A structural block diagram of a sensor unit according to an embodiment of the present invention is shown;

[0026] Figure 6A flowchart illustrating the updating of a rehabilitation training mode of a virtual reality device based on psychological state according to an embodiment of the present invention is shown. Detailed Implementation

[0027] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0030] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0031] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, invention, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0032] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this invention offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0033] Rehabilitation training can promote and induce plasticity in the central nervous system, thereby altering the functional connections and structure of brain regions to adapt to motor demands. However, due to the relatively low physical condition of patients and the limited range of rehabilitation training environments they encounter, they lack vivid and engaging training scenarios during the rehabilitation process.

[0034] According to an embodiment of the present invention, a lower limb exoskeleton robot system based on virtual reality feedback is provided, comprising: an exoskeleton robot device for collecting electromyographic (EMG) and electrocardiogram (ECG) signals from a user wearing the exoskeleton robot during rehabilitation training, and extracting features from the EMG and ECG signals to obtain physiological characteristics, including the following information: heart rate variability, EMG amplitude, and average EMG power; a communication server for transmitting the physiological characteristics to a virtual reality device; and a virtual reality device for inputting the physiological characteristics into a fitting model to obtain the user's psychological state, wherein the psychological function of the fitting model is obtained by fitting a preset set of psychological levels, a set of heart rate variability, a set of EMG amplitude, and a set of average EMG power, and the psychological function characterizes the relationship between the preset psychological levels and the physiological characteristics; and updating the rehabilitation training mode of the virtual reality device according to the psychological state.

[0035] Figure 1 A rehabilitation training scenario combining virtual reality and a lower limb exoskeleton robot according to an embodiment of the present invention is shown.

[0036] like Figure 1 As shown, the rehabilitation training scenario 100 combining virtual reality and a lower limb exoskeleton robot according to this embodiment may include a wearable device 101, a lower limb exoskeleton robot 102, and a network 103. The network 103 serves as a medium for providing a communication link between the wearable device 101 and the lower limb exoskeleton robot 102. The network 103 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0037] Wearable device 101 can provide users with a rich multi-dimensional experience encompassing visual, auditory, and tactile senses, immersing them in a synthetic virtual environment and increasing the enjoyment of rehabilitation training. Wearable device 101 can also display training tasks of varying intensities to users. For example, the wearable device could be glasses, a helmet, etc.

[0038] Users wear the lower limb exoskeleton robot 102 (also known as an exoskeleton robot) for rehabilitation training.

[0039] For example, the lower limb exoskeleton robot 102 can serve as an external support device for stroke patients during rehabilitation training. It can help patients train the movement of the hemiplegic limbs. However, due to patients' low physical condition and the limited rehabilitation training environment, the rehabilitation process can be monotonous and tedious. Furthermore, the lack of visual goals during rehabilitation training reduces its effectiveness.

[0040] For example, wearable device 101 provides users with multi-dimensional training paradigms encompassing vision, hearing, and touch. Lower limb exoskeleton robot 102 provides real-time motion power. The lower limb exoskeleton robot 102 drives the user to complete corresponding movements, and the completed work is compared with the standard movements provided by the virtual reality device. This allows for real-time adjustment of the driving method of the lower limb exoskeleton robot 102, resulting in effective training and ultimately achieving personalized, intelligent, and engaging rehabilitation training.

[0041] The following will be based on Figure 1 The described rehabilitation training scenario, through Figures 2-6 A detailed description is provided of the lower limb exoskeleton robot system based on virtual reality feedback, according to embodiments of the invention.

[0042] Figure 2 A structural block diagram of a lower limb exoskeleton robot system based on virtual reality feedback according to an embodiment of the present invention is shown.

[0043] like Figure 2 As shown, the lower limb exoskeleton robot system based on virtual reality feedback in this embodiment includes an exoskeleton robot device 110, a communication server 120, and a virtual reality device 130.

[0044] The exoskeleton robot device 110 is used to collect electromyographic and electrocardiographic signals of users wearing the exoskeleton robot during rehabilitation training, and to extract features from the electromyographic and electrocardiographic signals to obtain physiological characteristics.

[0045] According to embodiments of this disclosure, physiological characteristics include the following information: heart rate variability, electromyographic amplitude, and average electromyographic power.

[0046] According to embodiments of the present invention, exoskeleton devices may include exoskeleton robots.

[0047] According to embodiments of the present invention, electromyography (EMG) signals characterize electrical signals generated by muscle activity during a user's rehabilitation training. For example, EMG signals can be detected by attaching electrodes to the user's skin surface to detect muscle activity.

[0048] For example, filtering the electromyography (EMG) signal can remove high-frequency and low-frequency noise. Then, interpolation is performed on the filtered EMG signal to obtain a precise interpolated EMG signal. Feature extraction is then performed on the interpolated EMG signal to obtain the average amplitude, average power, and root mean square (RMS) value. The EMG amplitude can be considered as the average amplitude of the EMG signal.

[0049] For example, because the patient's physical condition is still relatively poor, electrocardiogram (ECG) signals cannot accurately determine changes in mood. Electromyographic (EMG) differential signals can partially reflect changes in muscle movement caused by emotional changes.

[0050] Therefore, the electromyographic (EMG) signal difference is calculated to obtain a stable EMG difference signal, reflecting the change in EMG signal amplitude over time. Feature extraction is then performed on the EMG difference signal to obtain its average amplitude, average frequency, and root mean square (RMS) value. The EMG amplitude can be considered as the average amplitude of the EMG difference signal, and the average EMG power can be considered as the average frequency of the EMG difference signal.

[0051] According to embodiments of the present invention, electrocardiogram (ECG) signals characterize electrical signals generated by the heart muscle activity of a user during rehabilitation training. For example, ECG signals from different parts of the user's body (such as the left and right legs) can be acquired by placing a motor on the user's skin surface.

[0052] For example, if the ECG signals are acquired from both legs, the corresponding ECG signals from each leg are time-aligned based on the signal timing information. Then, the aligned ECG signals from each leg are merged to obtain a combined signal. Merging can be achieved by weighted summation of the ECG signals from both legs, with the weights determined based on the user's leg recovery status to minimize the impact of leg recovery on the ECG signal. Feature extraction is then performed on the fused signal to obtain its time-domain and frequency-domain waveform features.

[0053] For example, if an electrocardiogram (ECG) signal from a single leg is acquired, waveform analysis is performed on the ECG signal to identify abnormal waveforms. Feature extraction is then performed on the ECG signal of the abnormal waveforms to obtain the ECG features of the abnormal waveforms.

[0054] For example, based on signal timing information, physiological features such as electromyography (EMG) signals and electrocardiogram (ECG) signals can be time-aligned to obtain physiological features based on time series.

[0055] A communication server 120 is used to transmit physiological characteristics to a virtual reality device 130.

[0056] According to an embodiment of the present invention, the communication server can be communicatively connected to the exoskeleton device 110 and the virtual reality device 130 respectively. The physiological characteristics extracted by the exoskeleton device 110 can be transmitted to the virtual reality device 130, reducing the data processing load of the virtual reality device 130.

[0057] The virtual reality device 130 is used to input physiological characteristics into a fitting model to obtain the user's psychological state. The psychological function of the fitting model is obtained by fitting a set of preset psychological levels, a set of heart rate variability, a set of electromyographic amplitude, and a set of average electromyographic power. The psychological function represents the relationship between preset psychological levels and physiological characteristics. The rehabilitation training mode of the virtual reality device is updated according to the psychological state.

[0058] According to an embodiment of the present invention, a preset psychological level characterizes the degree of a user's psychological state during rehabilitation training. For example, the preset psychological level set can be 10 levels, ranging from a pleasant psychological state to a depressed psychological state, with higher levels indicating a greater degree of depressed psychological state.

[0059] According to embodiments of this disclosure, the set of heart rate variability can be extracted from historical electrocardiogram (ECG) signals of different users. The set of electromyographic (EMG) amplitudes and the set of average EMG power can also be extracted from historical EMG signals of different users.

[0060] For example, if the user's psychological state indicates that they are in a relatively depressed state, the rehabilitation training mode of the virtual reality device should be updated to include more relaxing auditory audio, brightly colored visual scenes, and more relaxed training movements.

[0061] For example, if the user's psychological state indicates that they are in a relatively pleasant psychological state, more intense training movements in the virtual reality device can be updated to improve the training effect.

[0062] According to embodiments of the present invention, the fitting model can be a nonlinear model, a time series model, a machine learning model, etc.

[0063] For example, the fitting model can be a time series model. The set of physiological characteristics based on the time series and the set of preset psychological levels are input into the time series model, and the model parameters such as the autoregressive coefficient and the moving average coefficient in the time series model are optimized by the least squares method to obtain the psychological function.

[0064] The set of physiological features based on time series data can consist of a set of heart rate variability based on time series data, a set of electromyographic amplitude data, and a set of average electromyographic power data. The more accurate the data in the set of physiological features, the more accurate the predicted psychological state.

[0065] For example, since the relationship between psychological state and physiological characteristics (heart rate variability, electromyographic amplitude, average electromyographic power) is not linear, it may involve exponential or other nonlinear relationships. Therefore, the fitting model can also be a nonlinear model.

[0066] By utilizing virtual reality device 130 and exoskeleton device 110, a rich and immersive rehabilitation training environment is constructed, creating a rich and effective interactive loop for users during rehabilitation training. This provides personalized, intelligent, and fun rehabilitation training for the recovery of limb dysfunction and psychological restoration of stroke patients.

[0067] According to an embodiment of the present invention, an exoskeleton robot device is used to collect electromyographic (EMG) and electrocardiogram (ECG) signals of a user wearing the exoskeleton robot during rehabilitation training. Feature extraction is performed on the EMG and ECG signals during rehabilitation training to obtain physiological characteristics. These physiological characteristics are then transmitted to a virtual reality (VR) device using a communication server. The VR device inputs the physiological characteristics into a fitting model to obtain the user's psychological state. Since the psychological function of the fitting model is obtained by fitting a set of preset psychological levels, a set of heart rate variability, a set of EMG amplitudes, and a set of average EMG power, the psychological function can accurately predict the patient's psychological state. The rehabilitation training mode of the VR device is updated based on the psychological state to allow the patient to immerse themselves in personalized rehabilitation training.

[0068] Figure 3 A structural block diagram of an exoskeleton machine device according to an embodiment of the present invention is shown.

[0069] like Figure 3 As shown, the exoskeleton robot device 110 of this embodiment includes an electrophysiological module 111, an exoskeleton robot module 112, an execution module 113, a human-computer interaction module 114, a main processing module 115, and a communication module 116.

[0070] Electrophysiology module 111 is used to collect electromyographic and electrocardiographic signals from the user during rehabilitation training.

[0071] According to an embodiment of the present invention, the electrophysiological module 111 may consist of multiple electrodes to collect electromyographic signals and electrocardiographic signals from multiple sites of the user.

[0072] The exoskeleton robot module 112 is used to collect the user's movement information during rehabilitation training.

[0073] According to an embodiment of the present invention, the exoskeleton robot module 112 may include a variety of sensors.

[0074] According to embodiments of the present invention, motion information may include gait, joint angles, center of gravity changes, walking speed, etc.

[0075] According to embodiments of the present invention, heart rate variability characterizes the degree of variation in the heartbeat cycle. For example, features are extracted from the time-domain and frequency-domain information of the electrocardiogram (ECG) signal to obtain indicators of heart rate variability. Indicators of heart rate variability include, for example, low-frequency power, high-frequency power, and the standard deviation of the time interval between adjacent heartbeats.

[0076] According to an embodiment of the present invention, electromyography amplitude characterizes the strength of the electrical signal generated during muscle contraction.

[0077] According to embodiments of the present invention, the average electromyographic power characterizes the average power of electromyographic signals over a period of time. The average electromyographic power reflects the contraction intensity and fatigue intensity of the muscle.

[0078] The execution module 113 is used to drive the user wearing the exoskeleton robot to perform limb movements.

[0079] The human-computer interaction module 114 is used to operate the exoskeleton robot according to instructions from the user or guardian. For example, the human-computer interaction module 114 may include a touch screen and buttons.

[0080] The main processing module 115 is used to control the electrophysiology module 111, the exoskeleton robot module 112, the execution module 113, the human-computer interaction module 114, and the communication module 116.

[0081] The communication module 116 is connected to the main processing module 115 and the electrophysiology module 111, and is used to realize information interaction between the main processing module 115, the electrophysiology module 111 and the communication server 120.

[0082] Lower limb exoskeleton robot systems based on virtual reality feedback can be battery powered.

[0083] Figure 4 A structural block diagram of an electrophysiological module according to an embodiment of the present invention is shown.

[0084] like Figure 4 As shown, the electrophysiology module 111 of this embodiment includes an electromyography differential conditioning circuit 1111, an electrocardiogram differential conditioning circuit 1112, an electrophysiology main control unit 1113, an analog-to-digital conversion circuit 1114, and electrodes 1115.

[0085] The electromyography differential conditioning circuit 1111 is used to collect electromyography signals from the user during rehabilitation training.

[0086] According to an embodiment of the present invention, the electromyography differential conditioning circuit 1111 is connected to the analog-to-digital converter circuit 1114 and the electrode 1115. The electromyography differential conditioning circuit 1111 can amplify, filter and perform other signal conditioning on the electromyography signals picked up by the electrode 1115.

[0087] The differential ECG conditioning circuit 1112 is used to collect the user's ECG signals during rehabilitation training.

[0088] According to an embodiment of the present invention, the differential ECG conditioning circuit 1112 is connected to the analog-to-digital converter circuit 1114 and the electrode 1115 respectively. The differential ECG conditioning circuit 1112 can amplify, filter and perform other signal conditioning on the ECG signal picked up by the electrode 1115.

[0089] The electrophysiological control unit 1113 is used to extract features from electromyographic and electrocardiographic signals to obtain physiological features.

[0090] According to an embodiment of the present invention, the electrophysiological main control unit 1113 is connected to the communication module 116 and the analog-to-digital conversion circuit 1114 in the exoskeleton machine device 110, respectively. The electrophysiological main control unit 1113 is used to control the operation of the analog-to-digital conversion circuit, preprocess the converted data, extract features from the electromyography (EMG) and electrocardiogram (ECG) signals to obtain physiological features such as heart rate variability, EMG amplitude, and median EMG frequency.

[0091] The analog-to-digital converter circuit 1114 is connected to the electrophysiological main control unit 1113, the electromyography differential conditioning circuit 1111, and the electrocardiogram differential conditioning circuit 1112, respectively. The analog-to-digital converter circuit 1114 is used to convert the analog electromyography signals and electrocardiogram signals output by the electromyography differential conditioning circuit 1111 and the electrocardiogram differential conditioning circuit 1112 into digital signals.

[0092] Electrode 1115 is connected to electromyography differential conditioning circuit 1111 and electrocardiogram differential conditioning circuit 1112, respectively. Electrode 1115 is attached to the lower limb, with the electromyography electrode placed on the ipsilateral leg to collect electromyography signals, and one electrode taken from each of the two legs configured as an electrocardiogram electrode to collect electrocardiogram signals.

[0093] According to an embodiment of the present invention, the exoskeleton robot module 112 includes a sensor unit 1121. The sensor unit 1121 is used to collect motion information of the user during rehabilitation training, and the motion information is used to assess the balance function of the exoskeleton robot.

[0094] Figure 5 A structural block diagram of a sensor unit according to an embodiment of the present invention is shown.

[0095] like Figure 5As shown, the sensor unit 1121 in this embodiment includes a joint module encoder 11211, a ranging sensor 11212, and an attitude sensor 11213.

[0096] The joint module encoder 11211 is used to collect joint angles and motion information, including joint angles, from users wearing exoskeleton robots during rehabilitation training.

[0097] The ranging sensor 11212 is used to measure the limb length of the exoskeleton robot worn by the user.

[0098] The posture sensor 11213 is used to measure the posture and gait cycle of a user wearing an exoskeleton robot during rehabilitation training. The motion information also includes posture and gait cycle.

[0099] According to an embodiment of the present invention, the execution module includes: a joint module unit for driving the user's limb joints to rotate; and an electric push-pull unit for adjusting the limb length of the exoskeleton robot worn by the user to accommodate different users.

[0100] According to an embodiment of the present invention, the virtual reality device includes: a wearable device for presenting a virtual scene in a rehabilitation training mode to a user; a locator for locating the user's movement position during the rehabilitation training process; and a mixed reality processor for projecting the movement position onto the virtual scene and / or evaluating motion information during the rehabilitation training process.

[0101] Mixed reality processors can provide different virtual scenes and training objectives, and can evaluate the completion and accuracy of the movement based on the feedback motion state parameters.

[0102] According to embodiments of the present invention, the wearable device may be a helmet or glasses, used to present different virtual scenes to the user.

[0103] Mixed reality processors and locators can be powered by an electrical supply, while wearable devices can be powered by batteries.

[0104] According to an embodiment of the present invention, the heart rate variability is Electromyographic amplitude is The average power of electromyography is Then the predicted value of muscle fatigue for:

[0105] (1)

[0106] f1 is the muscle fatigue function.

[0107] F t To define a set of preset muscle fatigue levels, muscle strength tests are used to determine the preset muscle fatigue levels. Ft for Any combination of the above, simultaneously with There are n sets of data corresponding to each other. The coefficient matrix is ​​B, and B can characterize the muscle fatigue function.

[0108] According to an embodiment of the present invention, the mixed reality processor is further configured to: input a preset set of muscle fatigue levels, a set of electromyographic amplitudes, and a set of average electromyographic power into a fitting model to obtain a muscle fatigue function for the user; and use the muscle fatigue function to predict the user's muscle fatigue level, wherein the muscle fatigue function is characterized as follows:

[0109] (2)

[0110] B is the coefficient matrix of the muscle fatigue function, which can characterize the muscle fatigue function, F t This is a set of preset muscle fatigue levels. for The inverse matrix.

[0111] y t Given a preset psychological level and f2 as a psychological function, it can be expressed as:

[0112] (3)

[0113] The set of pre-defined psychological levels represents .

[0114] For example, using a scale test to determine a pre-set psychological level y t y t for Record any level of mental state, while simultaneously recording synchronized heart rate variability. Electromyographic amplitude Average power of electromyography .

[0115] According to an embodiment of the present invention, the formula between the preset psychological level and the physiological characteristic prediction is:

[0116] (4)

[0117] Y t X is a set of predefined psychological levels. HRV Let X be the set of heart rate variability. EMGF X is the collection of electromyographic amplitudes. EMGV Let C be the set of average electromyographic power, and let C be the psychological function predicted using the fitting model.

[0118] According to an embodiment of the present invention, the rehabilitation training mode includes training tasks and virtual scenes. The rehabilitation training mode of updating the virtual reality device according to the psychological state includes: updating the virtual scene of the virtual reality device in response to a preset psychological state, and obtaining the updated virtual scene. In response to the user being in the preset psychological state in the updated virtual scene, updating the training task of the virtual reality device.

[0119] According to an embodiment of the present invention, the preset psychological state is that the user is in a depressed or tense psychological state.

[0120] According to embodiments of the present invention, the training task may involve movement, exercise intensity, exercise standards, etc.

[0121] Figure 6 A flowchart illustrating the updating of a rehabilitation training mode of a virtual reality device based on psychological state according to an embodiment of the present invention is shown.

[0122] like Figure 6 As shown, the rehabilitation training mode of the virtual reality device is updated according to the psychological state, including operation S610~S680.

[0123] When operating the S610, in response to the virtual reality device, the user's movements in the rehabilitation training mode are evaluated, and features of the user's psychological state are extracted based on electromyography and electrocardiogram signals to obtain physiological characteristics.

[0124] When operating the S620, physiological characteristics are input into the fitting model to obtain the user's psychological state.

[0125] In operation S630, determine whether the user's psychological state is in the preset psychological state. If yes, execute operation S640; otherwise, execute operation S670.

[0126] By operating the S640, the virtual scene of the virtual reality device is updated, and the updated virtual scene is obtained.

[0127] In operation S650, determine whether the user is in a preset psychological state in the updated virtual scene. If yes, execute operation S660; otherwise, execute operation S670.

[0128] While operating the S660, update the training tasks of the virtual reality device.

[0129] In operation S670, determine whether the user has completed the training task. If yes, execute operation S680; otherwise, execute operation S610.

[0130] Using the S680, evaluate the user's exercise performance on this training task.

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0132] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0133] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A lower limb exoskeleton robot system based on virtual reality feedback, characterized in that, The system is used for lower limb rehabilitation training, and the system includes: An exoskeleton robot device is used to collect electromyographic (EMG) and electrocardiogram (ECG) signals from a user wearing the exoskeleton robot during lower limb rehabilitation training, and to extract features from the EMG and ECG signals to obtain physiological features, which include the following information: heart rate variability, EMG amplitude, and EMG average power. A communication server for transmitting the physiological characteristics to a virtual reality device; The virtual reality device is used to input the physiological characteristics into a fitting model to obtain the user's psychological state. The psychological function of the fitting model is obtained by fitting a set of preset psychological levels, a set of heart rate variability, a set of electromyographic amplitudes, and a set of average electromyographic power. The psychological function characterizes the relationship between the preset psychological levels and the physiological characteristics. The rehabilitation training mode of the virtual reality device is updated according to the psychological state. A mixed reality processor is used to project the user's movement position during the lower limb rehabilitation training process onto a virtual scene in the lower limb rehabilitation training mode, and / or to evaluate the motion information during the lower limb rehabilitation training process; input a set of preset muscle fatigue levels, a set of electromyographic amplitudes, and a set of average electromyographic power into the fitting model to obtain the user's muscle fatigue function; and use the muscle fatigue function to predict the user's muscle fatigue level. Wherein, the heart rate variability characterizes the degree of change in the heartbeat cycle, the electromyographic amplitude is the average amplitude of the electromyographic difference signal, the average electromyographic power is the average frequency of the electromyographic difference signal, and the electromyographic difference signal can partially reflect the changes in muscle movement caused by emotional changes; wherein, the formula between the preset psychological level and the physiological characteristic prediction is: Y t X is the set of preset psychological levels. HRV Let X be the set of heart rate variability. EMGF X is the set of electromyographic amplitudes. EMGV Let C be the set of average electromyographic powers, and let C be the psychological function predicted using the fitting model. The muscle fatigue function is characterized as follows: B is the coefficient matrix of the muscle fatigue function, which can characterize the muscle fatigue function, F t This is the set of preset muscle fatigue levels. for The inverse matrix.

2. The system according to claim 1, characterized in that, The exoskeleton machine device includes: An electrophysiological module is used to collect the electromyographic signals and electrocardiographic signals of the user during the lower limb rehabilitation training process; The exoskeleton robot module is used to collect the user's movement information during the lower limb rehabilitation training process.

3. The system according to claim 2, characterized in that, The electrophysiological module includes: Electromyographic differential conditioning circuit, used to acquire the electromyographic signals of the user during the lower limb rehabilitation training process; The differential ECG conditioning circuit is used to collect the ECG signals of the user during the lower limb rehabilitation training process; The electrophysiological control unit is used to extract features from the electromyographic signals and the electrocardiographic signals to obtain the physiological features.

4. The system according to claim 2, characterized in that, The exoskeleton robot module includes: A sensor unit is used to collect the user's motion information during the lower limb rehabilitation training process, and the motion information is used to evaluate the balance function of the exoskeleton robot.

5. The system according to claim 4, characterized in that, The sensor unit includes: A joint module encoder is used to collect the joint angles of the user wearing the exoskeleton robot during the lower limb rehabilitation training process, and the motion information includes the joint angles; A ranging sensor is used to measure the length of the limbs worn by the user on the exoskeleton robot; A posture sensor is used to measure the posture and gait cycle of the user wearing the exoskeleton robot during the lower limb rehabilitation training process, and the motion information also includes the posture and the gait cycle.

6. The system according to claim 5, characterized in that, The exoskeleton device further includes an execution module, which comprises: Joint module unit, used to drive the user's limb joints to rotate; An electric push-pull unit is used to adjust the limb length of the exoskeleton robot worn by the user.

7. The system according to claim 1, characterized in that, The virtual reality device includes: Wearable device for presenting the virtual scene under the lower limb rehabilitation training mode to the user; A locator is used to locate the user's position during the lower limb rehabilitation training.

8. The system according to claim 7, characterized in that, The lower limb rehabilitation training mode includes training tasks and the virtual scene, and updating the rehabilitation training mode of the virtual reality device according to the psychological state includes: In response to the preset psychological state represented by the psychological state, the virtual scene of the virtual reality device is updated to obtain an updated virtual scene; In response to the user being in the preset psychological state in the updated virtual scene, the training task of the virtual reality device is updated.

Images