Rehabilitation robot control method based on multi-source information perception and electronic device

Abstract

The application discloses a rehabilitation robot control method based on multi-source information perception and electronic equipment, and the method comprises the following steps: acquiring upper limb modal signals of a patient without motor function, wherein the upper limb modal signals comprise electroencephalogram signals, and eye movement signals and / or electromyogram signals; for each kind of upper limb modal signal, using an intention recognition model to perform motor intention recognition on the upper limb modal signal to obtain a corresponding motor intention recognition result; acquiring a confidence degree corresponding to each kind of upper limb modal signal; determining the motor intention of the patient according to the motor intention recognition result corresponding to each kind of upper limb modal signal and the corresponding confidence degree; determining a motor control signal of an upper limb rehabilitation exoskeleton according to the motor intention; and controlling the upper limb rehabilitation exoskeleton to move according to the motor control signal. The application can improve the motor intention recognition accuracy of the patient without motor function in the upper limb rehabilitation training process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical rehabilitation, in particular to a rehabilitation robot control method based on multi-source information perception and an electronic device. BACKGROUND

[0002] In the early stage after stroke surgery, the patient's upper limbs may have motor dysfunction, and severe cases may have limb motor function. In this state, the human upper limb cannot effectively generate electromyographic signals to control its movement, and its rehabilitation training can only be achieved through passive training. For the rehabilitation of motor function at this stage, the modern stroke rehabilitation view tends to be repetitive and task-oriented functional training. Upper limb rehabilitation exoskeleton robots can perform this task. However, since the motor function of the limbs of patients without motor function has been lost, the way for the robot to recognize human movement intention becomes very limited. Currently, the input of the developed intention recognition algorithm includes physical signals and bioelectric signals. Physical signals include inertial signals, angle signals, and pressure signals. Bioelectric signals include electroencephalogram signals and electromyographic signals. For patients with no motor ability in the early stage of stroke, only electroencephalogram signals can achieve patient movement intention recognition. However, due to the problem that electroencephalogram signals are easily disturbed and difficult to decode, the intention recognition accuracy of electroencephalogram signals is low. SUMMARY

[0003] Therefore, the embodiments of the present application provide a rehabilitation robot control method based on multi-source information perception and an electronic device to solve the problem of low accuracy of patient movement intention recognition based on single-mode signals in the upper limb rehabilitation training process of patients without motor function.

[0004] According to a first aspect, the embodiments of the present application provide a rehabilitation robot control method based on multi-source information perception, comprising:

[0005] Obtaining upper limb modal signals of a patient without motor function, the upper limb modal signals including electroencephalogram signals, and eye movement signals and / or electromyographic signals of the upper limbs;

[0006] For each type of upper limb modal signal, using an intention recognition model to recognize the movement intention of the upper limb modal signal to obtain a movement intention recognition result corresponding to the upper limb modal signal; wherein the intention recognition model is constructed based on a support vector machine algorithm;

[0007] Obtaining the confidence corresponding to each type of upper limb modal signal;

[0008] According to the movement intention recognition result corresponding to each type of upper limb modal signal and the corresponding confidence, determining the movement intention of the patient without motor function;

[0009] Determining the movement control signal of the upper limb rehabilitation exoskeleton according to the movement intention.

[0010] controlling the upper limb rehabilitation exoskeleton to move according to the motion control signal.

[0011] In some optional embodiments, the acquiring the upper limb modality signal of the patient without motor function includes:

[0012] controlling a virtual reality glasses worn by the patient without motor function to play a pre-designed upper limb rehabilitation training video for patients without motor function, so as to guide the patient to generate a corresponding upper limb movement intention;

[0013] acquiring the upper limb modality signal generated by the patient without motor function when watching the upper limb rehabilitation training video.

[0014] In some optional embodiments, when the upper limb modality signal includes the eye movement signal, the using the intention recognition model to recognize the movement intention of the upper limb modality signal includes:

[0015] determining a position and / or movement trajectory of a focus point of the eyes of the patient without motor function in a virtual space according to the eye movement signal;

[0016] inputting the position and / or movement trajectory into the intention recognition model for the eye movement signal to obtain a corresponding movement intention recognition result, wherein the intention recognition model for the eye movement signal is trained based on the upper limb rehabilitation training video.

[0017] In some optional embodiments

[0018] The upper limb rehabilitation exoskeleton is provided with a motion capture sensor, and the method further includes:

[0019] determining movement information of each joint of the upper limb rehabilitation exoskeleton according to information collected by the motion capture sensor;

[0020] reconstructing a human upper limb model in a virtual reality scene according to the movement information of each joint of the upper limb rehabilitation exoskeleton.

[0021] In some optional embodiments, the method further includes:

[0022] acquiring an acting force of an object in a virtual reality scene that conforms to a physical law on the human upper limb model;

[0023] controlling the upper limb rehabilitation exoskeleton to move according to the acting force, so as to feed back the acting force to the patient without motor function.

[0024] In some optional embodiments,

[0025] The motion intention of the patient without motor function is determined according to the motion intention recognition result corresponding to each of the upper limb modal signals and the corresponding confidence level.

[0026] The motion intention of the patient without motor function is determined according to the motion intention recognition result corresponding to each of the upper limb modal signals and the corresponding confidence level.

[0027] It is judged whether the motion intention determined preliminarily is consistent with the motion intention recognition result corresponding to the upper limb modal signal with the maximum confidence level.

[0028] If consistent, the motion intention determined preliminarily is taken as the motion intention of the patient without motor function.

[0029] In some optional specific embodiments, the upper limb rehabilitation exoskeleton comprises a plurality of joints, each joint corresponding to one or more driving motors.

[0030] The motion control signal of the upper limb rehabilitation exoskeleton is determined according to the motion intention, comprising:

[0031] The driving signal of the driving motor corresponding to at least part of the joint is determined according to the motion intention, the driving signal comprising at least one of driving time, rotating speed of the driving motor and running direction of the driving motor.

[0032] According to a second aspect, an embodiment of the present application provides a rehabilitation robot control system based on multi-source information perception, comprising:

[0033] A first acquisition module is configured to acquire upper limb modal signals of a patient without motor function, the upper limb modal signals comprising electroencephalogram signals, and eye movement signals and / or myoelectric signals of the upper limb.

[0034] An identification module is configured to, for each of the upper limb modal signals, use an intention identification model to perform motion intention identification on the upper limb modal signals, to obtain a motion intention recognition result corresponding to the upper limb modal signal; wherein the intention identification model is constructed based on a support vector machine algorithm.

[0035] A second acquisition module is configured to acquire a confidence level corresponding to each of the upper limb modal signals.

[0036] An intention determination module is configured to determine the motion intention of the patient without motor function according to the motion intention recognition result corresponding to each of the upper limb modal signals and the corresponding confidence level.

[0037] A control signal determination module is configured to determine a motion control signal of an upper limb rehabilitation exoskeleton according to the motion intention.

[0038] A control module is configured to control the upper limb rehabilitation exoskeleton to move according to the motion control signal.

[0039] According to a third aspect, an electronic device is provided, comprising:

[0040] A memory and a processor are communicatively connected, the memory is configured to store a computer program, and the computer program is configured to be executed by the processor to implement any of the rehabilitation robot control methods based on multi-source information perception according to the first aspect.

[0041] According to a fourth aspect, a computer readable storage medium is provided, the computer readable storage medium is configured to store a computer program, and the computer program is configured to be executed by a processor to implement any of the rehabilitation robot control methods based on multi-source information perception according to the first aspect.

[0042] The embodiment of the present application proposes a stroke early non-motor function patient intention recognition method fusing electroencephalogram and electromyogram and / or eye movement information, and performs motion intention recognition of each signal source through a support vector machine machine learning model (i.e., an intention recognition model constructed based on a support vector machine algorithm), thereby improving the recognition accuracy of the motion intention of the non-motor function patient. BRIEF DESCRIPTION OF DRAWINGS

[0043] The features and advantages of the present application will be more clearly understood through reference to the following drawings, which are presented as illustrative and should not be construed as limiting the present application, in which:

[0044] Figure 1 A flowchart of a rehabilitation robot control method based on multi-source information perception provided by the embodiment of the present application is shown in the figure;

[0045] Figure 2 A schematic diagram of a rehabilitation robot control process based on multi-source information perception provided by the embodiment of the present application is shown in the figure;

[0046] Figure 3 A process schematic diagram of multi-modal signal fusion intention recognition for non-motor function patients provided by the embodiment of the present application is shown in the figure;

[0047] Figure 4 A control process schematic diagram of an upper limb rehabilitation robot for rehabilitation training of non-motor function patients provided by the embodiment of the present application is shown in the figure;

[0048] Figure 5 A structural schematic diagram of a rehabilitation robot control system based on multi-source information perception provided by the embodiment of the present application is shown in the figure;

[0049] Figure 6A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present application.

[0051] It should be noted that the terms “comprising”, “containing” or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the phrase “comprising a” does not exclude the presence of additional identical elements in the process, method, article or device comprising the element. In addition, the terms “first”, “second” and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. In the description of the following embodiments, the meaning of “a plurality of” is two or more, unless otherwise specifically limited.

[0052] Please refer to Figure 1 The embodiment of the present application provides a rehabilitation robot control method based on multi-source information perception, comprising:

[0053] S101: acquiring upper limb modal signals of a patient without motor function, the upper limb modal signals comprising electroencephalogram signals, and eye movement signals and / or electromyogram signals of the upper limb; the upper limb modal signals can also be referred to as upper limb movement intention recognition signals;

[0054] S102: for each of the upper limb modal signals, using an intention recognition model to perform movement intention recognition on the upper limb modal signal to obtain a movement intention recognition result corresponding to the upper limb modal signal; wherein the intention recognition model is constructed based on a support vector machine algorithm; in the embodiment of the present application, each of the upper limb modal signals can correspond to an intention recognition model respectively;

[0055] S103: acquiring a confidence degree corresponding to each of the upper limb modal signals;

[0056] S104: determining a movement intention of the patient without motor function according to the movement intention recognition result corresponding to each of the upper limb modal signals and the corresponding confidence degree;

[0057] S105: determining a movement control signal of the upper limb rehabilitation exoskeleton according to the movement intention;

[0058] S106: controlling the upper limb rehabilitation exoskeleton to move according to the movement control signal. The rehabilitation robot comprises the upper limb rehabilitation exoskeleton.

[0059] Specifically, steps S101-S104 and steps S105-S106 can be executed by different devices, for example, steps S101-S104 are executed by an intention recognition device, and steps S105-S106 can be executed by an upper limb rehabilitation robot (including a processor and an upper limb rehabilitation exoskeleton), but the intention recognition device and the upper limb rehabilitation robot belong to the same upper limb rehabilitation system. At this time, after determining the movement intention of the patient without movement function, the intention recognition device needs to send the movement intention of the patient without movement function to the upper limb rehabilitation robot, so that the upper limb rehabilitation robot can control the movement of the upper limb rehabilitation exoskeleton according to the movement intention of the patient without movement function, and then drive the patient's upper limb to perform rehabilitation movement.

[0060] Steps S101-S104 and steps S105-S106 can also be executed by the upper limb rehabilitation robot, that is, no additional intention recognition device is needed, and the intention recognition is also realized by the upper limb rehabilitation robot.

[0061] The embodiment of the present application proposes an intention recognition method for early stroke patients without movement function by fusing electroencephalogram and electromyogram and / or eye movement information, and performs movement intention recognition of each signal source by constructing a support vector machine machine learning model (i.e. an intention recognition model based on a support vector machine algorithm), which improves the recognition accuracy of the movement intention of the patient without movement function. Then, the patient without movement function can perform upper limb rehabilitation training according to his own intention, which is more interesting than repetitive and task-oriented functional training, and can improve the enthusiasm of the patient for rehabilitation training.

[0062] In some specific embodiments, the upper limb modal signal of the patient without movement function is obtained by:

[0063] Please refer to Figure 2 controlling the virtual reality glasses worn by the patient without movement function to play a pre-designed upper limb rehabilitation training video for the patient without movement function, so as to guide the patient to generate corresponding upper limb movement intention;

[0064] The upper limb modal signal generated by the patient without movement function when watching the upper limb rehabilitation training video is collected.

[0065] In other optional specific embodiments, the electroencephalogram signal can be a steady-state visual evoked potential (SSVEP).

[0066] For electroencephalogram (EEG), the electroencephalogram is filtered by wavelet transform to filter out electromyogram, electrocardiogram, power frequency noise and other noises, extract the electroencephalogram therefrom, and perform baseline correction and bad channel processing. The processed electroencephalogram is segmented according to a time window segmentation scheme, and the characteristic values of each electroencephalogram unit after segmentation are calculated, to form a feature group, which is input into an intention recognition model trained in advance using electroencephalogram samples to obtain a movement intention recognition result. The electroencephalogram can be collected in real time, and the recognition of the electroencephalogram is also performed in real time. For multiple channels of electroencephalogram, after the movement intention corresponding to the electroencephalogram is recognized by the intention recognition model, the weight of each channel corresponding to the movement intention is determined, if the ratio of the root mean square value (RMS) of the electroencephalogram of the channel with the largest weight to the maximum root mean square value (RMSMax) of the channel in all movement intentions is much smaller than the weight of the channel, the recognition and classification are performed again, if the second recognition result is unchanged, the result is output, if the recognition result changes, but the value of RMS / RMSMax of the channel with the largest weight is still much smaller than the weight value, the re-recognition is performed again, until the recognition result is unchanged or the value of RMS / RMSMax of the channel with the largest weight is similar to the weight of the channel, and the recognition result is output as the final movement intention. The weight determination method is: the root mean square value (RMS) of the signal of each electroencephalogram channel in each movement intention is divided by the maximum root mean square value (RMSMax) of the channel in all movement intentions, and then the proportion of the values between channels is calculated as the weight of the electroencephalogram of the channel in the movement intention, and the sum of the weights of all channels is 1.

[0067] For electromyography (EMG), the surface electromyography of the upper limbs of a human body can be collected using a flexible sensor, the electromyography is filtered using a wavelet denoising algorithm, and relatively clean electromyography is extracted. The extracted relatively clean electromyography is segmented according to a time window segmentation scheme and characteristic values are calculated, a characteristic group is formed, and the trained intention recognition model is input to obtain a movement intention recognition result. The collection process and processing process of the electromyography sample used for training of the intention recognition model are consistent with the electromyography used in actual detection. After the movement intention corresponding to the electromyography is recognized using the intention recognition model, the weight of each channel corresponding to the movement intention is judged. If the ratio of the root mean square value (RMS) of the electromyography of the channel with the largest weight to the maximum voluntary contraction value (MVC) of the channel is much smaller than the weight of the channel, the recognition and classification are performed again. If the second recognition result is unchanged, the result is output. If the recognition result changes, but the ratio of the RMS / MVC value of the channel with the largest weight to the weight of the channel is still much smaller than the weight, the recognition is performed again until the recognition result is unchanged or the ratio of the RMS / MVC value of the channel with the largest weight to the weight of the channel is similar to the weight. The output recognition result is the movement intention obtained by the final recognition. The weight determination method is as follows: the root mean square value (RMS) of the signal of each electromyography channel in each movement intention is divided by the maximum voluntary contraction value (MVC) of the electromyography channel, and the ratio of the values between channels is calculated as the weight of the electromyography channel under the movement intention. The sum of the weights of the channels is 1.

[0068] In some specific embodiments, when the upper limb modality signal includes the eye movement signal, the movement intention recognition of the upper limb modality signal using the intention recognition model includes:

[0069] According to the eye movement signal, the position and / or movement trajectory of the focus of the eye of the patient without movement function are determined. Specifically, the eye movement signal can be collected using an eye tracker, and the position and / or movement trajectory of the focus of the eye of the patient without movement function are determined.

[0070] The position and / or movement trajectory are input to the intention recognition model for the eye movement signal to obtain a corresponding movement intention recognition result. The intention recognition model for the eye movement signal is trained based on the upper limb rehabilitation training video.

[0071] In the embodiments of the present application, the main principle of intention recognition based on eye movement signals is to determine the object that the patient wants to interact in the virtual space corresponding to the upper limb rehabilitation training video according to the position and / or movement trajectory of the focus of the eye of the patient without movement function, and then determine the movement intention of the patient according to the interactive function of the object and the position in the virtual space. The interactive function of the object is set by the program, so the interactive function of the object is limited.

[0072] In some specific embodiments, the upper limb rehabilitation exoskeleton is provided with a motion capture sensor (which can be referred to as a motion capture sensor for short), specifically, the motion capture sensor can be arranged at a central position between each joint of the upper limb rehabilitation exoskeleton, the motion capture sensor can be an inertial sensor (Inertial Measurement Unit, IMU), and the motion capture sensor is worn at a fixed position on a limb, such as a forearm, an upper arm, a hand, etc.; the method further comprises:

[0073] According to the information collected by the motion capture sensor, the motion information of each joint of the upper limb rehabilitation exoskeleton is determined;

[0074] Specifically, the process can be realized by a motion capture system. The information collected by the motion capture sensor is the kinematics data of the motion capture sensor itself, including a speed value, an acceleration value, an Euler angle, a magnetometer value, etc., the motion capture system calculates the displacement value of the sensor in three directions of its own coordinate system per unit time according to the collected sensor speed value, and the displacement value of the limb corresponding to the sensor per unit time is calculated by using this method, starting from the calibration position as the initial posture, the limb displacement value at each subsequent time is calculated in turn, and the magnetometer value is compensated with the earth coordinate system to obtain the kinematics data (i.e. motion information) of each joint of the upper limb rehabilitation exoskeleton in the earth coordinate system;

[0075] According to the motion information of each joint of the upper limb rehabilitation exoskeleton, the upper limb model in the virtual reality scene is reconstructed.

[0076] In the embodiment of the application, a virtual interaction scene, i.e. a virtual reality scene, is constructed by using virtual reality technology, the posture information of the upper limb rehabilitation exoskeleton is obtained by using the motion capture sensor arranged on the upper limb rehabilitation exoskeleton, specifically, the spatial position of each joint of the upper limb rehabilitation exoskeleton, and the upper limb model in the virtual reality scene is reconstructed according to the posture of the upper limb rehabilitation exoskeleton, so as to realize the interaction between the reality and the virtual reality scene.

[0077] In some specific embodiments, the method further comprises:

[0078] Obtaining an object in the virtual reality scene that conforms to the law of physics, and an acting force on the upper limb model;

[0079] Controlling the motion of the upper limb rehabilitation exoskeleton according to the acting force to feed back the acting force to the patient without motor function.

[0080] In the embodiment of the present application, the object conforming to the physical law in reality is constructed in the virtual reality scene, the force of the object on the upper limb model of the human body in the interaction process is obtained, and then the force is fed back to the upper limb of the patient without motor function through the movement control of the upper limb rehabilitation exoskeleton, so as to further improve the authenticity of the interaction with the virtual reality scene.

[0081] In some specific embodiments, the motion intention of the patient without motor function is determined according to the motion intention recognition result corresponding to each upper limb modal signal and the corresponding confidence.

[0082] The motion intention of the patient without motor function is preliminarily determined according to the motion intention recognition result corresponding to each upper limb modal signal and the corresponding confidence.

[0083] It is judged whether the preliminarily determined motion intention is consistent with the motion intention recognition result corresponding to the upper limb modal signal with the maximum confidence.

[0084] If consistent, the preliminarily determined motion intention is taken as the motion intention of the patient without motor function.

[0085] Please refer to Figure 3 In the embodiment of the present application, the intention recognition model is constructed based on support vector machine (SVM) and attention mechanism. The specific features are that a classification model corresponding to each modal signal (i.e. electroencephalogram signal, eye movement signal and electromyogram signal of the upper limb) is constructed respectively, and each classification model is subjected to identification test experiment (specifically, online identification test experiment). The identification accuracy of each model of each modality is obtained through multiple experiments, the confidence (i.e. weight) is distributed according to the identification accuracy of the three classification models in proportion, and the confidence of each modality is 1. The identification result based on the attention mechanism is that each model identifies the probability of each motion intention, which is expressed as a value of 0-1, and the larger the value, the greater the probability. After each probability is multiplied by the confidence and added, the final value of the maximum is determined as the final identification intention. It is judged whether the final intention is the same as the identification result of the classification model with the maximum confidence. If not, the identification classification is performed again. If the final intention obtained through the re-identification is the same, the intention is output.

[0086] In some specific embodiments, the upper limb rehabilitation exoskeleton includes a plurality of joints, and each joint corresponds to one or more driving motors.

[0087] The motion control signal of the upper limb rehabilitation exoskeleton is determined according to the motion intention, which includes:

[0088] The driving signal of the driving motor corresponding to each joint is determined according to the motion intention, and the driving signal includes at least one of a driving time, a rotating speed of the driving motor, and a running direction of the driving motor.

[0089] Specifically, the upper limb rehabilitation robot is an upper limb rehabilitation exoskeleton robot. The upper limb rehabilitation exoskeleton robot includes an upper limb rehabilitation exoskeleton and a processor (or a controller), or the upper limb rehabilitation exoskeleton robot is controlled by an external controller (for example, a computer). Please refer to Figure 4 After the motion intention of the patient without motor function is obtained, the controller obtains the motion requirements of each joint of the upper limb rehabilitation exoskeleton according to the motion intention, converts the motion requirements into driving signals (also referred to as control instructions) of the driving motors corresponding to each joint, and sends the driving signals to a multi-axis motion control card. The multi-axis motion control card sends the driving signals to the drivers of the corresponding driving motors, and the drivers control the driving motors to move according to the driving signals, so that the upper limb rehabilitation exoskeleton in the upper limb rehabilitation exoskeleton robot can perform upper limb rehabilitation movement according to the motion intention of the patient.

[0090] In some specific embodiments, please refer to Figure 3 The confidence corresponding to each of the upper limb modal signals is obtained, including:

[0091] The historical upper limb modal signals are obtained, the signal types of the historical upper limb modal signals are consistent with the signal types of the upper limb modal signals, and each of the historical upper limb modal signals includes a plurality of historical upper limb modal signals. The historical upper limb modal signals can be obtained by sampling other patients without motor function, that is, by sampling other patients without motor function, or by sampling healthy people, or by sampling the same patient without motor function. Of course, the historical upper limb modal signals can not be collected from the same object, and the collection objects can include healthy people, other patients without motor function, and / or the same patient without motor function.

[0092] For each of the historical upper limb modal signals, the real motion intention corresponding to each of the historical upper limb modal signals and the motion intention recognition result obtained by recognizing the historical upper limb modal signals using the intention recognition model are obtained. Each historical upper limb modal signal has a plurality of historical upper limb modal signals, and each historical upper limb modal signal can obtain a motion intention recognition result.

[0093] For each of the historical upper limb modal signals, the identification accuracy is determined according to the real motion intention and the motion intention recognition result corresponding to each of the plurality of historical upper limb modal signals.

[0094] For each of the historical upper limb modal signal, a corresponding confidence (i.e. weight) is determined according to the recognition accuracy.

[0095] In some specific embodiments, the motion intention of the patient without motor function is determined according to the motion intention recognition result of each of the upper limb modal signal and the corresponding confidence, including:

[0096] For the upper limb modal signal with consistent motion intention recognition result, the confidence of the corresponding motion intention recognition result is obtained by adding the electromyography signal corresponding to the upper limb modal signal; here, each of the upper limb modal signal has only one signal, which can be a signal with a certain time length;

[0097] For the upper limb modal signal with inconsistent motion intention recognition result with other upper limb modal signals, the confidence of the upper limb modal signal is directly used as the confidence of the corresponding motion intention recognition result.

[0098] The motion intention recognition result corresponding to the maximum confidence is determined as the motion intention of the patient without motor function.

[0099] For example, the upper limb modal signal includes three kinds of electroencephalogram signal, electromyography signal of upper limb and eye movement signal, wherein the motion intention recognized based on the electroencephalogram signal is the first kind of motion intention, the motion intention recognized based on the electromyography signal is the first kind of motion intention, and the motion intention based on the eye movement signal is the second kind of motion intention. The weight parameter corresponding to the electromyography signal is q1, the weight parameter corresponding to the electroencephalogram signal is q2 (greater than q1 and q3), and the weight parameter corresponding to the eye movement signal is q3. Then, the weight parameter q2 corresponding to the electroencephalogram signal is added to the weight parameter q1 corresponding to the electromyography signal to obtain (q2+q1) as the confidence of the first kind of motion intention, and the confidence of the second kind of motion intention is q3. The sizes of (q2+q1) and q3 are compared. If (q2+q1) is the maximum, then the motion intention of the patient without motor function is considered to be the first kind of motion intention.

[0100] In other specific embodiments, the determination of the motion intention recognition result corresponding to the maximum confidence as the motion intention of the patient without motor function includes:

[0101] It is judged whether the minimum confidence is greater than a preset threshold value.

[0102] If the minimum confidence is less than or equal to the preset threshold value, the motion intention recognition result corresponding to the maximum confidence is determined as the motion intention of the patient without motor function.

[0103] For example, if q3 is less than or equal to a preset threshold, the movement intention of the patient without movement function is considered to be the first movement intention (corresponding to the maximum confidence (q2+q1)), otherwise the movement intention of the patient without movement function cannot be considered to be the first movement intention.

[0104] In an embodiment of the present application, considering that the machine learning itself may have a recognition error, a judgment and feedback mechanism is designed to compensate for the fusion intention recognition result of the signal decision level by establishing a strong correlation between the intention recognition results of each signal source to determine whether the fusion intention recognition result is effective, thereby improving the accuracy of the fusion movement intention recognition of the patient without movement function based on multi-mode (electromyography, electroencephalogram, inertia, and force touch) signals.

[0105] In other specific embodiments, referring to Figure 3 , the method further comprises:

[0106] If the minimum confidence is greater than the preset threshold, the intention recognition model is used again to recognize the movement intention of the upper limb modal signal corresponding to each upper limb modal signal to obtain the movement intention recognition result corresponding to the upper limb modal signal, and the movement intention of the patient without movement function is determined based on the newly obtained movement intention recognition result.

[0107] That is, if the minimum confidence is greater than the preset threshold, the movement intention recognition result corresponding to the maximum confidence cannot be directly determined as the movement intention of the patient without movement function, and the movement intention recognition needs to be performed again.

[0108] In an embodiment of the present application, for the interaction problem in the rehabilitation training process, a virtual interaction scene is designed based on VR technology, a virtual training scene close to reality is constructed to improve the participation of the patient in the training process, and the training effect is improved.

[0109] In addition, the electromyography signal of the patient can be collected in real time during the training process, and the recovery of the upper limb electromyography signal of the human body is monitored to determine the rehabilitation effect of the patient without movement function of the upper limb.

[0110] Correspondingly, referring to Figure 5 , an embodiment of the present application provides a rehabilitation robot control system based on multi-source information perception, which comprises:

[0111] The first acquisition module 501 is configured to acquire the upper limb modal signal of the patient without movement function, and the upper limb modal signal comprises an electroencephalogram signal, and an eye movement signal and / or an electromyography signal of the upper limb.

[0112] The recognition module 502 is configured to, for each of the upper limb modality signals, perform motion intention recognition on the upper limb modality signal by using an intention recognition model to obtain a motion intention recognition result corresponding to the upper limb modality signal; and the intention recognition model is constructed based on a support vector machine algorithm.

[0113] The second acquisition module 503 is configured to acquire a confidence degree corresponding to each of the upper limb modality signals.

[0114] The intention determination module 504 is configured to determine the motion intention of the patient without motor function according to the motion intention recognition result corresponding to each of the upper limb modality signals and the corresponding confidence degree.

[0115] The control signal determination module 505 is configured to determine a motion control signal of the upper limb rehabilitation exoskeleton according to the motion intention.

[0116] The control module 506 is configured to control the upper limb rehabilitation exoskeleton to perform motion according to the motion control signal.

[0117] The embodiment of the present application proposes an intention recognition technology for a stroke patient without motor function in the early stage by fusing electroencephalogram and electromyogram and / or eye movement information, and performs motion intention recognition of each signal source by constructing a support vector machine machine learning model (i.e., an intention recognition model constructed based on a support vector machine algorithm), thereby improving the recognition accuracy of the motion intention of the patient without motor function. Furthermore, the patient without motor function can perform upper limb rehabilitation training according to his / her own intention, which is more interesting than repetitive and task-oriented functional training, and can improve the enthusiasm of the patient for rehabilitation training.

[0118] In some specific embodiments, the first acquisition module 501 comprises:

[0119] The playing unit is configured to control the virtual reality glasses worn by the patient without motor function to play a pre-designed upper limb rehabilitation training video for the patient without motor function, so as to guide the patient to generate a corresponding upper limb motion intention.

[0120] The acquisition unit is configured to acquire the upper limb modality signal generated by the patient without motor function when the patient watches the upper limb rehabilitation training video.

[0121] In some specific embodiments, the recognition module 502 comprises:

[0122] The first determination unit is configured to, in the case that the upper limb modality signal comprises the eye movement signal, determine the position and / or movement track of the focus point of the eye of the patient without motor function in the virtual space according to the eye movement signal.

[0123] The first identification unit is configured to input the position and / or the movement trajectory into the intention recognition model for the eye movement signal to obtain a corresponding movement intention recognition result, wherein the intention recognition model for the eye movement signal is trained based on the upper limb rehabilitation training video.

[0124] In some specific embodiments, the upper limb rehabilitation exoskeleton is provided with a motion capture sensor.

[0125] The system further comprises:

[0126] The motion information determination module is configured to determine the motion information of each joint of the upper limb rehabilitation exoskeleton according to the information collected by the motion capture sensor.

[0127] The reconstruction module is configured to reconstruct the upper limb model of the human body in the virtual reality scene according to the motion information of each joint of the upper limb rehabilitation exoskeleton.

[0128] In some specific embodiments, the system further comprises:

[0129] The force acquisition module is configured to acquire the force of the object in the virtual reality scene that conforms to the law of physics on the upper limb model of the human body.

[0130] The control module 506 is further configured to control the motion of the upper limb rehabilitation exoskeleton according to the force to feed back the force to the patient without motor function.

[0131] In some specific embodiments, the intention determination module 504 comprises:

[0132] The first determination unit is configured to preliminarily determine the movement intention of the patient without motor function according to the movement intention recognition result and the corresponding confidence level corresponding to each upper limb modal signal.

[0133] The judgment unit is configured to judge whether the preliminarily determined movement intention is consistent with the movement intention recognition result corresponding to the upper limb modal signal with the highest confidence level.

[0134] The second determination unit is configured to take the preliminarily determined movement intention as the movement intention of the patient without motor function in the case that the preliminarily determined movement intention is consistent with the movement intention recognition result corresponding to the upper limb modal signal with the highest confidence level.

[0135] In some specific embodiments, the upper limb rehabilitation exoskeleton comprises a plurality of joints, and each joint corresponds to one or more driving motors.

[0136] The control signal determination module 505 is specifically configured to determine a driving signal of the driving motor corresponding to at least part of the joint according to the motion intention, and the driving signal includes at least one of a driving time, a rotating speed of the driving motor, and a running direction of the driving motor.

[0137] The embodiment of the present application is a system embodiment based on the same inventive concept as the above-mentioned method embodiment, and therefore specific technical details and corresponding technical effects are described with reference to the above-mentioned method embodiment, which will not be described here again.

[0138] The embodiment of the present application also provides an electronic device, which can include an intention recognition device and a rehabilitation robot, or can be a rehabilitation robot (the intention recognition is realized by the rehabilitation robot), as shown in the figure, the electronic device can include a processor 61 and a memory 62, wherein the processor 61 and the memory 62 can be connected to each other through a bus or other means, Figure 6 for example, the bus connection is taken as an example. Figure 6

[0139] The processor 61 can be a central processing unit (CPU). The processor 61 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. chips, or combinations of the above various chips.

[0140] The memory 62 is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules of the rehabilitation robot control method based on multi-source information perception in the embodiment of the present application (for example, Figure 5 the first acquisition module 501, the identification module 502, the second acquisition module 503, the intention determination module 504, the control signal determination module 505 and the control module 506 shown in the figure). The processor 61 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 62, that is, realizes the rehabilitation robot control method based on multi-source information perception in the above-mentioned method embodiment.

[0141] ​The memory 62 can include a program storage area and a data storage area, where the program storage area can store an operating system, application programs required by at least one function, and the like, and the data storage area can store data created by the processor 61 and the like. In addition, the memory 62 can include a high-speed random access memory, and can also include a non-transitory memory such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 62 can optionally include a memory disposed remotely from the processor 61, which can be connected to the processor 61 through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0142] The one or more modules are stored in the memory 62 and, when executed by the processor 61, perform the above-mentioned method for controlling a rehabilitation robot based on multi-source information perception. Figures 1-4 The method for controlling a rehabilitation robot based on multi-source information perception in the illustrated embodiment.

[0143] The specific details of the above electronic device can be referred to in the above description of the method for controlling a rehabilitation robot based on multi-source information perception. Figures 1 to 4 The corresponding related description and effects in the illustrated embodiment are understood, and will not be repeated here.

[0144] Correspondingly, the embodiment of the present application also provides a computer readable storage medium for storing a computer program, which is executed by a processor to implement each process of the above-mentioned method for controlling a rehabilitation robot based on multi-source information perception, and can achieve the same technical effects. To avoid repetition, it will not be repeated here.

[0145] The computer readable medium includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer readable medium does not include transitory computer readable medium, such as modulated data signals and carriers.

[0146] The various embodiments in the specification are described in progressive manner, and the same or similar parts between the various embodiments can be mutually referred to, and each embodiment focuses on the difference from other embodiments.

[0147] The above merely provides an example of the present application, and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A rehabilitation robot control system based on multi-source information perception, characterized in that, include: The first acquisition module is used to acquire upper limb modal signals of patients without motor function, the upper limb modal signals including electroencephalogram (EEG) signals, eye movement signals and / or electromyographic signals of the upper limb; The recognition module is used to perform motion intention recognition on each of the upper limb modal signals using an intention recognition model, and obtain a motion intention recognition result corresponding to the upper limb modal signal; wherein, the intention recognition model is constructed based on the support vector machine algorithm; The second acquisition module is used to acquire the confidence level corresponding to each of the upper limb modal signals; The intent determination module is used to determine the motor intent of the patient without motor function based on the motor intent recognition result and the corresponding confidence level corresponding to each of the upper limb modal signals. A control signal determination module is used to determine the motion control signal of the upper limb rehabilitation exoskeleton based on the said motion intention. The control module is used to control the upper limb rehabilitation exoskeleton to move according to the motion control signal; Specifically, the intent determination module is used for: For the upper limb modal signals that have consistent motion intention recognition results, the confidence scores corresponding to the upper limb modal signals are summed to obtain the confidence score of the corresponding motion intention recognition result; For upper limb modal signals whose motion intention recognition results are inconsistent with other upper limb modal signals, the confidence level corresponding to the upper limb modal signal is directly used as the confidence level of the corresponding motion intention recognition result; The motor intention recognition result corresponding to the highest confidence level is determined as the motor intention of the patient without motor function.

2. The system according to claim 1, characterized in that, The first acquisition module includes: The playback unit is used to control the virtual reality glasses worn by the patient with no motor function to play pre-designed upper limb rehabilitation training videos for the patient with no motor function, so as to guide the patient to generate corresponding upper limb movement intentions; The acquisition unit is used to acquire the upper limb modal signals generated by the patient without motor function while watching the upper limb rehabilitation training video.

3. The system according to claim 2, characterized in that, The identification module includes: The first determining unit is configured to, when the upper limb modal signal includes the eye movement signal, determine the position and / or movement trajectory of the focal point of the eye of the patient without motor function in virtual space based on the eye movement signal; The first recognition unit is used to input the position and / or movement trajectory into the intention recognition model for the eye movement signal to obtain the corresponding movement intention recognition result, wherein the intention recognition model for the eye movement signal is obtained based on the upper limb rehabilitation training video.

4. The system according to claim 2, characterized in that, The upper limb rehabilitation exoskeleton is equipped with motion capture sensors; The system also includes: The motion information determination module is used to determine the motion information of each joint of the upper limb rehabilitation exoskeleton based on the information collected by the motion capture sensor. The reconstruction module is used to reconstruct the human upper limb model in the virtual reality scene based on the motion information of each joint of the upper limb rehabilitation exoskeleton.

5. The system according to claim 2 or 4, characterized in that, The system also includes: The force acquisition module is used to acquire the force exerted on the human upper limb model by objects that conform to the laws of physics in the virtual reality scene. The control module is also used to control the movement of the upper limb rehabilitation exoskeleton according to the force, so as to feed back the force to the patient without motor function.

6. The system according to claim 1, characterized in that, The upper limb rehabilitation exoskeleton includes multiple joints, each joint corresponding to one or more drive motors; The control signal determination module is specifically used to determine the drive signal of the drive motor corresponding to at least some joints according to the motion intention. The drive signal includes at least one of drive time, rotational speed of the drive motor, and running direction of the drive motor.

Images