Hand rehabilitation training method and device based on brain-computer interface and VR eye tracker

Abstract

The invention discloses a hand rehabilitation training method and device based on a brain-computer interface and a VR eye tracker, and the method comprises the steps: displaying the virtual reality scene information of hand rehabilitation training to a user through a virtual reality display module; a hand training module is used for carrying out motion training on the hands of the user, and a user hand motion information set is acquired and obtained; acquiring an electroencephalogram signal set of a user by using a brain-computer interface module; acquiring eyeball movement track information of the user during hand movement training by using a VR eye tracker; a data processing module is used for processing the hand movement information set, the electroencephalogram signal set and the eyeball movement track information of the user, and training effect information and feedback information are obtained; displaying the training effect information to a user by using a display module; the feedback information is used for adjusting the control torque of a flexible driver of the hand training module.

Description

Technical Field

[0001] The present invention relates to the fields of virtual reality data processing and rehabilitation medicine, and in particular to a hand rehabilitation training method and device based on a brain-computer interface and a VR eye tracker. Background Art

[0002] Hand dysfunction is a common sequela of many neurological diseases and injuries, such as stroke, brain trauma, Parkinson's disease, etc., which seriously affects patients' daily life and work ability. Traditional hand rehabilitation training methods mainly rely on manual assistance and guidance from physical therapists, with limited training effects, and the training process is relatively boring, so patients' enthusiasm and compliance are not high.

[0003] With the development of science and technology, brain-computer interface and virtual reality technology are gradually applied to the field of rehabilitation. Brain-computer interface can directly obtain neural signals from the brain and realize the interaction between the brain and external devices; virtual reality technology can provide patients with an immersive training environment, increasing the fun and attractiveness of training. However, the current methods and devices for applying brain-computer interface and virtual reality technology to hand rehabilitation training still have some shortcomings, such as single training mode, insufficient and inaccurate feedback, and inability to make personalized adjustments based on the patient's real-time status. Summary of the invention

[0004] The present invention mainly solves the problems of existing hand rehabilitation training devices, such as single training mode, inaccurate and timely feedback, and inability to make personalized adjustments according to the real-time status of patients. The present invention discloses a hand rehabilitation training method and device based on a brain-computer interface and a VR eye tracker.

[0005] In a first aspect of an embodiment of the present invention, a hand rehabilitation training method based on a brain-computer interface and a VR eye tracker is disclosed, comprising:

[0006] S1, using a virtual reality display module to display virtual reality scene information of hand rehabilitation training to the user;

[0007] S2, using the hand training module to perform motion training on the user's hand, and collect a set of user hand motion information; the user hand motion information set includes a speed information sequence, a position information sequence, and a force information sequence of each key position of the user's hand; the key positions of the user's hand include five fingers and a palm;

[0008] S3, using the brain-computer interface module to collect a set of brain electrical signals of the user; the set of brain electrical signals includes α wave signals, β wave signals, θ wave signals and δ wave signals;

[0009] S4, using a VR eye tracker to collect eye movement trajectory information of the user when performing hand movement training;

[0010] S5, using a data processing module to process the user's hand movement information set, EEG signal set, and eye movement trajectory information to obtain training effect information and feedback information;

[0011] S6, using a display module to display the training effect information to the user; using the feedback information to adjust the control torque of the flexible driver of the hand training module.

[0012] The processing of the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information and feedback information includes:

[0013] S51, evaluating and processing the user's hand movement information set, EEG signal set, and eye movement trajectory information to obtain training effect information;

[0014] S52, acquiring a control torque signal of the flexible actuator of the hand training module;

[0015] S53, performing feedback processing on the control torque signal and the training effect information to obtain feedback information.

[0016] The evaluation and processing of the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information includes:

[0017] S511, obtaining a set of standard hand motion information; the set of standard hand motion information includes speed standard information, position standard information and strength standard information of key positions of each user's hand;

[0018] S512, performing hand movement evaluation processing on the user hand movement information set and the hand standard movement information set to obtain a hand movement sub-effect value pg1;

[0019] S513, performing an EEG evaluation transformation on the EEG signal set to obtain a EEG effect value pg2;

[0020] S514, performing motion evaluation processing on the eye movement trajectory information to obtain an eye movement sub-effect value pg3;

[0021] S515, performing weighted summation on the hand movement sub-effect value, the brain electronic effect value and the eye movement sub-effect value to obtain training effect information.

[0022] The feedback processing of the control torque signal and the training effect information to obtain feedback information includes:

[0023] Performing torque effect transformation on the training effect information to obtain a torque equivalent value;

[0024] The expression of the torque effect transformation is:

[0025]

[0026] Among them, h is the training effect information, γ 0 and are preset constant factors, μ and are the second-order central moment and the second-order origin moment of the control torque signal respectively, and γ is the torque equivalent value;

[0027] Performing feedback calculation on the torque equivalent value and the control torque signal to obtain feedback information;

[0028] The expression of the feedback calculation is:

[0029]

[0030] in, is the feedback information, and f(t) is the control torque signal.

[0031] The hand movement evaluation process is performed on the user hand movement information set and the hand standard movement information set to obtain the hand movement sub-effect value pg1, including:

[0032] The speed information sequence, position information sequence and force information sequence of each key position of the user's hand are expressed as a hand motion matrix corresponding to the key position of the user's hand;

[0033] The standard hand motion information set is represented as a standard matrix; the row vector of the standard matrix is ​​the speed standard information, position standard information and force standard information of a key position of a user's hand;

[0034] Perform a first evaluation calculation on the hand motion matrix corresponding to the key positions of all users' hands and the standard matrix to obtain the hand motion sub-effect value pg1;

[0035] The expression of the first evaluation calculation is:

[0036]

[0037]

[0038] Among them, ω 1 is the preset weight value, a i,kj is the element of the kth row and jth column of the hand motion matrix corresponding to the key position of the hand of the i-th user, b ik is the element in the i-th row and k-th column of the standard matrix, is the variance value of the kth row of the hand motion matrix corresponding to the key position of the hand of the i-th user, ndj is the jth element of the motion evaluation vector, M, K, and J are the number of key hand positions of the user, the row dimension of the hand motion matrix, and the column dimension of the hand motion matrix, respectively.

[0039] In a second aspect of an embodiment of the present invention, a hand rehabilitation training device based on a brain-computer interface and a VR eye tracker is disclosed, which is used to implement the hand rehabilitation training method based on a brain-computer interface and a VR eye tracker, comprising: a virtual reality display module, a hand training module, a brain-computer interface module, a VR eye tracker, a data processing module and a display module;

[0040] The virtual reality display module is used to display virtual reality scene information of hand rehabilitation training to the user;

[0041] The hand training module is used to perform motion training on the user's hands and collect a set of user hand motion information;

[0042] The brain-computer interface module is used to collect a set of EEG signals from the user;

[0043] The VR eye tracker is used to collect and obtain the eye movement trajectory information of the user when performing hand movement training;

[0044] The data processing module is connected to the hand training module, the brain-computer interface module and the VR eye tracker respectively, and is used to process the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information and feedback information; and use the feedback information to adjust the control torque of the flexible driver of the hand training module;

[0045] The display module is connected to the data processing module and is used to display the training effect information to the user.

[0046] The VR eye tracker module is arranged inside the virtual reality display module and is used to collect the user's eye movement trajectory information;

[0047] The hand training module includes a glove, a flexible actuator, and a sensor group; the joints of the glove are equipped with a flexible actuator, and the flexible actuator is used to perform corresponding movement training on the patient's hand according to the control instruction;

[0048] The sensor group includes a mechanical sensor and an acceleration sensor; the mechanical sensor is arranged at a key position of the user's hand corresponding to the glove, and is used to collect a force information sequence of each key position of the user's hand;

[0049] The acceleration sensor is arranged at the key position of the user's hand corresponding to the glove, and is used to collect the speed information sequence and the position information sequence of each key position of the user's hand.

[0050] The third aspect of the present invention is to disclose a hand rehabilitation training device based on a brain-computer interface and a VR eye tracker, the device comprising:

[0051] A memory storing executable program code;

[0052] a processor coupled to the memory;

[0053] The processor calls the executable program code stored in the memory to execute the hand rehabilitation training method based on the brain-computer interface and VR eye tracker.

[0054] According to a fourth aspect of the present invention, a computer-storable medium is disclosed, wherein the computer-storable medium stores computer instructions. When the computer instructions are called by a computer, they are used to execute the hand rehabilitation training method based on a brain-computer interface and a VR eye tracker.

[0055] According to a fifth aspect of the present invention, an information data processing terminal is disclosed, which is used to implement the hand rehabilitation training method based on brain-computer interface and VR eye tracker.

[0056] The beneficial effects of the present invention are:

[0057] The present invention combines the technical advantages of brain-computer interface and VR eye tracker, which can more accurately obtain the patient's movement intention and attention state, and realize more personalized and precise hand rehabilitation training. Through virtual reality technology, patients are provided with rich and interesting training scenes to improve their training enthusiasm and compliance. Real-time monitoring and feedback of relevant information of hand movements help patients to understand the training effect in time, adjust training strategies, and improve training efficiency. The training plan can be dynamically adjusted according to the patient's training progress, adapt to the rehabilitation needs of different patients, and promote the recovery of hand function.

[0058] The present invention combines a brain-computer interface module and a hand motion sensor to collect the user's EEG signals and hand motion information in real time. By analyzing and processing the EEG signals, the brain electronic effect value is obtained, which directly reflects the patient's brain activity state during the training process. High-precision monitoring of hand motion information can fully capture the movement trajectory, speed and strength of the fingers and palms, and provide accurate physiological feedback for rehabilitation training. The user's eye movement trajectory information is collected through a VR eye tracker and incorporated into the training effect evaluation system. This innovative eye movement monitoring method can reveal the correlation between eye movement and hand movement, and provide more comprehensive physiological data support for rehabilitation training.

[0059] The present invention comprehensively evaluates a variety of physiological signals through a data processing module to obtain training effect information, and uses feedback information to dynamically adjust the control torque of the flexible driver, thereby realizing the personalization and intelligence of rehabilitation training. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 1 is a flow chart for implementing the method of the present invention;

[0061] Figure 2 It is a composition diagram of the device of the present invention. DETAILED DESCRIPTION

[0062] In order to better understand the content of the present invention, two embodiments are given here.

[0063] Embodiment 1

[0064] Figure 1 It is a flow chart for implementing the method of the present invention. Figure 2 It is a composition diagram of the device of the present invention.

[0065] In a first aspect of an embodiment of the present invention, a hand rehabilitation training method based on a brain-computer interface and a VR eye tracker is disclosed, which is implemented using a hand rehabilitation training device based on a brain-computer interface and a VR eye tracker, comprising:

[0066] S1, using a virtual reality display module to display virtual reality scene information of hand rehabilitation training to the user;

[0067] S2, using the hand training module to perform motion training on the user's hand, and collect a set of user hand motion information; the user hand motion information set includes a speed information sequence, a position information sequence, and a force information sequence of each key position of the user's hand; the key positions of the user's hand include five fingers and a palm;

[0068] S3, using the brain-computer interface module to collect a set of brain electrical signals of the user; the set of brain electrical signals includes α wave signals, β wave signals, θ wave signals and δ wave signals;

[0069] S4, using a VR eye tracker to collect eye movement trajectory information of the user when performing hand movement training;

[0070] S5, using a data processing module to process the user's hand movement information set, EEG signal set, and eye movement trajectory information to obtain training effect information and feedback information;

[0071] S6, using a display module to display the training effect information to the user; using the feedback information to adjust the control torque of the flexible driver of the hand training module.

[0072] The present invention provides users with an immersive rehabilitation training scene through a virtual reality display module, which can effectively improve patients' training enthusiasm and concentration. Virtual reality technology can dynamically adjust training tasks according to the patient's rehabilitation progress, allowing patients to perform rehabilitation training in an interesting environment and enhance the training effect.

[0073] The training effect information of the present invention is fed back to the user in real time through the display module, so that the patient can intuitively understand his or her own training status and progress, and further improve the initiative and pertinence of rehabilitation training.

[0074] The processing of the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information and feedback information includes:

[0075] S51, evaluating and processing the user's hand movement information set, EEG signal set, and eye movement trajectory information to obtain training effect information;

[0076] S52, acquiring a control torque signal of the flexible actuator of the hand training module;

[0077] S53, performing feedback processing on the control torque signal and the training effect information to obtain feedback information;

[0078] The evaluation and processing of the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information includes:

[0079] S511, obtaining a set of standard hand motion information; the set of standard hand motion information includes speed standard information, position standard information and strength standard information of key positions of each user's hand;

[0080] S512, performing hand movement evaluation processing on the user hand movement information set and the hand standard movement information set to obtain a hand movement sub-effect value pg1;

[0081] S513, performing an EEG evaluation transformation on the EEG signal set to obtain a EEG effect value pg2;

[0082] S514, performing motion evaluation processing on the eye movement trajectory information to obtain an eye movement sub-effect value pg3;

[0083] S515, performing weighted summation on the hand movement sub-effect value, the brain electronic effect value and the eye movement sub-effect value to obtain training effect information;

[0084] The feedback processing of the control torque signal and the training effect information to obtain feedback information includes:

[0085] Performing torque effect transformation on the training effect information to obtain a torque equivalent value;

[0086] The expression of the torque effect transformation is:

[0087]

[0088] Among them, h is the training effect information, γ 0 and are preset constant factors, μ and are the second-order central moment and the second-order origin moment of the control torque signal respectively, and γ is the torque equivalent value;

[0089] Performing feedback calculation on the torque equivalent value and the control torque signal to obtain feedback information;

[0090] The expression of the feedback calculation is:

[0091]

[0092] in, is the feedback information, and f(t) is the control torque signal.

[0093] The hand movement evaluation process is performed on the user hand movement information set and the hand standard movement information set to obtain the hand movement sub-effect value pg1, including:

[0094] The speed information sequence, position information sequence and force information sequence of each key position of the user's hand are expressed as a hand motion matrix corresponding to the key position of the user's hand;

[0095] The hand standard motion information set is represented as a standard matrix; the row vector of the standard matrix is ​​the speed standard information, position standard information and force standard information of a key position of a user's hand; the column number of the standard matrix corresponds to the row number of the hand motion matrix according to the information type, for example, the first column element of the standard matrix represents the speed standard information, and the first row element of the hand motion matrix represents the speed information sequence;

[0096] Perform a first evaluation calculation on the hand motion matrix corresponding to the key positions of all users' hands and the standard matrix to obtain the hand motion sub-effect value pg1;

[0097] The expression of the first evaluation calculation is:

[0098]

[0099] Among them, ω 1 is the preset weight value, a i,kjis the element of the kth row and jth column of the hand motion matrix corresponding to the key position of the hand of the i-th user, b ik is the element in the i-th row and k-th column of the standard matrix, is the variance value of the kth row of the hand motion matrix corresponding to the key position of the hand of the i-th user, nd j is the jth element of the motion evaluation vector, M, K, and J are the number of key hand positions of the user, the row dimension of the hand motion matrix, and the column dimension of the hand motion matrix, respectively.

[0100] The step of performing an EEG evaluation transformation on the EEG signal set to obtain an EEG effect value pg2 includes:

[0101] Perform Fourier transform on each signal in the EEG signal set to obtain the corresponding frequency domain signal;

[0102] For each frequency domain signal, statistical feature extraction is performed to obtain a statistical value set; the statistical value set includes the mean, variance, median, range value, and 1 / 4 quantile value of the frequency domain signal;

[0103] Performing a second evaluation calculation on the statistical value set of all frequency domain signals to obtain a brain electronic effect value;

[0104] The expression calculated by the second evaluation is:

[0105]

[0106] Among them, i takes values ​​from 1 to 4, representing the elements of the statistical value set of the frequency domain signals of the α wave signal, the β wave signal, the θ wave signal and the δ wave signal, respectively, and r i , β i , i , The mean, variance, median, range, and 1 / 4 quantile values ​​in the statistical value set of the i-th frequency domain signal, and are the average of the means and the average of the variances in all statistical value sets, L 2 is a quadratic Laguerre polynomial.

[0107] The first to fourth frequency domain signals are frequency domain signals of α wave signal, β wave signal, θ wave signal and δ wave signal respectively.

[0108] The step of performing motion evaluation processing on the eye movement trajectory information to obtain the eye movement sub-effect value pg3 includes:

[0109] Obtaining standard eye movement trajectory information;

[0110] Performing motion evaluation processing on the eyeball movement trajectory information and the eyeball standard movement trajectory information to obtain an eyeball movement sub-effect value pg3;

[0111] The expression of the motion evaluation process is:

[0112]

[0113] Among them, N1 is the number of coordinate points contained in the eye movement trajectory information, [x j ,y j ] is the coordinate of the jth coordinate point contained in the eye movement trajectory information, [x 0j ,y 0j ] is the coordinate of the jth coordinate point contained in the standard eye movement trajectory information.

[0114] The eyeball movement trajectory is represented by the two-dimensional position coordinates of the pupil in the eye plane.

[0115] The weights of the weighted sum of the hand movement sub-effect value, the brain electronic effect value and the eye movement sub-effect value are 0.5, 0.2 and 0.3 respectively;

[0116] The method of adjusting the control torque of the flexible actuator of the hand training module by using the feedback information is to set the control torque of the flexible actuator of the hand training module to F t is the control torque of the flexible actuator of the hand training module at time t, For feedback information.

[0117] In a second aspect of an embodiment of the present invention, a hand rehabilitation training device based on a brain-computer interface and a VR eye tracker is disclosed, which is used to implement the hand rehabilitation training method based on a brain-computer interface and a VR eye tracker, comprising: a virtual reality display module, a hand training module, a brain-computer interface module, a VR eye tracker, a data processing module and a display module;

[0118] The virtual reality display module is used to display virtual reality scene information of hand rehabilitation training to the user;

[0119] The hand training module is used to perform motion training on the user's hands and collect a set of user hand motion information;

[0120] The brain-computer interface module is used to collect a set of EEG signals from the user;

[0121] The VR eye tracker is used to collect and obtain the eye movement trajectory information of the user when performing hand movement training;

[0122] The data processing module is connected to the hand training module, the brain-computer interface module and the VR eye tracker respectively, and is used to process the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information and feedback information; and use the feedback information to adjust the control torque of the flexible driver of the hand training module;

[0123] The display module is used to display the training effect information to the user.

[0124] The VR eye tracker module is arranged inside the virtual reality display module, and acquires the user's eye movement trajectory information in real time through an infrared camera.

[0125] The hand training module includes a glove, a flexible actuator, and a sensor group; the joints of the glove are equipped with a flexible actuator, and the flexible actuator is used to perform corresponding movement training on the patient's hand according to the control instruction;

[0126] The sensor group includes a mechanical sensor and an acceleration sensor; the mechanical sensor is arranged at a key position of the user's hand corresponding to the glove, and is used to collect a force information sequence of each key position of the user's hand;

[0127] The acceleration sensor is arranged at the key position of the user's hand corresponding to the glove, and is used to collect the speed information sequence and the position information sequence of each key position of the user's hand;

[0128] The virtual reality display module can be implemented by using a virtual reality helmet;

[0129] The VR eye tracker module can be implemented using an eye tracker.

[0130] The processing of the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information and feedback information includes:

[0131] S51, evaluating and processing the user's hand movement information set, EEG signal set, and eye movement trajectory information to obtain training effect information;

[0132] S52, acquiring a control torque signal of the flexible actuator of the hand training module;

[0133] S53, performing feedback processing on the control torque signal and the training effect information to obtain feedback information;

[0134] The evaluation and processing of the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information includes:

[0135] S511, obtaining a set of standard hand motion information; the set of standard hand motion information includes speed standard information, position standard information and strength standard information of key positions of each user's hand;

[0136] S512, performing hand movement evaluation processing on the user hand movement information set and the hand standard movement information set to obtain a hand movement sub-effect value pg1;

[0137] S513, performing an EEG evaluation transformation on the EEG signal set to obtain a EEG effect value pg2;

[0138] S514, performing motion evaluation processing on the eye movement trajectory information to obtain an eye movement sub-effect value pg3;

[0139] S515, performing weighted summation on the hand movement sub-effect value, the brain electronic effect value and the eye movement sub-effect value to obtain training effect information;

[0140] The feedback processing of the control torque signal and the training effect information to obtain feedback information includes:

[0141] Performing torque effect transformation on the training effect information to obtain a torque equivalent value;

[0142] The expression of the torque effect transformation is:

[0143]

[0144] Among them, h is the training effect information, γ 0 and are preset constant factors, μ and are the second-order central moment and the second-order origin moment of the control torque signal respectively, and γ is the torque equivalent value;

[0145] Performing feedback calculation on the torque equivalent value and the control torque signal to obtain feedback information;

[0146] The expression of the feedback calculation is:

[0147]

[0148] in, is the feedback information, and f(t) is the control torque signal.

[0149] The hand movement evaluation process is performed on the user hand movement information set and the hand standard movement information set to obtain the hand movement sub-effect value pg1, including:

[0150] The speed information sequence, position information sequence and force information sequence of each key position of the user's hand are expressed as a hand motion matrix corresponding to the key position of the user's hand;

[0151] The hand standard motion information set is represented as a standard matrix; the row vector of the standard matrix is ​​the speed standard information, position standard information and force standard information of a key position of a user's hand; the column number of the standard matrix corresponds to the row number of the hand motion matrix according to the information type, for example, the first column element of the standard matrix represents the speed standard information, and the first row element of the hand motion matrix represents the speed information sequence;

[0152] Perform a first evaluation calculation on the hand motion matrix corresponding to the key positions of all users' hands and the standard matrix to obtain the hand motion sub-effect value pg1;

[0153] The expression of the first evaluation calculation is:

[0154]

[0155] Among them, ω 1 is the preset weight value, a i,kj is the element of the kth row and jth column of the hand motion matrix corresponding to the key position of the hand of the i-th user, b ik is the element in the i-th row and k-th column of the standard matrix, is the variance value of the kth row of the hand motion matrix corresponding to the key position of the hand of the i-th user, nd j is the jth element of the motion evaluation vector, M, K, and J are the number of key hand positions of the user, the row dimension of the hand motion matrix, and the column dimension of the hand motion matrix, respectively.

[0156] The step of performing an EEG evaluation transformation on the EEG signal set to obtain an EEG effect value pg2 includes:

[0157] Perform Fourier transform on each signal in the EEG signal set to obtain the corresponding frequency domain signal;

[0158] For each frequency domain signal, statistical feature extraction is performed to obtain a statistical value set; the statistical value set includes the mean, variance, median, range value, and 1 / 4 quantile value of the frequency domain signal;

[0159] Performing a second evaluation calculation on the statistical value set of all frequency domain signals to obtain a brain electronic effect value;

[0160] The expression calculated by the second evaluation is:

[0161]

[0162] Among them, i takes values ​​from 1 to 4, representing the elements of the statistical value set of the frequency domain signals of the α wave signal, the β wave signal, the θ wave signal and the δ wave signal, respectively, and r i , β i , i , The mean, variance, median, range, and 1 / 4 quantile values ​​in the statistical value set of the i-th frequency domain signal, and are the average of the means and the average of the variances in all statistical value sets, L 2 is a quadratic Laguerre polynomial.

[0163] The first to fourth frequency domain signals are frequency domain signals of α wave signal, β wave signal, θ wave signal and δ wave signal respectively.

[0164] The step of performing motion evaluation processing on the eye movement trajectory information to obtain the eye movement sub-effect value pg3 includes:

[0165] Obtaining standard eye movement trajectory information;

[0166] Performing motion evaluation processing on the eyeball movement trajectory information and the eyeball standard movement trajectory information to obtain an eyeball movement sub-effect value pg3;

[0167] The expression of the motion evaluation process is:

[0168]

[0169] Among them, N1 is the number of coordinate points contained in the eye movement trajectory information, [x j ,y j ] is the coordinate of the jth coordinate point contained in the eye movement trajectory information, [x 0j ,y 0j ] is the coordinate of the jth coordinate point contained in the standard eye movement trajectory information.

[0170] The third aspect of the present invention is to disclose a hand rehabilitation training device based on a brain-computer interface and a VR eye tracker, the device comprising:

[0171] A memory storing executable program code;

[0172] a processor coupled to the memory;

[0173] The processor calls the executable program code stored in the memory to execute the hand rehabilitation training method based on the brain-computer interface and VR eye tracker.

[0174] According to a fourth aspect of the present invention, a computer-storable medium is disclosed, wherein the computer-storable medium stores computer instructions. When the computer instructions are called by a computer, they are used to execute the hand rehabilitation training method based on a brain-computer interface and a VR eye tracker.

[0175] According to a fifth aspect of the present invention, an information data processing terminal is disclosed, which is used to implement the hand rehabilitation training method based on brain-computer interface and VR eye tracker.

[0176] Embodiment 2:

[0177] In a first aspect of an embodiment of the present invention, a hand rehabilitation training device based on a brain-computer interface and a VR eye tracker is disclosed, comprising a brain-computer interface module, a VR eye tracker module, a hand training module, a data processing module and a display module.

[0178] The brain-computer interface module uses a non-invasive electroencephalogram (EEG) sensor to collect the brain's neural electrical signals through electrodes placed on the patient's head, and uses signal processing algorithms to convert them into control commands, such as commands for hand extension, fist clenching, grasping, and other actions.

[0179] The VR eye tracker module is integrated into the virtual reality helmet. It uses infrared cameras and image processing technology to monitor the patient's eye movement trajectory in real time and obtain the patient's visual attention information in the virtual reality scene.

[0180] The hand training module includes a glove with sensors and flexible drivers installed at the joints of the glove. It can perform corresponding movement training on the patient's hand according to control instructions and collect data such as the position, speed and strength of the hand movement in real time.

[0181] The data processing module uses a high-performance computer and runs specially developed data processing software to receive and process data from the brain-computer interface module, VR eye tracker module and hand training module, and generate training plans and feedback information based on preset algorithms.

[0182] The display module is a high-definition virtual reality display that shows patients realistic virtual reality training scenes, including various daily life scenes and game scenes, such as grabbing objects, buttoning, playing with building blocks, etc., and displays training feedback information in real time, such as training progress, scores, movement accuracy, etc.

[0183] In a second aspect of the embodiment of the present invention, a hand rehabilitation training method based on the above device is disclosed, comprising the following steps:

[0184] Step 1: The patient wears the brain-computer interface device and VR eye tracker and puts his hand into the glove of the hand training module. The medical staff helps the patient adjust the equipment to ensure that the collected signals are accurate and reliable.

[0185] Step 2: Initialize the system and set training parameters and goals, such as training time, training intensity, rehabilitation stage (early, middle, late), etc., and set specific training goals according to the degree of hand dysfunction of the patient, such as finger extension angle, grip strength, etc.

[0186] Step 3: The patient performs hand movement tasks in a virtual reality scene. For example, grabbing fruit in a virtual kitchen scene, organizing documents in a virtual office scene, etc. The brain-computer interface module and VR eye tracker module collect the patient's neural electrical signals and eye movement trajectories in real time.

[0187] Step 4: The data processing module analyzes and processes the collected data. The patient's hand movement intention and attention state are judged through feature extraction and pattern recognition algorithms. For example, if the patient's EEG signal shows a strong intention to extend the hand, and the eye movement trajectory shows that the patient is paying attention to an object that needs to be grasped, it is judged that the patient wants to grasp. Adjust the training difficulty and training mode according to the preset algorithm. If the patient can easily complete the current task, increase the training difficulty, such as increasing the weight of the object, reducing the size of the grasping target, etc.; if the patient shows inattention, adjust the training scene and add stimulating elements to attract the patient's attention.

[0188] Step 5: The hand training module conducts movement training for the patient's hand according to the adjusted training plan, and provides real-time feedback on the position, speed, and strength of the hand movement. For example, when the patient successfully grasps an object, the glove will give a certain resistance feedback, allowing the patient to feel the grasping force; at the same time, the position and movement trajectory of the hand in the virtual scene are displayed in real time, allowing the patient to intuitively understand whether his or her movements are accurate.

[0189] Step 6: The data processing module evaluates the training effect based on the feedback information and generates a training report. The report includes the training time, the number and quality of completed tasks, various indicators of hand movements, etc., and adjusts the subsequent training plan according to the progress of the training. For example, if the patient's training effect is not obvious over a period of time, the training method will be adjusted to increase targeted training movements; if the patient has made significant progress, the auxiliary force can be gradually reduced to allow the patient to rely more on his own strength to perform hand movements.

[0190] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A hand rehabilitation training method based on brain-computer interface and VR eye tracker, characterized in that: include: S1, using a virtual reality display module to display virtual reality scene information of hand rehabilitation training to the user; S2, using the hand training module to perform motion training on the user's hand, and collect a set of user hand motion information; the user hand motion information set includes a speed information sequence, a position information sequence, and a force information sequence of each key position of the user's hand; the key positions of the user's hand include five fingers and a palm; S3, using the brain-computer interface module to collect a set of brain electrical signals of the user; the set of brain electrical signals includes α wave signals, β wave signals, θ wave signals and δ wave signals; S4, using a VR eye tracker to collect eye movement trajectory information of the user when performing hand movement training; S5, using a data processing module to process the user's hand movement information set, EEG signal set, and eye movement trajectory information to obtain training effect information and feedback information; S6, using a display module to display the training effect information to the user; The feedback information is used to adjust the control torque of the flexible actuator of the hand training module.

2. The hand rehabilitation training method based on brain-computer interface and VR eye tracker as claimed in claim 1, characterized in that: The processing of the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information and feedback information includes: S51, evaluating and processing the user's hand movement information set, EEG signal set, and eye movement trajectory information to obtain training effect information; S52, acquiring a control torque signal of the flexible actuator of the hand training module; S53, performing feedback processing on the control torque signal and the training effect information to obtain feedback information.

3. The hand rehabilitation training method based on brain-computer interface and VR eye tracker as claimed in claim 2, characterized in that: The evaluation and processing of the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information includes: S511, obtaining a set of standard hand motion information; the set of standard hand motion information includes speed standard information, position standard information and strength standard information of key positions of each user's hand; S512, performing hand movement evaluation processing on the user hand movement information set and the hand standard movement information set to obtain a hand movement sub-effect value pg1; S513, performing an EEG evaluation transformation on the EEG signal set to obtain a EEG effect value pg2; S514, performing motion evaluation processing on the eye movement trajectory information to obtain an eye movement sub-effect value pg3; S515, performing weighted summation on the hand movement sub-effect value, the brain electronic effect value and the eye movement sub-effect value to obtain training effect information.

4. The hand rehabilitation training method based on brain-computer interface and VR eye tracker as claimed in claim 2, characterized in that: The feedback processing of the control torque signal and the training effect information to obtain feedback information includes: Performing torque effect transformation on the training effect information to obtain a torque equivalent value; The expression of the torque effect transformation is: Among them, h is the training effect information, γ0 and are preset constant factors, μ and are the second-order central moment and the second-order origin moment of the control torque signal respectively, and γ is the torque equivalent value; Performing feedback calculation on the torque equivalent value and the control torque signal to obtain feedback information; The expression of the feedback calculation is: Among them, θ is the feedback information and f(t) is the control torque signal.

5. The hand rehabilitation training method based on brain-computer interface and VR eye tracker as claimed in claim 3, characterized in that: The hand movement evaluation process is performed on the user hand movement information set and the hand standard movement information set to obtain the hand movement sub-effect value pg1, including: The speed information sequence, position information sequence and force information sequence of each key position of the user's hand are expressed as a hand motion matrix corresponding to the key position of the user's hand; The standard hand motion information set is represented as a standard matrix; the row vector of the standard matrix is ​​the speed standard information, position standard information and force standard information of a key position of a user's hand; Perform a first evaluation calculation on the hand motion matrix corresponding to the key positions of all users' hands and the standard matrix to obtain the hand motion sub-effect value pg1; The expression of the first evaluation calculation is: Among them, ω1 is the preset weight value, a i,kj is the element of the kth row and jth column of the hand motion matrix corresponding to the key position of the hand of the i-th user, b ik is the element in the i-th row and k-th column of the standard matrix, is the variance value of the kth row of the hand motion matrix corresponding to the key position of the hand of the i-th user, nd j is the jth element of the motion evaluation vector, M, K, and J are the number of key hand positions of the user, the row dimension of the hand motion matrix, and the column dimension of the hand motion matrix, respectively.

6. A hand rehabilitation training device based on a brain-computer interface and a VR eye tracker, characterized in that: A hand rehabilitation training method based on a brain-computer interface and a VR eye tracker for implementing any one of claims 1 to 5, comprising: a virtual reality display module, a hand training module, a brain-computer interface module, a VR eye tracker, a data processing module and a display module; The virtual reality display module is used to display virtual reality scene information of hand rehabilitation training to the user; The hand training module is used to perform motion training on the user's hands and collect a set of user hand motion information; The brain-computer interface module is used to collect a set of EEG signals from the user; The VR eye tracker is used to collect and obtain the eye movement trajectory information of the user when performing hand movement training; The data processing module is connected to the hand training module, the brain-computer interface module and the VR eye tracker respectively, and is used to process the user's hand movement information set, the EEG signal set, and the eye movement trajectory information to obtain training effect information and feedback information; and use the feedback information to adjust the control torque of the flexible driver of the hand training module; The display module is connected to the data processing module and is used to display the training effect information to the user.

7. The hand rehabilitation training device based on brain-computer interface and VR eye tracker as claimed in claim 6, characterized in that: The VR eye tracker module is arranged inside the virtual reality display module and is used to collect the user's eye movement trajectory information; The hand training module includes a glove, a flexible actuator, and a sensor group; the joints of the glove are equipped with a flexible actuator, and the flexible actuator is used to perform corresponding movement training on the patient's hand according to the control instruction; The sensor group includes a mechanical sensor and an acceleration sensor; the mechanical sensor is arranged at a key position of the user's hand corresponding to the glove, and is used to collect a force information sequence of each key position of the user's hand; The acceleration sensor is arranged at the key position of the user's hand corresponding to the glove, and is used to collect the speed information sequence and the position information sequence of each key position of the user's hand.

8. A hand rehabilitation training device based on a brain-computer interface and a VR eye tracker, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the hand rehabilitation training method based on brain-computer interface and VR eye tracker as described in any one of claims 1 to 5.

9. A computer storable medium, characterized in that: The computer storable medium stores computer instructions, which, when called by a computer, are used to execute the hand rehabilitation training method based on a brain-computer interface and a VR eye tracker as described in any one of claims 1 to 5.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the hand rehabilitation training method based on brain-computer interface and VR eye tracker as described in any one of claims 1 to 5.

Images