Fine motion imagination method and system based on hand motion parameter change

Abstract

The invention provides a fine motor imagery method and system based on hand motion parameter variation, the fine motor imagery method based on hand motion parameter variation can be applied to a cerebral apoplexy rehabilitation experiment / training system, the system comprises an electroencephalogram acquisition device, a motion capture device, a first computer, a second computer and a bearing platform, the fine motion imagination method based on the hand motion parameter change comprises the following steps: S1, building a high-precision motion capture and electroencephalogram synchronous acquisition experiment platform; S2, carrying out a speed-single parameter fine motion task; and S3, carrying out a speed-direction double-vector parameter fine motion task. Based on the fine motion tasks of different motion parameters, the degree of freedom of the motion imagination task is increased, the available normal form of the motion imagination task is expanded, and the brain coding and decoding rule of the vector space motion intention rich in direction, speed and other information can be further understood by adopting the speed-direction fine motion intention experimental design scheme.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electroencephalogram signal processing, and particularly relates to a fine motor imagination method and system based on hand movement parameter change. BACKGROUND

[0002] A brain computer interface (BCI) provides a direct connection between the human brain and electronic devices, realizing information exchange between the human brain and external devices.

[0003] Motor intention is divided into a motor preparation process of motor execution and motor imagination without actual limb movement.

[0004] Motor imagery (MI) refers to an imagined behavior performed only by the brain without actual limb movement.

[0005] In the aspect of motor intention BCI rehabilitation application, in recent years, under the guidance of the neurofeedback training mechanism, the research results of motor intention BCI combined with physical stimulation, auxiliary robots and the like for clinical rehabilitation training are remarkable. Stroke patients usually suffer from motor dysfunction and thus lose the ability to freely move and control muscles, but the plasticity of brain neurons makes it possible to realize motor function rehabilitation. Therefore, the rehabilitation training of neurofeedback combined with motor intention BCI has become a popular method for stroke treatment. Neurofeedback training is a kind of biofeedback training, which monitors the activity mode of the brain in real time, gives appropriate sound, image and tactile feedback to the trainee through computer system assistance, so that the trainee changes the activity mode of the brain by subjectively feeling and consciously changing the biological signals of the trainee, enhances the self-regulation ability of the brain, and improves or even repairs the function of the damaged brain. Motor intention BCI shows broad research prospects and great application potential in the rehabilitation treatment of motor dysfunction caused by central nervous system diseases such as stroke, and has attracted widespread attention from scholars at home and abroad.

[0006] However, the traditional motor imagery brain-computer interface (MI-BCI) usually only realizes the motor intention decoding of different limb parts (such as left and right hands), and the recognition accuracy is low, which restricts the fine motor control and rehabilitation effect. At present, the scalp electroencephalogram coding and decoding research of motor intention has experienced decades of development, and with the updating of related paradigm design and decoding algorithm, the traditional MI-BCI instruction set based on motor intention decoding is expected to break through the problems such as limited instruction set and unnatural brain-computer interaction. There are still some problems to be solved in the prior art: (1) In the existing research on motor intention, there is almost no research and analysis on the speed as a kinematic parameter; (2) The existing motor intention coding and decoding usually only targets the electroencephalogram features of a certain specific kinematic or dynamic parameter, and lacks the exploration of a vector space electroencephalogram response mapping model with multiple parameter coupling; (3) In addition, the existing motor intention BCI research is mostly in offline mode, and further optimization design of the paradigm and algorithm is needed to apply it to the online motor intention BCI system to prove its application feasibility. SUMMARY

[0007] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a fine motor imagination method and system based on hand movement parameter change, which can at least solve one of the above-mentioned problems in the prior art.

[0008] To achieve the above-mentioned purposes and other related purposes, the present application provides a fine motor imagination method based on hand movement parameter change, which is applied to a stroke rehabilitation experiment / training system, the stroke rehabilitation experiment / training system comprising an electroencephalogram acquisition device, a motion capture device, a first computer, a second computer and a support platform, and the fine motor imagination method based on hand movement parameter change comprises the following steps: S1, building a high-precision motion capture and electroencephalogram synchronous acquisition experiment platform; S2, performing a speed-single parameter fine motor task; S3, performing a speed-direction double vector parameter fine motor task.

[0009] In some embodiments, step S1 specifically comprises: S11, connecting the electroencephalogram acquisition device, the motion capture device, the first computer and the second computer; S12, building a high-precision motion capture and electroencephalogram synchronous acquisition experiment platform based on the motion capture device and the electroencephalogram acquisition device.

[0010] In some embodiments, step S2 specifically comprises: S21, presetting two modes of fine motor tasks, i.e. uniform speed mode and variable speed mode, the fine motor task at least comprising a motor imagination stage and a motor execution stage; S22, first performing a fine motor task in a uniform motion mode, in the uniform motion mode, the subject is required to move at a certain speed to the target position in the motion execution phase; S23, later performing a fine motor task in a variable speed motion mode, in the variable speed motion mode, the subject is required to move at the fastest speed to the target position after the motion execution phase starts.

[0011] In some embodiments, step S22 specifically comprises: S22a, preparation: the palm of the left hand or the right hand of the subject is placed on the supporting platform; S22b, trial experiment: the experiment is divided into four stages of calibration stage, motion imagination stage, motion execution stage and resting stage, the calibration stage is the preparation work before the experiment, and the motion capture calibration needs to be performed, the time is 0-2s; the motion imagination stage is the prompt period, prompting the subject to prepare for the corresponding task, the time is 2-4s, the motion execution stage is the task period, the subject needs to move the palm to the target position following the uniform speed moving prompt displayed on the stimulation screen of the first computer, the motion capture device can obtain the motion data generated by the palm in the moving process, and the first computer can form the motion trajectory based on the motion data collected by the motion capture device and feedback to the stimulation screen of the first computer; In the motion execution stage of step S22b, the uniform speed moving prompt includes two speed levels of first speed and second speed, and the first speed is less than the second speed, and the target position is 2, wherein the uniform speed moving prompt moves from the center starting point to the first target position at the first speed, and moves from the first target position to the second target position at the second speed.

[0012] In some embodiments, step S23 specifically comprises: S23a, preparation: the palm of the left hand or the right hand of the subject is placed on the supporting platform; S23b, trial experiment: the experiment is divided into four stages of calibration stage, motion imagination stage, motion execution stage and resting stage, the calibration stage is the preparation work before the experiment, and the motion capture calibration needs to be performed, the time is 0-2s; the motion imagination stage is the prompt period, prompting the subject to prepare for the corresponding task, the time is 2-4s, the motion execution stage is the task period, the subject needs to move the palm to the target position following the uniform speed moving prompt displayed on the stimulation screen of the first computer, the motion capture device can obtain the motion data generated by the palm in the moving process, and the first computer can form the motion trajectory based on the motion data collected by the motion capture device and feedback to the stimulation screen of the first computer; In the motion execution stage of step S23b, the prompt of the variable uniform velocity movement includes two speed levels of the third speed and the fourth speed, and the third speed is less than the fourth speed, and the target position is 2, wherein the prompt of the variable uniform velocity movement moves from the center starting point to the first target position at the third speed, and moves from the first target position to the second target position at the fourth speed.

[0013] In some embodiments, after step S3, further comprising: S4, offline modeling: performing multiple tasks to establish an offline model according to the electroencephalogram data; S5, online experiment / training: performing online experiment / training based on the offline model.

[0014] In some embodiments, step S4 specifically comprises: S41, repeating the above steps S2-S3 to perform multiple sets of fine motor tasks; S42, data preprocessing; S43, electroencephalogram data feature extraction; S44, pattern recognition; S45, two / four classification modeling; S46, outputting the offline model.

[0015] In some embodiments, step S5 specifically comprises: S51, performing multiple sets of online experiment / training according to the offline model outputted in step S46, each set of online experiment / training including multiple online experiment / training; S52, outputting the results of all sets after all the sets of online experiment / training are completed.

[0016] In some embodiments, step S51 specifically comprises: S51a, continuing to use the high-precision motion capture and electroencephalogram synchronous acquisition experiment platform built in step S1 to synchronously perform motion capture and electroencephalogram acquisition, and performing real-time motion trajectory feedback according to the acquired motion data; S51b, performing online processing task state data according to the acquired real-time electroencephalogram data: S51c, judging whether the total number of completed online experiment / training of the current set is greater than or equal to M, M=8-15: If yes, go to step S52; If no, go to step S51d; S51d, further judging whether the total number of times of the current set exceeds N times, N=10-15: If no, go to step S51a; If yes, go to step S51e; S51e, feeding back the results of the current completed set.

[0017] To achieve the above object and other related objects, the application further provides a fine motor imagination system based on hand motion parameter change, comprising a first computer, a second computer, an electroencephalogram acquisition device, an electroencephalogram amplifier, a motion capture device and a support platform, the electroencephalogram acquisition device is used to acquire electroencephalogram signals of a subject, the electroencephalogram amplifier is connected with the electroencephalogram acquisition device and the second computer respectively, the first computer and the second computer are in communication connection, the motion capture device is arranged directly above the support platform and is in wired connection with the first computer, the first computer is equipped with a stimulation screen used to interact with the subject, a high-precision motion capture and electroencephalogram synchronous acquisition experiment platform is built based on the motion capture device and the electroencephalogram acquisition device, the subject performs a fine motor task containing at least a motor imagination stage and a motor execution stage according to the guidance of the stimulation screen. In the process of the motor execution stage of the fine motor task, the palm of the left hand or the right hand of the subject is placed on the support platform and moves according to the guidance of the stimulation screen, the motion capture device is used to acquire motion data generated by the palm of the left hand or the right hand of the subject in the moving process, the first computer can form a motion trajectory based on the motion data acquired by the motion capture device and feed back to the stimulation screen of the first computer, and the second computer can at least perform feature extraction, offline modeling and online feedback based on the electroencephalogram data acquired by the electroencephalogram acquisition device.

[0018] As described above, the fine motor imagination method based on hand motion parameter change has the following beneficial effects: The application focuses on the kinematic intention-induced electroencephalogram response in BCI, and explores three aspects of problems, i.e., uniform speed and variable speed motion mode, speed-direction double-vector parameter coding model and vector space motion intention online feasibility: firstly, the motion intention paradigm of different motion speed modes of a single limb is studied, the neural response mode and law under different motion speed modes are studied, and a new speed-direction coupled kinematic intention experiment paradigm is designed according to the differences in electroencephalogram coding and decoding effects of the optimal motion speed mode, the double-vector parameter kinematic intention coding and decoding law model is further explored, and a speed-direction double-vector parameter online BCI system is further built.

[0019] Through the application, the beneficial effects include: 1. The simple motion intention task is extended to the fine motor task based on different motion parameters, the freedom degree of the motor imagination task is increased, and the available paradigm of the motor imagination task is expanded.

[0020] 2. The speed-direction fine motor intention experiment design scheme is helpful to further understand the brain coding and decoding law of the vector space motion intention rich in direction, speed and other information.

[0021] 3. By designing a complex motor imagery task, the application increases the possibility of motor imagery brain-computer interface in practical application.

[0022] 4. The offline experiment is expanded to online experiment, forming an online verification platform with good interaction. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 It is a simplified system framework diagram of the application.

[0024] Figure 2 It is an application scenario diagram of the application.

[0025] Figure 3 It is a schematic diagram of the high-precision motion capture and EEG synchronous acquisition experiment platform of the application.

[0026] Figure 4 It is one of the electroencephalograms of the application.

[0027] Figure 5 It is one of the timing diagrams of the trial experiment under the uniform motion mode of the application.

[0028] Figure 6 It is a guide displayed in the stimulation interface under the variable speed motion mode of the application.

[0029] Figure 7 It is one of the timing diagrams of the trial experiment under the variable speed motion mode of the application.

[0030] Figure 8 It is a running flowchart of the offline mode and the online mode of the application.

[0031] Figure 1 The reference signs in the drawings: 100-electroencephalogram acquisition device; 200-motion capture device; 300-first computer; 400-second computer; 500-electroencephalogram amplifier. DETAILED DESCRIPTION

[0032] The embodiments of the application are described below by specific concrete examples, and those skilled in the art can easily understand other advantages and effects of the application from the disclosure of the specification. The application can also be implemented or applied by different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the application. It should be noted that the following examples and features in the examples can be combined with each other without conflict.

[0033] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concepts of the present application, and only the components related to the present application are shown in the diagrams, rather than being drawn according to the number, shape and size of the components in actual implementation. The shapes, number and proportions of the components in actual implementation can be arbitrarily changed, and the layout pattern of the components can be more complex.

[0034] Referring to Figures 1-8 The present application provides a fine motor imagery method and system based on hand motion parameter change.

[0035] The fine motor imagery system based on hand motion parameter change can be used as a stroke rehabilitation experiment / training system, and the fine motor imagery method based on hand motion parameter change is applied to the stroke rehabilitation experiment / training system.

[0036] As Figure 1 shown, the stroke rehabilitation experiment / training system includes an electroencephalogram acquisition device 100, a motion capture device 200, a first computer 300, a second computer 400 and a support platform. The electroencephalogram acquisition device 100 is used to acquire the electroencephalogram signal of a subject. An electroencephalogram amplifier 500 is connected with the electroencephalogram acquisition device 100 and the second computer 400 respectively. The first computer 300 and the second computer 400 are connected through a UDP (User Datagram Protocol) protocol. The electroencephalogram amplifier 500 communicates with the second computer 400 through a parallel port. The motion capture device 200 is arranged directly above the support platform and is wiredly connected with the first computer 300. The first computer 300 is equipped with a stimulation screen used for interacting with the subject. A high-precision motion capture and electroencephalogram synchronous acquisition experiment platform is built based on the motion capture device 200 and the electroencephalogram acquisition device 100. The subject performs a fine motor task including at least a motor imagery stage and a motor execution stage according to the guidance of the stimulation screen. The electroencephalogram acquisition device 100 adopts an electroencephalogram cap produced by Compumedics Neuroscan Company. The electroencephalogram amplifier 500 adopts SYNAMPS RT, which is a high-density electroencephalogram amplifier system produced by Compumedics Neuroscan Company.

[0037] During the experiment / training process of the fine motor task, the subject sits on a comfortable experimental chair according to the requirements, and the right hand is placed in a relaxed state on the central position of the support platform on the right side of the body. The motion capture device 200 is directly above the hand. The display of the first computer 300 is placed 1 m in front of the subject. The motion data and the electroencephalogram data are synchronously acquired by using the built motion capture device 200 and the electroencephalogram acquisition device 100, and the EEG signal is labeled. Figure 2As shown, the experimental environment as a whole comprises four parts of a stimulation screen, motion capture, a support platform and electroencephalogram acquisition. The overall operation process of the experiment is as follows: first, a preset motion intention induction prompt picture is presented on the stimulation screen, then the subject performs corresponding hand movement on the support platform according to the prompt, the motion capture device 200 records the motion data such as position, speed and acceleration of each sampling point of the hand in the two-dimensional plane, the data is saved to the corresponding folder of the subject and the real-time motion trajectory of the hand is transmitted and presented to the stimulation screen.

[0038] In the process of performing the fine motor task in the motion execution phase, the palm of the left hand or the right hand of the subject is placed on the support platform and moves according to the guidance of the stimulation screen. The motion capture device 200 is used to acquire the motion data generated by the palm of the left hand or the right hand of the subject in the moving process. The first computer 300 can form a motion trajectory based on the motion data collected by the motion capture device 200 and feed back to the stimulation screen of the first computer 300. The second computer 400 can at least extract features, offline modeling and online feedback based on the electroencephalogram data collected by the electroencephalogram acquisition device 100.

[0039] The fine motor imagination method based on hand motion parameter change comprises the following steps: S1, build a high-precision motion capture and electroencephalogram synchronous acquisition experimental platform: The specific implementation mode of this step is as follows: S11, connect the electroencephalogram acquisition device 100, the motion capture device 200, the first computer 300 and the second computer 400; S12, build a high-precision motion capture and electroencephalogram synchronous acquisition experimental platform based on the motion capture device 200 and the electroencephalogram acquisition device 100, as shown in Figure 3 .

[0040] It should be noted that the high-precision motion capture and EEG synchronous acquisition experiment platform is built based on the motion capture device 200 and the EEG acquisition device 100, and the actual motion trajectory of the hand of the subject can be fed back in the stimulation screen in real time. The motion capture device 200 adopts a commercially available somatosensory sensor, such as a Leap Motion somatosensory capturer of the Leap company, which is connected to the host of the first computer 300 through a USB connection line, captures and transmits high-precision hand motion data of the subject to the host of the first computer 300, including time information, speed, acceleration and three-dimensional position information corresponding to the task, and the motion data signal acquisition frequency is 100 Hz, which is synchronous with the EEG data recording. The motion capture system can make the motion trajectory appear in the interactive screen in real time: first, the official development kit of the motion capture device 200 is used to develop a Leap Motion motion data stream extraction program, the motion data of the palm center module is extracted and stored in a temporary json file, and real-time coverage update is maintained, then the self-developed motion trajectory real-time presentation program reads the temporary file to process the motion coordinate data to generate a motion trajectory stimulation screen, and the effective linkage of the LM data stream and the stimulation interface is realized. In the experiment, the motion capture system sends a task start tag instruction to the EEG acquisition device 100 at the moment of starting the motion according to the kinematic data of the subject, and sends a task end tag instruction when the subject moves to the target position, as shown in Figure 4 The function of the tag is to accurately record the EEG position under the preset motion intention, and facilitate subsequent data processing. It should be noted that the experiment platform has several fixed parameters: the Leap distance support platform is about 30 cm (which can be adjusted accordingly according to the actual situation), the Leap data refresh rate is 9 ms (110 Hz), and the Leap motion capture delay average time is 11.2 ms (actual measurement). The scaling ratio of the support platform in the screen is 1:3.

[0041] S2, a fine motor task of speed-single parameter is performed, specifically including: S21, the fine motor task has two modes of uniform motion mode and variable speed motion mode, and the fine motor task at least includes a motor imagination stage and a motor execution stage; S22, the fine motor task is first performed in the uniform motion mode, and in the uniform motion mode, the subject is required to move to the target position at a certain speed in the motor execution stage; S23, the fine motor task is then performed in the variable speed motion mode, and in the variable speed motion mode, the subject is required to move to the target position at the fastest speed after the motor execution stage starts.

[0042] As preferred, step S22 specifically includes: S22a, preparation: the palm of the left hand or the right hand of the subject is placed on the support platform; S22b, test experiment: the experiment is divided into four stages of calibration stage, motor imagination stage, motor execution stage and resting stage. The calibration stage is the preparation work before the experiment, which needs to be calibrated by motion capture, and the time is 2s (0-2s); the motor imagination stage is the prompt period, which prompts the subjects to prepare for the corresponding task, and the time is 2s (2-4s); the motor execution stage is the task period, and the time is 2s (4-6s), the subjects need to move the palm to the target position following the uniform speed moving prompt displayed on the stimulus screen of the first computer 300, the motion capture device 200 can obtain the motion data generated by the palm during the movement, and the first computer 300 can form the motion trajectory based on the motion data collected by the motion capture device 200 and feedback to the stimulus screen of the first computer 300; the resting stage is the resting period, and the white circle appears in the center of the screen again, and the duration is 2s (6-8s), in this stage, the subjects do not perform any task, and the purpose is to make the subjects adjust the mode and wait for the start of the next test between two task tests.

[0043] As another embodiment, as shown in Figure 5 the experiment is divided into five stages of calibration stage, recovery stage, motor imagination stage, motor execution stage and resting stage: The first stage is the calibration stage, which is mainly the preparation work before the experiment / training, which needs to be calibrated by motion capture, and the time is 2s (0-2s); The second stage is the recovery stage, which is the resting period, which needs to be recovered from the calibration state (2-4s); The third stage is the motor imagination stage, which is the prompt period, which is used to prompt the subjects to prepare for the corresponding task, and the time is 2s (4-6s); The fourth stage is the motor execution stage, which is the task period, and the time is 2s (4-6s), the subjects need to move the palm to the target position following the uniform speed moving prompt displayed on the stimulus screen of the first computer 300, the motion capture device 200 can obtain the motion data generated by the palm during the movement, and the first computer 300 can form the motion trajectory based on the motion data collected by the motion capture device 200 and feedback to the stimulus screen of the first computer 300; The fifth stage is the resting stage, which is the resting period, and the white circle appears in the center of the screen again, and the duration is 2s (6-8s), in this stage, the subjects do not perform any task, and the purpose is to make the subjects adjust the mode and wait for the start of the next test between two task tests.

[0044] In the movement execution stage of step S22b, the uniform speed movement prompt includes two speed levels of a first speed and a second speed, and the first speed is less than the second speed, and the target position is 2, wherein the uniform speed movement prompt moves from the center starting point to the first target position at the first speed, and moves from the first target position to the second target position at the second speed.

[0045] As preferred, step S23 specifically includes: S23a, preparation: the palm of the left hand or the right hand of the subject is placed on the supporting platform; S23b, trial experiment: as shown in Figure 6 and 7 , the experiment is divided into four stages of calibration stage, movement imagination stage, movement execution stage and resting stage. The calibration stage is the preparation work before the experiment, which needs to be calibrated by motion capture, and the time is 0-2s; the movement imagination stage is the prompt period, prompting the subject to prepare for the corresponding task, the time is 2-4s, the movement execution stage is the task period, after the subject sees the variable speed uniform speed movement prompt displayed on the stimulus screen of the first computer 300, the subject needs to move the palm to the target position at the fastest speed he thinks, the motion capture device 200 can obtain the movement data generated by the palm during the movement, and the first computer 300 can form a movement trajectory based on the movement data collected by the motion capture device 200 and feedback to the stimulus screen of the first computer 300; In the movement execution stage of step S23b, the variable speed uniform speed movement prompt includes two speed levels of a third speed and a fourth speed, and the third speed is less than the fourth speed, and the target position is 2, wherein the variable speed uniform speed movement prompt moves from the center starting point to the first target position at the third speed, and moves from the first target position to the second target position at the fourth speed.

[0046] Specifically, the experiment of each speed mode includes 10 groups, and each group includes 20 trials (2*10), so in the uniform speed or variable speed experiment, each subject completes 100 fast tasks and 100 slow tasks. After each group of experiments is completed, the subject will choose whether to rest and relieve fatigue according to his own situation to maintain good mental state as a whole.

[0047] On the basis of single parameter experiment, it can be determined that variable speed movement mode (preferably fast mode of variable speed movement mode) is more suitable for designing motor intention brain electrical coding and decoding paradigm, and then step S3 is entered: S3, fine motor task of speed-direction double vector parameters.

[0048] In the step, the variable speed movement parameters and the up, down, left and right four direction movement parameters are coupled to design the speed-direction double parameter kinematic intention experiment paradigm, as shown in Figure 6The timing diagram of the experiment / training process is shown as Figure 7 The subjects follow the movement intention response prompts on the stimulus screen to collect movement intention EEG data and construct different speed-direction kinematic intention EEG decoding models. The decoding parameters such as the optimal frequency band, the best classification combination, and the best decoding time window are screened and optimized to finally realize the best speed-direction double-parameter kinematic intention offline decoding classification effect.

[0049] As preferred, after step S3, it further includes: S4, offline modeling: multiple tasks are performed, and an offline model is established according to the EEG data, as shown in Figure 8 The specific steps of this step include: S41, repeat the above steps S2-S3 to perform multiple sets of fine motor tasks; As preferred, S42, data preprocessing; S43, EEG data feature extraction; S44, pattern recognition; S45, two / four classification modeling; S46, output offline model.

[0050] S5, online experiment / training: based on the offline model, online experiment / training is performed, as shown in Figure 8 The specific steps of this step include: S51, according to the offline model output by step S46, multiple sets of online experiment / training are performed, and each set of online experiment / training includes multiple online experiment / training; S52, after all sets of online experiment / training are completed, the results of all sets are output.

[0051] As preferred, step S51 specifically includes: S51a, continue to use the high-precision motion capture and EEG synchronous acquisition experiment platform built in step S1 to synchronously perform motion capture and EEG acquisition, and perform real-time motion trajectory feedback according to the acquired motion data; S51b, perform online processing task state data according to the acquired real-time EEG data: S51c, judge whether the total number of completed online experiment / training of the current set is greater than or equal to M: If yes, go to step S52; If no, go to step S51d; In this step, M=8-15, preferably M=10.

[0052] S51d, further judge whether the total number of the current set is more than N times: If no, go to step S51a; If yes, go to step S51e; In this step, N=10-15, preferably N=10.

[0053] S51e, feedback the result of the current completed group.

[0054] The specific implementation of the above steps is illustrated as follows: First, 3 groups of trial experiments are carried out offline, each group of trial experiment including 12 experiments, preferably using a four-class mode (i.e. 3 experiments in each direction), and the offline data is sequentially preprocessed, feature extracted, pattern recognized, and a classification model is constructed. The decoding parameters screened in the foregoing are directly used in modeling, and finally an offline model for speed-direction online experiment is obtained. There are two experimental sequences for online experiment, sequence one is online experiment of two-classification first and four-classification second, and sequence two is online experiment of four-classification first and two-classification second. The two experimental sequences are randomly executed in 15 subjects. The online experiment continues to use the motion capture and electroencephalogram synchronous acquisition system built in the foregoing step S1, the motion data collected is used for real-time motion trajectory feedback, the real-time electroencephalogram data collected is used to generate a 30s buffer zone, and 1s buffer data is used for online task-state decoding. The actual online accuracy is fed back after each online experiment / training group is completed, so that the subject can adjust his / her motion mode according to the actual situation to achieve more ideal experimental results, and finally all the results are output after M groups of experiments are completed.

[0055] The above two-classification refers to only up-down direction fast movement experiment; The above four-classification refers to up-down-left-right four direction fast movement experiment, wherein the fast movement is fast level (fourth speed) in variable speed movement mode.

[0056] The motion capture device 200 of the application is used for capturing hand motion information, which can obtain detailed motion data information under high-speed motion condition, can accurately record the motion information of the current object, and restore the spatial position and action information of the hand or other object through the built-in algorithm. Based on the above functions, a program of hand real-time motion trajectory matched with the electroencephalogram acquisition device 100 is further developed, and the feedback interaction of electroencephalogram and motion is realized. The two kinds of features of MRCP (Movement Related Cortical Potentials) and ERD (Event-related desynchronization) are extracted by using discriminant spatial pattern (DSP) and common spatial pattern (CSP) algorithms respectively to perform feature classification, and the effectiveness of the kinematic intention electroencephalogram coding paradigm designed by the application in different movement speed modes is verified.

[0057] As described above, the fine motor imagination method based on hand movement parameter change has the following beneficial effects: The application focuses on the kinematic intention induced electroencephalogram response in BCI, and explores the problems in three aspects of uniform speed and variable speed movement mode, speed-direction double vector parameter coding model and vector space movement intention online feasibility: firstly, the movement intention paradigm of single limb in different movement speed modes is studied, the neural response mode and rule in different movement speed modes are studied, and a new speed-direction coupled kinematic intention experiment paradigm is designed according to the difference of brain electroencephalogram coding and decoding effect of the optimal movement speed mode, the double vector parameter kinematic intention coding and decoding rule model is further explored, and a speed-direction double vector parameter online BCI system is further built.

[0058] The fine motor imagination method and system based on hand movement parameter change of the application have at least the following advantages relative to the prior art: 1、Generally, the motor imagination task is generally simple imagination of left and right hand grasping movement, the application extends from simple movement intention task to fine motor task based on different movement parameters, increases the degree of freedom of motor imagination task, expands the available paradigm of motor imagination task, and provides theoretical basis and technical support for high degree of freedom motor intention electroencephalogram coding and efficient brain-computer interaction.

[0059] 2、The fine motor intention experiment based on speed-direction double kinematic parameters is helpful to further understand the brain coding and decoding rule of vector space movement intention rich in direction, speed and other information, and further helps to restore the movement function of patients with movement disorders.

[0060] 3、By designing a complex motor imagination paradigm, more output instructions of the motor imagination brain-computer interface can be increased, and when expanding to practical application, more output instructions can change more human-computer interaction functions, and lay a foundation for more natural human-computer interaction.

[0061] 4. From the offline experimental system to the online experimental system verification, it is proved that the practical application of the present application is feasible, and a good online interactive verification platform is a prerequisite for turning to practical application.

[0062] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought disclosed by the present application should be covered by the claims of the present application.

Claims

1. A fine motor imagery method based on changes in hand movement parameters, characterized in that: Applied to a stroke rehabilitation experiment / training system, the stroke rehabilitation experiment / training system includes an electroencephalogram (EEG) acquisition device, a motion capture device, a first computer, a second computer, and a support platform. The fine motor imagery method based on changes in hand motion parameters includes the following steps: S1. Build a high-precision motion capture and EEG synchronous acquisition experimental platform: S2: Perform a speed-single-parameter fine motor task: S3. Perform fine motor tasks with speed-direction dual vector parameters.

2. The fine motor imagery method based on changes in hand movement parameters according to claim 1, characterized in that: Step S1 specifically includes: S11, connecting the EEG acquisition device, the motion capture device, the first computer and the second computer; S12. Build a high-precision motion capture and EEG synchronous acquisition experimental platform based on motion capture equipment and EEG acquisition equipment.

3. The fine motor imagery method based on hand motion parameter changes according to claim 1, characterized in that: Step S2 specifically includes: S21, presetting a fine motor task with two modes: a uniform speed motion mode and a variable speed motion mode, wherein the fine motor task includes at least a motor imagination stage and a motor execution stage; S22, first perform the fine motor task in the uniform motion mode. In the uniform motion mode, the subjects are required to move to the target position at a constant speed during the movement execution phase; S23, then performed the fine motor task in the variable speed movement mode. In the variable speed movement mode, the subjects were required to move the target position as fast as possible after the start of the movement execution phase.

4. The fine motor imagery method based on changes in hand movement parameters according to claim 3, characterized in that: Step S22 specifically includes: S22a. Preparation: The subject's left or right palm is placed on the support platform; S22b, trial experiment: The experiment is divided into four stages: calibration stage, motor imagery stage, motor execution stage, and resting stage. The calibration stage is the preparatory work before the experiment, which requires motion capture calibration and lasts for 0-2 seconds. The motor imagery stage is the prompt period, which prompts the subject to prepare for the corresponding task and lasts for 2-4 seconds. The motor execution stage is the task period, in which the subject needs to move the palm to the target position following the uniform movement prompt displayed on the stimulation screen of the first computer. The motion capture device can obtain motion data generated by the palm during the movement. The first computer can form a motion trajectory based on the motion data collected by the motion capture device and feed it back to the stimulation screen of the first computer; In the motion execution stage of step S22b, the prompt for uniform movement includes two speed levels, the first speed and the second speed, and the first speed is less than the second speed. There are two target positions, among which the prompt for uniform movement moves from the center starting point to the first target position at the first speed, and moves from the first target position to the second target position at the second speed.

5. The fine motor imagery method based on hand motion parameter changes according to claim 3, characterized in that: Step S23 specifically includes: S23a. Preparation: The subject's left or right palm is placed on the support platform; S23b, trial experiment: The experiment is divided into four stages: calibration stage, motor imagery stage, movement execution stage and rest stage. The calibration stage is the preparatory work before the experiment, which requires motion capture calibration and lasts for 0-2 seconds. The motor imagery stage is the prompt period, which prompts the subjects to prepare for the corresponding task and lasts for 2-4 seconds. The movement execution stage is the task period. After seeing the prompt of variable speed and uniform speed displayed on the stimulation screen of the first computer, the subjects need to follow the prompt and move their palm to the target position at the fastest speed they think. The motion capture device can obtain the motion data generated by the palm during the movement. The first computer can form a motion trajectory based on the motion data collected by the motion capture device and feedback it to the stimulation screen of the first computer; In the motion execution stage of step S23b, the prompt for variable speed and uniform speed movement includes two speed levels, the third speed and the fourth speed, and the third speed is less than the fourth speed. There are two target positions, among which the prompt for variable speed and uniform speed movement moves from the center starting point to the first target position at the third speed, and moves from the first target position to the second target position at the fourth speed.

6. The fine motor imagery method based on hand motion parameter changes according to any one of claims 1 to 5, characterized in that: After step S3, the method further includes: S4, offline modeling: perform multiple tasks and build an offline model based on EEG data; S5. Online experiment / training: Conduct online experiment / training based on the offline model.

7. The fine motor imagery method based on hand motion parameter changes according to claim 6, characterized in that: Step S4 specifically includes: S41, repeat steps S2-S3 to perform multiple sets of fine motor tasks; S42, data preprocessing; S43, EEG data feature extraction; S44, pattern recognition; S45, binary / quadruple classification modeling; S46: Output the offline model.

8. The fine motor imagery method based on hand motion parameter changes according to claim 7, characterized in that: Step S5 specifically includes: S51, performing multiple groups of online experiments / training according to the offline model output in step S46, each group of online experiments / training including multiple online experiments / training; S52. After all the online experiments / trainings are completed, the results of all the groups are output.

9. The fine motor imagery method based on hand motion parameter changes according to claim 8, characterized in that: Step S51 specifically includes: S51a, continue to use the high-precision motion capture and EEG synchronous acquisition experimental platform built in step S1 to synchronously perform motion capture and EEG acquisition, and provide real-time motion trajectory feedback based on the acquired motion data; S51b, process task state data online based on the collected real-time EEG data: S51c. Determine whether the total number of groups of the currently completed online experiment / training is greater than or equal to M, where M = 8-15: If yes, proceed to step S52; If not, proceed to step S51d; S51d, further determine whether the total number of times of the current group exceeds N times, N=10-15: If not, proceed to step S51a; If yes, proceed to step S51e; S51e: Feedback the result of the currently completed group.

10. A fine motor imagery system based on changes in hand movement parameters, characterized in that: The apparatus comprises a first computer, a second computer, an EEG acquisition device, an EEG amplifier, a motion capture device, and a support platform, wherein the EEG acquisition device is used to acquire EEG signals of a subject, the EEG amplifier is connected to the EEG acquisition device and the second computer, respectively, the first computer and the second computer are in communication connection, the motion capture device is arranged directly above the support platform and is connected to the first computer by wire, the first computer is equipped with a stimulation screen for interacting with the subject, a high-precision motion capture and EEG synchronous acquisition experimental platform is constructed based on the motion capture device and the EEG acquisition device, and the subject performs a fine motor task including at least a motor imagery stage and a motor execution stage according to the guidance of the stimulation screen; During the movement execution phase of the fine motor task, the subject's left or right palm is placed on the support platform and moves according to the guidance of the stimulation screen. The motion capture device is used to obtain the motion data generated by the subject's left or right palm during the movement. The first computer can form a motion trajectory based on the motion data collected by the motion capture device and feed it back to the stimulation screen of the first computer. The second computer can at least perform feature extraction, offline modeling, and online feedback based on the EEG data collected by the EEG acquisition device.

Images