Brain-computer interface three-dimensional space motion rehabilitation training system and method

Abstract

The invention discloses a brain-computer interface three-dimensional space motion rehabilitation training system and method. The system comprises a naked eye 3D stimulation display module, a space motion training scene module, an electroencephalogram signal acquisition and analysis module and a rehabilitation robot module. A stimulation interface is presented based on a naked eye 3D technology, a user is supported to operate a target indication point on a naked eye 3D display screen to perform three-dimensional motion through a steady-state visual evoked potential brain control method, and then a rehabilitation robot in an actual environment follows the indication point to drive an affected limb to move in a three-dimensional space, so that virtual-real combined training is achieved; and the immersive three-dimensional scene is provided for the user, three-dimensional space motion rehabilitation training is completed comfortably and efficiently, and the effects of being good in interaction experience, low in dizziness and more stable are achieved.

Description

Technical Field

[0001] The present invention relates to the field of rehabilitation training for stroke patients in biomedical engineering and brain-computer interface technology, and in particular to a brain-computer interface three-dimensional space motion rehabilitation training system and method. Background Art

[0002] In order to speed up the recovery of stroke patients in rehabilitation training and improve the problems of physical therapy, exercise training and drug therapy in traditional rehabilitation training, such as the need for professional guidance and high costs, the recently booming brain-computer interface (BCI) technology has been applied to the rehabilitation treatment of stroke patients. Brain-computer interface rehabilitation technology enables stroke patients to autonomously control rehabilitation robots directly through EEG signals, thereby driving the corresponding affected limb movement of stroke patients to complete rehabilitation training. Among the related technologies that stimulate the corresponding EEG signals that can control the movement of rehabilitation training robots, Steady-State Visual Evoked Potential (SSVEP) has attracted much attention because of its outstanding response and low sensitivity to noise interference. In the application of SSVEP, how to present stable and accurate induced stimulation images that can bring a certain sense of immersion to users is the key to improving the accuracy and sensitivity of system recognition.

[0003] Traditional SSVEP is stimulated by light-emitting diodes or computers presenting visual stimulation screens. However, these stimulation devices are bulky and the system has poor mobility. It also makes the interaction with real-world objects unintuitive and unsuitable for rehabilitation training. In addition, traditional devices can only present two-dimensional SSVEP stimulation, which makes it difficult to support the completion of three-dimensional space training tasks, weakening the training results. For example, the patented multimodal asynchronous BCI and visual fusion robot collaborative control method and system (publication number: CN117260714A) arranges 15 brain-controlled visual stimulation targets in the screen space, and each instruction is mapped to the robot arm action instruction in 3D space. When used, the sense of space is weak, and it is difficult to induce users to correspond visual stimulation to action instructions.

[0004] With the continuous development of 3D display technology, 3D visual stimulation can present interactive training scenes and visual stimulation targets in three-dimensional space, with a strong sense of immersion, which helps patients to carry out intensive training. For example, the common ways of presenting 3D stimulation are that the observer wears 3D glasses based on the principle of polarized light, VR (Virtual Reality) glasses or AR (Augmented Reality) glasses. For example, the patented visual perception function evaluation system based on virtual reality-brain-computer interface (publication number: CN118236074A) presents three-dimensional visual stimulation of fingers moving in front of the eyes of the human eye through polarized displays and glasses, inducing stimulation-related features in EEG signals; but this technology is not very friendly to people who usually need to wear myopia glasses. For stroke patients, long-term wearing can also cause physical discomfort, which is not conducive to the patient's recovery. The patented hybrid brain-computer interface online system with multi-scene autonomous control based on VR (publication number: CN112990074B) uses a VR helmet to present a virtual brain-computer interface 3D application scene control. Since it is a completely virtual scene presentation, the interactivity with the real environment is poor, which can easily cause users to feel dizzy when using it. The patented brain-computer interface technology method induced by augmented reality (publication number: CN113434040B) uses augmented reality technology to superimpose virtual three-dimensional objects as flickering stimulation sources on real scene pictures, construct brain-computer interface control scenes and SSVEP virtual three-dimensional object stimulation targets, interact with real space, and have a stimulation effect of virtual-real integration. However, the brain-computer interface stimulation used by this technology is still in a planar flickering form, without a strong sense of three-dimensional space, and the AR device is relatively heavy. After wearing it for a long time, it is easy to cause physical fatigue, such as neck pain, and poor comfort, which is not conducive to the recovery of rehabilitation patients.

[0005] Naked-eye 3D technology means that users can use only naked-eye 3D display screens to present a three-dimensional effect that can be perceived by the human eye from a flat two-dimensional image source or video source without any additional glasses or wearable devices. It is easy and comfortable to use, supports the rapid construction of three-dimensional space training scenes, and people who have not received relevant training can also quickly adapt to it. It is a visual stimulation presentation carrier for brain-computer interfaces that is more suitable for three-dimensional space rehabilitation training. At present, there is no literature published on its use in brain-computer interface three-dimensional space movement rehabilitation training. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a brain-computer interface three-dimensional space motion rehabilitation training system and method, which presents a stimulation interface based on naked-eye 3D technology, supports users to operate the three-dimensional movement of the target indication point on the naked-eye 3D display screen through the steady-state visual evoked potential brain control method, and then the rehabilitation robot in the actual environment follows the guidance point to drive the affected limb to move in three-dimensional space, thereby achieving virtual and real combined training, providing users with an immersive three-dimensional scene, so as to complete three-dimensional space motion rehabilitation training comfortably and efficiently; it has good interactive experience, low dizziness, and more stable effects.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A brain-computer interface three-dimensional space motion rehabilitation training system, comprising a naked-eye 3D stimulation display module, a space motion training scene module, an electroencephalogram signal acquisition and analysis module, and a rehabilitation robot module;

[0009] Naked-eye 3D stimulation display module: The naked-eye 3D stimulation display module is mainly a naked-eye 3D display screen based on lenticular lens technology. The bottom of the naked-eye 3D display screen is equipped with an eyeball capture camera, which focuses the right-eye image on the viewer's right eye and the left-eye image on the viewer's left eye. Due to the existence of parallax, the user can observe a stereoscopic image or video.

[0010] Spatial motion training scene module: its function is to present the SSVEP visual stimulation induction training scene interface through the naked eye 3D stimulation display module. The drawing of the visual stimulation induction training scene interface is divided into two parts, visible to the left eye and visible to the right eye. The two parts have the same content, but there is parallax, so a three-dimensional sense appears after being displayed by the naked eye 3D stimulation display module. In the visual stimulation induction training scene interface, there are SSVEP induced stimulation targets and spatial position feedback targets. The SSVEP induced stimulation target is used to induce SSVEP brain characteristics, and the spatial position feedback target is used to display the motion position in space, and move in three-dimensional space according to the control instructions corresponding to the characteristic brain response.

[0011] EEG signal acquisition and analysis module: including EEG signal amplifier, EEG cap and signal processing algorithm; the user generates corresponding EEG response signals by looking at different stimuli, which are then collected by the EEG signal acquisition device and saved in real time as corresponding experimental data; the collection location of EEG induced signals is located near the occipital area of ​​the brain, and the corresponding electrode leads of the brain are selected; after the EEG response signals are collected, the real-time data is analyzed and processed by the signal processing algorithm, and after the stimulation frequency results are obtained, the corresponding control instructions are output to the rehabilitation robot module;

[0012] Rehabilitation robot module: its function is to receive the control instructions sent by the EEG signal acquisition and analysis module, drive the affected limb to move to the real three-dimensional spatial position where the spatial position feedback target is located, so as to complete the movement training; the rehabilitation robot module hardware includes an end-traction rehabilitation robot type, which pulls the upper limb movement and tracks the movement to the spatial position feedback target position in the spatial movement training scene module.

[0013] The naked-eye 3D display screen in the naked-eye 3D stimulation display module is achieved by adding a multi-lens screen in front of the display screen, and using a row of vertically arranged cylindrical lenses to control the projection directions of the left and right images.

[0014] The visual stimulation induction training scene interface of the spatial motion training scene module has multiple SSVEP-induced stimulation targets, and each SSVEP-induced stimulation target performs 3D reciprocating stereoscopic motion or planar reciprocating motion or flashes at different frequencies.

[0015] The SSVEP-induced stimulation target is in the shape of an arrow-shaped, circular, rectangular or radial plane or three-dimensional shape.

[0016] The 3D reciprocating stereoscopic motion is composed of the reciprocating plane motion of the stimulus target zooming in and out and the reciprocating depth motion of the parallax change, which can induce the EEG perception of stereoscopic motion during the stimulation process; the plane reciprocating motion is composed of the stimulus target zooming in and out or reciprocating rotational motion or back-and-forth linear or curved motion; the flicker is the periodic change of brightness, contrast or color at a certain frequency.

[0017] The adjustable parameters of the SSVEP-induced stimulation target include stimulation image texture, stimulation image change frequency, stimulation image size and frequency change waveform.

[0018] The signal processing algorithm of the EEG signal acquisition and analysis module adopts a canonical correlation analysis signal processing algorithm (Canonical Correlation Analysis, CCA).

[0019] A method for utilizing a brain-computer interface three-dimensional space motion rehabilitation training system comprises the following steps:

[0020] Step 1: Wear the EEG cap in the EEG signal acquisition and analysis module and adjust the distance between the user and the naked-eye 3D display screen;

[0021] Step 2, running the initialization program, the rehabilitation robot in the rehabilitation robot module moves to the initial position, and fixes the user's affected limb at the end of the rehabilitation robot;

[0022] Step 3, run the program, and the spatial motion training scene module opens the spatial motion training scene;

[0023] Step 4: The spatial motion training scene is presented, and the user looks at different SSVEP-induced stimulation targets to control the spatial position feedback target position;

[0024] Step 5, the EEG signal acquisition and analysis module acquires and analyzes brain signals, and calculates the CCA coefficient corresponding to each frequency according to the canonical correlation analysis signal processing algorithm (CCA). If the CCA coefficient does not exceed the set threshold, it is considered that the user is in an idle state. If the CCA coefficient exceeds the set threshold, it is considered that the user is in an active training state. The frequency corresponding to the largest CCA coefficient is selected as the response recognition result to the stimulation signal. The SSVEP induced stimulation target and control instruction that the user is looking at corresponding to the frequency, and the spatial position feedback target moves according to the control instruction;

[0025] Step 6: The rehabilitation robot starts to respond synchronously according to the judgment result of the brain control signal, follows the spatial position feedback target in real time, drives the user's affected limb, and thus completes the active training task;

[0026] Step 7, repeat steps 4-6, conduct continuous rehabilitation training, and finally end the training task after a specified time.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] Since the present invention adopts SSVEP-induced brain-computer rehabilitation based on naked-eye 3D presentation technology, and utilizes the characteristic of naked-eye 3D display that does not require wearing VR, AR or polarized glasses to reduce 3D dizziness, it has the characteristics of high ease of use and high comfort; since the SSVEP-induced stimulation and its training scene interface designed by the present invention correspond the brain-controlled virtual stimulation targets and induced targets to the real space coordinates one by one, the problem of virtual-real combined three-dimensional motion interactive training is solved, so it has the ability to associate rehabilitation training tasks with real-life actions, and can enhance the real action training effect; the SSVEP stimulation form includes reciprocating three-dimensional motion, which can induce EEG perception of three-dimensional motion during the stimulation process, enhance the user's three-dimensional space immersion, and improve the SSVEP-induced EEG response characteristic intensity.

[0029] The present invention is easy to operate, and users can master it without complicated training. In addition, it has a high recognition accuracy and supports complex three-dimensional space motion training modes, and is an efficient brain-controlled rehabilitation training method. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of a system according to an embodiment of the present invention.

[0031] Figure 2Schematic diagram of a spatial motion training scenario module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention is described in detail below with reference to the accompanying drawings and embodiments. This embodiment demonstrates the system rehabilitation training for a stroke patient.

[0033] Reference Figure 1 , a brain-computer interface three-dimensional space motion rehabilitation training system, including a naked-eye 3D stimulation display module, a space motion training scene module, an EEG signal acquisition and analysis module, and a rehabilitation robot module;

[0034] Naked-eye 3D stimulation display module: Using the principle of parallax, the effect of presenting naked-eye 3D stimulation interface and rehabilitation training scene can be realized. The naked-eye 3D stimulation display module includes a naked-eye 3D display screen based on cylindrical lens technology. By adding a multi-lens screen in front of the display screen, a row of vertically arranged cylindrical lenses is used to control the direction of the left and right images; the bottom of the naked-eye 3D display screen is equipped with an eyeball capture camera to focus the right eye image on the viewer's right eye and the left eye image on the viewer's left eye. Due to the existence of parallax, when the parallax angle is appropriate, the user can observe a stereoscopic image or video;

[0035] Spatial motion training scene module: its function is to present the SSVEP visual stimulation induction training scene interface through the naked eye 3D stimulation display module. The drawing of the visual stimulation induction training scene interface is divided into two parts, visible to the left eye and visible to the right eye. The two parts have the same content, but there is parallax, so a three-dimensional sense appears after being displayed by the naked eye 3D stimulation display module; in the visual stimulation induction interface (visual stimulation induction training scene interface), there are SSVEP induced stimulation targets and spatial position feedback targets; the SSVEP induced stimulation target functions to induce SSVEP brain characteristics, and there are multiple SSVEP induced stimulation targets in the interface (visual stimulation induction training scene interface). The shapes of the SSVEP induced stimulation targets are arrow-shaped, circular, rectangular, radial, etc. Plane or three-dimensional shape; each SSVEP-induced stimulus target performs 3D reciprocating stereoscopic motion or plane reciprocating motion or flickering at different frequencies; 3D reciprocating stereoscopic motion is composed of reciprocating plane motion of the stimulus target zooming in and out and reciprocating depth motion of parallax change; the spatial position feedback target functions to display the motion position in space and moves in three-dimensional space according to the control instructions corresponding to the characteristic brain response; the plane reciprocating motion is composed of the stimulus target zooming in and out or reciprocating rotation motion or back-and-forth straight line and curve motion; flickering is the periodic change of brightness, contrast or color at a certain frequency; the adjustable parameters of the SSVEP-induced stimulus target include stimulus image texture, stimulus image change frequency, stimulus image size, and frequency change waveform;

[0036] EEG signal acquisition and analysis module: including an EEG signal amplifier, an EEG cap and a signal processing algorithm; the user generates a corresponding EEG response signal by looking at different stimuli, and then the signal is collected by the EEG signal acquisition device and saved in real time as the corresponding experimental data; the acquisition position of the EEG induced signal is located near the occipital area of ​​the brain, and the corresponding electrode leads of the brain are selected, such as O1, Oz, O2, Po3, Po4, Poz and other occipital areas; after the EEG response signal is collected, the real-time data is analyzed and processed by a canonical correlation analysis signal processing algorithm (Canonical Correlation Analysis, CCA), and after obtaining the stimulation frequency result, the corresponding control instruction is output to the rehabilitation robot module;

[0037] Rehabilitation robot module: its function is to receive the control instructions sent by the EEG signal acquisition and analysis module, drive the affected limb to move to the real three-dimensional spatial position where the spatial position feedback target is located, so as to complete the movement training; the rehabilitation robot module hardware includes an end-traction rehabilitation robot type, which pulls the upper limb movement and tracks the movement to the spatial position feedback target position in the spatial movement training scene module.

[0038] A method for utilizing a brain-computer interface three-dimensional space motion rehabilitation training system comprises the following steps:

[0039] Step 1: Wear the EEG cap in the EEG signal acquisition and analysis module, adjust the distance between the user and the naked-eye 3D display screen to about 80 cm, and face the center of the naked-eye 3D display screen;

[0040] Step 2, running the initialization program, the rehabilitation robot in the rehabilitation robot module uses the upper limb end traction machine to move the rehabilitation robot to the initial position, and fix the user's arm at the end of the rehabilitation robot;

[0041] Step 3, run the program, the space motion training scene module opens the space motion training scene, and reminds the user to be ready;

[0042] Step 4: The spatial motion training scene is presented, and the user looks at different SSVEP-induced stimulation targets to control the spatial position feedback target position;

[0043] Reference Figure 2The spatial motion training scene of this embodiment includes three SSVEP-induced stimulation targets and one spatial position feedback target; the SSVEP-induced stimulation targets are in the shape of arrows and emission lines. The upward arrow-shaped SSVEP-induced stimulation target continuously performs 3D reciprocating stereoscopic motion inside and outside the screen at 5Hz, and the corresponding control instruction is that the spatial position feedback target moves upward by 1 cm; the rightward arrow-shaped SSVEP-induced stimulation target continuously performs 3D reciprocating stereoscopic motion inside and outside the screen at 6Hz, and the corresponding control instruction is that the spatial position feedback target moves right by 1 cm; the emission line-shaped SSVEP-induced stimulation target continuously performs 3D reciprocating stereoscopic motion outside the screen at 7Hz, and the corresponding control instruction is that the spatial position feedback target moves The feedback target moves 1 cm outside the screen; the spatial position feedback target is in the shape of a lunar texture. When the spatial position feedback target reaches the left, right, upper and lower limit positions (determined by the size of the 3D naked eye display screen and the range of motion of the rehabilitation robot), the corresponding arrow target will reverse, and the control instruction direction will reverse accordingly (that is, the upward arrow-shaped SSVEP-induced stimulation target will change to a downward arrow-shaped SSVEP-induced stimulation target, and the control instruction will also change accordingly); when the spatial position feedback target reaches the inner and outer limit positions (determined by the size of the 3D naked eye display screen and the range of motion of the rehabilitation robot), the corresponding emission line target will change from 3D reciprocating stereoscopic motion outside the screen to 3D reciprocating stereoscopic motion inside the screen, and the control instruction direction will reverse accordingly;

[0044] Step 5, the EEG signal acquisition and analysis module acquires and analyzes brain signals, and analyzes them once every 0.5s. According to the canonical correlation analysis signal processing algorithm (CCA), the CCA coefficient corresponding to each frequency is calculated. If the CCA coefficient does not exceed the set threshold, it is considered that the user is in an idle state. If the CCA coefficient exceeds the set threshold, it is considered that the user is in an active training state. The frequency corresponding to the largest CCA coefficient is selected as the response recognition result to the stimulation signal. The SSVEP induced stimulation target and control instruction that the user is looking at are corresponding to the frequency, and the spatial position feedback target moves according to the control instruction;

[0045] Step 6, the rehabilitation robot starts to respond synchronously according to the brain control signal judgment result, follows the spatial position feedback target in real time, drives the user's affected limb, and thus completes the active training task; Step 7, repeat steps 4-6, conduct continuous rehabilitation training, and finally end the training task after the specified time.

Claims

1. A brain-computer interface three-dimensional space motion rehabilitation training system, characterized in that: It includes naked eye 3D stimulation display module, spatial motion training scene module, EEG signal acquisition and analysis module and rehabilitation robot module; Naked-eye 3D stimulation display module: The naked-eye 3D stimulation display module includes a naked-eye 3D display screen based on lenticular lens technology. The bottom of the naked-eye 3D display screen is equipped with an eyeball capture camera, so that the right-eye image is focused on the viewer's right eye, and the left-eye image is focused on the viewer's left eye. Due to the existence of parallax, the user can observe a stereoscopic image or video; Spatial motion training scene module: its function is to present the SSVEP visual stimulation induction training scene interface through the naked eye 3D stimulation display module. The drawing of the visual stimulation induction training scene interface is divided into two parts: the left eye visible part and the right eye visible part. The two parts have the same content, but there is parallax, so a three-dimensional sense appears after being displayed by the naked eye 3D stimulation display module; in the visual stimulation induction training scene interface, there are SSVEP induced stimulation targets and spatial position feedback targets; The target role of SSVEP-induced stimulation is to induce SSVEP brain features, and the target role of spatial position feedback is to display the movement position in space, and to move in three-dimensional space according to the control instructions corresponding to the characteristic brain response; EEG signal acquisition and analysis module: including EEG signal amplifier, EEG cap and signal processing algorithm; The user generates corresponding EEG response signals by looking at different stimuli, which are then collected by the EEG signal acquisition device and saved in real time as corresponding experimental data; the collection location of the EEG induced signal is located near the occipital area of ​​the brain, and the corresponding electrode leads of the brain are selected; after the EEG response signal is collected, the real-time data is analyzed and processed by the signal processing algorithm, and after obtaining the stimulation frequency result, the corresponding control command is output to the rehabilitation robot module; Rehabilitation robot module: its function is to receive the control instructions sent by the EEG signal acquisition and analysis module, drive the affected limb to move to the real three-dimensional spatial position where the spatial position feedback target is located, so as to complete the movement training; the rehabilitation robot module hardware includes an end-traction rehabilitation robot type, which pulls the upper limb movement and tracks the movement to the spatial position feedback target position in the spatial movement training scene module.

2. A brain-computer interface three-dimensional space motion rehabilitation training system according to claim 1, characterized in that: The naked-eye 3D display screen in the naked-eye 3D stimulation display module is achieved by adding a multi-lens screen in front of the display screen, and using a row of vertically arranged cylindrical lenses to control the projection directions of the left and right images.

3. A brain-computer interface three-dimensional space motion rehabilitation training system according to claim 1, characterized in that: The visual stimulation induction training scene interface of the spatial motion training scene module has multiple SSVEP-induced stimulation targets, and each SSVEP-induced stimulation target performs 3D reciprocating stereoscopic motion or planar reciprocating motion or flashes at different frequencies.

4. A brain-computer interface three-dimensional space motion rehabilitation training system according to claim 3, characterized in that: The SSVEP-induced stimulation target is in the shape of an arrow-shaped, circular, rectangular or radial plane or three-dimensional shape.

5. The brain-computer interface three-dimensional space motion rehabilitation training system according to claim 3, characterized in that: The 3D reciprocating stereoscopic motion is composed of the reciprocating plane motion of the stimulus target zooming in and out and the reciprocating depth motion of the parallax change, which induces the brain's electrical perception of stereoscopic motion during the stimulation process; the plane reciprocating motion is composed of the stimulus target zooming in and out or reciprocating rotational motion or back-and-forth linear or curved motion; the flicker is a periodic change in brightness, contrast or color at a certain frequency.

6. The brain-computer interface three-dimensional space motion rehabilitation training system according to claim 1, characterized in that: The adjustable parameters of the SSVEP-induced stimulation target include stimulation image texture, stimulation image change frequency, stimulation image size and frequency change waveform.

7. The brain-computer interface three-dimensional space motion rehabilitation training system according to claim 1, characterized in that: The signal processing algorithm of the EEG signal acquisition and analysis module adopts a canonical correlation analysis signal processing algorithm (Canonical Correlation Analysis, CCA).

8. A method for utilizing a brain-computer interface three-dimensional space motion rehabilitation training system according to any one of claims 1 to 7, characterized in that: The following steps are involved: Step 1: Wear the EEG cap in the EEG signal acquisition and analysis module and adjust the distance between the user and the naked-eye 3D display screen; Step 2, running the initialization program, the rehabilitation robot in the rehabilitation robot module moves to the initial position, and fixes the user's affected limb at the end of the rehabilitation robot; Step 3, run the program, and the spatial motion training scene module opens the spatial motion training scene; Step 4: The spatial motion training scene is presented, and the user looks at different SSVEP-induced stimulation targets to control the spatial position feedback target position; Step 5, the EEG signal acquisition and analysis module acquires and analyzes brain signals, and calculates the CCA coefficient corresponding to each frequency according to the canonical correlation analysis signal processing algorithm (CCA). If the CCA coefficient does not exceed the set threshold, it is considered that the user is in an idle state. If the CCA coefficient exceeds the set threshold, it is considered that the user is in an active training state. The frequency corresponding to the largest CCA coefficient is selected as the response recognition result to the stimulation signal. The SSVEP induced stimulation target and control instruction that the user is looking at corresponding to the frequency, and the spatial position feedback target moves according to the control instruction; Step 6: The rehabilitation robot starts to respond synchronously according to the judgment result of the brain control signal, follows the spatial position feedback target in real time, drives the user's affected limb, and thus completes the active training task; Step 7, repeat steps 4-6, conduct continuous rehabilitation training, and finally end the training task after a specified time.

Images