A brain-computer interaction-based upper limb exoskeleton robot motion assistance method

Abstract

The application discloses a kind of upper limb exoskeleton robot motion auxiliary methods based on brain-computer interaction, the method first, the instruction of two-layer LED stimulation method is collected electroencephalogram by electrode cap, obtains electroencephalogram by analog-digital conversion, signal amplifier, simultaneously, based on the detection result of line of sight that face image is acquired by external camera acquisition, again through SSVEP information is classified and identified, and with line of sight detection result is decision fusion and obtains electroencephalogram control instruction, according to the first layer instruction determines plane moving direction, second layer instruction is set to move step and return first layer.Last, based on the distribution of target area estimated by Kalman filtering, when peripheral camera detects target object, intelligent planning control instruction is generated, and electroencephalogram control instruction is fused to obtain shared control instruction to control upper limb exoskeleton robot motion, when it moves to target object and meets target area certain threshold value, target object is automatically grabbed, so as to realize motion auxiliary function.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface (BCI) application technology, and more specifically to a method for assisting the movement of an upper limb exoskeleton robot based on BCI. Background Technology

[0002] Brain-computer interface (BCI) has received widespread attention and development due to its ability to directly understand and respond to human intentions by collecting and processing brain signals, especially for specific groups such as paralyzed individuals, patients with depression, or those who need to enhance their interactive abilities in special environments. BCI plays a crucial role in assisting and rehabilitating motor function in people with disabilities, and also provides a feasible approach to alleviating the social care challenges brought about by an aging population.

[0003] Due to the low signal-to-noise ratio and unstable characteristics of human brain signals, simple brain-computer interface (BCI) control alone is insufficient to effectively assist motor function in completing tasks, severely limiting its application scenarios and hindering its development and widespread adoption. Furthermore, it fails to fully leverage the advantages of BCI and robotics technologies. Therefore, combining BCI with robotics technology, BCI-based upper limb exoskeleton robots for motor assistance can provide active rehabilitation therapy for paralyzed and depressed patients, enhancing their initiative and proactivity in movement. Summary of the Invention

[0004] Given the low signal-to-noise ratio and instability of human brain signals, simple brain-computer interfaces are insufficient to effectively assist paralyzed or depressed patients with motor function tasks. Furthermore, they pose significant challenges in terms of initiative and motivation in movement, and their applicability is very limited. Therefore, a brain-computer interface-based upper limb exoskeleton robot motor assistance method is proposed.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A brain-computer interface-based method for assisting movement with an upper limb exoskeleton robot includes the following steps:

[0007] S1. Establish two-layer LED stimulation paradigm rules, and preprocess, extract features, and classify the EEG signals generated by the instructions of the stimulation paradigm rules.

[0008] S2. Simultaneously, the user's eye signals are identified online in real time, and the SSVEP EEG signal classification results and gaze detection results are fused through a decision fusion method.

[0009] S3. Set the base coordinates of the upper limb exoskeleton robot. Using the base coordinates of the upper limb exoskeleton robot as the origin of the xy-plane coordinates, estimate the initial target area online, and determine the planar movement direction according to the first-level instructions. The second-level instructions determine the step size and variance of the user's region of interest or return to the first level to obtain the current EEG control instructions. ;

[0010] S4. Obtain the robot's intelligent planning instructions by online estimation of the target region based on the Kalman filter method. Transmit brainwave control commands and intelligent planning and control commands Weighted fusion to obtain shared control commands When the peripheral camera detects a target object in the target area and the target area threshold is met, the upper limb exoskeleton robot automatically moves the user's arm to the target area to complete the assisted grasping motion; otherwise, step S1 is executed.

[0011] Furthermore, in step S1, each layer of the two-layer LED stimulus paradigm rule contains four LEDs. The first layer of LEDs is green, with four commands: forward, backward, left, and right. The second layer of LEDs is white, with four commands, including three different numerical values ​​for the movement step size. , , And the instruction to return to the first level, in which Each movement step size instruction also represents a different target area variance. , , .

[0012] Furthermore, the data preprocessing, feature extraction, and classification in step S1 include butterworth filtering and canonical correlation analysis for SSVEP signal classification.

[0013] Furthermore, step S2 specifically includes:

[0014] The S21 uses an external camera to capture facial images and uses the VJ face detector to perform face detection. It extracts the eye region through feature point detection and uses the feature vector constructed by the aspect ratio of the eyes to realize the detection of vertical gaze and blinking.

[0015] S22 achieves coarse iris center localization through template matching, and then uses the Snakuscule energy method for precise iris center localization, and initializes with the results of template matching; the horizontal gaze direction is determined by the distance between the iris center and the reference point, and combined with the vertical gaze detection results, the user's four gaze directions (up, down, left, and right) can be identified.

[0016] S23. While generating SSVEP through the user's gaze stimulus, the direction of the user's gaze is obtained by tracking the eyeball, and the correlation coefficients of the frequency directions consistent with the gaze direction are weighted and fused by a weighted averaging method.

[0017] Furthermore, step S3 specifically includes:

[0018] S31 sets the base coordinates of the upper limb exoskeleton robot, with the upper limb exoskeleton robot base coordinates as the origin of the xy plane coordinates. Assuming that the x and y directions of the user's target area satisfy Gaussian distributions and are independent of each other, during task execution, according to the rules of the two-layer control instructions, each step can obtain the step size and variance in a certain direction of the x and y axes, while still maintaining the independence of the x and y directions;

[0019] S32 uses Kalman filtering to estimate the user's initial target region online. At the initial time t=0, let the position of the upper limb exoskeleton robot's end effector in the xy plane be... At this point, the probability distribution of the user's initial target area is established. Gaussian distribution , The variance of the initial target region distribution;

[0020] S33 Further, when t=k steps, two layers of instructions are obtained through the fusion of EEG signals and gaze detection. The first layer of instructions is the planar movement direction, and the second layer of instructions determines the step size and variance of the user's region of interest or returns to the first layer, thereby obtaining the current EEG control instructions. ;

[0021] Furthermore, step S4 specifically includes:

[0022] S41 according to the step size of the instruction and variance information The probability distribution of the user's target region is updated based on Kalman filtering. First, the probability distribution is obtained based on the motion model of the target region. for ,in This represents the state of the user's target area distribution at time t=k-1. Let V be the variance of the target region distribution at the previous time step. The variance of Gaussian noise inherent in the motion model;

[0023] S42. After obtaining the final instruction at time t=k, update the probability distribution of the target region. for ,in , , To adjust the weighting coefficients between the model's predicted and measured values, and ;

[0024] S43. The probability distribution of the user's target region at time t=k is the probability distribution of the updated target. for And the coordinates of the end effector of the upper limb rehabilitation robot in the xy plane are ;

[0025] The S44's externally mounted cameras detect and identify objects in real time, generating intelligent planning and control commands. The system considers EEG state assessment values ​​to obtain the control weights at the current moment, and then uses a linear weighted fusion method to integrate the EEG control commands. and intelligent planning and control commands The current control weights are used for fusion to obtain the final shared control commands output to the robot. This allows the end effector of the upper limb rehabilitation robot to move;

[0026] There are multiple target objects on the S45 xy plane. The peripheral camera detects the target objects on the plane using a target detection method. When the center coordinates of the target objects are located... It falls within the target region distribution and satisfies At that time, the upper limb rehabilitation robot helps the user's arm to automatically grasp the target object, thereby achieving motion assistance.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] This invention combines hybrid brain-computer interface (BCI) control with machine intelligence control. By controlling the movement of an upper limb exoskeleton robot, it simultaneously estimates the user's target area online. Compared to traditional exoskeleton robot methods, this hybrid BCI approach, with its online target area estimation, significantly reduces the user's burden. Through interactive processes, the upper limb exoskeleton robot can estimate the user's intentions, achieving mutual adaptation and enhancing the comfort of active movement through more comfortable motion assistance. Furthermore, because it can change the exoskeleton robot's trajectory in real time according to the user's intentions, different movement paths can be set as needed, enhancing the patient's active control over movement. This approach holds promise for improving the motion assistance effect for paralyzed patients or improving hormone levels in patients with depression through exercise intervention. Attached Figure Description

[0029] Figure 1 This is a flowchart of a motion assistance method for upper limb exoskeleton robots based on a hybrid brain-computer interface;

[0030] Figure 2 It is a two-layer LED stimulation paradigm;

[0031] Figure 3This is a diagram showing the probability distribution of the region of interest in the xy plane in an embodiment of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0033] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0034] The invention will now be described in further detail with reference to the accompanying drawings:

[0035] Reference Figure 1 A brain-computer interface-based method for assisting movement with an upper limb exoskeleton robot includes the following steps:

[0036] S1. Establish the two-layer LED stimulation paradigm rules (refer to...) Figure 2 A two-layer LED stimulation paradigm is set up. The visual stimulation device contains a total of 4 LEDs as stimuli. To facilitate the user's differentiation between the two layers of control commands, the first layer is set with green LEDs, with 4 commands including: forward, backward, left, and right; the second layer is set with white LEDs, with 4 commands including three different movement step sizes. , , (and The instructions also include instructions to return to the first level, and each movement step size instruction represents a different target area variance. , , Before ( ) , ),forward( , ),forward( , ),back( , ),back( , ),back( , ),Left( , ),Left( , ),Left( , ),right( , ),right( , ),right( , There are 13 commands in total, including returning to the first level.

[0037] Using an EEG acquisition amplifier and electrode cap to acquire EEG signals, and based on the two-layer LED stimulation paradigm rules, the EEG signals are obtained through analog-to-digital conversion and signal amplification. The SSVEP information is then processed by butterworth or hilbert filtering, and the signals are identified using the common SSVEP signal canonical correlation analysis (CCA) method. This process yields classification results for the two layers of instructions, and the rules of the two layers of instructions are combined to form the final instruction.

[0038] S2. Use an external camera to capture facial images, and use the VJ face detector to perform face detection. Extract the eye region through feature point detection. The feature vector constructed based on the aspect ratio of the eyes can be used to detect vertical gaze and blinking.

[0039] Furthermore, a template matching method is used to achieve coarse iris center localization, followed by precise iris center localization using the Snakuscule energy method, and the template matching results are used for initialization. The horizontal gaze direction is determined by the distance between the iris center and a reference point, and combined with the vertical gaze detection results, the user's four gaze directions (up, down, left, and right) can be identified.

[0040] By generating SSVEP through user gaze stimuli and combining it with eye tracking to obtain the direction of the user's gaze, and by using a weighted averaging method to weight and fuse the correlation coefficients of the frequency directions consistent with the gaze direction, the accuracy of recognition can be effectively improved.

[0041] Reference Figure 3

[0042] S3. Set the base coordinates of the upper limb exoskeleton robot, using the upper limb exoskeleton robot's base coordinates as the origin of the xy-plane coordinates. Assume that the x and y directions of the user's target area satisfy Gaussian distributions and are mutually independent. In a single task (a single task refers to the process from the start to the object being grasped, which requires multiple steps of instruction control), according to the rules of the two-layer control instructions, each step can obtain the step size and variance in a certain direction of the x and y axes, while still maintaining the independence of the x and y directions.

[0043] Online estimation of the user's initial region based on Kalman filtering. At the initial time t=0, let the position of the end effector of the upper limb exoskeleton robot on the xy plane be... At this point, the probability distribution of the user's initial target region is established. Gaussian distribution , This represents the variance of the initial target region distribution.

[0044] Furthermore, at t=k steps, two layers of instructions can be obtained through the fusion of EEG signals and gaze detection. The first layer of instructions is the planar movement direction, and the second layer of instructions determines the step size and variance of the user's target area or returns to the first layer to obtain the current EEG control instructions. ;

[0045] S4 further, based on the step size of the instruction and variance information The probability distribution of the user's target region is updated based on Kalman filtering. First, the probability distribution is obtained based on the motion model of the target region. for ,in This represents the state of the user's target region distribution at the previous time step (i.e., time t=k-1). Let V be the variance of the target region distribution at the previous time step. This represents the Gaussian noise variance inherent in the motion model.

[0046] Furthermore, after obtaining the final instruction at time t=k, the probability distribution of the target region is updated. for ,in , , To adjust the weighting coefficients between the model's predicted and measured values, and .

[0047] Furthermore, the probability distribution of the user's target region at time t=k is the probability distribution of the updated target. for And the coordinates of the end effector of the upper limb rehabilitation robot in the xy plane are .

[0048] The peripheral cameras detect and identify objects in real time, generating intelligent planning and control commands. The system considers EEG state assessment values ​​to obtain the control weights at the current moment, and then uses a linear weighted fusion method to integrate the EEG control commands. and intelligent planning and control commands The current control weights are used for fusion to obtain the final shared control commands output to the robot. This allows the end effector of the upper limb rehabilitation robot to move;

[0049] Assuming there are multiple objects in the xy plane, an external camera can detect these objects using a detection method. The coordinates of the object's center are then determined. It falls within the target area distribution and satisfies and At 0.9, the upper limb rehabilitation robot automatically guides the user's arm to grasp objects, thereby achieving motion assistance.

[0050] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for assisting movement with an upper limb exoskeleton robot based on brain-computer interaction, characterized in that, Includes the following steps: S1. Establish two-layer LED stimulation paradigm rules, and preprocess, extract features, and classify the EEG signals generated by the instructions of the stimulation paradigm rules. S2. Simultaneously, the user's eye signals are identified online in real time, and the SSVEP EEG signal classification results and gaze detection results are fused through a decision fusion method. S3. Set the base coordinates of the upper limb exoskeleton robot. Using the base coordinates of the upper limb exoskeleton robot as the origin of the xy-plane coordinates, estimate the initial target area online, and determine the planar movement direction according to the first-level instructions. The second-level instructions determine the step size and variance of the user's region of interest or return to the first level to obtain the current EEG control instructions. ; S4. Obtain the robot's intelligent planning instructions by online estimation of the target region based on the Kalman filter method. Transmit brainwave control commands and intelligent planning and control commands Weighted fusion to obtain shared control commands When the peripheral camera detects a target object in the target area and the target area threshold is met, the upper limb exoskeleton robot automatically moves the user's arm to the target area to complete the assisted grasping motion. Otherwise, proceed to step S1.

2. The method for motion assistance of an upper limb exoskeleton robot based on brain-computer interface according to claim 1, characterized in that, In step S1, the two-layer LED stimulus paradigm rule contains four LEDs in each layer. The first layer of LEDs is green, with four commands: forward, backward, left, and right. The second layer of LEDs is white, with four commands, including three different numerical values ​​for the movement step size. , , And the instruction to return to the first level, in which Each movement step size instruction also represents a different target area variance. , , .

3. The method for motion assistance of an upper limb exoskeleton robot based on brain-computer interface according to claim 2, characterized in that, Step S1 involves data preprocessing, feature extraction, and classification, including Butterworth filtering and canonical correlation analysis for SSVEP signal classification.

4. The method for motion assistance of an upper limb exoskeleton robot based on brain-computer interface according to claim 1, characterized in that, Step S2 is as follows: The S21 uses an external camera to capture facial images and uses the VJ face detector to perform face detection. It extracts the eye region through feature point detection and uses the feature vector constructed by the aspect ratio of the eyes to realize the detection of vertical gaze and blinking. S22 achieves coarse iris center localization through template matching, and then uses the Snakuscule energy method for precise iris center localization, and initializes with the results of template matching; the horizontal gaze direction is determined by the distance between the iris center and the reference point, and combined with the vertical gaze detection results, the user's four gaze directions (up, down, left, and right) can be identified. S23. While generating SSVEP through the user's gaze stimulus, the direction of the user's gaze is obtained by tracking the eyeball, and the correlation coefficients of the frequency directions consistent with the gaze direction are weighted and fused by a weighted averaging method.

5. The method for motion assistance of an upper limb exoskeleton robot based on brain-computer interface according to claim 1, characterized in that, Step S3 is as follows: S31 sets the base coordinates of the upper limb exoskeleton robot, with the base coordinates of the upper limb exoskeleton robot as the origin of the xy plane coordinates; assuming that the x and y directions of the user's target area satisfy Gaussian distribution and are independent of each other, during task execution, according to the rules of the two-layer control instructions, each step can obtain the step size and variance in a certain direction of the x and y axes, while still maintaining the independence of the x and y directions. S32 uses Kalman filtering to estimate the user's initial target region online. At the initial time t=0, let the position of the upper limb exoskeleton robot's end effector in the xy plane be... At this point, the probability distribution of the user's initial target area is established. Gaussian distribution , The variance of the initial target region distribution; S33 Further, when t=k steps, two layers of instructions are obtained through the fusion of EEG signals and gaze detection. The first layer of instructions is the planar movement direction, and the second layer of instructions determines the step size and variance of the user's region of interest or returns to the first layer, thereby obtaining the current EEG control instructions. .

6. The method for motion assistance of an upper limb exoskeleton robot based on brain-computer interaction according to any one of claims 1-5, wherein step S4 specifically comprises: S41 according to the step size of the instruction and variance information The probability distribution of the user's target region is updated based on Kalman filtering. First, the probability distribution is obtained based on the motion model of the target region. for ,in This represents the state of the user's target area distribution at time t=k-1. Let V be the variance of the target region distribution at the previous time step. The variance of Gaussian noise inherent in the motion model; S42. After obtaining the final instruction at time t=k, update the probability distribution of the target region. for ,in , , To adjust the weighting coefficients between the model's predicted and measured values, and ; S43. The probability distribution of the user's target region at time t=k is the probability distribution of the updated target. for And the coordinates of the end effector of the upper limb rehabilitation robot in the xy plane are ; The S44's externally mounted cameras detect and identify objects in real time, generating intelligent planning and control commands. The system considers EEG state assessment values ​​to obtain the control weights at the current moment, and then uses a linear weighted fusion method to integrate the EEG control commands. and intelligent planning and control commands The current control weights are used for fusion to obtain the final shared control commands output to the robot. This allows the end effector of the upper limb rehabilitation robot to move; There are multiple target objects on the S45 xy plane. The peripheral camera detects the target objects on the plane using a target detection method. When the center coordinates of the target objects are located... It falls within the target region distribution and satisfies At that time, the upper limb rehabilitation robot helps the user's arm to automatically grasp the target object, thereby achieving motion assistance.

Images