Upper limb exoskeleton robot motion assisting method based on brain-computer interaction

Abstract

The invention discloses an upper limb exoskeleton robot movement assisting method based on brain-computer interaction, which comprises the following steps of: firstly, acquiring an electroencephalogram signal through an electrode cap by adopting an instruction of a two-layer LED stimulation method, acquiring the electroencephalogram signal through analog-to-digital conversion and a signal amplifier, acquiring a human face image based on an external camera, and acquiring a sight line detection result; classifying and identifying the SSVEP information, carrying out decision fusion on the SSVEP information and a sight line detection result to obtain an electroencephalogram control instruction, determining a plane moving direction according to a first-layer instruction, setting a second-layer instruction as a moving step length, and returning to the first layer. And finally, estimating the distribution of a target area based on Kalman filtering, generating an intelligent planning control instruction when a peripheral camera detects a target object, fusing the intelligent planning control instruction with an electroencephalogram control instruction to obtain a shared control instruction to control the upper limb exoskeleton robot to move, and automatically grabbing the target object when the upper limb exoskeleton robot moves until the target object meets a certain threshold value of the target area. Therefore, the exercise assisting function is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of brain-computer interaction applications, and in particular to an upper limb exoskeleton robot motion assistance method based on brain-computer interaction. Background Art

[0002] Brain-computer interaction (BCI) has gained widespread attention and development due to its ability to directly understand and respond to human intentions by collecting and processing brain signals. This is particularly true for populations with disabilities, such as those with paralysis, those suffering from depression, or those who need enhanced interactive capabilities in specialized environments. BCI plays a crucial role in assisting and rehabilitating motor function for people with disabilities, while also providing a viable approach to alleviating the social care challenges associated with an aging population.

[0003] Due to the relatively low signal-to-noise ratio of human brain signals and its unstable characteristics, it is difficult to effectively achieve motor function assistance to complete tasks by relying solely on brain-computer interaction for control. The application scenarios are very limited, which to a certain extent restricts the development and application promotion of brain-computer interaction. At the same time, the advantages of brain-computer interaction and robotic intelligence technology are not fully utilized. Therefore, combined with robotic intelligence technology, upper limb exoskeleton robot motion assistance based on brain-computer interaction can achieve active motor rehabilitation treatment methods for paralyzed patients and patients with depression through the combination of brain-computer interaction and robotic technology, enhancing the initiative and initiative of movement. Summary of the Invention

[0004] Given the low signal-to-noise ratio and instability of current human brain signals, it is difficult to effectively assist paralyzed and depressed patients with motor function by relying solely on brain-computer interaction. There are also great problems with their initiative and enthusiasm for movement, and the usage scenarios are also very limited. This paper proposes an upper limb exoskeleton robot motion assistance method based on brain-computer interaction.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A method for assisting upper limb exoskeleton robot motion based on brain-computer interaction, comprising the following steps:

[0007] S1. Establish a two-layer LED stimulation paradigm rule, and perform preprocessing, feature extraction, and classification recognition on the EEG signals generated by the instructions of the stimulation paradigm rule;

[0008] S2. Simultaneously, the user's eye signals are recognized online in real time, and the SSVEP EEG signal classification results and gaze detection results are fused through a decision fusion method;

[0009] S3. Set the upper limb exoskeleton robot base coordinates, take the upper limb exoskeleton robot base coordinates as the origin of the xy plane coordinates, estimate the initial target area online, and determine the plane movement direction according to the first layer instructions. The second layer instructions determine the step size and variance of the user's area of interest or return to the first layer to obtain the current EEG control instructions. ;

[0010] S4. Based on the Kalman filter method, online estimation is used to obtain the distribution space of the target area and obtain the robot intelligent planning instructions. , the brain electrical control instructions and intelligent planning control instructions Weighted fusion to obtain shared control instructions ; When the peripheral camera detects the target object in the target area and meets the target area threshold, the upper limb exoskeleton robot automatically drives the user's arm to reach the target area, thereby completing the assisted grasping movement; otherwise, execute step S1.

[0011] Furthermore, in step S1, the two-layer LED stimulation paradigm rules contain 4 LED lights in each layer. The first layer LED lights are green, with a total of 4 instructions, namely forward, backward, left, and right; the second layer LED lights are white, with a total of 4 instructions, including three different numerical values of the moving step length 、 、 and an instruction to return to the first layer, where Each moving step instruction also represents a different target area variance 、 、 .

[0012] Furthermore, the data preprocessing, feature extraction and classification identification in step S1 include Butterworth filtering processing and canonical correlation analysis method to classify and identify SSVEP signals.

[0013] Furthermore, step S2 is specifically as follows:

[0014] The S21 uses an external camera to capture facial images, uses the VJ face detector for face detection, extracts the eye area through feature point detection, and uses the feature vector constructed from the eye aspect ratio to detect vertical gaze and blinking.

[0015] The S22 uses template matching to roughly locate the iris center, then uses the Snakuscule energy method to precisely locate the iris center, and uses the template matching results for initialization. The horizontal gaze direction is determined by the distance between the iris center and a reference point. Combined with the vertical gaze detection results, it can identify the user's gaze direction (up, down, left, and right).

[0016] S23. While generating SSVEP by the user gazing at the stimulus source, the user's gaze direction is obtained by tracking the eyeballs, and the correlation coefficients of the frequency direction consistent with the gaze direction are weightedly fused by a weighted averaging method.

[0017] Furthermore, step S3 is specifically as follows:

[0018] S31 sets the upper limb exoskeleton robot base coordinates, with the upper limb exoskeleton robot base coordinates as the origin of the xy plane coordinates. Assuming that the x and y directions of the user's target area satisfy Gaussian distributions and are independent of each other, during task execution, according to the rules of the two-layer control instructions, each step can obtain the step length and variance in a certain direction of the x and y axes, while still maintaining the independence of the x and y directions;

[0019] S32 estimates the user's initial target area online based on Kalman filtering. At the initial time t=0, the position of the upper limb exoskeleton robot end on the xy plane is , at this time, the probability distribution of the user's initial target area is established Gaussian distribution , is the variance of the initial target area distribution;

[0020] S33 further, when t=k steps, two layers of instructions are obtained through the fusion results of EEG signals and gaze detection, that is, the first layer of instructions obtained are the plane movement direction, and the second layer of instructions obtained determine the step size and variance of the user's area of interest or return to the first layer, thereby obtaining the current EEG control instructions ;

[0021] Furthermore, step S4 is specifically as follows:

[0022] S41 according to the step size of the instruction and variance information , based on Kalman filtering to update the probability distribution of the user's target area, first obtain the probability distribution according to the motion model of the target area for ,in is the state of the user's target area distribution at t=k-1, is the variance of the target area distribution at the previous moment, is the Gaussian noise variance inherent in the motion model;

[0023] S42. After obtaining the final instruction at time t=k, update the probability distribution of the target area for ,in , , is the weight coefficient between the model prediction value and the measured value, and ;

[0024] S43. The probability distribution of the user's target area at time t=k is the probability distribution of the updated target. for , and the coordinates of the xy plane of the upper limb rehabilitation robot end are ;

[0025] The S44's peripheral cameras detect and identify objects in real time, generating intelligent planning and control instructions. , considering the EEG state evaluation value to obtain the current moment control weight, and then, through the linear weighted fusion method, the EEG control instruction and intelligent planning control instructions Use the current control weights to fuse and obtain the shared control instructions that are finally output to the robot , thereby moving the end of the upper limb rehabilitation robot;

[0026] S45, there are multiple target objects on the xy plane, and the peripheral cameras detect the target objects on the plane through the target detection method. When the center positioning coordinates of the target objects are Falls within the target area distribution and satisfies When the user is in a state of physical discomfort, the upper limb rehabilitation robot drives the user's arm to automatically grasp the target object, thereby achieving motion assistance.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] By combining hybrid brain-computer interaction control and machine intelligent control, the user's target area is estimated online while controlling the movement of the upper limb exoskeleton robot. Compared with the traditional exoskeleton robot method, the hybrid brain-computer interaction method and the online estimation of the user's target area greatly reduce the user's usage burden. For the upper limb exoskeleton robot, through the interaction of the process, it can estimate the user's purpose, thereby achieving mutual adaptation between the two, and improving the comfort of the active movement process with a more comfortable movement assistance method. At the same time, because it can change the exoskeleton robot's movement trajectory in real time according to the user's purpose, different movement paths can be set according to needs, thereby enhancing the patient's active control ability during the movement process. It is expected to be used to improve the movement assistance effect of paralyzed patients or to increase the hormone level of patients with depression through movement intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of the upper limb exoskeleton robot motion assistance method based on a hybrid brain-computer interface;

[0030] Figure 2 It is a two-layer LED stimulation paradigm;

[0031] Figure 33 is a probability distribution effect diagram of the region of interest in the xy plane in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0033] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0034] The present invention will be described in detail below with reference to the accompanying drawings:

[0035] Reference Figure 1 A method for assisting upper limb exoskeleton robot motion based on brain-computer interaction, comprising the following steps:

[0036] S1. Set the rules of the two-layer LED stimulation paradigm (refer to Figure 2 A two-layer LED stimulation paradigm was set up. The visual stimulation device contained a total of 4 LED lights as stimuli. To facilitate the user to distinguish the two layers of control instructions, the first layer was set to green LED, with a total of 4 instructions including: forward, backward, left, and right; the second layer was set to white LED, with a total of 4 instructions including three different sizes of movement steps. 、 、 (and ) and the instruction to return to the first layer. Each moving step instruction also represents a different target area variance. 、 、 , before the total production ( , ),forward( , ),forward( , ),back( , ),back( , ),back( , ),Left( , ),Left( , ),Left( , ),right( , ),right( , ),right( , ), return to the first layer, a total of 13 instructions.

[0037] Using an EEG acquisition amplifier and an electrode cap to collect EEG signals, according to the rules of the two-layer LED stimulation paradigm, EEG signals are obtained through analog-to-digital conversion and signal amplifier. Butterworth filtering or Hilbert filtering is performed on the SSVEP information, and the common canonical correlation analysis (CCA) method of SSVEP signals is used to identify the signal. These operations can obtain the classification results of the two layers of instructions respectively, and combine the rules of the two layers of instructions to form the final instruction.

[0038] S2. Use an external camera to capture facial images, perform face detection using the VJ face detector, and extract the eye region through feature point detection. The feature vector constructed from the eye aspect ratio enables detection of vertical gaze and blinking.

[0039] Furthermore, template matching is used to coarsely locate the iris center. The Snakuscule energy method is then used to precisely locate the iris center, and the template matching results are used for initialization. The horizontal gaze direction is determined by the distance between the iris center and a reference point. Combined with the vertical gaze detection results, the user's gaze can be identified in four directions: up, down, left, and right.

[0040] While generating SSVEP by the user's gaze at the stimulus source, the gaze direction is obtained by tracking the eyeballs. The correlation coefficients of the frequency direction consistent with the gaze direction are weightedly fused through the weighted average method, which can effectively improve the recognition accuracy.

[0041] Reference Figure 3

[0042] S3. Set the upper-limb exoskeleton robot base coordinates, with the upper-limb exoskeleton base coordinates as the origin of the xy plane. Assume that the x and y directions of the user's target area follow a Gaussian distribution and are independent of each other. Within a single task (a single task refers to the process from initiation to object grasping, which requires multiple steps of command control), the two-layer control command rules are used to determine the step length and variance in each x and y direction, while maintaining independence in the x and y directions.

[0043] Based on the online estimation of the user's initial area based on Kalman filtering, at the initial time t=0, the position of the upper limb exoskeleton robot end on the xy plane is , at this time, the probability distribution of the user's initial target area is established Gaussian distribution , is the variance of the initial target region distribution.

[0044] Furthermore, when t=k steps, two layers of instructions can be obtained through the fusion results of EEG signals and gaze detection. The first layer of instructions is the plane movement direction, and the second layer of instructions determines the step size and variance of the user's target area or returns to the first layer to obtain the current EEG control instructions. ;

[0045] S4 further, according to the step length of the instruction and variance information , based on Kalman filtering to update the probability distribution of the user's target area, first obtain the probability distribution according to the motion model of the target area for ,in is the state of the user's target area distribution at the previous moment (i.e., t=k-1), is the variance of the target area distribution at the previous moment, is the Gaussian noise variance inherent in the motion model.

[0046] Furthermore, after obtaining the final instruction at time t=k, the probability distribution of the target area is updated for ,in , , is the weight coefficient between the model prediction value and the measured value, and .

[0047] Furthermore, the probability distribution of the user's target area at time t=k is the probability distribution of the updated target. for , and the coordinates of the xy plane of the upper limb rehabilitation robot end are .

[0048] The cameras installed on the periphery can detect and identify objects in real time and generate intelligent planning and control instructions. , considering the EEG state evaluation value to obtain the current moment control weight, and then, through the linear weighted fusion method, the EEG control instruction and intelligent planning control instructions Use the current control weights to fuse and obtain the shared control instructions that are finally output to the robot , thereby moving the end of the upper limb rehabilitation robot;

[0049] Assume that there are multiple objects on the xy plane. The peripheral cameras can detect the objects on the plane through the detection method. When the center of the object is located at the coordinate Falling in the target area distribution, satisfying and At 0.9, the upper limb rehabilitation robot drives the user's arm to automatically grasp objects, thereby achieving motion assistance.

[0050] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A method for assisting upper limb exoskeleton robot motion based on brain-computer interaction, characterized in that: The following steps are involved: S1. Establish a two-layer LED stimulation paradigm rule, and perform preprocessing, feature extraction, and classification recognition on the EEG signals generated by the instructions of the stimulation paradigm rule; S2. Simultaneously, the user's eye signals are recognized online in real time, and the SSVEP EEG signal classification results and gaze detection results are fused through a decision fusion method; S3. Set the upper limb exoskeleton robot base coordinates, take the upper limb exoskeleton robot base coordinates as the origin of the xy plane coordinates, estimate the initial target area online, and determine the plane movement direction according to the first layer instructions. The second layer instructions determine the step size and variance of the user's area of interest or return to the first layer to obtain the current EEG control instructions. ; S4. Based on the Kalman filter method, online estimation is used to obtain the distribution space of the target area and obtain the robot intelligent planning instructions. , the brain electrical control instructions and intelligent planning control instructions Weighted fusion to obtain shared control instructions When the peripheral camera detects the target object in the target area and meets the target area threshold, the upper limb exoskeleton robot automatically drives the user's arm to the target area, thereby completing the assisted grasping movement; Otherwise, execute step S1.

2. The upper limb exoskeleton robot motion assistance method based on brain-computer interaction according to claim 1 is characterized in that: In step S1, the two-layer LED stimulation paradigm rules contain 4 LED lights in each layer. The first layer of LED lights is green, with a total of 4 instructions, namely forward, backward, left, and right; the second layer of LED lights is white, with a total of 4 instructions, including three different numerical values of the moving step length. 、 、 and an instruction to return to the first layer, where Each moving step instruction also represents a different target area variance 、 、 .

3. The upper limb exoskeleton robot motion assistance method based on brain-computer interaction according to claim 2 is characterized in that: The data preprocessing, feature extraction and classification identification in step S1 include Butterworth filtering processing and canonical correlation analysis method to classify and identify SSVEP signals.

4. The upper limb exoskeleton robot motion assistance method based on brain-computer interaction according to claim 1 is characterized in that: Step S2 is specifically as follows: The S21 uses an external camera to capture facial images, uses the VJ face detector for face detection, extracts the eye area through feature point detection, and uses the feature vector constructed from the eye aspect ratio to detect vertical gaze and blinking. The S22 uses template matching to roughly locate the iris center, then uses the Snakuscule energy method to precisely locate the iris center, and uses the template matching results for initialization. The horizontal gaze direction is determined by the distance between the iris center and a reference point. Combined with the vertical gaze detection results, it can identify the user's gaze direction (up, down, left, and right). S23. While generating SSVEP by the user gazing at the stimulus source, the user's gaze direction is obtained by tracking the eyeballs, and the correlation coefficients of the frequency direction consistent with the gaze direction are weightedly fused by a weighted averaging method.

5. The upper limb exoskeleton robot motion assistance method based on brain-computer interaction according to claim 1 is characterized in that: Step S3 is specifically as follows: S31 sets the upper limb exoskeleton robot base coordinates, with the upper limb exoskeleton robot base coordinates as the origin of the xy plane coordinates. Assuming that the x and y directions of the user's target area satisfy Gaussian distributions and are independent of each other, during task execution, according to the rules of the two-layer control instructions, each step can obtain the step length and variance in a certain direction of the x and y axes, while still maintaining the independence of the x and y directions; S32 estimates the user's initial target area online based on Kalman filtering. At the initial time t=0, the position of the upper limb exoskeleton robot end on the xy plane is , at this time, the probability distribution of the user's initial target area is established Gaussian distribution , is the variance of the initial target area distribution; S33 further, when t=k steps, two layers of instructions are obtained through the fusion results of EEG signals and gaze detection, that is, the first layer of instructions obtained are the plane movement direction, and the second layer of instructions obtained determine the step size and variance of the user's area of interest or return to the first layer, thereby obtaining the current EEG control instructions .

6. According to the brain-computer interaction-based upper limb exoskeleton robot motion assistance method according to any one of claims 1 to 5, step S4 specifically comprises: S41 according to the step size of the instruction and variance information , based on Kalman filtering to update the probability distribution of the user's target area, first obtain the probability distribution according to the motion model of the target area for ,in is the state of the user's target area distribution at t=k-1, is the variance of the target area distribution at the previous moment, is the Gaussian noise variance inherent in the motion model; S42. After obtaining the final instruction at time t=k, update the probability distribution of the target area for ,in , , is the weight coefficient between the model prediction value and the measured value, and ; S43. The probability distribution of the user's target area at time t=k is the probability distribution of the updated target. for , and the coordinates of the xy plane of the upper limb rehabilitation robot end are ; The S44's peripheral cameras detect and identify objects in real time, generating intelligent planning and control instructions. , considering the EEG state evaluation value to obtain the current moment control weight, and then, through the linear weighted fusion method, the EEG control instruction and intelligent planning control instructions Use the current control weights to fuse and obtain the shared control instructions that are finally output to the robot , thereby moving the end of the upper limb rehabilitation robot; S45, there are multiple target objects on the xy plane, and the peripheral cameras detect the target objects on the plane through the target detection method. When the center positioning coordinates of the target objects are Falls within the target area distribution and satisfies When the user is in a state of physical discomfort, the upper limb rehabilitation robot drives the user's arm to automatically grasp the target object, thereby achieving motion assistance.

Images