Hand rehabilitation training method and device based on brain-computer interface and AR glasses

Abstract

The invention discloses a hand rehabilitation training device and method based on a brain-computer interface and AR glasses. The device comprises the AR glasses, brain-computer interface equipment, a hand motion sensor, a data processing module and a training feedback module. The AR glasses are used for displaying a virtual training scene and task guidance information for a user; the brain-computer interface equipment is used for acquiring an electroencephalogram signal set of a user during hand rehabilitation training; the hand motion sensor is used for monitoring the motion state of the hand of the user in real time to obtain a motion track information set of the hand of the user; the data processing module is used for analyzing and processing the electroencephalogram signal set and the motion trail information set to obtain training evaluation result information; and the training feedback module is used for generating corresponding display result information according to the training evaluation result value, and sending the display result information to the AR glasses for display.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation training, and in particular to a hand rehabilitation training method and device based on augmented reality (AR) glasses and a brain-computer interface. Background Art

[0002] Hand dysfunction is a common sequelae of many neurological diseases and trauma, such as stroke, brain injury, hand trauma, etc. Traditional hand rehabilitation training methods usually rely on manual assistance and guidance from physical therapists, and have problems such as low training efficiency, boring training process, and difficulty in quantifying and evaluating training effects.

[0003] With the development of science and technology, some new technologies such as virtual reality (VR), augmented reality (AR) and brain-computer interface (BCI) have been applied to the field of rehabilitation training, bringing new ideas and methods to hand rehabilitation training. However, the current related technologies still have some limitations. For example, VR devices may cause dizziness in users and lack interaction with the real environment; the application of brain-computer interface technology alone in hand rehabilitation training is not mature enough, and it is difficult to accurately capture and interpret complex hand movement intentions.

[0004] Therefore, it is necessary to develop a hand rehabilitation training method and device that combines AR glasses and brain-computer interface to improve the efficiency and fun of rehabilitation training. Summary of the invention

[0005] The present invention mainly solves the problems existing in existing hand rehabilitation training methods, such as low training efficiency, lack of interest, and difficulty in accurately evaluating training effects. The present invention discloses a hand rehabilitation training method and device based on a brain-computer interface and AR glasses.

[0006] In a first aspect of an embodiment of the present invention, a hand rehabilitation training device based on a brain-computer interface and AR glasses is disclosed, comprising: AR glasses, a brain-computer interface device, a hand motion sensor, a data processing module, and a training feedback module;

[0007] The AR glasses are used to display virtual training scenes and task guidance information for users;

[0008] The brain-computer interface device is used to collect a set of EEG signals of the user when performing hand rehabilitation training;

[0009] The hand motion sensor is used to monitor the motion state of the user's hand in real time to obtain a set of motion trajectory information of the user's hand; the set of motion trajectory information includes the motion trajectory information of five fingers and the motion trajectory information of the palm;

[0010] The data processing module is connected to the brain-computer interface device and the hand motion sensor respectively, and is used to analyze and process the EEG signal set and the motion trajectory information set to obtain training evaluation result information;

[0011] The training feedback module is connected to the AR glasses and the data processing module respectively, and is used to generate corresponding display result information according to the training evaluation result value, and send the display result information to the AR glasses for display.

[0012] The hand motion sensor comprises: a glove, an acceleration sensor arranged at the fingertips of the glove, an acceleration sensor arranged at the middle of the palm of the glove, and a communication submodule;

[0013] The acceleration sensors arranged at the fingertips of the gloves are used to collect the motion trajectory information of the five fingers;

[0014] The acceleration sensor disposed in the middle of the palm of the glove is used to collect the palm motion trajectory information;

[0015] The communication submodule is used to send the motion track information of the five fingers and the motion track information of the palm to the data processing module.

[0016] The data processing module analyzes and processes the EEG signal set and the motion trajectory information set to obtain training evaluation result information, including:

[0017] Performing electroencephalogram (EEG) evaluation on the EEG signal set to obtain an EEG evaluation value;

[0018] Performing motion evaluation on the motion trajectory information set to obtain a motion evaluation value and a maximum deviation;

[0019] The training evaluation result information is constructed using the EEG evaluation value, the movement evaluation value and the maximum deviation.

[0020] In a second aspect of an embodiment of the present invention, a hand rehabilitation training method based on a brain-computer interface and AR glasses is disclosed, which is implemented using the hand rehabilitation training device based on the brain-computer interface and AR glasses, and includes:

[0021] S1, using the AR glasses to display virtual training scenes and task guidance information for the user;

[0022] S2, using the brain-computer interface device to collect the EEG signal set of the user when performing hand rehabilitation training;

[0023] S3, using the hand motion sensor to monitor the motion state of the user's hand in real time to obtain a set of motion trajectory information of the user's hand; the set of motion trajectory information includes motion trajectory information of five fingers and palm motion trajectory information;

[0024] S4, using the data processing module to analyze and process the EEG signal set and the motion trajectory information set to obtain training evaluation result information;

[0025] S5, using the training feedback module to generate corresponding display result information according to the training evaluation result value, and sending the display result information to the AR glasses for display.

[0026] The analyzing and processing the EEG signal set and the motion trajectory information set to obtain training evaluation result information includes:

[0027] S41, performing electroencephalogram (EEG) evaluation on the EEG signal set to obtain an EEG evaluation value; the EEG signal set includes α wave signals, β wave signals, θ wave signals, and δ wave signals;

[0028] S42, performing motion evaluation on the motion trajectory information set to obtain a motion evaluation value and a maximum deviation;

[0029] S43, constructing training evaluation result information using the EEG evaluation value, the movement evaluation value and the maximum deviation.

[0030] The performing EEG evaluation on the EEG signal set to obtain an EEG evaluation value comprises:

[0031] S411, obtaining an α wave standard signal, a β wave standard signal, a θ wave standard signal, and a δ wave standard signal;

[0032] S412, subtracting each type of signal in the EEG signal set from the corresponding standard signal to obtain a corresponding difference signal;

[0033] S413, performing statistical processing on each type of difference signal to obtain a corresponding statistical value set; the statistical value set includes variance, mean, median, harmonic mean, characteristic variance value, second-order origin moment and third-order central moment; the harmonic mean is the mean of all harmonic frequencies of the FFT sequence of the difference signal; the characteristic variance value is the square root of the variance of the first-order derivative of the FFT sequence of the difference signal divided by the variance of the difference signal;

[0034] S414, constructing a difference statistical matrix using the statistical value set of each type of difference signal; the row vector of the difference statistical matrix is ​​a statistical value set of a type of difference signal;

[0035] S415, performing deviation characteristic calculation on the difference statistical matrix to obtain deviation characteristic value;

[0036] S416, determining the deviation characteristic value as an EEG evaluation value.

[0037] The expression for calculating the deviation characteristic is:

[0038]

[0039] Where, ∈ is the trace value of the difference statistics matrix, P1 and P2 are the first difference value and the second difference value respectively, σ ij is the element of the i-th row and j-th column of the difference statistics matrix, is the mean of the ith row of the difference statistics matrix, is the mean of all elements of the difference statistics matrix, m and n are the row dimension and column dimension of the difference statistics matrix respectively, and P3 is the deviation eigenvalue.

[0040] The performing motion evaluation on the motion trajectory information set to obtain a motion evaluation value and a maximum deviation includes:

[0041] S421, obtaining a standard motion trajectory information set; the standard motion trajectory information set includes a standard motion trajectory sequence of five fingers and a palm;

[0042] S422, representing the motion trajectory information set as a motion matrix; the first to sixth row vectors of the motion matrix are the motion trajectory information of five fingers and the motion trajectory information of the palm, respectively;

[0043] S423, representing the standard motion trajectory information set as a standard matrix; the first to sixth row vectors of the standard matrix are the standard motion trajectory sequence of the five fingers and the palm motion trajectory sequence respectively;

[0044] S424, performing fusion evaluation calculation on the motion matrix and the standard matrix to obtain a motion evaluation value;

[0045] S425, subtracting the motion matrix from the standard matrix to obtain a difference matrix;

[0046] S426, extracting characteristic rows from the difference matrix to obtain a row characteristic value set; the row characteristic value set includes a row characteristic value of each row;

[0047] S427, determining the row number of the row corresponding to the largest row eigenvalue as the maximum deviation.

[0048] The expression for extracting the feature line is:

[0049]

[0050] in, is the i-th element of the first difference vector, is the i-th element of the second difference vector, g ij is the element of the i-th row and j-th column of the difference matrix, g j is the mean of the jth column of the difference matrix, g ii is the element in the ith row and ith column of the difference matrix, N is the column dimension of the difference matrix, k i is the row eigenvalue of the i-th row.

[0051] The generating corresponding display result information according to the training evaluation result value includes:

[0052] The display result information includes task display information and prompt action display information; the training feedback module stores a value range set and task display information corresponding to each value range; the value range set includes several value ranges;

[0053] S51, performing weighted summation on the EEG evaluation value and the movement evaluation value to obtain a comprehensive evaluation value;

[0054] S52, determining a value range corresponding to the comprehensive evaluation value according to a preset value range set, and determining task display information corresponding to the value range;

[0055] S53, determining the user hand information corresponding to the maximum deviation, and displaying the information as a prompt action.

[0056] The beneficial effects of the present invention are:

[0057] The virtual training scenes and tasks provided by the AR glasses of the present invention are highly interesting and attractive, and can stimulate the user's enthusiasm for active participation, thereby improving the efficiency of training. The data processing module can perform quantitative analysis on the training data and generate a detailed training evaluation report, providing a scientific basis for rehabilitation therapists to evaluate the training effect and adjust the training plan.

[0058] The present invention provides users with virtual training scenes and task guidance information through AR glasses, allowing users to perform rehabilitation training in an immersive environment. This immersive experience can effectively improve users' training enthusiasm and concentration and enhance the training effect. At the same time, the virtual scene can be dynamically adjusted according to the user's rehabilitation progress to meet the training needs of different stages.

[0059] The brain-computer interface device can collect the user's EEG signals in real time during rehabilitation training, and obtain the EEG evaluation value by analyzing and processing the EEG signals. This EEG evaluation method can directly reflect the user's brain activity state during training and provide important physiological feedback for rehabilitation training. Combined with the motion trajectory information collected by the hand motion sensor, the data processing module can comprehensively evaluate the user's rehabilitation status and generate accurate training evaluation result information. The hand motion sensor adopts a glove-type design, and accelerometers are set at the fingertips and the middle of the palm, which can monitor the motion trajectory of the five fingers and the palm with high precision. This high-precision motion monitoring method can capture the subtle movements of the user's hand and provide accurate data support for motion evaluation. By comparing and analyzing the standard motion trajectory information, the motion evaluation value and the maximum deviation can be obtained, thereby providing quantitative evaluation indicators for the user's hand motor function recovery. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 1 is a flow chart for implementing the method of the present invention;

[0061] Figure 2 It is a schematic diagram of the composition of the device of the present invention. DETAILED DESCRIPTION

[0062] In order to better understand the content of the present invention, an embodiment is given here.

[0063] Figure 1 It is a flow chart for implementing the method of the present invention. Figure 2 It is a schematic diagram of the composition of the device of the present invention.

[0064] In a first aspect of an embodiment of the present invention, a hand rehabilitation training device based on a brain-computer interface and AR glasses is disclosed, comprising: AR glasses, a brain-computer interface device, a hand motion sensor, a data processing module, and a training feedback module;

[0065] The AR glasses are used to display virtual training scenes and task guidance information for users.

[0066] The brain-computer interface device is used to collect a set of EEG signals of a user when performing hand rehabilitation training; obtain a motor imagery signal of the user; the motor imagery signal of the user is obtained by determining the brain area that emits the EEG signal, and determining the corresponding motor imagery signal according to the brain area; the types of the motor imagery signal include left upper limb upward movement, left upper limb downward movement, left upper limb forward movement, left upper limb backward movement, right upper limb upward movement, right upper limb downward movement, right upper limb forward movement, and right upper limb backward movement;

[0067] The hand motion sensor is used to monitor the motion state of the user's hand in real time to obtain a set of motion trajectory information of the user's hand; the set of motion trajectory information includes the motion trajectory information of five fingers and the motion trajectory information of the palm;

[0068] The data processing module is connected to the brain-computer interface device and the hand motion sensor respectively, and is used to analyze and process the EEG signal set and the motion trajectory information set to obtain training evaluation result information;

[0069] The training feedback module is connected to the AR glasses and the data processing module respectively, and is used to generate corresponding display result information according to the training evaluation result value, and send the display result information to the AR glasses for display.

[0070] The brain-computer interface device can be implemented using BLueBrain-Hands from Xi'an Blue Brain.

[0071] The hand rehabilitation training device based on the brain-computer interface and AR glasses also includes a motion training auxiliary device, which includes a driving motor arranged on the fingers of the glove, and the driving motor drives the fingers of the glove to perform corresponding movements according to the user's motion imagination signal.

[0072] The training feedback module of the present invention can generate corresponding display result information according to the training evaluation results, and provide real-time feedback to the user through AR glasses. This real-time feedback mechanism enables users to understand their training effects in a timely manner, adjust their training strategies, and further improve the pertinence and effectiveness of rehabilitation training.

[0073] The hand motion sensor comprises: a glove, an acceleration sensor arranged at the fingertips of the glove, an acceleration sensor arranged at the middle of the palm of the glove, and a communication submodule;

[0074] The acceleration sensors arranged at the fingertips of the gloves are used to collect the motion trajectory information of the five fingers;

[0075] The acceleration sensor disposed in the middle of the palm of the glove is used to collect the palm motion trajectory information;

[0076] The communication submodule is used to send the motion track information of the five fingers and the motion track information of the palm to the data processing module;

[0077] When the user performs rehabilitation training using the hand rehabilitation training device based on the brain-computer interface and AR glasses, the user wears gloves and obtains the motion trajectory information of the five fingers and the palm through the acceleration sensor.

[0078] The hand motion sensor may be implemented by a gesture sensor;

[0079] The data processing module analyzes and processes the EEG signal set and the motion trajectory information set to obtain training evaluation result information, including:

[0080] Performing electroencephalogram (EEG) evaluation on the EEG signal set to obtain an EEG evaluation value;

[0081] Performing motion evaluation on the motion trajectory information set to obtain a motion evaluation value and a maximum deviation;

[0082] The training evaluation result information is constructed using the EEG evaluation value, the movement evaluation value and the maximum deviation.

[0083] The performing EEG evaluation on the EEG signal set to obtain an EEG evaluation value comprises:

[0084] The EEG signal set includes α wave signals, β wave signals, θ wave signals and δ wave signals;

[0085] Acquiring an alpha wave standard signal, a beta wave standard signal, a theta wave standard signal, and a delta wave standard signal;

[0086] Subtracting each type of signal in the EEG signal set from the corresponding standard signal to obtain a corresponding difference signal;

[0087] For each type of difference signal, statistical processing is performed separately to obtain a corresponding set of statistical values; the set of statistical values ​​includes variance, mean, median, harmonic mean, characteristic variance value, second-order origin moment and third-order central moment; the harmonic mean is the mean of all harmonic frequencies of the FFT sequence of the difference signal; the characteristic variance value is the square root of the variance of the first-order derivative of the FFT sequence of the difference signal divided by the variance of the difference signal;

[0088] Using the statistical value set of each type of difference signal, a difference statistical matrix is ​​constructed; the row vector of the difference statistical matrix is ​​a statistical value set of a type of difference signal;

[0089] Calculating the deviation characteristics of the difference statistical matrix to obtain deviation eigenvalues;

[0090] Determine the deviation characteristic value as an EEG evaluation value;

[0091] The expression for calculating the deviation characteristic is:

[0092]

[0093] Where, ∈ is the trace value of the difference statistics matrix, P1 and P2 are the first difference value and the second difference value respectively, σ ij is the element of the i-th row and j-th column of the difference statistics matrix, is the mean of the ith row of the difference statistics matrix, is the mean of all elements of the difference statistics matrix, m and n are the row dimension and column dimension of the difference statistics matrix respectively, and P3 is the deviation eigenvalue.

[0094] The difference signals of the first to sixth categories are difference signals of α wave signals, β wave signals, θ wave signals and δ wave signals respectively;

[0095] The performing motion evaluation on the motion trajectory information set to obtain a motion evaluation value and a maximum deviation includes:

[0096] Acquire a standard motion trajectory information set; the standard motion trajectory information set includes a standard motion trajectory sequence of five fingers and a palm;

[0097] The motion trajectory information set is represented as a motion matrix; the first to sixth row vectors of the motion matrix are the motion trajectory information of the five fingers and the motion trajectory information of the palm, respectively;

[0098] The standard motion trajectory information set is represented as a standard matrix; the first to sixth row vectors of the standard matrix are the standard motion trajectory sequence of the five fingers and the palm motion trajectory sequence respectively;

[0099] Performing fusion evaluation calculation on the motion matrix and the standard matrix to obtain a motion evaluation value;

[0100] Subtracting the motion matrix from the standard matrix to obtain a difference matrix;

[0101] Extracting characteristic rows from the difference matrix to obtain a set of row eigenvalues;

[0102] The expression for extracting the feature line is:

[0103]

[0104] in, is the i-th element of the first difference vector, is the i-th element of the second difference vector, g ij is the element of the i-th row and j-th column of the difference matrix, g j is the mean of the jth column of the difference matrix, g ii is the element in the ith row and ith column of the difference matrix, N is the column dimension of the difference matrix, k i is the row eigenvalue of the i-th row; using the row eigenvalues ​​of all rows, a row eigenvalue set is constructed;

[0105] Determine the row number of the row eigenvalue with the maximum value, which is the maximum deviation;

[0106] The fusion evaluation calculation includes:

[0107] P=Y 1 / 2 TY -1 / 2 ,

[0108] Among them, Y is the transformation matrix, T is the cross-correlation matrix, and P is the motion evaluation matrix;

[0109] Performing eigenvalue decomposition processing on the motion evaluation matrix to obtain eigenvectors and eigenvalues;

[0110] The median value of the eigenvector corresponding to the maximum eigenvalue is determined as the motion evaluation value.

[0111] The training feedback module generates corresponding display result information according to the training evaluation result value, including:

[0112] The display result information includes task display information and prompt action display information; the training feedback module stores a value range set and task display information corresponding to each value range; the value range set includes several value ranges;

[0113] Performing weighted summation on the EEG evaluation value and the movement evaluation value to obtain a comprehensive evaluation value;

[0114] Determine the value range corresponding to the comprehensive evaluation value according to a preset value range set, and determine the task display information corresponding to the value range;

[0115] Determine the user hand information corresponding to the maximum deviation, and display the information for the prompt action; when the maximum deviation is 1 to 6, the corresponding user hand information is the thumb, index finger, middle finger, ring finger, little finger and palm respectively;

[0116] The AR glasses store an action prompt video corresponding to each user's hand information, and display the corresponding action prompt video according to the received prompt action display information;

[0117] The weight values ​​for weighted summation of the EEG evaluation value and the movement evaluation value may be 0.2 and 0.4 respectively.

[0118] The AR glasses store a plurality of task guidance information corresponding to the task display information; the task guidance information is used to guide users with different training effects to complete hand rehabilitation training;

[0119] Each task displays information, with corresponding task guidance information

[0120] The task display information is determined according to the comprehensive evaluation value; the larger the comprehensive evaluation value, the worse the completion effect of the user's hand rehabilitation training, so the task guidance information corresponding to the task display information is more detailed, which is used to guide the user to complete the hand rehabilitation training; the smaller the comprehensive evaluation value, the better the completion effect of the user's hand rehabilitation training, so the task guidance information corresponding to the task display information is simpler;

[0121] The present invention uses advanced signal processing algorithms and statistical analysis methods in the process of EEG assessment and motion assessment. For example, by constructing a difference statistical matrix and calculating the deviation eigenvalue, the abnormality of the EEG signal can be accurately assessed; by matrixing the motion trajectory information with the standard motion trajectory, the motion deviation can be quickly calculated. These innovative processing methods not only improve the accuracy of the assessment, but also provide a scientific basis for the personalized adjustment of rehabilitation training.

[0122] The second aspect of the present invention is to disclose a hand rehabilitation training method based on a brain-computer interface and AR glasses, which is implemented by using the hand rehabilitation training device based on the brain-computer interface and AR glasses, including:

[0123] S1, using the AR glasses to display virtual training scenes and task guidance information for the user;

[0124] S2, using a brain-computer interface device to collect a set of EEG signals of the user when performing hand rehabilitation training; obtaining a motor imagery signal of the user; the obtaining of the motor imagery signal of the user is to determine the brain area that sends the EEG signal, and determine the corresponding motor imagery signal according to the brain area; the types of the motor imagery signal include left upper limb upward movement, left upper limb downward movement, left upper limb forward movement, left upper limb backward movement, right upper limb upward movement, right upper limb downward movement, right upper limb forward movement, and right upper limb backward movement;

[0125] S3, using the hand motion sensor to monitor the motion state of the user's hand in real time to obtain a set of motion trajectory information of the user's hand; the set of motion trajectory information includes motion trajectory information of five fingers and palm motion trajectory information;

[0126] S4, using the data processing module to analyze and process the EEG signal set and the motion trajectory information set to obtain training evaluation result information;

[0127] S5, using the training feedback module to generate corresponding display result information according to the training evaluation result value, and sending the display result information to the AR glasses for display.

[0128] The analyzing and processing the EEG signal set and the motion trajectory information set to obtain training evaluation result information includes:

[0129] S41, performing electroencephalogram (EEG) evaluation on the EEG signal set to obtain an EEG evaluation value; the EEG signal set includes α wave signals, β wave signals, θ wave signals, and δ wave signals;

[0130] S42, performing motion evaluation on the motion trajectory information set to obtain a motion evaluation value and a maximum deviation;

[0131] S43, constructing training evaluation result information using the EEG evaluation value, the movement evaluation value and the maximum deviation.

[0132] The performing EEG evaluation on the EEG signal set to obtain an EEG evaluation value comprises:

[0133] S411, obtaining an α wave standard signal, a β wave standard signal, a θ wave standard signal, and a δ wave standard signal;

[0134] S412, subtracting each type of signal in the EEG signal set from the corresponding standard signal to obtain a corresponding difference signal;

[0135] S413, performing statistical processing on each type of difference signal to obtain a corresponding statistical value set; the statistical value set includes variance, mean, median, harmonic mean, characteristic variance value, second-order origin moment and third-order central moment; the harmonic mean is the mean of all harmonic frequencies of the FFT sequence of the difference signal; the characteristic variance value is the square root of the variance of the first-order derivative of the FFT sequence of the difference signal divided by the variance of the difference signal;

[0136] S414, constructing a difference statistical matrix using the statistical value set of each type of difference signal; the row vector of the difference statistical matrix is ​​a statistical value set of a type of difference signal;

[0137] S415, performing deviation characteristic calculation on the difference statistical matrix to obtain deviation characteristic value;

[0138] S416, determining the deviation characteristic value as an EEG evaluation value;

[0139] The expression for calculating the deviation characteristic is:

[0140]

[0141] Where, ∈ is the trace value of the difference statistics matrix, P1 and P2 are the first difference value and the second difference value respectively, σ ij is the element of the i-th row and j-th column of the difference statistics matrix, is the mean of the ith row of the difference statistics matrix, is the mean of all elements of the difference statistics matrix, m and n are the row dimension and column dimension of the difference statistics matrix respectively, and P3 is the deviation eigenvalue.

[0142] The difference signals of the first to sixth categories are difference signals of α wave signals, β wave signals, θ wave signals and δ wave signals respectively;

[0143] The performing motion evaluation on the motion trajectory information set to obtain a motion evaluation value and a maximum deviation includes:

[0144] S421, obtaining a standard motion trajectory information set; the standard motion trajectory information set includes a standard motion trajectory sequence of five fingers and a palm;

[0145] S422, representing the motion trajectory information set as a motion matrix; the first to sixth row vectors of the motion matrix are the motion trajectory information of five fingers and the motion trajectory information of the palm, respectively;

[0146] S423, representing the standard motion trajectory information set as a standard matrix; the first to sixth row vectors of the standard matrix are the standard motion trajectory sequence of the five fingers and the palm motion trajectory sequence respectively;

[0147] S424, performing fusion evaluation calculation on the motion matrix and the standard matrix to obtain a motion evaluation value;

[0148] S425, subtracting the motion matrix from the standard matrix to obtain a difference matrix;

[0149] S426, extracting characteristic rows from the difference matrix to obtain a row characteristic value set; the row characteristic value set includes a row characteristic value of each row;

[0150] The expression for extracting the feature line is:

[0151]

[0152] in, is the i-th element of the first difference vector, is the i-th element of the second difference vector, g ij is the element of the i-th row and j-th column of the difference matrix, g j is the mean of the jth column of the difference matrix, g ii is the element in the ith row and ith column of the difference matrix, N is the column dimension of the difference matrix, k i is the row eigenvalue of the i-th row;

[0153] S427, determine the row number of the row corresponding to the largest row eigenvalue, which is the maximum deviation

[0154] The fusion evaluation calculation includes:

[0155] P=Y 1 / 2 TY -1 / 2 ,

[0156] Among them, Y is the transformation matrix, T is the cross-correlation matrix, and P is the motion evaluation matrix;

[0157] Performing eigenvalue decomposition processing on the motion evaluation matrix to obtain eigenvectors and eigenvalues;

[0158] The median value of the eigenvector corresponding to the maximum eigenvalue is determined as the motion evaluation value.

[0159] The generating corresponding display result information according to the training evaluation result value includes:

[0160] The display result information includes task display information and prompt action display information; the training feedback module stores a value range set and task display information corresponding to each value range; the value range set includes several value ranges;

[0161] S51, performing weighted summation on the EEG evaluation value and the movement evaluation value to obtain a comprehensive evaluation value;

[0162] S52, determining a value range corresponding to the comprehensive evaluation value according to a preset value range set, and determining task display information corresponding to the value range;

[0163] S53, determining the user hand information corresponding to the maximum deviation, and displaying the prompt action information; when the maximum deviation is 1 to 6, the corresponding user hand information is the thumb, index finger, middle finger, ring finger, little finger and palm respectively.

[0164] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A hand rehabilitation training device based on brain-computer interface and AR glasses, characterized in that: include: AR glasses, brain-computer interface devices, hand motion sensors, data processing modules, and training feedback modules; The AR glasses are used to display virtual training scenes and task guidance information for users; The brain-computer interface device is used to collect a set of EEG signals of the user when performing hand rehabilitation training; The hand motion sensor is used to monitor the motion state of the user's hand in real time to obtain a set of motion trajectory information of the user's hand; the set of motion trajectory information includes the motion trajectory information of five fingers and the motion trajectory information of the palm; The data processing module is connected to the brain-computer interface device and the hand motion sensor respectively, and is used to analyze and process the EEG signal set and the motion trajectory information set to obtain training evaluation result information; The training feedback module is connected to the AR glasses and the data processing module respectively, and is used to generate corresponding display result information according to the training evaluation result value, and send the display result information to the AR glasses for display.

2. The hand rehabilitation training device based on brain-computer interface and AR glasses as claimed in claim 1, characterized in that: The hand motion sensor comprises: a glove, an acceleration sensor arranged at the fingertips of the glove, an acceleration sensor arranged at the middle of the palm of the glove, and a communication submodule; The acceleration sensors arranged at the fingertips of the gloves are used to collect the motion trajectory information of the five fingers; The acceleration sensor disposed in the middle of the palm of the glove is used to collect the palm motion trajectory information; The communication submodule is used to send the motion track information of the five fingers and the motion track information of the palm to the data processing module.

3. The hand rehabilitation training device based on brain-computer interface and AR glasses as claimed in claim 2, characterized in that: The data processing module analyzes and processes the EEG signal set and the motion trajectory information set to obtain training evaluation result information, including: Performing electroencephalogram (EEG) evaluation on the EEG signal set to obtain an EEG evaluation value; Performing motion evaluation on the motion trajectory information set to obtain a motion evaluation value and a maximum deviation; The training evaluation result information is constructed using the EEG evaluation value, the movement evaluation value and the maximum deviation.

4. A hand rehabilitation training method based on brain-computer interface and AR glasses, characterized in that: The method is implemented by using the hand rehabilitation training device based on a brain-computer interface and AR glasses according to any one of claims 1 to 3, comprising: S1, using the AR glasses to display virtual training scenes and task guidance information for the user; S2, using the brain-computer interface device to collect the EEG signal set of the user when performing hand rehabilitation training; S3, using the hand motion sensor to monitor the motion state of the user's hand in real time to obtain a set of motion trajectory information of the user's hand; the set of motion trajectory information includes motion trajectory information of five fingers and palm motion trajectory information; S4, using the data processing module to analyze and process the EEG signal set and the motion trajectory information set to obtain training evaluation result information; S5, using the training feedback module to generate corresponding display result information according to the training evaluation result value, and sending the display result information to the AR glasses for display.

5. The hand rehabilitation training method based on brain-computer interface and AR glasses as claimed in claim 4, characterized in that: The analyzing and processing the EEG signal set and the motion trajectory information set to obtain training evaluation result information includes: S41, performing electroencephalogram (EEG) evaluation on the EEG signal set to obtain an EEG evaluation value; the EEG signal set includes α wave signals, β wave signals, θ wave signals, and δ wave signals; S42, performing motion evaluation on the motion trajectory information set to obtain a motion evaluation value and a maximum deviation; S43, constructing training evaluation result information using the EEG evaluation value, the movement evaluation value and the maximum deviation.

6. The hand rehabilitation training method based on brain-computer interface and AR glasses as claimed in claim 5, characterized in that: The performing EEG evaluation on the EEG signal set to obtain an EEG evaluation value comprises: S411, obtaining an α wave standard signal, a β wave standard signal, a θ wave standard signal, and a δ wave standard signal; S412, subtracting each type of signal in the EEG signal set from the corresponding standard signal to obtain a corresponding difference signal; S413, performing statistical processing on each type of difference signal to obtain a corresponding statistical value set; the statistical value set includes variance, mean, median, harmonic mean, characteristic variance value, second-order origin moment and third-order central moment; the harmonic mean is the mean of all harmonic frequencies of the FFT sequence of the difference signal; the characteristic variance value is the square root of the variance of the first-order derivative of the FFT sequence of the difference signal divided by the variance of the difference signal; S414, constructing a difference statistical matrix using the statistical value set of each type of difference signal; the row vector of the difference statistical matrix is ​​a statistical value set of a type of difference signal; S415, performing deviation characteristic calculation on the difference statistical matrix to obtain deviation characteristic value; S416, determining the deviation characteristic value as an EEG evaluation value.

7. The hand rehabilitation training method based on brain-computer interface and AR glasses as claimed in claim 6, characterized in that: The expression for calculating the deviation characteristic is: Where, ∈ is the trace value of the difference statistics matrix, P1 and P2 are the first difference value and the second difference value respectively, σ ij is the element of the i-th row and j-th column of the difference statistics matrix, is the mean of the ith row of the difference statistics matrix, is the mean of all elements of the difference statistics matrix, m and n are the row dimension and column dimension of the difference statistics matrix respectively, and P3 is the deviation eigenvalue.

8. The hand rehabilitation training method based on brain-computer interface and AR glasses as claimed in claim 5, characterized in that: The performing motion evaluation on the motion trajectory information set to obtain a motion evaluation value and a maximum deviation includes: S421, obtaining a standard motion trajectory information set; the standard motion trajectory information set includes a standard motion trajectory sequence of five fingers and a palm; S422, representing the motion trajectory information set as a motion matrix; the first to sixth row vectors of the motion matrix are the motion trajectory information of five fingers and the motion trajectory information of the palm, respectively; S423, representing the standard motion trajectory information set as a standard matrix; the first to sixth row vectors of the standard matrix are the standard motion trajectory sequence of the five fingers and the palm motion trajectory sequence respectively; S424, performing fusion evaluation calculation on the motion matrix and the standard matrix to obtain a motion evaluation value; S425, subtracting the motion matrix from the standard matrix to obtain a difference matrix; S426, extracting characteristic rows from the difference matrix to obtain a row characteristic value set; the row characteristic value set includes a row characteristic value of each row; S427, determining the row number of the row corresponding to the largest row eigenvalue as the maximum deviation.

9. The hand rehabilitation training method based on brain-computer interface and AR glasses as claimed in claim 5, characterized in that: The expression for extracting the feature line is: in, is the i-th element of the first difference vector, is the i-th element of the second difference vector, g ij is the element of the i-th row and j-th column of the difference matrix, g j is the mean of the jth column of the difference matrix, g ii is the element in the ith row and ith column of the difference matrix, N is the column dimension of the difference matrix, k i is the row eigenvalue of the i-th row.

10. The hand rehabilitation training method based on brain-computer interface and AR glasses as claimed in claim 4, characterized in that: The generating corresponding display result information according to the training evaluation result value includes: The display result information includes task display information and prompt action display information; the training feedback module stores a value range set and task display information corresponding to each value range; the value range set includes several value ranges; S51, performing weighted summation on the EEG evaluation value and the movement evaluation value to obtain a comprehensive evaluation value; S52, determining a value range corresponding to the comprehensive evaluation value according to a preset value range set, and determining task display information corresponding to the value range; S53, determining the user hand information corresponding to the maximum deviation, and displaying the information as a prompt action.

Images