EEG Wheelchair Control Method Based on Comprehensive Stimulation of SSVEP and P300

Abstract

The present invention discloses an electroencephalogram wheelchair control method based on comprehensive stimulation of SSVEP and P300, belonging to the technical field of intelligent control. The method optimizes the linear discriminant analysis algorithm by correcting the membership threshold of the Fisher criterion function iteration to converge, solves the problem that a large number of electroencephalogram signals of users need to be collected to construct a personal database before user target recognition in a brain-controlled wheelchair, which consumes a large amount of time and labor costs and is not universal, and achieves the effect of high recognition accuracy of the electroencephalogram wheelchair control system and fast adaptation to different users; at the same time, the present application adopts parallel mode synthesis, the stimulation target is within the same flashing square, and SSVEP and P300 signals are induced simultaneously. It solves the problems of large interference between multiple targets on a small-size screen, small number of instruction sets, hierarchical display of multi-level mixed stimulation paradigms, and cumbersome operations.

Description

Technical Field

[0001] The invention relates to an electroencephalogram wheelchair control method based on SSVEP and P300 comprehensive stimulation, and belongs to the technical field of intelligent control. Background Art

[0002] Brain-Computer Interface (BCI), also known as brain-computer interface, is a control system that converts brain electrical signals collected by acquisition equipment into commands. The emergence and development of the technology of connecting the human brain to a computer has become a new way of information exchange and control, and has made great contributions to improving the ability of disabled people to take care of themselves.

[0003] BCI systems usually use different stimulation paradigms to effectively collect the intentions or instructions expressed by users, accurately convert them into relevant instructions and output them to the outside world. The evoked paradigm is often used in traditional single-paradigm BCI systems based on electroencephalogram (EEG), which can achieve a high recognition accuracy and information transmission rate. However, due to the single thinking type, the small control instruction set and the difficulty in balancing the number of instruction sets, stimulation interference and classification accuracy, the information transmission rate is still low and it is difficult to meet practical needs. In order to increase the types of thinking modes, expand the scale of instructions and further improve the information transmission rate and system performance of BCI, a hybrid paradigm brain-computer interface system composed of multiple single-thinking type BCI systems has been developed. It has various EEG advantages and has the ability to expand the instruction set, so that it can further improve the system work efficiency when performing multiple tasks.

[0004] Mixed-mode BCI can be divided into two basic types according to its mixed control method: serial mode and parallel mode. In the serial mode, two different sensory modes are controlled in sequence, such as serial connection based on P300 and SSVEP, using SSVEP as the system switch signal, and then using P300 signal for command output. This serial connection method reduces the false positive rate of the system, but the system is cumbersome and increases the time cost of specific instructions; while in the parallel mode, the two sensory modes are controlled simultaneously and collaboratively, which is equivalent to increasing the number of tasks that the system can recognize, but in the mixed stimulation of P300 and SSVEP signals, SSVEP signals are still often used as a system switch signal, which cannot improve the output efficiency of specific instructions, and the number of recognizable tasks is reduced.

[0005] The presentation methods of the hybrid stimulation paradigm of P300 and SSVEP signals are mostly matrix - type dense arrangements. The comprehensive degree of the presentation of signals in different dimensions is not high. Such an arrangement increases the interference of each stimulation target on the signal and also makes the subject prone to fatigue, thus affecting the subject's concentration of thinking and the quality of brain signals. In addition, the existing presentation of the hybrid stimulation paradigm is limited by the display area and scenario. In the case of a small - size portable BCI system with a dense - arrangement - type stimulation presentation, as the display area of the stimulation decreases, the interference of the stimulation gradually increases, and the classification accuracy and the number of instruction sets of the BCI are reduced exponentially.

[0006] Asynchronous BCI is relative to synchronous BCI. It can judge the state of the user when using BCI and only realizes command output when the user has an output intention, while synchronous BCI believes that the user is always in the state of intention output. For example, in the use of traditional stimulation paradigms, even if the user does not look at the BCI interface, the system will still output characters, resulting in false triggers. Traditional solutions usually use a kind of signal as a "brain switch" to control serial output, but this approach limits the operation flexibility. There are also studies that achieve asynchronous control by using the difference in brain - electrical characteristics induced in the control state and the idle state.

[0007] Currently, the research on BCI systems mostly stays at the laboratory research level, ignoring the situation in specific application scenarios. In practical life scenarios such as portable and mobile ones, the design and control schemes of many stimulation paradigms cannot be adapted. Moreover, due to individual differences in brain - electricity, the brain - electrical characteristics of different individuals vary greatly. Existing brain - electricity classification algorithms need to collect a large amount of brain - electrical signals from each subject in advance to construct a personal database, consuming a large amount of time and labor costs and not being universal, which is not suitable for mass production and large - scale use. Summary of the Invention

[0008] To solve the problem that the existing electroencephalogram (EEG) wheelchair control method needs to collect a large amount of EEG signals from each subject to construct a personal database, consuming a large amount of time and labor costs and not being universal, the present invention provides an EEG wheelchair control method based on comprehensive stimulation of SSVEP and P300. The method includes:

[0009] Step 1: Real - time collect the EEG signals generated by the current wheelchair user under the stimulation of a multi - level hybrid stimulation paradigm, and extract SSVEP and P300 channel data; among them, the SSVEP signals in the SSVEP channel data correspond to the four commands of forward, left - turn, right - turn, and stop of the first - level commands, and the P300 signals in the P300 channel data correspond to the forward speed, left - turn steering angle, right - turn steering angle, or braking degree of the second - level commands;

[0010] Step 2: Perform canonical correlation analysis on the SSVEP signals in the SSVEP channel data, calculate the canonical correlation coefficients between the SSVEP signals and the templates corresponding to the four commands of forward, left turn, right turn, and stop in the reference template respectively, and the target corresponding to the maximum correlation coefficient is the primary command that the current wheelchair user is gazing at;

[0011] Step 3: Perform linear discriminant analysis on the P300 channel data. During the analysis process, use the modified Fisher criterion function to iteratively converge the membership threshold to a predetermined value to determine the standard P300 data set for the current wheelchair user, so as to determine the secondary command he / she is gazing at according to the P300 channel data in the electroencephalogram signal of the current wheelchair user;

[0012] Step 4: Control the movement state of the wheelchair according to the primary command and secondary command determined in Step 2 and Step 3.

[0013] Optionally, the multi-level hybrid stimulation paradigm is: sequentially display four indicating graphics with different blinking frequencies on the electroencephalogram wheelchair control display screen to induce SSVEP signals; the four indicating graphics are respectively distributed at the upper, left, right, and lower positions of the display, corresponding to the four commands of forward, left turn, right turn, and stop of the primary command;

[0014] While the four blinking indicating graphics of the SSVEP signal are being displayed in brightness, randomly display six different colors at a predetermined frequency, and respectively preset three of them as three target stimuli to induce P300 signals. The three target stimuli correspond to three levels of forward speed, left turn steering angle, right turn steering angle, or braking degree in the secondary command;

[0015] Preset the corresponding relationship between the four indicating graphics at the upper, left, right, and lower positions on the display and the forward speed, left turn steering angle, right turn steering angle, and braking degree of the secondary command.

[0016] Optionally, the standard P300 data set for the current wheelchair user in Step 3 includes a training set and a test set. For different wheelchair users, replace the sample data in the test set by iteratively converging the membership threshold;

[0017] Step 3 includes:

[0018] Step 3.1: Pre-collect standard n segments of P300 signals and n segments of non-P300 signals as sample data in the training set, and preprocess the sample data in the training set;

[0019] Step 3.2: Determine whether the P300 channel data collected in real time needs to be retained according to the sample data in the training set and the iterative initial value p0 of the preset membership threshold;

[0020] When the amount of data to be retained is 2n, pack this part of the data to be retained and mark its membership threshold as p0; and add this part of the data to the test set as sample data in the test.

[0021] Step 3.3: Update the membership threshold to p i+1 , i = 0, 1, 2, 3,...; Determine whether the P300 channel data to be collected subsequently needs to be retained according to the data samples in the training set and the test set at this time and the updated membership threshold p i+1 ; When the amount of data to be retained in the subsequently collected P300 channel data reaches 2 i+2 n, pack this part of the data to be retained and mark its membership threshold as p i+1 ; And add this part of the data to the test set as sample data in the test as well.

[0022] Step 3.4: When the membership threshold is updated to the preset determination value, use the sample data in the test and the sample data in the training set at this time as the standard P300 data set for the current wheelchair user.

[0023] During subsequent use, match the P300 signal in the electroencephalogram signal of the current wheelchair user collected in real time with the sample data in the standard P300 data set to determine the secondary command that the current wheelchair user is gazing at.

[0024] Optionally, during subsequent use, if the P300 channel data in the electroencephalogram signal of the current wheelchair user collected in real time is determined to not need to be retained continuously for M times, remind the current wheelchair user whether to delete the sample data in the test set of the standard P300 data set. If the current wheelchair user chooses to delete, re - execute Step 3.2 to Step 3.4 to update the standard P300 data set to adapt to the current wheelchair user.

[0025] Optionally, Step 3.1 includes:

[0026] Step 3.1.1: Collect standard n segments of P300 signals and n segments of non - P300 signals.

[0027] Step 3.1.2: Calculate the mean vectors m j :

[0028]

[0029] Among them, N j is the number of samples of class w j When j = 1, class w j represents P300 data, and when j = 2, class w j represents non - P300 data, and X represents the sample data of the corresponding class;

[0030] Step 3.1.3: Calculate the within-class scatter matrix S of the samples j and the total within-class scatter matrix S w :

[0031]

[0032] S w = S1 + S2

[0033] Step 3.1.4: Calculate the between-class scatter matrix S of the training samples b :

[0034] S b = (m1 - m2)(m1 - m2) T

[0035] Step 3.1.5: Define the Fisher criterion function J F (w), and find the projection vector w* corresponding to the maximum value of J F (w):

[0036]

[0037] w* = S w -1 (m1 - m2)

[0038] Step 3.1.6: According to the obtained projection vector w*, project the two types of samples in the training set to obtain the projection values y of the two types of sample data j :

[0039] y j = (w*) T X j

[0040] Step 3.1.7: Project the real-time collected P300 channel data to obtain the corresponding projection value y0; calculate the distances d1 and d2 between the projection values y of the two types of sample data j and the projection value y0 of the real-time collected P300 channel data, and normalize them to obtain D1 and D2 respectively:

[0041]

[0042] Normalize the distances to obtain D1 and D2

[0043]

[0044] where, when j = 1, D j class represents the normalized distance from the standard P300 data, and when j = 2, D jDenote the normalized distance from the non - P300 data, X represents the sample data of the corresponding class; u1 is the mean vector of the P300 data, u2 is the mean vector of the non - P300 data, and ∑ is the corresponding covariance matrix;

[0045] Construct the membership function f(x):

[0046]

[0047] where exp represents the exponential function with the natural constant e as the base;

[0048] Substitute the normalized distances D1 and D2 into the membership function f(x) to obtain the corresponding membership degrees P1 and P2.

[0049] Optionally, step 3.2 includes:

[0050] Determine whether the P300 channel data collected in real - time needs to be retained according to the relationship between the membership degrees P1 and P2 and a preset membership degree threshold p0;

[0051] If both P1 and P2 are less than the preset membership degree threshold p0, then do not retain this data; otherwise, save this data, and if P1 > P2, then this data segment contains the P300 signal, that is, contains the target stimulus signal, and obtain a secondary instruction; if P1 < P2, then this data segment contains the non - P300 signal, that is, does not contain the target stimulus signal;

[0052] When the amounts of P300 and non - P300 data to be retained are both n, pack this part of the data to be retained and mark its membership degree threshold as p0; and add this part of the data to the test set as sample data in the test, and distinguish whether it contains the P300 signal.

[0053] Optionally, in step 3.3, according to the data samples in the training set and the test set at this time and the updated membership degree threshold p i+1 Determine whether the subsequent P300 channel data to be collected needs to be retained, including:

[0054] Step 3.3.1 Mark the P300 and non - P300 data in the packed data with the membership degree threshold marked as p i Calculate the mean vectors m of the two types of data j ;

[0055]

[0056] where N j is the number of samples of class w j When j = 1, class w j represents the P300 data, and when j = 2, class w jThe class represents non-P300 data, and X represents the data corresponding to the class;

[0057] Step 3.3.2 Calculate the within-class scatter matrix S of the packed data j and the total within-class scatter matrix

[0058]

[0059] Step 3.3.3 Calculate the between-class scatter matrix of the P300 data and non-P300 data of the packed data

[0060]

[0061] Step 3.3.4 According to the membership threshold p at this time i+1 and the scatter matrix, trim the Fisher criterion function:

[0062]

[0063] And denote as the between-class scatter matrix of the samples at this time, with the symbol S b,i+1 ; Denote {(1 - as the total within-class scatter matrix of the samples at this time, with the symbol S w,i+1 .

[0064] Step 3.3.5 Determine the new projection vector w* according to the trimmed Fisher criterion function i+1 ;

[0065] Step 3.3.6 According to the new projection vector w* i+1 obtain the projection values of the two types of sample data; at the same time, calculate the projection values of the P300 channel data collected subsequently;

[0066] Step 3.3.7 Calculate the distances d1 and d2 between the projection values of the two types of sample data and the projection values of the P300 channel data collected subsequently, and normalize them to obtain D1 and D2 respectively; substitute the normalized distances D1 and D2 into the membership function f(x) to obtain the corresponding membership degrees P1 i+1 and P2 i+1 , substitute P1 i+1 and P2 i+1 compare with the updated membership threshold p i+1 to determine whether the P300 channel data collected subsequently needs to be retained. If it needs to be retained, further distinguish whether it contains P300 signals.

[0067] Optionally, the updated membership threshold p i+1 is:

[0068] p i+1 = p i + P

[0069] where P is a fixed value.

[0070] Optionally, the updated membership threshold p i+1 is:

[0071] p i+1 = p i + P i

[0072] where P i is a value that increases or decreases according to a predetermined rule.

[0073] This application also provides an electroencephalogram wheelchair control system based on the comprehensive stimulation of SSVEP and P300. It is characterized in that the system controls the movement of the wheelchair by using the above method.

[0074] The beneficial effects of the present invention are:

[0075] By proposing a new comprehensive stimulation paradigm of SSVEP and P300 and adopting parallel mode synthesis, with the stimulation target in the same flashing square, the SSVEP and P300 signals are induced simultaneously. This solves the problems of large interference between multiple targets on a small screen, small number of instruction sets, hierarchical display of multi-level hybrid stimulation paradigms, cumbersome operation, and reduction of the number of instruction sets when one sensory signal is often used as a "switch" in a parallel mode synthesis asynchronous system. It achieves the effects of realizing a multi-level and multi-instruction set number with fewer stimulation targets on a small screen, small interference of stimulation targets, simple operation, and asynchronous control of the wheelchair by the parallel comprehensive stimulation paradigm.

[0076] Furthermore, by modifying the Fisher criterion function to iteratively converge the membership threshold to optimize the linear discriminant analysis algorithm, this application solves the problems that a large number of electroencephalogram signals need to be collected from users to construct a personal database before user target recognition in a brain-controlled wheelchair, consuming a large amount of time and labor costs and lacking universality. It achieves the effects of high recognition accuracy of the electroencephalogram wheelchair control system, rapid adaptation to different users, and mass production and large-scale use. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0078] Figure 1 It is a schematic diagram of the overall application process in an embodiment of the present invention.

[0079] Figure 2 Schematic diagram of the SSVEP and P300 combined stimulation paradigm in an embodiment of the present invention.

[0080] Figure 3 Flow chart of the optimized linear discriminant analysis method in an embodiment of the present invention.

[0081] Figure 4 Schematic diagram of the distribution of EEG electrode positions. Detailed implementation manners

[0082] To make the objectives, technical solutions and advantages of the present invention clearer, the following will further describe the embodiments of the present invention in detail with reference to the accompanying drawings.

[0083] Embodiment 1:

[0084] This embodiment provides an EEG wheelchair control method based on SSVEP and P300 combined stimulation, and the method includes:

[0085] Step 1: Real-time collect the EEG signals generated by the current wheelchair user under the stimulation of a multi-level mixed stimulation paradigm, and extract the SSVEP and P300 channel data; among them, the SSVEP signals in the SSVEP channel data correspond to the four commands of forward, left turn, right turn, and stop of the first-level commands, and the P300 signals in the P300 channel data correspond to the forward speed, left turn steering angle, right turn steering angle, or braking degree of the second-level commands; the P300 channel data includes P300 data and non-P300 data; among them, the P300 data is the data corresponding to the P300 signal.

[0086] The multi-level mixed stimulation paradigm is: sequentially display four indication graphics with different flashing frequencies on the EEG wheelchair control display screen to induce SSVEP signals; the four indication graphics are respectively distributed at the upper, left, right, and lower positions of the display, corresponding to the four commands of forward, left turn, right turn, and stop of the first-level commands;

[0087] While the four flashing indication graphics of the SSVEP signal are being displayed, randomly display six different colors at a predetermined frequency, and respectively preset three of them as three target stimuli to induce P300 signals, and the three target stimuli correspond to three levels of forward speed, left turn steering angle, right turn steering angle, or braking degree in the second-level commands;

[0088] Pre-set the corresponding relationship between the four indication graphics at the upper, left, right, and lower positions on the display and the forward speed, left turn steering angle, right turn steering angle, and braking degree of the second-level instructions.

[0089] Step 2: Perform canonical correlation analysis on the SSVEP signals in the SSVEP channel data to calculate the canonical correlation coefficients between the SSVEP signals and the templates corresponding to the four commands of forward, left turn, right turn, and stop in the first-level commands of the reference template. The target corresponding to the maximum correlation coefficient is the first-level command that the current wheelchair user is gazing at;

[0090] Step 3: Perform linear discriminant analysis on the P300 channel data. During the analysis process, use the modified Fisher criterion function to iteratively converge the membership threshold to a predetermined value to determine the standard P300 data set for the current wheelchair user, so as to determine the second-level command he is gazing at according to the P300 channel data in the electroencephalogram signal of the current wheelchair user; the standard P300 data set includes a training set and a test set. For different wheelchair users, the sample data in the test set is replaced by iteratively converging the membership threshold.

[0091] Step 4: Control the motion state of the wheelchair according to the first-level command and the second-level command determined in Step 2 and Step 3.

[0092] Embodiment 2:

[0093] This embodiment provides an electroencephalogram wheelchair control method based on comprehensive stimulation of SSVEP and P300, as Figure 1 shown, the method includes:

[0094] Step 1: Place measurement electrodes in the parietal lobe region, occipital lobe region, frontal lobe region, and temporal lobe region of the subject's brain, and place a reference electrode and a ground electrode at the positions of the left and right earlobes respectively. The lead positions conform to the international 10-20 standard. Subsequently, the measured electroencephalogram data is amplified and processed by analog-to-digital conversion and then sent to the electroencephalogram acquisition device in real time.

[0095] Step 2: The wheelchair microprocessor controls the display screen to sequentially display four indication graphics with different flashing frequencies to induce SSVEP signals; the four indication graphics are respectively distributed at the upper, left, right, and lower positions of the display, corresponding to the four commands of forward, left turn, right turn, and stop in the first-level commands;

[0096] While the four flashing indication graphics of the SSVEP signal are being displayed, six different colors are randomly displayed at a predetermined frequency, and three of them are respectively preset as three target stimuli to induce P300 signals. The three target stimuli correspond to three levels of forward speed, left turn steering angle, right turn steering angle, or braking degree in the second-level commands;

[0097] The corresponding relationships between the four indication graphics at the upper, left, right, and lower positions on the display and the forward speed, left turn steering angle, right turn steering angle, and braking degree of the second-level instructions are preset.

[0098] In practical applications, the wheelchair microprocessor can determine the flashing frequencies of the four indicating graphics according to the display refresh rate. For example, according to the current situation where the display refresh rate in the market is 60 Hz, the present application can set the flashing frequencies of the four indicating graphics as: 8.57, 10, 12, 15 Hz, that is, the display generates flashing frequencies of 8.57, 10, 12, 15 Hz in sequence to display the four indicating graphics in brightness; the indicating graphics can be four squares or arrows pointing to four directions (the present application does not limit this), which are used to induce SSVEP signals.

[0099] In this embodiment, the four indicating graphics are four squares respectively distributed at the upper, left, right, and lower parts of the display, corresponding to the four commands of forward, left turn, right turn, and stop of the first-level command respectively.

[0100] In this embodiment, while the four flashing squares of the SSVEP signal are flashing in brightness, six colors of red, blue, yellow, green, orange, and purple are randomly displayed at a frequency of 6 Hz, and blue, red, and yellow are respectively preset as three target stimuli to induce the secondary commands controlled by the P300 signal. The three target stimuli correspond to three levels of secondary forward speed or secondary steering angle or secondary braking (it is preset that the four target squares on the display respectively correspond to the forward speed, steering angle, steering angle, and stop degree of the secondary command), as Figure 2 shown.

[0101] Step3: After generating the stimulation paradigm, the subject pre-determines the target command and then observes the flashing square corresponding to the command. The SSVEP signal of the first-level command is induced by a specific brightness flashing frequency. At the same time, the subject pays attention to the color change of the flashing square. When the color of the target target stimulus appears, the subject silently counts the number of flashes of the target stimulus to induce the P300 signal of the secondary command. When the subject needs to change the secondary command, the subject can change the color of the target stimulus in the heart at any time and then observe the color of the square. When the stimulus is presented, the electroencephalogram acquisition device acquires the corresponding electroencephalogram data.

[0102] Step4: The electroencephalogram data of the subject collected is digitally processed. According to the channel positions of SSVEP and P300, the electroencephalogram data is divided into two groups, and the two groups of data are segmented, downsampled, and band-pass filtered. Then, the preprocessed electroencephalogram data is subjected to feature extraction to extract the SSVEP channel data and the P300 channel data respectively; in the SSVEP channel data, it contains both the SSVEP signal, that is, the target stimulus signal of the first-level command, and non-SSVEP signals; similarly, in the P300 channel data, it contains both the P300 signal, that is, the target stimulus signal of the secondary command, and non-P300 signals.

[0103] For the SSVEP channel data, canonical correlation analysis is performed on the SSVEP channel data according to the reference template to calculate the canonical correlation coefficient. The target corresponding to the maximum correlation coefficient is the first-level command at which the subject is gazing. For specific details, please refer to the introduction in "Liu, X. (2018). Research on the construction method and key technologies of a portable brain-computer interface system based on SSVEP. (Doctoral dissertation, Xiamen University)", which will not be elaborated here in this application.

[0104] For the P300 channel data, linear discriminant analysis is performed, as Figure 3 shown, including:

[0105] Step4.1: Pre-collect standard n-segment P300 signals and n-segment non-P300 signals as sample data in the training set, and preprocess the sample data in the training set;

[0106] The preprocessing includes:

[0107] Calculate the mean vector m j :

[0108]

[0109] where N j is the number of samples in class w j , when j = 1, class w j represents P300 data, and when j = 2, class w j represents non-P300 data, and X represents the sample data of the corresponding class;

[0110] Calculate the within-class scatter matrix S j and the total within-class scatter matrix S w :

[0111]

[0112] S w = S1 + S2

[0113] Calculate the between-class scatter matrix S b :

[0114] S b = (m1 - m2)(m1 - m2) T

[0115] Define the Fisher criterion function J F (w), and find the projection vector w* corresponding to the maximum value of J F (w):

[0116]

[0117] Project the two types of samples in the training set to obtain the projection values y of the two types of sample data j :

[0118] y j =(w*) T X j

[0119] Project the P300 channel data collected in real time to obtain the corresponding projection value y0; calculate the distances d1 and d2 between the projection values y of the two types of sample data j and the projection value y0 of the P300 channel data collected in real time, and normalize them to obtain D1 and D2 respectively:

[0120]

[0121] Normalize the distances to obtain D1 and D2

[0122]

[0123] where D when j = 1 j class represents the normalized distance from the standard P300 data, and D when j = 2 j represents the normalized distance from the non-P300 data, X represents the sample data of the corresponding class; u1 is the mean vector of the P300 data, u2 is the mean vector of the non-P300 data, and ∑ is the corresponding covariance matrix;

[0124] Construct the membership function f(x):

[0125]

[0126] where exp represents the exponential function with the natural constant e as the base;

[0127] Substitute the normalized distances D1 and D2 into the membership function f(x) to obtain the corresponding membership degrees P1 and P2.

[0128] Step4.2: Determine whether the P300 channel data collected in real time needs to be retained according to the sample data in the training set and the initial iteration value p0 of the preset membership degree threshold;

[0129] When the amount of data to be retained is 2n, pack this part of the data to be retained and mark its membership degree threshold as p0; and add this part of the data to the test set as the sample data in the test;

[0130] Specifically, if both P1 and P2 are less than a preset membership threshold p0, the data is not retained; otherwise, the data is saved. And if P1 > P2, this data segment contains the P300 signal, that is, it contains the target stimulus signal, and a secondary instruction is obtained; if P1 < P2, this data segment contains a non-P300 signal, that is, it does not contain the target stimulus signal.

[0131] When the amount of data to be retained is 2n, the data to be retained in this part is packaged and its membership threshold is marked as p0; and this part of the data is added to the test set as sample data in the test, and it is distinguished whether it contains the P300 signal.

[0132] In this embodiment, the membership threshold p0 can be set to 70%. In practical applications, technicians can set it to other values according to experience.

[0133] Step4.3: Update the membership threshold p0 to p i+1 , i = 0, 1, 2, 3,...; Determine whether the P300 channel data to be collected subsequently needs to be retained according to the data samples in the training set and the test set at this time and the updated membership threshold p i+1 ; When the amount of data to be retained in the subsequently collected P300 channel data reaches 2 i+2 n, the data to be retained in this part is packaged and its membership threshold is marked as p i+1 ; And this part of the data is also added to the test set as sample data in the test;

[0134] Specifically, update the membership threshold p0 to p1, p1 > p0;

[0135] Step 4.3.1 Mark the P300 and non-P300 data in the packaged data according to the membership threshold marked as p i and calculate the mean vectors m of the two types of data j ;

[0136]

[0137] where N j is the number of samples in class w j , when j = 1, class w j represents P300 data, and when j = 2, class w j represents non-P300 data, and X represents the data corresponding to the class;

[0138] Step 4.3.2 Calculate the within-class scatter matrix S j of the packaged data and the total within-class scatter matrix

[0139]

[0140] Step 4.3.3 Calculate the between-class scatter matrix of the packed data P300 data and non-P300 data

[0141]

[0142] Step 4.3.4 Trim the Fisher criterion function according to the membership threshold p i+1 at this time and the scatter matrix:

[0143]

[0144] And denote as the between-class scatter matrix of the samples at this time, with the symbol S b,i+1 ; Denote as the total within-class scatter matrix of the samples at this time, with the symbol S w,i+1 ;

[0145] Step 4.3.5 Determine the new projection vector w* according to the trimmed Fisher criterion function i+1 ;

[0146] Step 4.3.6 Obtain the projection values of the two types of sample data according to the new projection vector w* i+1 ; At the same time, calculate the projection values of the P300 channel data collected subsequently;

[0147] Step 4.3.7 Calculate the distances d1 and d2 between the projection values of the two types of sample data and the projection values of the P300 channel data collected subsequently, and normalize them to obtain D1 and D2 respectively; Substitute the normalized distances D1 and D2 into the membership function f(x) to obtain the corresponding membership degrees P1 i+1 and P2 i+1 , and substitute P1 i+1 and P2 i+1 Compare with the updated membership threshold p i+1 to determine whether the P300 channel data collected subsequently needs to be retained. If it needs to be retained, further distinguish whether it contains P300 signals.

[0148] Until the amount of data to be retained reaches 2 i+2 n, pack this part of the data to be retained and mark its membership threshold as p i+1 ; And add this part of the data to the test set as sample data in the test;

[0149] Update the membership threshold again, and repeat the above Step4.3 until the membership threshold is updated to the preset decision value;

[0150] In this embodiment, the preset determination value is 94%. In practical applications, technicians can set it to other values according to experience.

[0151] When updating the membership threshold as described above, a fixed value can be selected to increase each time, such as p i+1 = p i + P, where P is taken as 2% or other fixed values, or the value that changes each time can be increased, such as p i+1 = p i + P i , P i takes variable values, such as values that increase in sequence, 2%, 4%, 6%, etc., or values that decrease in sequence, 20%, 15%, 10%, etc. The specific change rule is not limited in this application.

[0152] In this embodiment, it is selected to increase the fixed value of 2% each time.

[0153] Step4.4: When the membership threshold p i is updated to the preset determination value, the sample data in the current test and the sample data in the training set are used as the standard P300 data set for the current wheelchair user;

[0154] During subsequent use, the P300 signal in the EEG signal of the current wheelchair user collected in real time is matched with the sample data in the standard P300 data set to determine the secondary command gazed at by the current wheelchair user.

[0155] Specifically, when the membership threshold p i is updated to the preset determination value of 94%, the sample data in the current test and the sample data in the training set are used as the standard P300 data set for the current wheelchair user, and the projection vector w* i is calculated at this time. Suppose it is denoted as w* 终 ,

[0156] According to the final projection vector w* i the projection values of the two types of sample data corresponding to the sample data in the training set and the sample data in the test at this time are obtained and determined as the final projection values of the two types of sample data;

[0157] During subsequent wheelchair use, after collecting the P300 channel data, its projection value is calculated according to the following formula:

[0158] y = (w* 终 ) T X

[0159] where X represents the collected P300 channel data;

[0160] Calculate the final projection values of two types of sample data and the projection values of the subsequently collected P300 channel data;

[0161] Calculate the distances between the final projection values of the two types of sample data and the projection values of the subsequently collected P300 channel data respectively, and normalize them to obtain the final normalized distance D1 终 , D2 终 ; Substitute the normalized distances D1 终 , D2 终 into the membership function f(x) to obtain the corresponding membership degrees P1 i+1 and P2 i+1 . Compare P1 i+1 and P2 i+1 with the final membership degree threshold p i to determine whether the subsequently collected P300 channel data needs to be retained. If it needs to be retained, further distinguish whether it contains the P300 signal. If it contains the P300 signal, that is, it contains the target stimulus signal, further match it with the data representing each secondary command in the standard P300 dataset to determine the specific secondary command.

[0162] It should be noted that for the EEG data of the P300 channel collected, mark the corresponding stimulation color for the corresponding EEG data segment according to the flashing time of each stimulation color. When a certain data segment is identified as containing P300 data, determine the corresponding secondary command according to the stimulation color marked on the data segment.

[0163] Send the identified primary command and secondary command to the wheelchair controller. If only one level of instruction is identified, no instruction is sent.

[0164] Step5: To overcome the problem that different individuals need to spend a large amount of time collecting sample data in advance when using the EEG wheelchair control system and make the system more universal, when the method of this application determines whether to retain the current data, if there are M consecutive non-retainments, remind the user whether to clear all the data in the test set and re-identify and package and iterate to adapt to the EEG characteristics of the new subject.

[0165] In practical applications, the value of M can be set by technicians according to experience. For example, in this embodiment, M = 5.

[0166] Step 6: The microprocessor of the wheelchair takes the first-level command and the second-level command as the input of the fuzzy control system, and calculates the accurate speed values of the left and right motors of the wheelchair through the fuzzy inference decision algorithm. The control algorithm of the wheelchair includes a speed PI controller and an angle PID controller. For non-turning instructions, only the PI controller is used. The microprocessor converts the data collected by the encoder into the actual speed and inputs it into the PI controller to form a closed loop, so as to accurately control the driving speed of the wheelchair to reach the target speed. For turning instructions, a two-stage control algorithm with the PID controller in series with the PI controller is adopted. The input of the first-stage PID controller is the actual rotation angle measured by the gyroscope, the actual driving speed and the target angle, and the output is the target speed of the left and right motors of the wheelchair. The second-stage PI controller takes the output of the previous stage as the input to accurately control the rotation speed of the left and right motors of the wheelchair. The specific control implementation method is not limited in this application.

[0167] Further, the sampling rate of the electroencephalogram acquisition device is 250 Hz. The measurement electrodes for SSVEP are O1, O2, P3, P4, Pz, and the measurement electrodes for P300 are: Pz, Fz, Cz, C3, C4, as Figure 4 shown.

[0168] Further, the induced electroencephalogram regions of the SSVEP signal and the P300 signal are different, and the two types of data can be quickly classified. The stimulation paradigms of SSVEP and P300 are in the same flashing square, and the stimulation is carried out simultaneously to induce the first-level command and the second-level command at the same time.

[0169] Some steps in the embodiments of the present invention can be implemented by software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk, etc.

[0170] The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for controlling an electroencephalogram wheelchair based on a comprehensive stimulus of SSVEP and P300, characterized in that, The method includes: Step 1: Collect the electroencephalogram (EEG) signals generated by the current wheelchair user under the multi-level hybrid stimulation paradigm in real time, and extract the SSVEP and P300 channel data. Among them, the SSVEP signals in the SSVEP channel data correspond to the four commands of forward, left turn, right turn, and stop of the primary command, and the P300 signals in the P300 channel data correspond to the forward speed, left turn angle, right turn angle, or braking degree of the secondary command. Step 2: Perform canonical correlation analysis on the SSVEP signals in the SSVEP channel data to calculate the canonical correlation coefficients between the SSVEP signals and the templates corresponding to the four commands of forward, left turn, right turn, and stop of the primary command in the reference template. The target corresponding to the maximum correlation coefficient is the primary command gazed at by the current wheelchair user. Step 3: Perform linear discriminant analysis on the P300 channel data. During the analysis process, the modified Fisher criterion function is used to iteratively converge the membership threshold to a predetermined value to determine the standard P300 data set for the current wheelchair user, so as to determine the secondary command gazed at by the current wheelchair user according to the P300 channel data in the EEG signals of the current wheelchair user. Step 4: Control the motion state of the wheelchair according to the primary command and the secondary command determined in Step 2 and Step 3. The standard P300 data set for the current wheelchair user in Step 3 includes a training set and a test set. For different wheelchair users, the sample data in the test set is replaced by iteratively converging the membership threshold. Step 3 includes: Step 3.1: Pre-collect standard n segments of P300 signals and n segments of non-P300 signals as sample data in the training set, and preprocess the sample data in the training set. Step 3.2: Determine whether the P300 channel data collected in real time needs to be retained according to the sample data in the training set and the initial iteration value p0 of the preset membership threshold. When the amount of data to be retained is 2n, pack this part of the data to be retained and mark its membership threshold as p0, and add this part of the data to the test set as sample data for testing. Step 3.3: Update the membership threshold to p i+1 , i = 0, 1, 2, 3,...; According to the data samples in the training set and the test set at this time and the updated membership threshold p i+1 Determine whether the subsequently collected P300 channel data needs to be retained; When the data that needs to be retained in the subsequently collected P300 channel data reaches 2 i+2 n, pack this part of the data that needs to be retained and mark its membership threshold as p i+1 ; And add this part of the data to the test set as sample data in the test; Step 3.4: When the membership threshold is updated to the preset determination value, use the sample data in the test and the sample data in the training set as the standard P300 data set for the current wheelchair user. During subsequent use, match the P300 signals in the EEG signals of the current wheelchair user collected in real time with the sample data in the standard P300 data set to determine the secondary command gazed at by the current wheelchair user.

2. The method according to claim 1, wherein The multi-level hybrid stimulation paradigm is as follows: Four indication graphics with different flashing frequencies are sequentially and brightly displayed on the EEG wheelchair control display screen to induce SSVEP signals. The four indication graphics are respectively distributed at the upper, left, right, and lower positions of the display, corresponding to the four commands of forward, left turn, right turn, and stop of the primary command. While four flashing indicator graphics of the SSVEP signal are being displayed with brightness, six different colors are randomly displayed at a predetermined frequency, and three of them are respectively preset as three target stimuli to induce P300 signals. The three target stimuli correspond to the forward speed, left turn steering angle, right turn steering angle, or braking degree of the three levels in the secondary command. The corresponding relationships between the four indicator graphics of up, left, right, and down on the display and the forward speed, left turn steering angle, right turn steering angle, and braking degree of the secondary command are preset in advance.

3. The method according to claim 2, wherein During subsequent use, if the P300 channel data in the EEG signal of the current wheelchair user collected in real time is continuously determined not to be retained for M times, the current wheelchair user is reminded whether to delete the sample data in the test set of the standard P300 data set. If the current wheelchair user chooses to delete, steps 3.2 to 3.4 are re-executed to update the standard P300 data set to adapt to the current wheelchair user.

4. The method according to claim 3, wherein The said step 3.1 includes: Step 3.1.1: Collect standard n segments of P300 signals and n segments of non-P300 signals. Step 3.1.2: Calculate the mean vectors m of the two types of sample data in the training set j : Among them, N j is the number of samples of class w j When j = 1, class w j represents P300 data. When j = 2, class w j represents non-P300 data, and X represents the sample data of the corresponding class; Step 3.1.3: Calculate the within-class scatter matrix S of the samples j and the total within-class scatter matrix S w : S w = S1 + S2 Step 3.1.4: Calculate the between-class scatter matrix S of the training samples b : S b = (m1 - m2)(m1 - m2) T Step 3.1.5: Define the Fisher criterion function \(J\) F F (w), and find the projection vector \(w^*\) corresponding to the maximum value of \(J\) F F (w): w* = S w -1 (m1 - m2) Step 3.1.6: According to the obtained projection vector w*, project the two types of samples in the training set to obtain the projection values y of the two types of sample data j : y j =(w*) T X j Step 3.1.7: Project the real-time collected P300 channel data to obtain the corresponding projection value y0; calculate the distances d1 and d2 between the projection values y of the two types of sample data and the projection value y0 of the real-time collected P300 channel data respectively, and normalize them to obtain D1 and D2 accordingly: j and the distances d1 and d2 between the projection values y of the two types of sample data and the projection value y0 of the real-time collected P300 channel data respectively, and normalize them to obtain D1 and D2 accordingly: The normalized distances are obtained as D1, D2. where D when j = 1 j class represents the normalized distance from the standard P300 data, and D when j = 2 j represents the normalized distance from the non-P300 data, X represents the sample data of the corresponding class; u1 is the mean vector of the P300 data, u2 is the mean vector of the non-P300 data, and ∑ is the corresponding covariance matrix; Construct the membership function f(x): Among them, exp represents the exponential function with the natural constant e as the base. The normalized distances D1 and D2 are substituted into the membership function f(x) to obtain the corresponding membership degrees P1 and P2.

5. The method according to claim 4, characterized in that, The said step 3.2 includes: Determine whether the P300 channel data collected in real time needs to be retained according to the relationship between the membership degrees P1 and P2 and the preset membership degree threshold p0. If both P1 and P2 are less than the preset membership degree threshold p0, the data is not retained; otherwise, the data is saved. And if P1 > P2, this data segment contains a P300 signal, that is, it contains a target stimulus signal, and a secondary command is obtained; if P1 < P2, this data segment contains a non-P300 signal, that is, it does not contain a target stimulus signal. When the amounts of P300 and non-P300 data to be retained are both n, this part of the data to be retained is packaged and marked with its membership degree threshold as p0; and this part of the data is added to the test set as sample data in the test, and it is distinguished whether it contains a P300 signal.

6. The method according to claim 5, wherein In step 3.3, based on the data samples in the training set and the test set at this time and the updated membership threshold p i+1 Determine whether the P300 channel data to be collected subsequently needs to be retained, including: Step 3.3.1 Mark as p according to the membership threshold at this time i For the P300 and non-P300 data in the packed data, calculate the mean vectors m of the two types of data j ; Among them, N j is the number of samples of class w j When j = 1, class w j represents P300 data. When j = 2, class w j represents non-P300 data, and X represents the data corresponding to the class; Step 3.3.2 Calculate the within-class scatter matrix S of the packed data j and the total within-class scatter matrix Step 3.3.3 Calculate the between-class scatter matrix of the packed data P300 data and non-P300 data Step 3.3.4 Modify the Fisher criterion function according to the membership threshold p at this time i+1 and the divergence matrix: And denote as the between-class scatter matrix of the samples at this time, denoted by S b,i+1 ; Denote as the total within-class scatter matrix of the samples at this time, denoted by S w,i+1 ; Step 3.3.5 Determine a new projection vector w* according to the trimmed Fisher criterion function i+1 ; Step 3.3.6 According to the new projection vector w* i+1 Obtain the projection values of the two types of sample data; at the same time, calculate the projection values of the P300 channel data collected subsequently; Step 3.3.7 Calculate the distances d1 and d2 between the projection values of the two types of sample data and the projection values of the subsequently collected P300 channel data respectively, and normalize them to obtain D1 and D2 correspondingly; Substitute the normalized distances D1 and D2 into the membership function f(x) to obtain the corresponding membership degrees P1 i+1 and P2 i+1 , Substitute P1 i+1 and P2 i+1 Compare with the updated membership degree threshold p i+1 to determine whether the subsequently collected P300 channel data needs to be retained. If it needs to be retained, further distinguish whether it contains P300 signals.

7. The method according to claim 6, characterized in that, The updated membership threshold p i+1 is as follows: p i+1 = p i + P Among them, P is a fixed value.

8. The method according to claim 6, wherein The updated membership threshold p i+1 is as follows: p i+1 = p i + P i where P i is a value that increases or decreases according to a predetermined rule.

9. An EEG wheelchair control system based on a comprehensive stimulus of SSVEP and P300, characterized in that, The said system controls the movement of the wheelchair by using the method described in any one of claims 1-8.

Images