Brain-computer interface equipment, control method thereof, medium and electronic equipment

Abstract

The invention provides brain-computer interface equipment and a control method thereof, a medium and electronic equipment. The control method of the brain-computer interface equipment comprises the following steps: acquiring a brain signal generated by a subject in any stimulation state; performing feature recognition on the brain signals to obtain brain response features contained in the brain signals; performing instruction identification under the brain response characteristics on the brain signal to obtain a target instruction represented by the brain signal; and configuring a preset execution probability based on the target instruction, and outputting the target instruction based on the execution probability to control external controlled equipment. According to the method and the device, the target instruction can be identified from the brain signal in real time through the response feature identification of the brain signal and the instruction identification under the response feature, and the target instruction is output in real time to control the external controlled equipment, so that the requirements of application fields with relatively high real-time requirements can be met.

Description

Technical Field

[0001] The present application belongs to the field of brain-computer interface technology, and relates to a brain-computer interface device and a control method, medium and electronic device thereof. Background Art

[0002] Brain Computer Interface (BCI) is a technology that establishes direct communication between the human brain and external devices, bypassing traditional nerve and muscle channels and allowing brain activity to directly control external devices. Control systems based on non-invasive brain-computer interfaces have been able to achieve operations such as character spelling, robotic arm control, and drone control, which has attracted widespread attention in society. Non-invasive brain-computer interfaces have the advantages of being applicable to a wider range of people, having fewer ethical issues, and being non-invasive.

[0003] However, existing BCI systems rely on collecting weak electrical signals from the surface of the brain and are easily affected by external noise. Advanced signal processing technology is needed to improve the accuracy and stability of the signal. Signal processing takes a lot of time, resulting in slower command output and unable to meet actual application needs. Summary of the invention

[0004] The present application provides a brain-computer interface device and its control method, medium and electronic device, which are used to solve the problem that the command output of existing brain-computer interface devices is slow and cannot meet the needs of application fields with high real-time requirements.

[0005] In a first aspect, the present application provides a method for controlling a brain-computer interface device, comprising: acquiring a brain signal generated by a subject under any stimulation state; performing feature recognition on the brain signal to obtain a brain response feature contained in the brain signal; performing instruction recognition on the brain signal under the brain response feature to obtain a target instruction represented by the brain signal; configuring a preset execution probability based on the target instruction, and outputting the target instruction based on the execution probability to control an external controlled device.

[0006] In this implementation, by identifying the response characteristics of brain signals and identifying the instructions under the response characteristics, the target instructions can be identified from the brain signals in real time, and the target instructions can be output in real time to control the external controlled devices, which can meet the needs of application fields with high real-time requirements.

[0007] In an implementation of the first aspect, the execution probability of the identified target instruction is iteratively calculated based on the Kalman filter; the target instruction is identified once, and the corresponding iterative calculation increases by one generation; it is determined whether the execution probability of the target instruction is greater than or equal to a preset value; if so, the target instruction is output to control the external controlled device; if not, the target instruction is not output.

[0008] In this implementation, by utilizing the Kalman filter to set execution probability control on the output of the target instruction and realize smooth processing of the target instruction, the false touch rate can be greatly reduced. Not only can the accuracy of the execution and / or termination of the target instruction be improved, but also the delay of the execution and / or termination of the target instruction can be accurately controlled.

[0009] In one implementation of the first aspect, it is determined whether the target instruction identified in the current generation is the same as the target instruction identified in the previous generation; if they are the same, the execution probability of the current target instruction increases; if they are different, the execution probability of the current target instruction decreases; and the execution probability of the target instruction is updated to the execution probability of the current target instruction.

[0010] In this implementation, the accuracy and effectiveness of the target instruction output can be further improved by adjusting the execution probability of the target instruction upward or downward. A separate upward / downward adjustment strategy can be set for each target instruction, which can be further matched with the actual control scenario of the external controlled device. It can be flexibly configured according to the actual situation of the external controlled device, and the effectiveness of the brain-computer interface control device is not affected by changes in the external controlled device. It has a wide range of applications and is more versatile.

[0011] In an implementation of the first aspect, the accuracy of the execution and / or termination of the target instruction is controlled based on a probability function; the delay of the execution and / or termination of the target instruction is controlled based on the calculation parameters of the probability function; the probability function includes an increasing probability function and a decreasing probability function; the execution probability of the contemporary target instruction is controlled to increase based on the increasing probability function; the execution probability of the contemporary target instruction is controlled to decrease based on the decreasing probability function; each target instruction corresponds to its own probability function.

[0012] In this implementation, the problem of "false touch" in target instruction output in actual applications can be solved by setting the probability function and calculation parameters.

[0013] In an implementation of the first aspect, the brain response characteristics include visual brain response characteristics and non-visual brain response characteristics; the non-visual brain response characteristics include muscle activity brain response characteristics and resting state brain response characteristics; the target instruction recognition model identifies the brain signal based on the visual brain response characteristics, and obtains the operation instructions and / or movement instructions represented by the brain signal; the target instruction recognition model identifies the brain signal based on the muscle activity brain response characteristics, and obtains the movement instructions and / or control instructions represented by the brain signal; the target instruction recognition model identifies the brain signal based on the resting state brain response characteristics, and obtains the resting state instructions represented by the brain signal; wherein the resting state instructions refer to instructions that do not execute the operation instructions, the movement instructions and the control instructions; the control instructions realize the start, stop or / and switching of the execution of other types of instructions.

[0014] In this implementation, operation instructions and / or movement instructions can be obtained based on the visual brain response feature recognition, movement instructions and / or control instructions can be obtained based on the muscle activity brain response feature recognition, and resting state instructions can be obtained based on the resting state brain response feature recognition. By identifying the muscle activity brain response feature and the resting state brain response feature, the target instructions corresponding to the muscle activity and the target instructions corresponding to the resting state can be increased, further increasing the number of instructions, thereby expanding the degree of freedom of system control.

[0015] In an implementation of the first aspect, the static state instruction further includes retaining the original action of the previous instruction under the condition that the operation instruction, the movement instruction and the control instruction are not executed.

[0016] In this implementation, the resting state instruction is a very important instruction type, which can be implemented in conjunction with other types of instructions. While improving the execution effect of other types of instructions, it can also relieve the visual control fatigue of the subject.

[0017] In an implementation of the first aspect, a viewing direction signal of a subject is collected based on an inertial measurement unit; command recognition is performed on the viewing direction signal, and a target instruction represented by the viewing direction signal is obtained as a viewing direction instruction and / or a direction instruction; the viewing direction instruction and / or the direction instruction can be executed simultaneously with the operation instruction, the movement instruction, the control instruction or / and the resting state instruction.

[0018] In this implementation, the dimension of the target instruction can be further enriched by introducing the viewing direction signal collected by the inertial measurement unit, thereby achieving more control degrees of freedom of the target instruction.

[0019] In an implementation of the first aspect, feature recognition is performed on the brain signal based on a response feature recognition model to obtain brain response features contained in the brain signal, including: acquiring a training data set, wherein the training data set includes brain signals generated by a subject under various stimulation states; performing feature extraction on the training data set to obtain a response feature template data set; and inputting the response feature template data set and the brain signal into a response feature recognition model for processing to obtain brain response features of the brain signal.

[0020] In an implementation of the first aspect, the response feature recognition model includes a first spatial filter, a correlation coefficient unit and a response feature output unit; the first spatial filter performs filtering processing on a preset response feature template data set to obtain a response feature filtering data set; the first spatial filter performs filtering processing on the brain signal to obtain a first filtering feature; the correlation coefficient unit calculates the correlation value between the filtering feature and each response feature filtering data in the response feature filtering data set, and determines that the response feature corresponding to the response feature filtering data with the largest correlation value is the brain response feature of the brain signal; the response feature output unit outputs the brain response feature.

[0021] In this implementation, the target response features in the brain signal are accurately identified through the response feature recognition model, while non-target response features are suppressed, thereby providing a basic reference for the subsequent implementation of more accurate target command recognition.

[0022] In an implementation of the first aspect, the brain signal is subjected to instruction recognition under the brain response feature based on a target instruction recognition model to obtain the target instruction represented by the brain signal; the target instruction recognition model includes a second spatial filter, a correlation processing unit, a classifier and an instruction output unit; the second spatial filter performs filtering processing on the response feature template data corresponding to the brain response feature to obtain corresponding response feature filtered data; the second spatial filter performs filtering processing on the brain signal to obtain second filtered data; the correlation processing unit calculates and obtains a coefficient vector of the second filtered data and the response feature filtered data; the classifier classifies the coefficient vector to identify the target instruction represented by the brain signal; and the instruction output unit outputs the target instruction.

[0023] In this implementation, based on the accurate identification of the target response features in the brain signal by the response feature recognition model, the target command recognition model is further used to more accurately identify the target command in the brain signal, thereby improving the accuracy of command recognition and greatly improving the information transmission rate of the target command.

[0024] In an implementation of the first aspect, a response feature analysis is performed on the acquired training data set to obtain a spatial filter; the response feature analysis includes task discriminant component analysis, task-related component analysis or canonical correlation analysis.

[0025] In the second aspect, the present application provides a brain-computer interface device, including: an input interface module, which obtains brain signals generated by a subject under any stimulation state; a response feature recognition model, which performs response feature recognition on the brain signal to obtain the stimulation response feature contained in the brain signal; a target instruction recognition model, which performs instruction recognition under the response feature on the brain signal to obtain the target instruction represented by the brain signal; and an output interface module, which outputs the target instruction to control an external controlled device.

[0026] In a third aspect, the present application provides an electronic device comprising: one or more processors; and one or more memories, wherein the memories store computer readable codes, and when the computer readable codes are executed by the one or more processors, the control method of the brain-computer interface device as described above is implemented.

[0027] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the control method of the brain-computer interface device as described above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Shown is a schematic diagram of an exemplary application scenario of the brain-computer interface-based control method described in an embodiment of the present application.

[0029] Figure 2 Shown is an exemplary flow chart of a method for controlling a brain-computer interface device as described in an embodiment of the present application.

[0030] Figure 3A Shown is an exemplary flow chart of step S202 of the control method described in an embodiment of the present application.

[0031] Figure 3B Display as Figure 3A A schematic diagram of an exemplary result of the control method.

[0032] Figure 3C Another exemplary flow chart of step S202 of the control method described in an embodiment of the present application is shown.

[0033] Figure 4A-1 Shown is an exemplary flow chart of step S204 of the control method described in an embodiment of the present application.

[0034] Figure 4A-2Shown is a schematic diagram of the 3-state Kalman filter described in an embodiment of the present application.

[0035] Figure 4A-3 Shown is a schematic diagram of the 8-state Kalman filter described in an embodiment of the present application.

[0036] Figure 4A-4 Shown is a schematic diagram of brain signal filtering during a specific brain response period described in an embodiment of the present application.

[0037] Figure 4B-1 Shown is an exemplary flow chart of step S410 of the control method described in an embodiment of the present application.

[0038] Figure 4B-2 Shown is an exemplary schematic diagram of the increase / decrease in execution probability described in an embodiment of the present application.

[0039] Figure 4C-1 and Figure 4C-2 An exemplary schematic diagram showing the execution mode of the target instruction described in an embodiment of the present application is shown.

[0040] Figure 5 Shown is another exemplary flow chart of the control method of the brain-computer interface device described in an embodiment of the present application.

[0041] Figure 6 Shown is an exemplary structural diagram of the response feature recognition model described in an embodiment of the present application.

[0042] Figure 7 Shown is an exemplary structural diagram of the target instruction recognition model described in an embodiment of the present application.

[0043] Fig. 8A Shown is an exemplary flow chart of the method for online decoding, analysis and identification of brain signals described in an embodiment of the present application.

[0044] Figure 8B Display as Fig. 8A A schematic diagram of a recognition result of the method.

[0045] Figure 8C Display as Fig. 8A Another schematic diagram of recognition results of the method.

[0046] Fig. 9 Shown is an exemplary structural diagram of the brain-computer interface device described in an embodiment of the present application.

[0047] Fig.10 Shown is an exemplary structural diagram of an electronic device described in an embodiment of the present application.

[0048] Component number description

[0049] 110 External Stimulation Module

[0050] 120 Brain signal acquisition module

[0051] 130 Brain Signal Processing Module

[0052] 140 conversion control module

[0053] 150 External controlled devices

[0054] 600 Response Feature Identification Model

[0055] 610 First spatial filter

[0056] 620 Correlation coefficient unit

[0057] 630 Response characteristic output unit

[0058] 700 Target Instruction Recognition Model

[0059] 710 Second spatial filter

[0060] 720 related processing units

[0061] 730 Classifier

[0062] 740 Command output unit

[0063] 900 Brain-computer interface devices

[0064] 910 Input Interface Module

[0065] 920 Response Feature Identification Model

[0066] 930 Target Instruction Recognition Model

[0067] 940 Output Interface Module

[0068] 100 Electronic devices

[0069] Steps S201 to S204

[0070] Steps S301 to S304

[0071] Steps S410 to S440

[0072] Steps S411 to S414

[0073] Steps S501 to S503 DETAILED DESCRIPTION

[0074] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0075] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0076] Brain-Computer Interface (BCI) is a technology that establishes a direct connection between the human or animal brain and an external controlled device (such as a computer, robot, etc.) to achieve information interaction and functional integration between the nervous system and the external controlled device. Brain-computer interface technology includes invasive brain-computer interface and non-invasive brain-computer interface. Invasive brain-computer interface obtains signals by implanting electrodes into the brain. This method can obtain more accurate neural signals, but there are high safety risks and technical barriers. Non-invasive brain-computer interface records brain activity from the surface of the head through a scalp wearable device, without the need for surgery and device implantation. This technology is relatively safe, reduces the risk of brain surgery, and can be used to treat a variety of neurological diseases and manage biological health. However, the existing BCI system has problems such as slow command output and false touch of command output, which limits its practical application in related fields.

[0077] The technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application.

[0078] like Figure 1 As shown, it shows a schematic diagram of an application scenario of the present application. The control scenario based on the brain-computer interface includes: an external stimulation module 110, a brain signal acquisition module 120, a brain signal processing module 130, a conversion control module 140 and an external controlled device 150.

[0079] The external stimulation module 110 refers to an object that outputs specific stimulation, not limited to environment, device or / and equipment. The types of stimulation include physical stimulation such as sound, touch, smell and vision, as well as mental stimulation such as emotion, thought and feeling.

[0080] The brain signal acquisition module 120 is configured to collect brain signals generated by the subject (human or animal) under the action of specific stimulation. Brain signals include electroencephalogram (EEG), magnetoencephalogram (MEG), functional magnetic resonance imaging (fMRI), functional near-infrared spectroscopy (fNIRS) or / and neuronal spikes and local field potentials (LFP). Among them, brain waves are brain electrical activity waves with spatiotemporal characteristics formed by the synchronous discharge of local groups of neurons in the brain. According to the frequency, brain waves can be divided into the following main categories: Delta waves have a frequency of 0.5 to 3 Hz and mainly appear in the deep sleep stage; Theta waves have a frequency of 4 to 7 Hz and are related to meditation, drowsiness, hypnosis or sleep; Alpha waves have a frequency of 8 to 13 Hz and mainly appear when awake, quiet and with eyes closed; Beta waves have a frequency of 13 to 34 Hz ​​and are related to active and busy thinking activities; Gamma waves have a frequency of more than 35 Hz and are related to cognitive processing and perception. Magnetoencephalogram (MEG) is a signal that reflects changes in the brain's magnetic field and can provide similar temporal resolution to EEG, but with higher spatial resolution. Functional magnetic resonance imaging (fMRI) indirectly reflects neural activity by measuring changes in blood oxygen levels caused by brain activity. It has high spatial resolution but relatively low temporal resolution. Functional near-infrared spectroscopy (fNIRS) is a non-invasive brain imaging technology that monitors neural activity by measuring changes in hemoglobin concentration caused by brain activity. Neuronal spikes and local field potentials (LFP) are commonly used signal types in invasive or semi-invasive brain-computer interfaces and can provide more direct information on neuronal activity.

[0081] For example: a stimulation interface is used to present SSVEP stimulations of various frequencies (8.5-12Hz, step length 0.5Hz). When the subject is looking at the stimulation interface, the brain signal acquisition module collects his physiological electrical signals (collectively referred to as brain signals). When the subject looks at the stimulation interface of a specific frequency, the brain will produce a corresponding brain response, which is manifested in the brain signal as an increase in the amplitude of a specific frequency component. The response amplitude refers to the intensity of a specific frequency component in a brain signal, which can reflect the intensity of the subject's response to the stimulus.

[0082] The brain signal processing module 130 is configured to process and analyze the collected brain signals, extract effective features from the brain signals, identify the effective features to obtain brain intention information for subsequent applications.

[0083] The brain control module 140 is configured to convert the brain intention information identified by the brain signal processing module 130 into a specific intention instruction, and control the external controlled device 150 through the intention instruction, so that the external controlled device 150 performs the action corresponding to the intention instruction. The brain control module outputs a specific instruction, and controls the virtual keyboard or handle through the specific instruction to realize the control of the external controlled device (such as a computer terminal device, a game terminal device).

[0084] like Figure 2 As shown, an embodiment of the present application provides a control method for a brain-computer interface device, comprising the following steps S201 to S204.

[0085] S201, obtaining brain signals generated by the subject under any stimulation state.

[0086] In one implementation of this embodiment, brain signals may be generated based on a BCI paradigm. A BCI paradigm refers to a set of specific psychological tasks or external stimuli carefully selected / designed by BCI developers under specific brain imaging technology to represent the intention of a subject or user (hereinafter collectively referred to as a user).

[0087] In one implementation of this embodiment, the stimulus state includes any one or more specific stimuli such as sound, touch, smell, or / and vision, or / and mental stimuli such as emotion, thought, and feeling. Brain signals are formed in response to specific stimuli, so brain signals will contain response characteristics corresponding to specific stimuli.

[0088] S202, performing feature recognition on the brain signal to obtain brain response features included in the brain signal.

[0089] In an implementation of this embodiment, the brain response feature may be a frequency response feature, a phase response feature, or / and an amplitude response feature, etc. The brain response feature includes a visual brain response feature and a non-visual brain response feature; the non-visual brain response feature includes a muscle activity brain response feature, a resting state brain response feature, or other brain-like response features.

[0090] S203, performing command recognition on the brain signal under the brain response characteristics to obtain the target command represented by the brain signal.

[0091] In one implementation of this embodiment, if the brain response feature is a visual brain response feature, the identified instructions may include at least 8 types; if the brain response feature is a muscle activity brain response feature, the identified instructions may include at least 5 types, such as clenching teeth, blinking, closing eyes, coughing or resting, etc.; if the brain response feature is a resting state brain response feature, the identified instructions include at least 1 type, such as resting state instructions. Based on the above three brain response features, instruction encoding with at least 14 degrees of freedom can be achieved.

[0092] S204: configuring a preset execution probability based on the target instruction, and outputting the target instruction based on the execution probability to control an external controlled device.

[0093] This embodiment can extract the response characteristics of brain signals to specific stimuli, identify the target instructions reflecting the brain's intention in the response characteristics, and output the target instructions to control the external controlled device. This embodiment can identify the target instructions from the brain signals in real time by identifying the response characteristics of the brain signals and the instructions under the response characteristics, and output the filtered target instructions in real time to control the external controlled device, which can meet the needs of application fields with high real-time requirements.

[0094] In one embodiment of the present application, the brain response features include visual brain response features and non-visual brain response features; the non-visual brain response features include muscle activity brain response features and resting state brain response features. Figure 3A As shown, the control method of the brain-computer interface device described in the embodiment of the present application may also include steps S301 to S303.

[0095] S301, identifying the brain signal based on the visual brain response feature, and obtaining the operation instructions and / or movement instructions represented by the brain signal.

[0096] S302, identifying the brain signal based on the muscle activity brain response characteristics, and obtaining movement instructions and / or control instructions represented by the brain signal; wherein the target instructions corresponding to the muscle activity brain response characteristics include preset instructions obtained by any non-visually induced actions such as clenching teeth, blinking, closing eyes, coughing or resting; wherein each non-visually induced action corresponds to a preset instruction.

[0097] S303, identifying the brain signal based on the resting-state brain response characteristics, and obtaining the resting-state instruction represented by the brain signal; wherein the resting-state instruction refers to an instruction not to execute the operation-type instruction, the movement-type instruction and the control-type instruction; the control-type instruction realizes the start, stop and / or switching of the execution of other types of instructions.

[0098] In this embodiment, operation instructions and / or movement instructions can be obtained based on the visual brain response feature recognition, movement instructions and / or control instructions can be obtained based on the muscle activity brain response feature recognition, and resting state instructions can be obtained based on the resting state brain response feature recognition. Operation instructions include commands for operations such as walking and attacking, movement instructions include commands for moving in various directions such as forward, backward, left, and right, control instructions include commands for control such as starting, stopping, switching, etc., and resting state instructions are equivalent to no instructions.

[0099] For example: Example Figure 3B Provided is a method for implementing the recognition of the brain signal based on visual brain response features, muscle activity brain response features and resting state brain response features, including: collecting 2 seconds of real-time brain signals (step length 20 milliseconds) by a brain signal acquisition terminal. After collecting the brain signal, response feature recognition is performed. If the resting state brain response feature is recognized, the resting state instruction (i.e., the target instruction of the specific feature) can be directly output; if the muscle activity response feature is recognized, the muscle activity (such as clenching teeth) state instruction (i.e., the target instruction of the specific feature) can be output; if the visual brain response feature is recognized, it can be determined as an SSVEP instruction, and then further identified through a multi-instruction recognition module (i.e., the target instruction recognition model described below) A specific instruction among the multiple instructions contained in the SSVEP instruction. The target instruction is decoded into a specific external controlled device execution instruction in the form of a virtual keyboard or handle.

[0100] For example: if the target instruction is applied to game control and the external controlled device is a game device, the rest state instruction can be used to control the game character to keep executing the previous action, such as walking, and the gritting teeth instruction can be used to control the opening and closing of the entire game control system, or control the start, stop or switch of the target instruction execution, and the SSVEP instruction can be used to control the internal operations of the game, such as: light attack, heavy attack, rolling, jumping, reply, skill 1 / 2 / 3, etc.

[0101] In this implementation, operation instructions and / or movement instructions can be obtained based on the visual brain response feature recognition, movement instructions and / or control instructions can be obtained based on the muscle activity brain response feature recognition, and resting state instructions can be obtained based on the resting state brain response feature recognition. By identifying the muscle activity brain response feature and the resting state brain response feature, the target instructions corresponding to the muscle activity and the target instructions corresponding to the resting state can be increased, further expanding the degree of freedom of control, and at the same time increasing the number of target instructions that can be implemented corresponding to the visual brain response feature.

[0102] In one embodiment of the present application, the rest state instruction also includes retaining the original action of the previous instruction under the condition that the operation instruction, the movement instruction and the control instruction are not executed.

[0103] In this implementation, the resting state instruction is a very important instruction type, which can be implemented in conjunction with other types of instructions. While improving the execution effect of other types of instructions, it can also relieve the visual control fatigue of the subject. Through the implementation of resting state instructions, the fluency of control can be significantly improved, visual fatigue can be relieved, the previous action can be continued, and a short rest can be taken before the instruction is switched. For example, in game control, if no other instructions are recognized during the attack, it can be regarded as a resting state instruction to continue the attack and will not be stopped; or if a command blank appears during the transfer process and is stuck in the middle state with no output, the intermediate state can also be compensated by the resting state instruction to smoothly transition to the next instruction state, rather than passively stop.

[0104] In one embodiment of the present application, Figure 3C As shown, the control method of the brain-computer interface device described in the embodiment of the present application may also include step S304, collecting the viewing angle direction signal of the subject based on the inertial measurement unit, performing command recognition on the viewing angle direction signal, and obtaining the target instruction represented by the viewing angle direction signal as a viewing angle instruction and / or a direction instruction; the viewing angle instruction and / or the direction instruction may be executed simultaneously with the operation instruction, the movement instruction, the control instruction or / and the resting state instruction.

[0105] In one embodiment of the present application, the inertial measurement unit includes a three-axis inertial measurement unit and a six-axis inertial measurement unit. The three-axis inertial measurement unit (Inertial Measurement Unit, referred to as IMU) is used to detect and measure the motion state of an object on three spatial axes (usually X, Y, and Z axes), including acceleration, tilt, impact, vibration, rotation, and multi-degree-of-freedom motion. IMU usually includes a three-axis gyroscope, a three-axis accelerometer, and a three-axis magnetometer, which are used to measure the angular velocity, acceleration, and orientation information of the object, respectively. The six-axis inertial measurement unit (IMU) is a sensor that integrates a three-axis accelerometer and a three-axis gyroscope, and is used to measure the acceleration and angular velocity of an object in space.

[0106] For 2.5 / 3D games that require perspective switching, this application can also receive and process gyroscope data and convert it into virtual keyboard / controller signals to achieve perspective switching.

[0107] In this implementation, the dimension of the target instruction can be further enriched by introducing the viewing direction signal collected by the inertial measurement unit, thereby achieving more control degrees of freedom of the target instruction.

[0108] In one embodiment of the present application, Figure 4A-1 As shown, step S204 described in the embodiment of the present application may further include steps S410 to S440.

[0109] S410, iteratively calculating the execution probability of the identified target instruction based on the Kalman filter; the target instruction is identified once, and the corresponding iterative calculation increases by one generation;

[0110] S420, determining whether the execution probability of the target instruction is greater than or equal to a preset value;

[0111] S430, if yes, output the target instruction to control the external controlled device;

[0112] S440, if not, then the target instruction is not output.

[0113] In this implementation, due to reasons such as resting state instructions are difficult to identify and line of sight shifting easily leading to mis-touch of instructions, this embodiment utilizes a Kalman filter to set execution probability control on the output of the target instructions to achieve smooth processing of the target instructions, which can greatly reduce the mis-touch rate and can not only improve the accuracy of the execution and / or termination of the target instructions, but also can accurately control the delay of the execution and / or termination of the target instructions, so that the technical solution described in this application can effectively achieve precise control of the external controlled device.

[0114] For example: Figure 4A-2 The 3-state Kalman filter schematic diagram shown is used as an example. In order from top to bottom, the first picture at the top is the brain signal waveform diagram, the second picture is the 3-state instruction output diagram, the third picture is the 3-state Kalman filter output diagram, and the fourth picture is the instruction output diagram after 3-state filtering. Among them, the black line at the bottom of the 3-state Kalman filter output diagram is the output "threshold". Only when the filtered value exceeds the threshold, the system will determine that it is an SSVEP instruction, which greatly reduces the system's false touch rate and improves the system's stability. This can be clearly seen from the fourth 3-state filtered instruction output diagram.

[0115] For example: Figure 4A-3 The 8-state Kalman filter diagram is shown as an example. From top to bottom, the first picture at the top is the output probability diagram of each instruction after filtering, and a convex waveform represents an instruction; the second picture is the correlation coefficient diagram of each instruction. When the value of the first picture is 0, it is the stage of no instruction output or rest state. The instruction output at other times is the instruction represented by the convex waveform with the highest value at that moment.

[0116] The 8-state Kalman filter in this embodiment is applied inside the SSVEP to avoid the situation where the keys are strung inside the instructions during the instruction period. The formula is as follows:

[0117] x(t)=A1x(t-1)+A2[F(x(t),x(t-1))]+B

[0118] Among them, the output probability x(t) of the instruction is an n×1 vector, n is the total number of instructions in the system, t represents the current moment, t-1 represents the previous moment, and the value of each row of x(t) represents the probability of each SSVEP instruction being output at the current time. A1 and A2 are the control matrices of the system, which are responsible for controlling and coordinating the relationship between the previous instruction and the currently detected instruction and outputting it based on the actual situation. B is an n×1 constant vector, which is used to make subtle adjustments to the system according to the actual situation. F is the conditional function of the current instruction output probability x(t) based on the previous instruction output probability x(t-1). F means that the output probability x(t) of the current instruction is calculated based on the output probability x(t-1) of the previous instruction and various actual situations.

[0119] For example, Figure 4A-4 The schematic diagram shown is used as an example. If the diagram is during the user's resting period, the first diagram is the original classification diagram of the brain signal, and the second diagram is the Kalman filter diagram. From the comparison of the two diagrams, it can be seen that the false touch of SSVEP has disappeared and a stable resting state is maintained. Figure 4A-4 The displayed false positive rate is 0%.

[0120] like Figure 4A-4 When the user is fixating on a stimulus block, it can be seen that the delay of the output command is very low and the consistency is very high. The data that can be given at this time are: the delay of entering the command after the left vertical line is 0, and the execution rate of the command during the entire fixation period is 100%.

[0121] In one embodiment of the present application, Figure 4B-1 As shown, step S410 described in the embodiment of the present application may further include steps S411 to S414.

[0122] S411, determining whether the target instruction recognized in the current generation is the same as the target instruction recognized in the previous generation;

[0123] S412, if the two are the same, the execution probability of the contemporary target instruction increases;

[0124] S413, if different, the execution probability of the contemporary target instruction decreases;

[0125] S414, updating the execution probability of the target instruction to the execution probability of the current target instruction.

[0126] In this implementation, by adjusting the probability of execution of the target instruction upward or downward, the accuracy and effectiveness of the target instruction output can be further improved. A separate upward / downward adjustment strategy can be set for each target instruction, which can be further matched with the actual control scenario of the external controlled device. It can be flexibly configured according to the actual situation of the external controlled device, and the effectiveness of the brain-computer interface control device is not affected by changes in the external controlled device. It has a wide range of applications and is more versatile. For example, it is also possible to achieve the effect of fast rise and slow exit for a certain target instruction, which can be customized according to personal habits or system needs.

[0127] In one embodiment of the present application, Figure 4B-2 As shown, the curve on the left represents the process of the execution probability increasing, and the curve on the right represents the process of the execution probability decreasing. The difference between them is achieved by selecting the "increasing" probability function and the "decreasing" probability function respectively, wherein the delay degree achieved by the "increasing" probability function and the "decreasing" probability function is controlled by the calculation parameters in the probability function.

[0128] In one embodiment of the present application, the accuracy of the execution and / or termination of the target instruction is controlled based on a probability function; the delay of the execution and / or termination of the target instruction is controlled based on the calculation parameters of the probability function; the probability function includes an increasing probability function and a decreasing probability function; the execution probability of the contemporary target instruction is controlled to increase based on the increasing probability function; the execution probability of the contemporary target instruction is controlled to decrease based on the decreasing probability function; each target instruction corresponds to its own probability function.

[0129] In one embodiment of the present application, Figure 4C-1 and Figure 4C-2 As shown, it shows a schematic diagram of two modes of target instruction execution. This embodiment controls the "mode" of target instruction execution through a probability function, and the probability function can be flexibly set as needed, which can be a curve mode, a straight line mode, or any other required linear mode. This embodiment controls the "steepness" of the display curve of the probability function through the calculation parameters in the probability function. If it is particularly steep, the execution delay of the target instruction will be low (because the speed of reaching the threshold will be fast), and if it is particularly flat, the execution delay of the target instruction will be high (because the speed of reaching the threshold will be slow).

[0130] Figure 4C-1 A schematic diagram of a mode of starting execution of a target instruction is shown. The black curve in the figure is the execution mode curve of the target instruction, indicating that the target instruction is executed smoothly with a slightly higher execution delay.

[0131] Figure 4C-2Another schematic diagram of the start of execution of the target instruction is shown. The black curve in the figure is the execution mode curve of the target instruction, indicating that the target instruction is executed abruptly with a low execution delay.

[0132] Different execution mode curves allow users to customize according to their own circumstances, so that the system can better fit the habits of different users and achieve a balance between fewer false touches and fast response according to their own circumstances.

[0133] In this implementation, by setting the probability function and the calculation parameters, the problem of "false touch" in the output of the target instruction in practical applications can be solved. Taking game control as an example, the false touch phenomenon refers to the unexpected operation of the game character at certain moments. The source of this false touch phenomenon is: in reality, there will always be a state that is misjudged as another state. In order to avoid the occurrence of false touches, this embodiment can realize the matching design of the output of the target instruction and the game control system based on the Kalman filter. The matching design will consider the relationship between the current output instruction and the previous multiple executed instructions, and finally the output of the target instruction is performed.

[0134] In one embodiment of the present application, step S202 may perform feature recognition on the brain signal based on a response feature recognition model to obtain the brain response feature contained in the brain signal. Figure 5 As shown, the control method of the brain-computer interface device described in the embodiment of the present application may also include steps S501 to S503.

[0135] S501, obtaining a training data set, wherein the training data set includes brain signals generated by a subject under various stimulation states;

[0136] S502, extracting features from the training data set (for example, superimposing and averaging the same response feature data) to obtain a response feature template data set;

[0137] S503: Input the response feature template data set and the brain signal into a response feature recognition model for processing to obtain brain response features of the brain signal.

[0138] In one embodiment of the present application, Figure 6 As shown, the response feature recognition model 600 may include a first spatial filter 610 , a correlation coefficient unit 620 and a response feature output unit 630 .

[0139] The first spatial filter 610 performs filtering processing on the preset response feature template data set to obtain a response feature filtered data set.

[0140] The first spatial filter 610 performs filtering processing on the brain signal to obtain a first filtering feature.

[0141] The correlation coefficient unit 620 calculates the correlation value between the filter feature and each response feature filter data in the response feature filter data set, and determines that the response feature corresponding to the response feature filter data with the largest correlation value is the brain response feature of the brain signal.

[0142] The response feature output unit 630 outputs the brain response feature.

[0143] This embodiment accurately identifies the target response features in brain signals through the response feature recognition model, while suppressing non-target response features, thereby providing a basic reference for subsequent more accurate target command recognition.

[0144] In one embodiment of the present application, step S203 may perform command recognition on the brain signal under the brain response feature based on a target command recognition model to obtain the target command represented by the brain signal. Figure 7 As shown, the target instruction recognition model 700 includes a second spatial filter 710 , a correlation processing unit 720 , a classifier 730 and an instruction output unit 740 .

[0145] The second spatial filter 710 performs filtering processing on the response feature template data corresponding to the brain response feature to obtain corresponding response feature filtered data.

[0146] The second spatial filter 710 performs filtering processing on the brain signal to obtain second filtered data.

[0147] The correlation processing unit 720 calculates and obtains the coefficient vector of the second filtered data and the response characteristic filtered data.

[0148] The classifier 730 classifies the coefficient vector and identifies the target instruction represented by the brain signal.

[0149] The instruction output unit 740 outputs the target instruction.

[0150] Based on the accurate identification of the target response features in the brain signals by the response feature identification model, this embodiment further more accurately identifies the target commands in the brain signals through the target command identification model, thereby greatly improving the information transmission rate of the target commands in the control method of the brain-computer interface device.

[0151] In one embodiment of the present application, the first spatial filter 610 and the second spatial filter 710 have the same structure and function and are the same type of spatial filters. The spatial filter acquisition process includes: performing response feature analysis on the acquired training data set to obtain the spatial filter; the response feature analysis includes task discriminant component analysis, task-related component analysis or canonical correlation analysis; the training data set includes brain signals generated by the subject under various stimulation states.

[0152] For example: Example Fig. 8A A specific implementation diagram of online decoding and analysis of brain signals is provided, including: the process before online decoding and analysis of brain signals and the process during online decoding and analysis of brain signals, which are described in detail as follows:

[0153] Before online decoding and analysis of brain signals, template data of each frequency are first generated based on the data set collected by offline experiments (i.e., training data set). Then, the spatial filter w is trained using the Task-Discriminant Component Analysis (TDCA) algorithm. Finally, the linear discriminant analysis model corresponding to each frequency is generated by combining the spatial filter w and the training data set.

[0154] During the online decoding and analysis process, the response characteristics of the brain signal to be tested are first identified, and then the target instructions of the brain signal to be tested under the response characteristics are identified. In this way, the target instructions represented by the brain signal can be identified very accurately and quickly. The specific process includes:

[0155] 1) For a section of test data to be detected (i.e., brain signal), by calculating the correlation coefficient between the template data and the test data after spatial filtering, the correlation matching with each frequency template can be obtained. The larger the correlation coefficient, the greater the correlation, and it indicates that the test data is very likely to belong to the EEG response induced by the steady-state data stimulation of the instruction corresponding to the template data. By selecting the largest correlation coefficient, the encoding frequency of the target instruction can be determined.

[0156] 2) The target command is identified in each target frequency. The identification method extracts features from the test data and the target frequency template data after spatial filtering, and uses the linear discriminant analysis model corresponding to the target frequency to identify and classify the extracted features. In the feature extraction, five features are selected, namely, the correlation coefficient between the template data and the test data after spatial filtering, the mutual correlation coefficient between the template data and the test data after spatial filtering, the frequency domain amplitude at the encoding frequency of the test data after spatial filtering, the sum of the high-frequency frequency domain amplitudes of the test data after spatial filtering (including 55-72Hz), and the time domain variance of the test data after spatial filtering. Among them, the three features of the correlation coefficient, the mutual correlation coefficient and the frequency domain amplitude at the encoding frequency reflect the response characteristics of the Steady-State Visual Evoked Potentials (SSVEP) in the brain signal, and the two features of the sum of the high-frequency frequency domain amplitudes and the time domain variance reflect the response characteristics of the clenching of teeth in the brain signal.

[0157] For example, taking the decoding analysis of 12 rounds of offline experimental data of a subject as an example, the recognition results are as follows: Figure 8B As shown in the figure, the recognition results of 10 categories are shown. The collected offline experimental data are divided into training and test data sets according to the leave-one-out cross-validation method. Figure 8B Labels 1-8 correspond to the instructions for frequencies 8.5-12 Hz (i.e., SSVEP instructions), label 9 corresponds to the resting instructions, and label 10 corresponds to the teeth-gripping instructions. Figure 8B The displayed results show that the accuracy of target command recognition (ACC) can reach 96.6%. When the induced SSVEP response is weak, the command of a certain coding frequency is easily recognized as the command of resting coding.

[0158] For example, taking the decoding analysis of 12 rounds of offline experimental data of a subject as an example, the recognition results are as follows: Figure 8C As shown, the recognition results of 8 categories are displayed. Figure 8C Labels 1-8 correspond to the instructions for frequencies 8.5-12 Hz, i.e., 8 SSVEP instructions. Figure 8C It can be seen from the displayed results that the accuracy of target command recognition (ACC) can reach 95.8%.

[0159] Figure 8B and Figure 8CThe displayed confusion matrix is ​​a classification model evaluation tool that can intuitively show the relationship between the model's predicted results (Predicted label) and the true label (True label). The confusion matrix can implement multi-class classification and intuitively show the performance of the model by comparing the distribution of predicted labels and true labels. In multi-classification problems, the confusion matrix becomes an n×n matrix, where n is the number of categories. Each row of the confusion matrix represents the actual category and the column represents the predicted category. Each element (i, j) in the confusion matrix represents the number of samples that are actually class i but are predicted to be class j.

[0160] Depend on Figure 8B and Figure 8C The displayed results show that the recognition accuracy of frequency coding instructions in this embodiment is 95.8%, and the accuracy of overall instruction recognition after including resting and clenching teeth is 96.6%. It can be seen that after including resting and clenching teeth instructions, this embodiment can increase the number of system instructions while ensuring the recognition accuracy, thereby improving the system's degree of freedom.

[0161] In one implementation of the present application, the present application embodiment provides a brain-computer interface device, such as Fig. 9 As shown, the brain-computer interface device 900 includes an input interface module 910, a response feature recognition model 920, a target instruction recognition model 930, and an output interface module 940. The input interface module 910 obtains the brain signal generated by the subject under any stimulation state; the response feature recognition model 920 performs response feature recognition on the brain signal to obtain the stimulation response feature contained in the brain signal; the target instruction recognition model 930 performs instruction recognition under the response feature on the brain signal to obtain the target instruction represented by the brain signal; the output interface module 940 outputs the target instruction to control the external controlled device.

[0162] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0163] In one implementation of the present application, an embodiment of the present application provides an electronic device, such as Fig.10As shown, the electronic device includes one or more processors and one or more memories; the memory stores a computer-readable code, and when the computer-readable code is executed by the one or more processors, the control method of the brain-computer interface device as described above is implemented. In addition, the electronic device may also include conventional electronic devices such as an I / O interface and a communication module, which will not be elaborated here.

[0164] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0165] In one implementation of the present application, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the control method of the brain-computer interface device as described above is implemented.

[0166] A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing a processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above-mentioned storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid-state disk (SSD)), etc.

[0167] In an implementation of the present application, the present application embodiment may also provide a computer program product, the computer program product including one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer or data center. When the computer program product is executed by a computer, the computer executes the method described in the aforementioned method embodiment. The computer program product may be a software installation package, and in the case where the aforementioned method is required, the computer program product may be downloaded and executed on a computer.

[0168] In the several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules or units, which can be electrical, mechanical or other forms.

[0169] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0170] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A control method for a brain-computer interface device, characterized in that: include: Obtain brain signals generated by the subject under any stimulation state; Performing feature recognition on the brain signal to obtain brain response features contained in the brain signal; Performing command recognition on the brain signal under the brain response feature to obtain a target command represented by the brain signal; A preset execution probability is configured based on the target instruction, and the target instruction is output based on the execution probability to control an external controlled device.

2. The control method of the brain-computer interface device according to claim 1, characterized in that: Also includes: Iteratively calculating the execution probability of the identified target instruction based on the Kalman filter; the target instruction is identified once, and the corresponding iterative calculation increases by one generation; Determining whether the execution probability of the target instruction is greater than or equal to a preset value; If yes, output the target instruction to control the external controlled device; If not, the target instruction is not output.

3. The control method of the brain-computer interface device according to claim 1, characterized in that: Also includes: Determine whether the target instruction identified in the current generation is the same as the target instruction identified in the previous generation; If they are the same, the execution probability of the contemporary target instruction increases; If they are different, the execution probability of the contemporary target instruction decreases; The execution probability of the target instruction is updated to the execution probability of the current target instruction.

4. The control method of the brain-computer interface device according to claim 3, characterized in that: Also includes: Controlling the accuracy of execution and / or termination of the target instruction based on a probability function; controlling the delay of the execution and / or termination of the target instruction based on the calculated parameters of the probability function; The probability function includes an increasing probability function and a decreasing probability function; based on the increasing probability function, the execution probability of the contemporary target instruction is controlled to increase; based on the decreasing probability function, the execution probability of the contemporary target instruction is controlled to decrease; each target instruction corresponds to its own probability function.

5. The control method of the brain-computer interface device according to claim 1, characterized in that: Also includes: The brain response characteristics include visual brain response characteristics and non-visual brain response characteristics; the non-visual brain response characteristics include muscle activity brain response characteristics and resting state brain response characteristics; The target instruction recognition model recognizes the brain signal based on the visual brain response feature to obtain the operation instruction and / or movement instruction represented by the brain signal; The target instruction recognition model recognizes the brain signal based on the muscle activity brain response characteristics to obtain movement instructions and / or control instructions represented by the brain signal; The target instruction recognition model recognizes the brain signal based on the resting state brain response characteristics to obtain the resting state instruction represented by the brain signal; Among them, the static state instruction refers to an instruction that does not execute the operation instruction, the movement instruction and the control instruction; the control instruction realizes the start, stop and / or switching of the execution of other types of instructions.

6. The control method of the brain-computer interface device according to claim 5, characterized in that: The static state instruction also includes retaining the original action of the previous instruction under the condition that the operation instruction, the movement instruction and the control instruction are not executed.

7. The control method of the brain-computer interface device according to claim 5, characterized in that: Also includes: Collecting the subject's visual direction signal based on an inertial measurement unit; The viewing direction signal is subjected to command recognition, and the target command represented by the viewing direction signal is obtained as a viewing direction command and / or a direction command; the viewing direction command and / or the direction command can be executed simultaneously with the operation command, the movement command, the control command or / and the rest state command.

8. The control method of the brain-computer interface device according to claim 1, characterized in that: Also includes: Performing feature recognition on the brain signal based on the response feature recognition model to obtain brain response features contained in the brain signal includes: Acquire a training data set, wherein the training data set includes brain signals generated by a subject under various stimulation states; Performing feature extraction on the training data set to obtain a response feature template data set; The response feature template data set and the brain signal are input into a response feature recognition model for processing to obtain the brain response feature of the brain signal.

9. The control method of the brain-computer interface device according to claim 8, characterized in that: The response feature recognition model includes a first spatial filter, a correlation coefficient unit and a response feature output unit; The first spatial filter performs filtering processing on a preset response feature template data set to obtain a response feature filtered data set; The first spatial filter performs filtering processing on the brain signal to obtain a first filtering feature; The correlation coefficient unit calculates the correlation value between the filter feature and each response feature filter data in the response feature filter data set, and determines that the response feature corresponding to the response feature filter data with the largest correlation value is the brain response feature of the brain signal; The response feature output unit outputs the brain response feature.

10. The control method of the brain-computer interface device according to claim 9, characterized in that: Also includes: Performing command recognition on the brain signal under the brain response characteristics based on a target command recognition model to obtain the target command represented by the brain signal; The target instruction recognition model includes a second spatial filter, a correlation processing unit, a classifier and an instruction output unit; The second spatial filter performs filtering processing on the response feature template data corresponding to the brain response feature to obtain corresponding response feature filtering data; The second spatial filter performs filtering processing on the brain signal to obtain second filtering data; The correlation processing unit calculates and obtains a coefficient vector of the second filter data and the response characteristic filter data; The classifier classifies the coefficient vector to identify the target instruction represented by the brain signal; The instruction output unit outputs the target instruction.

11. The control method of the brain-computer interface device according to claim 9 or 10, characterized in that: Also includes: Perform response characteristic analysis on the acquired training data set to obtain a spatial filter; The response characteristic analysis includes task discriminant component analysis, task-related component analysis or canonical correlation analysis.

12. A brain-computer interface device, characterized in that: include: An input interface module is used to obtain brain signals generated by the subject under any stimulation state; A response feature recognition model is used to perform response feature recognition on the brain signal to obtain stimulus response features contained in the brain signal; a target instruction recognition model, which performs instruction recognition on the brain signal under the response characteristics to obtain the target instruction represented by the brain signal; The output interface module outputs the target instruction to control the external controlled device.

13. An electronic device, characterized in that: include: one or more processors; and One or more memories, wherein the memories store computer readable codes, and when the computer readable codes are executed by the one or more processors, the control method according to any one of claims 1 to 11 is implemented.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the control method according to any one of claims 1 to 11 is implemented.

Images