Asynchronous brain-computer interface switch control method based on periodic visual fixation

Abstract

The application discloses a kind of asynchronous brain-computer interface switch control methods based on periodic visual fixation, it is related to the field of automatic control, the method includes the following steps: performing periodic fixation visual stimulation task, generates periodically changed electroencephalogram signal;Collect scalp electroencephalogram signal, signal amplification is carried out to electroencephalogram signal, analog-digital conversion and filtering processing;Characteristic signal containing fixation visual stimulation state is extracted by decoding algorithm;Characteristic signal is converted into the numerical value of virtual force;Virtual force is applied to virtual dynamics system, drives virtual dynamics system state change and oscillation;When the state of virtual dynamics system reaches the set threshold, brain-computer interface switch output instruction is triggered, and the control of brain-computer interface switch is realized;Visual stimulation and virtual dynamics system state are displayed on display device.The application constructs a new switch trigger mode with interactive trigger mode, improves the reliability of switch control, and improves the practicability of brain-computer interface system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic control, and in particular to an asynchronous brain-computer interface switch control method based on periodic visual fixation. BACKGROUND

[0002] At present, the quality of life of the disabled is widely concerned by all sectors of society. Certain diseases such as motor neuron disease and other diseases can cause patients to have movement disorders, be unable to take care of themselves, and have a serious decline in quality of life. Brain-computer interface (BCI) is a technology that directly establishes an information interaction channel between the brain and an external device. Through brain-computer interface technology, disabled patients can bypass the traditional neuromuscular pathway and directly communicate and control the outside world through brain activity. Through brain-computer interface technology, disabled patients can drive wheelchairs, control robotic arms and other devices to complete some simple actions, which is expected to improve the quality of life of this group.

[0003] Steady state visual evoked potential (SSVEP) is a phenomenon that when visual stimulation is presented with a periodicity higher than 6Hz, the occipital region of the human brain will produce a periodic response phenomenon with the same frequency as the stimulation frequency and its harmonic frequency. Brain-computer interface based on steady state visual evoked potential has realized applications such as brain-computer interface typewriter, wheelchair control, robotic arm control, etc. Similar to its principle are visual evoked potential (VEP) produced by fixation visual stimulation, coding visual potential (C-VEP), P300 and other technologies that produce electroencephalogram response through fixation visual stimulation.

[0004] In order to apply the brain-computer interface system controlled by fixation visual stimulation to the life of the disabled, it is necessary to improve the effect of asynchronous brain-computer interface switch control, so that the user can turn on the brain-computer interface system as soon as possible when he or she wants to use it; and when the user is doing other things and does not want to use the brain-computer interface system, it is not easy to trigger an error and turn on the system by mistake. However, due to the low signal-to-noise ratio of electroencephalogram, the effect of brain-computer interface switch control based on fixation visual stimulation is not ideal, and such switches are difficult to effectively avoid the interference of other irrelevant actions in the user's visual attention decoding, and are easy to trigger an error when paying attention to the stimulus near the position, which limits its practicality.

[0005] Therefore, the person skilled in the art is committed to developing an asynchronous brain-computer interface switch control method based on periodic visual fixation, providing two potential visual fixation state decoding results by using a periodic fixation visual stimulation task, and mapping as an input signal to drive a virtual dynamics system, realizing multiple periodic feature fusion and attenuation by using a virtual dynamics system, constructing a trigger threshold, thereby improving the feature separability of the brain-computer interface switch, only the periodic fixation according to the rhythm will trigger the brain switch, and the active trigger speed is still at an acceptable level. SUMMARY

[0006] In view of the above defects of the prior art, the technical problem to be solved by the present application is to improve the reliability of brain-computer interface switch control based on fixation visual stimulation and reduce the probability of switch false triggering.

[0007] To achieve the above-mentioned purpose, the present application provides an asynchronous brain-computer interface switch control method based on periodic visual fixation, characterized in that the method comprises the following steps:

[0008] S101: performing a periodic fixation visual stimulation task to generate a periodically changing electroencephalogram signal;

[0009] S103: brain electrical data acquisition and preprocessing: collecting scalp electroencephalogram signals and preprocessing the scalp electroencephalogram signals, the preprocessing including signal amplification, analog-to-digital conversion and filtering processing;

[0010] S105: extracting the feature signal of the electroencephalogram signal online: extracting the feature signal containing the fixation visual stimulation state by a decoding algorithm, the fixation visual stimulation state being obtained by extracting time domain or frequency domain features, the feature signal representing the fixation state currently being performed and changing after switching the line of sight position;

[0011] S107: mapping the feature signal to a virtual force: converting the feature signal to a numerical value of the virtual force, the conversion method including filtering, baseline removal and function mapping;

[0012] S109: virtual dynamics system control: applying the virtual force to the virtual dynamics system to drive the virtual dynamics system state change and oscillation;

[0013] S111: triggering the control of the brain-computer interface switch: the state of the virtual dynamics system reaches a set threshold to trigger the output instruction of the brain-computer interface switch, thereby realizing the control of the brain-computer interface switch;

[0014] S113: visual stimulus display and virtual dynamics system state feedback: display the visual stimulus and the state of the virtual dynamics system on a display device, the visual stimulus flashes according to a frequency or sequence, and the virtual dynamics system state feedback displays the vibration amplitude of the system.

[0015] Further, the periodic gaze visual stimulus task in step S101 refers to periodically switching the line of sight position according to a certain rhythm, thereby switching the visual gaze task of gazing at the left or right visual stimulus, and the visual stimulus is a visual stimulus with distinguishable signal characteristics.

[0016] Further, the method of the visual stimulus includes at least one of steady-state visual evoked potential, coded visual evoked potential, visual evoked potential, and P300.

[0017] Further, in step S103, the EEG data is collected by a non-invasive scalp EEG collection device, which includes an EEG cap, and the EEG cap is arranged with different electrode channels at different positions of the brain region to collect scalp EEG signals at different positions, the scalp EEG signals include EEG signals generated when gazing at left and right stimuli, and the EEG signals come from the electrode channels of the occipital region.

[0018] Further, in step S103, the filtering process includes band-pass filtering and notch filtering steps, the band-pass filtering uses a Butterworth filter for 1-70Hz band-pass filtering, and the notch filtering uses a 50Hz notch filter for power frequency noise filtering.

[0019] Further, in step S105, the feature signal is extracted from the EEG signal online, including intercepting the current time window of N milliseconds before the visual-related electrode channel, and extracting the feature using canonical correlation analysis to obtain the feature signal related to the left gaze visual stimulus state and the right gaze visual stimulus state, the electrode channel includes multiple channels in the left and right brain regions, and step S105 includes the following sub-steps:

[0020] S1051: establish two sinusoidal frequency base frequencies and harmonic frequencies corresponding to the positive and negative sine signals of the visual stimulus, with a time length of N milliseconds, to form two template signal matrices;

[0021] S1052: form a current time EEG signal matrix with a time length of N milliseconds from the EEG signals of the multiple channels;

[0022] S1053: performing canonical correlation analysis on the electroencephalogram signal matrix and the template signal matrix, and obtaining correlation coefficients of the electroencephalogram signal matrix and the template signal matrix, the correlation coefficients including left correlation coefficients and right correlation coefficients, the correlation coefficients being the characteristic signals, and the characteristic signals respectively representing states of whether the left side stimulation and the right side stimulation are currently gazed.

[0023] Further, in the step S107, the left characteristic signal and the right characteristic signal are respectively filtered using a filter to reduce unnecessary interference; a function mapping method is used to extract the characteristic signal amplitude difference corresponding to the left visual stimulation and the right visual stimulation, obtain a decoding result of a current gazing visual stimulation state, and map estimation of the left gazing visual stimulation state and the right gazing visual stimulation state into a direction and an amplitude of the virtual force, and then subtract an average value of the virtual force in a predetermined time from the current virtual force to remove a baseline.

[0024] Further, in the step S109, the virtual dynamics system is composed of a physical or electrical system with oscillation and attenuation characteristics, and a response of the virtual dynamics system to an input signal is solved by a numerical integration method, and an inherent oscillation period of the virtual dynamics system is a period of the periodic gazing visual stimulation task.

[0025] Further, the virtual dynamics system is composed of a spring-mass-damper, the virtual dynamics system is an under-damped system, and the numerical integration method obtains a state of the virtual dynamics system at a current time according to a state of the system at a previous time of S milliseconds ago and according to physical quantity values of the virtual dynamics system by using an Euler method to recursively calculate the system state.

[0026] Further, in the step S111, the threshold value is a threshold value of a state quantity of the virtual dynamics system, the state quantity is a swing amplitude of the virtual dynamics system, and an output instruction of the brain-computer interface switch is a switch instruction output to an external device or a computer program.

[0027] In the preferred embodiment of the present application, compared with the prior art, the brain-computer interface switch control method based on the periodic gazing visual stimulation task fully utilizes the periodic characteristic signals generated by the periodic gazing visual stimulation task, fuses characteristic information of electroencephalogram signals in a period of time, converts a method of directly decoding a visual gazing state of a user in a traditional brain-computer interface to distinguish a task-idle state into a distinction between the periodic gazing visual stimulation task and other states, and constructs a new switch triggering mode in an interactive mode, thereby improving the reliability of switch control, realizing reliable control of a switch of an external device, and improving the practicability of a brain-computer interface system.

[0028] The concept, specific structure and generated technical effects of the present application will be further described below in combination with the drawings, so as to fully understand the purposes, features and effects of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a brain-computer interface switch control method step of a preferred embodiment of the present application;

[0030] Figure 2 is an electrode schematic diagram of a non-invasive scalp electroencephalogram acquisition device of a preferred embodiment of the present application;

[0031] Figure 3 is a virtual dynamics system and its visual stimulation and feedback interface composition schematic diagram of a preferred embodiment of the present application. DETAILED DESCRIPTION

[0032] The following describes the preferred embodiments of the present application with reference to the drawings, so as to make the technical content of the present application more clear and easy to understand. The present application can be embodied in many different forms of embodiments, and the protection scope of the present application is not limited to the embodiments mentioned herein.

[0033] In the drawings, the same components have the same reference numerals, and components with similar structures or functions have similar reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present application does not limit the size and thickness of each component. In order to make the drawing clearer, the thickness of some components is appropriately exaggerated in some places in the drawings.

[0034] As shown in the drawings, the present application provides an asynchronous brain-computer interface switch control method based on periodic visual fixation, which includes the following steps: Figure 1

[0035] S101: Perform a periodic fixation visual stimulation task to generate a periodically changing electroencephalogram signal.

[0036] ​The embodiment of the present application provides a user operation when performing a periodic visual fixation task: when a switch is not desired to be triggered, the user keeps idle or performs irrelevant matters, and does not need to pay attention to the system state; or when the switch needs to be triggered, the user performs the periodic visual fixation task, and generates a periodically changed electroencephalogram signal. The scalp electroencephalogram signal is generated by the user performing the periodic visual fixation task or keeping idle, the periodic visual fixation task refers to periodically switching the visual line position according to a certain rhythm, so as to switch the visual fixation on the left or right visual stimulation, the visual stimulation is visual stimulation which can be distinguished from each other in signal characteristics, and the rhythm of the periodic visual fixation task is determined according to the state of a virtual dynamic system. The periodic visual fixation task can select a computer screen to display rectangular white visual stimulation on the left and right sides, and flash at different frequencies respectively, so as to facilitate decoding of a steady-state visual evoked potential, and other visual stimulation forms can also be used, including coding visual evoked potential, visual evoked potential, P300 and the like. The idle state refers to a state in which the user keeps awake or performs activities irrelevant to the switch control.

[0037] S103: Electroencephalogram data acquisition and preprocessing: scalp electroencephalogram signals are collected, and the scalp electroencephalogram signals are preprocessed, the preprocessing includes signal amplification, analog-to-digital conversion and filtering processing.

[0038] The electroencephalogram data is collected by a non-invasive scalp electroencephalogram collection device, the non-invasive scalp electroencephalogram collection device includes an electroencephalogram cap, different electrode channels are arranged at different positions of the brain area, and scalp electroencephalogram signals at different positions are collected, the scalp electroencephalogram signals include brain electrical signals generated when fixing left and right sides of the stimulation, the brain electrical signals come from the electrode channels in the occipital region, the visual brain area related electrode channels include a plurality of channels in the left and right brain areas, preferably the channels of POZ and OZ and the surrounding channels, and the non-invasive scalp electroencephalogram collection device is as shown in the drawing. Figure 2

[0039] When the collected electroencephalogram data is filtered, a plurality of steps including band-pass filtering and notch filtering are included, preferably a Butterworth filter is used for 1-70Hz band-pass filtering, and a 50Hz notch filter is used for power frequency noise filtering.

[0040] S105: Online extraction of characteristic signals of electroencephalogram signals: characteristic signals containing fixation visual stimulation states are extracted by a decoding algorithm, the fixation visual stimulation states are obtained by extracting time domain or frequency domain characteristics, the characteristic signals represent the fixation state currently being performed, and change after the visual line position is switched.

[0041] ​When extracting the feature signal of the brain electrical signal online, the method includes the following steps: intercepting the time window of N milliseconds before the current time for the electrode channel related to vision, and extracting the feature by using canonical correlation analysis to obtain the feature signal related to the left and right visual stimulation states, and the electrode channel includes multiple channels in the left and right brain regions.

[0042] The method for extracting the online feature is determined according to the used visual stimulation, and if the steady-state visual evoked potential is used, the method includes the following steps:

[0043] The sine signals of the fundamental frequency and the harmonic frequency corresponding to the sine wave frequencies of the two visual stimulations are established, for example, the sine signals of the fundamental frequency and the harmonic frequency of 15 Hz and 20 Hz are established, and the time length is N milliseconds, and two template signal matrices are formed;

[0044] The brain electrical signals of multiple channels are combined to form a brain electrical signal matrix of the current time, and the time length is N milliseconds;

[0045] The brain electrical signal matrix and the template signal matrix are subjected to canonical correlation analysis to obtain two correlation coefficients of the brain electrical signal matrix and the two template signal matrices, and the left and right correlation coefficients are used as the feature signals, which respectively represent the current state of whether the user is watching the left stimulation and the right stimulation.

[0046] S107: Map the feature signal to virtual force: convert the feature signal to the numerical value of the virtual force, and the conversion method includes filtering, baseline removal and function mapping;

[0047] When filtering the feature signal, a filter is used to filter the left feature signal and the right feature signal (for example, the correlation coefficient) respectively to reduce unnecessary interference. The function mapping method is used to calculate the amplitude difference of the feature signals corresponding to the left and right stimulations, obtain the decoding result of the current visual fixation state of the user, and map the estimation of the left and right fixation states to the direction and amplitude of the virtual force. By subtracting the average value of the virtual force in the past period of time from the current virtual force, the virtual force has the statistical property of zero mean, so as to facilitate the driving of the virtual dynamics system.

[0048] S109: Virtual dynamics system control: the virtual force is applied to the virtual dynamics system to drive the state change and oscillation of the virtual dynamics system.

[0049] The virtual dynamics system is composed of a physical or electrical system with oscillation and attenuation characteristics. The state change of the dynamics system over time is virtually calculated on a computer or mobile electronic device, and the response of the dynamics system to the input signal is solved by a numerical integration method. A virtual dynamics system composed of a spring-mass-damper is selected, wherein the virtual mass is M, the spring stiffness coefficient is K, and the natural oscillation frequency of the virtual dynamics system can be obtained by the following formula:

[0050]

[0051] The natural frequency of the virtual dynamic system can be obtained by calculation, and the natural period can be obtained. In the brain-computer interface switch control, the dynamic system is an under-damped system, and the damping coefficient can be determined according to the signal quality of the user and the requirement of the false trigger frequency. In the preferred case, the influence of the damping on the oscillation period of the system can be ignored, and thus the natural oscillation period of the system will determine the period of the periodic fixation visual stimulation task in use.

[0052] The numerical integration method calculates the system state at the current time in the computer according to the system state at the last time before S milliseconds and the physical quantity value input into the system, and preferably uses the Euler method to recursively calculate the system state.

[0053] S111: Triggering control of the brain-computer interface switch: triggering the output instruction of the brain-computer interface switch when the state of the virtual dynamic system reaches a set threshold, so as to realize control of the brain-computer interface switch;

[0054] The set threshold of the virtual dynamic system is a threshold of a state quantity in the virtual dynamic system, and the state quantity is the swing amplitude of the virtual dynamic system. Preferably, the virtual dynamic system triggers detection when the swing amplitude of the virtual dynamic system reaches 1 m. The virtual dynamic system outputs the brain-computer interface switch instruction and clears the state quantity of the virtual dynamic system. The output instruction of the brain-computer interface switch is an output switch instruction to an external device or a computer program, such as controlling video playing or stopping, a television switch, etc.

[0055] S113: Visual stimulation display and virtual dynamic system state feedback.

[0056] The visual stimulation and the state of the virtual dynamic system are displayed on a display device. The visual stimulation flashes according to a frequency or a sequence, and the state of the virtual dynamic system includes various states. For a spring-mass-damper system, the state includes the swing position of the mass block, the movement speed of the mass block, the acceleration of the mass block, etc. The virtual dynamic system state feedback displays the vibration amplitude of the system.

[0057] The brain-computer interface switch control method based on the periodic fixation visual stimulation task fully utilizes the periodically changing characteristic signal generated by the periodic fixation visual stimulation task, fuses the characteristic information of the electroencephalogram signal in a period of time, changes the method of directly decoding the visual fixation state of the user in the traditional brain-computer interface to distinguish the task-idle state into the discrimination of the periodic fixation visual stimulation task and other states, and constructs a new switch trigger mode in an interactive mode, improves the reliability of the switch control, realizes reliable control of the switch of the external device, and improves the practicability of the brain-computer interface system.

[0058] AsFigure 2 As shown, the non-invasive scalp EEG acquisition device provided by the embodiment of the application is provided with a hat (EEG cap) to be worn on the head of a user, different electrode channels are arranged at different positions on the hat according to the positions of brain regions, and scalp EEG signals at different positions are acquired. The electrode positions of the EEG cap are distributed according to an extended version of the international standard 10-20 system, Figure 2 Each circle shown in the figure represents a channel, and the symbol in the circle represents the channel name. Nasion represents the nasion position (the nasion is located at the top of the nose and is flush with the eyes), and Inion represents the inion direction (the inion tip is located on the midline of the back of the head and is at the bottom of the skull). The skull circumference is measured on the cross section in the middle of the circles, and the circumference is divided into intervals of 10% and 20% to determine the positions of the electrodes. The system also specifies the naming of the electrodes at the corresponding positions, in which A represents "mastoid", C represents "central", F represents "frontal", Fp represents "frontal pole", O represents "occipital", P represents "parietal", and T represents "temporal". The electrodes deployed in the system include: mastoid electrode positions including A1 / A2, central electrode positions including Cz, C1-C6, frontal electrode positions including Fz, F1-F10, occipital electrode positions including Oz, O1-O2, parietal electrode positions including Pz, P1-P10, temporal electrode positions including T7-T10, mastoid temporal electrode positions including AFZ, AF3 / AF4 / AF7 / AF8, parietal central electrode positions including CPZ, CP1-CP6, frontal central electrode positions including FCZ, FC1-FC6, frontal pole electrode positions including FpZ, Fp1-Fp2, frontal temporal electrode positions including FT7-FT10, occipital parietal electrode positions including POZ, PO3 / PO4 / PO7 / PO8, temporal parietal electrode positions including TP7-TP10, and other electrodes including NZ and IZ.

[0059] The EEG signals acquired by the application are related to visual stimulation, and the electrode channels mainly from the occipital region are used in the switching control method proposed by the application to carry the EEG signals generated when the left and right stimuli are gazed. The EEG signals of these channels are acquired, amplified, and converted from analog to digital, and then are subjected to filtering processing, which includes 1-70Hz band-pass filtering by a Butterworth filter and power frequency noise filtering by a 50Hz notch filter.

[0060] As shown in the figure, the EEG cap is provided with a plurality of electrode channels, and the electrode channels are arranged according to the positions of brain regions. Figure 3As shown, the virtual dynamics system and its visual stimulation and feedback interface provided by the embodiment of the present application are composed of a virtual spring-mass-damper system, which is a system with oscillation and damping properties. The left part of the figure is a virtual spring with a spring stiffness coefficient of K, the middle part is a virtual mass M, and the right part is a damping system B. fin(t) is the force input to the system, and the natural period of the system is obtained from the natural frequency of the virtual dynamics system. The system in this example is an under-damped system, and the effect of damping on the oscillation period of the system can be ignored. Therefore, the natural oscillation period of the system will determine the period of the periodic fixation visual stimulation task in use. After the virtual force is updated, the system state before S milliseconds is used to perform numerical solution in a variety of numerical methods. In this example, Euler method is used for numerical integration to complete the system state calculation

[0061] The visual stimulation and some states of the virtual dynamics system are displayed to the user on the screen or LED. The state feedback of the virtual dynamics system is realized by displaying the vibration amplitude of the system, so that the user can master the current state of the virtual dynamics system in operation, and complete the periodic fixation visual stimulation task according to the rhythm and period of the system. In this embodiment, the position of the mass block of the system is represented by a triangle on the screen, and the position between the visual stimulation rectangles represents the threshold of the switch trigger, as shown in the user interface part of Figure 3 The position of the triangle is updated every S milliseconds, which is consistent with the position of the mass block of the virtual physical system, and the visual stimulation remains unchanged.

[0062] In the virtual system solved by the computer, the virtual force is applied to the virtual dynamics system to drive the system state change and oscillation. Specifically, every S milliseconds, after the value of the virtual force is updated, the current virtual force is input into the virtual system, and the system state before S milliseconds is used to perform numerical solution to calculate the system change and obtain the current state of the system.

[0063] The present application will be described in detail below in combination with the preferred embodiments of the present application.

[0064] As shown in Figure 1 The present application provides an asynchronous brain-computer interface switch control method based on periodic visual fixation, which comprises the following steps:

[0065] Step 1: The user starts the periodic fixation visual stimulation task:

[0066] When the user does not want to trigger the switch, the user keeps idle or performs irrelevant matters, and does not need to pay attention to the system state. At this time, although the system is in an online state, the swing amplitude is generally insufficient, and the output of the switch command trigger is generally not generated;

[0067] Or when the switch needs to be triggered, the user performs the periodic gaze visual stimulation task, and when the system is in a state of almost static, the periodic gaze visual stimulation is performed at the user's own pace, or when the system has a certain swing amplitude, the periodic gaze visual stimulation task is performed according to the system state to follow the system rhythm, thereby generating a periodically changing electroencephalogram signal consistent with the swing speed of the system, and the system accumulates energy to gradually reach a threshold to complete the switch triggering.

[0068] Step 2: Non-invasive scalp electroencephalogram data acquisition and preprocessing:

[0069] The scalp electroencephalogram signal of the user can be collected by a non-invasive scalp electroencephalogram acquisition device, and basic signal amplification, analog-to-digital conversion, filtering and other preprocessing are performed.

[0070] The distribution of the non-invasive scalp electroencephalogram acquisition device is shown in Figure 2 , which is by wearing a hat (electroencephalogram cap) on the user's head, arranging different electrode channels on the hat according to the brain region position, and collecting scalp electroencephalogram signals at different positions. The electrode position of the electroencephalogram cap is distributed according to the extended version of the international standard 10-20 system, wherein Figure 2 Each circle represents a channel, and the symbol in the circle represents the channel name. Nasion represents the nasion position (nasion is located at the top of the nose, level with the eyes), and Inion represents the direction of the external occipital protuberance (the tip of the external occipital protuberance is located on the midline of the back of the head, the bottom of the skull). The skull circumference is measured on the cross section in the middle of the circle, and the circumference is divided into 10% and 20% intervals to determine the position of the electrode. The system also specifies the electrode naming at the corresponding position, wherein A represents "mastoid", C represents "central", F represents "frontal", Fp represents "frontal pole", O represents "occipital", P represents "top", and T represents "temporal".

[0071] The electroencephalogram signal collected by the present application is related to visual stimulation, and the electrode channels from the occipital lobe region are used in the switch control method proposed by the present application, carrying the electroencephalogram signals generated when gazing at the left and right side stimuli. After the electroencephalogram signals of these channels are collected, amplified, and digitized, they will be filtered. The filtering process includes 1-70Hz band-pass filtering by a Butterworth filter, and 50Hz notch filtering for power frequency noise filtering.

[0072] Step 3: Online feature extraction of electroencephalogram signal:

[0073] The pre-processed scalp EEG signal is decoded to obtain the current gaze visual stimulus of the user. The current gaze visual stimulus state of the user is obtained by extracting time domain or frequency domain features. Specifically, the visual related electrode channel is intercepted at the current time window of N milliseconds (N = 400 in this example), and the features are extracted by canonical correlation analysis (CCA), thereby obtaining the feature signals related to the left and right gaze visual stimulus states. In this example, the process is performed in a sliding window, and is performed every S milliseconds (S = 31.25 in this example), with overlap with the previous intercepted time window, so that the feature signals are updated at a sufficient frequency. Similarly, other visual stimuli commonly used in the art and their improved methods are used to generate visual evoked signals, and the corresponding feature extraction of the visual stimuli is also included in the embodiments of the present application.

[0074] Step 4: Mapping the feature signals to virtual forces:

[0075] The feature signals are converted into numerical values of virtual forces by filtering, baseline removal, function mapping, etc.

[0076] In this example, the filter is used to filter the left and right feature signals (correlation coefficients) respectively to reduce unnecessary interference. Further, the function mapping method is used to extract the amplitude difference of the left and right visual stimulus corresponding feature signals, obtain the decoding result of the current gaze visual stimulus state of the user, and map the estimation of the left and right gaze visual stimulus states to the direction and amplitude of the virtual force. Then, the current virtual force is subtracted by its average value in the past 10 seconds to remove the baseline.

[0077] Step 5: Virtual dynamics system control:

[0078] In the virtual system solved by the computer, the virtual force is applied to the virtual dynamics system to drive the system state change and oscillation. Specifically, every S milliseconds, after the value of the virtual force is updated, the current virtual force is input into the virtual system, and according to the original state of the system S milliseconds ago, the numerical solution is performed to calculate the system change and obtain the current state of the system. In this example, a virtual spring-damper-mass system is used as the dynamics system, as shown in Figure 3

[0079] Step 6: System state triggering control of switch:

[0080] When the state of the virtual dynamics system reaches a set threshold, the output command of the brain-computer interface switch is triggered, thereby realizing the control of the brain-computer interface switch. In this example, the brain switch is used to control the pause and start of the video playback software.

[0081] ​Step 7: Visual stimulus display and system state feedback:

[0082] The visual stimulus and some states of the virtual dynamics system are displayed to the user on the screen or LED, etc. Preferably, the system state feedback is displayed by displaying the system vibration amplitude, so that the user can master the current state of the virtual dynamics system during operation, and facilitate the completion of the periodic fixation visual stimulus task according to the system rhythm and period. In this example, the tip of the triangle on the screen represents the position of the mass of the system, and the position between the visual stimulus rectangles represents the threshold of the switch trigger. The triangle updates the position every S milliseconds, which is consistent with the position of the mass of the virtual physical system, and the visual stimulus remains unchanged, as shown in the schematic diagram. Figure 3

[0083] In step 1, the periodic fixation visual stimulus task refers to the user periodically switching the line of sight position according to a certain rhythm, thereby switching the fixation state of the left or right visual stimulus. The selected visual stimulus is a visual stimulus with distinguishable signal characteristics in the field. In this example, using the steady-state visual evoked potential method, the user performs the visual fixation task of alternating left and right stimulus in the periodic fixation visual stimulus task, and the switching frequency and rhythm are determined according to the state of the virtual dynamics system.

[0084] In step 3, the visual-related electrodes selected in this example are: POZ, PO3, PO4, OZ, O1, O2, as shown in Figure 2 The steps of feature extraction include:

[0085] 1. Establishing sine wave frequencies of base frequency and harmonic frequency corresponding to 2 visual stimuli, for example, establishing 15Hz, 20Hz base frequency and harmonic frequency sine wave, time length N (this example takes 400) milliseconds, forming 2 template signal matrices;

[0086] 2. The EEG signal matrix of the current time is composed of the multiple channels of the EEG signal, and the time length is N (this example takes 400) milliseconds;

[0087] 3. After the EEG signal matrix and the template signal matrix are subjected to canonical correlation analysis (CCA), the correlation coefficients of the EEG signal matrix and the two template signal matrices are obtained, and the left and right side correlation coefficients are used as characteristic signals, respectively representing whether the user is currently fixating on the left stimulus and the right stimulus;

[0088] In step 4, this example uses the feature difference corresponding to the left and right visual stimuli to map the virtual force, forming a virtual force that points to the left when fixating on the left visual stimulus and points to the right when fixating on the right visual stimulus, and limiting the virtual force amplitude within a certain range to prevent noise or other interference from causing the system to fluctuate violently.

[0089] ​In step 5, the virtual dynamic system should be a system with oscillation and damping properties, and a virtual spring-mass-damper system is selected in this example, in which the virtual mass M is 4.06 kg, the spring stiffness coefficient K is 10 N / m, and the natural frequency of the virtual dynamic system is 0.25 Hz, i.e. the natural period is 4 s. The system in this example is an under-damped system, and the effect of damping on the oscillation period of the system can be ignored, so the natural oscillation period of the system will determine the period of the periodic fixation visual stimulation task in use. After the virtual force is updated, according to the original state of the system before S milliseconds, a variety of numerical methods can be used in the numerical solution process, and the Euler method is used for numerical integration in this example to complete the system state calculation.

[0090] In step 6, the threshold is the threshold of a state in the physical system, and when the system reaches or exceeds the threshold at a certain time, the system will output a switch switching instruction and clear the current state of the system. In this example, the swing amplitude of the system is selected as the control quantity, and the swing amplitude is triggered when it reaches 1 m (corresponding to the size on the screen according to screen conversion), and the swing amplitude and speed are cleared after triggering.

[0091] In step 7, the visual stimulation includes two different and distinguishable visual stimuli, so as to generate different characteristics in the electroencephalogram to facilitate the distinction of the user's fixation state of the visual stimulation. Preferably, 15 Hz is selected as the frequency of the left visual stimulation, and 20 Hz is selected as the frequency of the right visual stimulation. In this example, each stimulation is displayed as a 4x2 cm rectangle on the screen, and the brightness changes with time, with a frequency of 15 or 20 Hz, which can be represented by the formula: light(f, t) = 1 / 2{1+sin[2πft]}, wherein light represents the brightness of the stimulation with a frequency of f displayed at time t, and sin represents the sine function.

[0092] The brain-computer interface switch control method based on the periodic fixation visual stimulation task provided by the present application realizes the switch control of the video playing process through the following steps: user operation, non-invasive scalp electroencephalogram data acquisition and preprocessing, online feature extraction of electroencephalogram, feature signal mapping to virtual force, virtual dynamic system control, system state triggering control of switch, visual stimulation display and system state feedback. The use of the periodic fixation visual stimulation task generates periodic changes in characteristics, combined with the swing and damping characteristics of the virtual dynamic system, and the feature information of the electroencephalogram in a period of time, the method of directly decoding whether the user is fixating the visual stimulation state in the traditional brain-computer interface to distinguish the task-idle state is changed into the discrimination of the periodic fixation visual stimulation task and other states, and an interactive trigger mode is constructed, which improves the reliability of the switch control, realizes the reliable control of the switch of external devices, and improves the practicability of the brain-computer interface system.

[0093] The preferred embodiments of the application have been described above in detail. It should be understood that modifications and variations to the preferred embodiments could be made by those skilled in the art without departing from the spirit and scope of the application. Accordingly, it is intended that there be included within the scope of the application, all such modifications and variations as would be apparent to those skilled in the art upon reading this disclosure. It is intended to obtain for the inventors such patent rights as are available in any country on the world.

Claims

1. A switching control method for an asynchronous brain-computer interface based on periodic visual gaze, characterized in that, The method includes the following steps: S101: Performs a periodic fixation visual stimulus task, generating periodically changing EEG signals; S103: EEG data acquisition and preprocessing: Acquire scalp EEG signals and preprocess the scalp EEG signals, the preprocessing including signal amplification, analog-to-digital conversion and filtering; S105: Online extraction of feature signals from the EEG signal: The feature signal containing the gaze visual stimulus state is extracted by a decoding algorithm. The gaze visual stimulus state is obtained by extracting time-domain or frequency-domain features. The feature signal represents the gaze state currently being executed and changes after switching gaze positions. S107: Mapping the feature signal to a virtual force: converting the feature signal into a value of the virtual force, the conversion method including filtering, baseline removal, and function mapping; S109: Virtual dynamic system control: Applying the virtual force to the virtual dynamic system to drive the state changes and oscillations of the virtual dynamic system; S111: Triggering control of the brain-computer interface switch: When the state of the virtual dynamics system reaches a set threshold, the output command of the brain-computer interface switch is triggered, thereby realizing control of the brain-computer interface switch; S113: Visual stimulus display and virtual dynamic system state feedback: The visual stimulus and the state of the virtual dynamic system are displayed on a display device. The visual stimulus flashes according to frequency or sequence, and the virtual dynamic system state feedback displays the system vibration amplitude.

2. The method as described in claim 1, characterized in that, The periodic fixation visual stimulus task in step S101 refers to a visual fixation task in which the gaze position is periodically switched according to a certain rhythm, thereby switching the gaze to the left or right visual stimulus. The visual stimulus is a visual stimulus with mutually distinguishable signal features.

3. The method as described in claim 2, characterized in that, The visual stimulation method includes at least one of steady-state visual evoked potentials, encoded visual evoked potentials, visual evoked potentials, and P300.

4. The method as described in claim 3, characterized in that, In step S103, the EEG data is acquired using a non-invasive scalp EEG acquisition device. The non-invasive scalp EEG acquisition device includes an EEG cap, which has different electrode channels arranged at different locations in the brain region to acquire the scalp EEG signals at different locations. The scalp EEG signals include EEG signals generated when looking at stimuli on the left and right sides, and the EEG signals come from the electrode channels in the occipital lobe region.

5. The method as described in claim 4, characterized in that, In step S103, the filtering process includes bandpass filtering and notch filtering steps. The bandpass filtering uses a Butterworth filter to perform 1-70Hz bandpass filtering, and the notch filtering uses a 50Hz notch filter to perform power frequency noise filtering.

6. The method as described in claim 5, characterized in that, In step S105, the feature signals are extracted online from the EEG signals, including truncating the visual-related electrode channels within a time window N milliseconds before the current moment, and extracting features using canonical correlation analysis to obtain the feature signals related to the left-side gaze visual stimulus state and the right-side gaze visual stimulus state. The electrode channels include multiple channels in the left and right brain regions. Step S105 includes the following sub-steps: S1051: Establish sine and cosine signals of the fundamental and harmonic frequencies of two visual stimuli, with a duration of N milliseconds, to form two template signal matrices; S1052: The EEG signals from the multiple channels are combined into an EEG signal matrix for the current moment, with a duration of N milliseconds; S1053: Perform canonical correlation analysis on the EEG signal matrix and the template signal matrix to obtain the correlation coefficient between the EEG signal matrix and the template signal matrix. The correlation coefficient includes a left-side correlation coefficient and a right-side correlation coefficient. The correlation coefficient serves as the feature signal, which respectively characterizes whether the current state is focused on the left-side stimulus and the right-side stimulus.

7. The method as described in claim 6, characterized in that, In step S107, filters are used to filter the left and right feature signals respectively to reduce unnecessary interference; the amplitude difference of the feature signals corresponding to the left and right visual stimuli is extracted using a function mapping method to obtain the decoding result of the current gaze visual stimulus state, and the estimates of the left and right gaze visual stimulus states are mapped to the direction and amplitude of the virtual force. Then, the current virtual force is subtracted from its average value over a predetermined period of time to remove the baseline.

8. The method as described in claim 7, characterized in that, In step S109, the virtual dynamic system is composed of a physical or electrical system with oscillation and decay characteristics. The response of the virtual dynamic system to the input signal is solved by numerical integration method. The inherent oscillation period of the virtual dynamic system is the period of the periodic gaze visual stimulus task.

9. The method as described in claim 8, characterized in that, The virtual dynamic system consists of a spring-mass-damping system and is an underdamped system. The numerical integration method calculates the current state of the virtual dynamic system by recursively using the Euler method based on the system state at the previous moment S milliseconds ago and the physical quantities of the virtual dynamic system.

10. The method as described in claim 9, characterized in that, In step S111, the threshold is the threshold of a certain state quantity in the virtual dynamics system, the state quantity is the swing amplitude of the virtual dynamics system, and the output command of the brain-computer interface switch refers to the output of a switch command to an external device or computer program.

Images