Asynchronous brain-computer interface switch control method based on periodic motor imagery task

Abstract

The application discloses a kind of asynchronous brain-computer interface switch control methods based on periodic motor imagery task, it is related to automatic control field, method includes the following steps: user operation, when needing to trigger brain-computer interface switch, user executes periodic motor imagery task, and periodic variation brain electric signal is generated;Non-invasive scalp EEG data acquisition and pretreatment;Extracting EEG signal online feature;Characteristic signal is mapped as virtual force;Virtual force is applied to virtual dynamics system to drive virtual dynamics system state change and oscillation;According to virtual dynamics system state, brain-computer interface switch trigger control;The state of virtual dynamics system is fed back to user, and user completes periodic motor imagery task according to the rhythm and period of virtual dynamics system.The application constructs a new switch trigger mode, improves the reliability of switch control, realizes the switch reliable control to external equipment etc., improves the practicability of brain-computer interface system.

Description

Technical Field

[0001] This invention relates to the field of automatic control, and more particularly to an asynchronous brain-computer interface switching control method based on a periodic motor imagery task. Background Technology

[0002] Currently, the quality of life of people with disabilities has received widespread attention from all sectors of society. Certain diseases, such as motor neuron disease, can cause mobility impairments, leading to dependence on others for daily living and a severe decline in their quality of life. Brain-computer interfaces (BCIs), a technology that establishes a direct information exchange pathway between the brain and an external device, can help people with disabilities communicate and control their surroundings directly through brain activity, bypassing traditional neuromuscular pathways. Through BCI technology, people with disabilities can drive wheelchairs, control robotic arms, and other devices to perform simple actions, potentially improving their quality of life.

[0003] Motor imagery (MI) is defined as the phenomenon where, during the execution of actions, the intention to perform actions, the imagination of actions, or other mental activities, the alpha (8-13Hz) and beta (14-26Hz) bands of the sensorimotor cortex of the brain exhibit energy decreases, while the gamma (>30Hz) band shows an increase in energy. Brain-computer interfaces based on motor imagery have already enabled cursor movement control, robotic arm motion control, and wheelchair motion control, among others.

[0004] To apply motor imagery-based brain-computer interface (BCI) systems to the lives of people with disabilities, it is necessary to improve the effectiveness of asynchronous BCI on / off control. This would allow users to quickly activate the BCI system when they wish to use it, and minimize accidental activation when users are engaged in other activities and do not wish to use the system. However, due to the low signal-to-noise ratio of EEG signals, the current on / off control performance of motor imagery-based BCIs is not ideal, and it remains difficult to achieve a good performance balance between accidental and active triggering times.

[0005] Therefore, those skilled in the art are dedicated to developing an asynchronous brain-computer interface switch control method based on a periodic motor imagery task. By using a periodic motor imagery task and a virtual dynamics system, multiple periodic features are fused and attenuated to construct a trigger threshold, thereby improving the feature separability of the brain-computer interface switch. This significantly reduces false triggering while keeping the speed of active triggering at an acceptable level. Summary of the Invention

[0006] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is how to reduce the false triggering of brain-computer interface switches while keeping the speed of active triggering at an acceptable level, and how to prevent other irrelevant actions or imaginations that do not conform to the rhythm from causing false triggering of the system.

[0007] To achieve the above objectives, the present invention provides an asynchronous brain-computer interface switching control method based on a periodic motor imagery task, the method comprising the following steps:

[0008] Step 1: The user performs an operation. When it is not necessary to trigger the brain-computer interface switch, the user remains idle or performs irrelevant tasks. When it is necessary to trigger the brain-computer interface switch, the user performs the periodic motor imagery task, generating periodically changing EEG signals.

[0009] Step 2: Collect the user's scalp EEG signals using a non-invasive scalp EEG device and perform preprocessing.

[0010] Step 3: Extract online features from the EEG signal to obtain the feature signal;

[0011] Step 4: Map the feature signals as virtual forces;

[0012] Feature 5: Applying the virtual force to the virtual dynamics system to drive the state changes and oscillations of the virtual dynamics system;

[0013] Step 6: When the state of the virtual dynamics system reaches a set threshold, the output command of the brain-computer interface switch is triggered to control the brain-computer interface switch.

[0014] Step 7: Feedback the state of the virtual dynamics system to the user, and the user completes the periodic motion visualization task according to the rhythm and period of the virtual dynamics system.

[0015] Furthermore, the idle state refers to a state in which the user remains awake or performs activities unrelated to the brain-computer interface switch control without moving; the EEG signal is generated by the user performing periodic motor imagery tasks or remaining in the idle state.

[0016] Furthermore, the periodic motion imagery task refers to the user periodically switching between two limbs performing motion imagery according to a certain rhythm; the rhythm of the periodic motion imagery task is determined according to the state of the virtual dynamics system.

[0017] Furthermore, step 3 includes: extracting the imagined limb movement state through a decoding algorithm; the imagined limb movement state is obtained by extracting time-domain or frequency-domain features.

[0018] Furthermore, step 3 also includes: extracting a time window of N milliseconds before the current moment for the motion imagery-related electrode channel, and performing spatial filtering and power spectrum estimation to extract features; the features represent the motion imagery limb that the user is currently executing, and change after the user switches limbs.

[0019] Furthermore, the electrode channels related to motor imagery include multiple channels in the left and right hemispheres.

[0020] Further, the extraction of time-domain or frequency-domain features includes: combining one of the multiple channels and its surrounding electrodes in a linear combination to form a single left-side electrode channel, representing the neural activity of the left sensorimotor area; combining another of the multiple channels and its surrounding electrodes in a linear combination to form a single right-side electrode channel, representing the neural activity of the right sensorimotor area; and performing spectral analysis on the single left-side electrode channel and the single right-side electrode channel respectively to calculate the frequency-domain features of the sensorimotor area.

[0021] Furthermore, step 4 also includes:

[0022] Step 4.1: Filter the left and right frequency domain features separately using filters to reduce unnecessary interference;

[0023] Step 4.2: Extract the amplitude difference of the frequency domain features of the sensory motor zone between the left and right electrode channels using the function mapping method, obtain the decoding result of the user's current motion imagination state, and map the estimation of the motion imagination state into the direction and amplitude of the virtual force.

[0024] Step 4.3: Subtract the average value of the virtual force over a period of time from the current virtual force using the baseline removal method, so that the virtual force has the statistical characteristic of zero mean, in order to facilitate the driving of the virtual dynamic system.

[0025] Furthermore, step 5 also includes: every S milliseconds, after the value of the virtual force is updated, the current virtual force is input into the virtual dynamics system, and numerical solutions are performed based on the original state of the system S milliseconds ago to calculate the changes in the system and obtain the current state of the system.

[0026] Furthermore, the virtual dynamic system is composed of a physical or electrical system with oscillation and decay characteristics. The state change of the virtual dynamic system over time is virtually calculated on a computer or mobile electronic device, and the response of the virtual dynamic system to the input signal is solved by numerical integration method.

[0027] Compared with the prior art, the present invention has the following beneficial technical effects:

[0028] This invention fully utilizes the periodic changes generated by periodic motor imagery tasks and integrates the characteristic information of EEG signals over a period of time. It transforms the traditional brain-computer interface method of directly decoding the user's motor imagery state to distinguish between task-idle states into the differentiation between periodic motor imagery tasks and other states. The interactive triggering mode constructs a novel switch triggering mode, which improves the reliability of switch control, realizes reliable switch control of external devices, and improves the practicality of brain-computer interface systems.

[0029] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0030] Figure 1 This is a flowchart of a preferred embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of the electrodes of a non-invasive scalp electroencephalogram device according to a preferred embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of the dynamic system composition of a preferred embodiment of the present invention. Detailed Implementation

[0033] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0034] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0035] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the present invention. This embodiment is used to control the on / off state of video playback and includes the following steps:

[0036] S1, User Operations.

[0037] When the user does not want to trigger the switch, they can remain idle or perform unrelated tasks and do not need to pay attention to the system status. At this time, although the system is online, the output of the switch command will generally not be triggered due to insufficient swing amplitude.

[0038] When the switch needs to be triggered, the user performs a periodic motion visualization task. When the system is in a nearly static state, the user performs a periodic motion visualization task with a customizable progress, or when the system has a certain swing amplitude, the user performs a periodic motion visualization task according to the system state and the system rhythm. This generates periodic EEG signals that match the swing speed of the system, allowing the system to accumulate energy and gradually reach the threshold to complete the switch triggering.

[0039] The periodic motor imagery task refers to the user periodically switching between two limbs performing motor imagery according to a certain rhythm (determined by the state of the virtual dynamics system). The selected limbs are those that can be distinguished or categorized in the motor imagery brain-computer interface. In this embodiment, the left and right hands are used as the two limbs to be periodically switched. The user performs a motor imagery task in which the left and right hands alternate in a cycle, that is, imagining the left or right hand continuously making grasping movements until switching to the other hand. In other embodiments of the present invention, the limbs performing motor imagery can be imagining the feet, the tongue, or mental arithmetic or music.

[0040] S2. Non-invasive scalp EEG data acquisition and preprocessing.

[0041] The scalp EEG signal is collected by a non-invasive scalp EEG device and preprocessed by basic signal amplification, analog-to-digital conversion, filtering and other processes. The EEG signal is generated by the user performing periodic motor imagery tasks or maintaining an idle state.

[0042] Distribution of non-invasive scalp EEG acquisition devices as follows Figure 2 As shown, it involves fitting a cap (EEG cap) onto the user's head, with different electrode channels arranged at different locations on the cap according to brain regions, to collect EEG signals from different areas of the scalp. The electrode positions of the EEG cap are distributed according to an extended version of the international standard 10-20 system. Figure 2 Each circle in the diagram represents a channel, and the symbol inside the circle represents the channel name. "Nasion" indicates the location of the nasal root (located at the top of the nose, level with the eyes), and "Inion" indicates the direction of the external occipital protuberance (the tip of the external occipital protuberance is located on the midline of the back of the head, at the base of the skull). The circumference of the skull is measured on a cross-section directly in the center of these circles, and the circumference is divided into 10% and 20% intervals to determine the electrode placement. Electrode names are also defined for the corresponding locations: C for "central," P for "parietal," T for "temporal," F for "frontal," Fp for "frontal pole," O for "occipital," and A for "mastoid."

[0043] The EEG signals acquired in this embodiment are related to motor imagery and mainly come from electrode channels C3, C4, and their surrounding areas. These channels are used in the switching control method proposed in this invention and correspond to the EEG signals of the left and right sensorimotor brain regions, respectively. After the EEG signals from these channels are acquired, amplified, and converted from digital to analog, they will be filtered. The filtering process includes bandpass filtering and notch filtering. In this embodiment, a Butterworth filter is used for 1-70Hz bandpass filtering, and a 50Hz notch filter is used for power frequency noise filtering.

[0044] S3. Online feature extraction of EEG signals.

[0045] The preprocessed scalp EEG signal is decoded to obtain an estimate of the user's currently imagined limb movement (in other embodiments of the invention, the decoding algorithm can also extract other valid mental activities such as mental arithmetic states). The user's current imagined limb movement state is obtained by extracting time-domain or frequency-domain features. Specifically, a time window (N=400) prior to the current moment is extracted from the electrode channels related to the movement imagination, and spatial filtering and power spectrum estimation are performed to extract frequency-domain features, thereby obtaining feature signals related to the left and right hand movement imagination states. This process is performed using a sliding window, occurring every S milliseconds (S=31.25), overlapping with the previously extracted time window to ensure the feature signals are updated at a sufficient frequency. In other embodiments of the invention, common spatial pattern filtering and its improved methods can also be used for channel dimensionality reduction before energy feature extraction.

[0046] In step S3, for electrodes related to motion imagery, this embodiment selects: FC3, FC4, C1, C2, C3, C4, C5, C6, CP3, and CP4. The feature extraction steps include:

[0047] S31. Linearly combine C3 and the surrounding FC3, C1, C5, and CP3 channels (linear combination coefficients: 1, -0.25, -0.25, -0.25, -0.25) to obtain a single left-side electrode channel, which characterizes the neural activity of the left sensorimotor area.

[0048] S32. Linearly combine C4 and the surrounding FC4, C2, C6, and CP4 channels (with the same linear combination coefficients as above) to obtain a single right-side electrode channel, which characterizes the neural activity of the right sensorimotor area.

[0049] S33. Perform spectrum analysis on the single left electrode channel and the single right electrode channel respectively, and calculate the frequency domain characteristics of the sensory motor zone such as the energy of each frequency band. In this embodiment, the 8-13Hz alpha frequency band characteristics are selected to characterize the state of motion imagination.

[0050] S4, the characteristic signal is mapped to a virtual force.

[0051] The characteristic signal is converted into a virtual force value through methods such as filtering, baseline removal, and function mapping.

[0052] In this embodiment, filters are used to filter the frequency domain feature signals of the left and right sides respectively to reduce unnecessary interference. Furthermore, a function mapping method is used to extract the amplitude difference of the frequency domain features of the sensorimotor zones of the left and right electrode channels to obtain the decoding result of the user's current motor imagination state. The estimated motor imagination states of the left and right hands are mapped to the direction and amplitude of the virtual force. Then, the current virtual force is subtracted from its average value over the past 10 seconds to remove the baseline.

[0053] The difference in frequency domain characteristics between the left and right sensorimotor zones is used to map virtual forces, forming a virtual force that points to the left when imagining left-hand movement and to the right when imagining right-hand movement. The amplitude of the virtual force is limited to a certain range to prevent noise or other interference from causing drastic fluctuations in the system.

[0054] S5, Virtual Dynamics System Control.

[0055] In a computer-solved virtual system, virtual forces are applied to the virtual dynamic system to drive its state changes and oscillations. Specifically, every S milliseconds, after the value of the virtual force is updated, the current virtual force is input into the virtual system. Based on the original state of the system S milliseconds ago, numerical solutions are performed to calculate the system changes and obtain the current state of the system.

[0056] The virtual dynamic system should be a system with oscillatory and damped properties. In this embodiment, a virtual dynamic system consisting of a virtual spring-mass-damping system is selected, such as... Figure 3 As shown, the virtual mass M is 4.06 kg, the spring constant K is 10 N / m, and the natural oscillation frequency of the virtual dynamic system can be obtained by the following formula:

[0057]

[0058] At this point, the natural frequency of the virtual dynamics system is 0.25Hz, meaning its natural period is 4s. The system in this embodiment is an underdamped system, and its damping coefficient can be determined based on the user's signal quality and requirements for the false trigger frequency. Ideally, the damping's influence on the system's oscillation period is negligible. Therefore, the system's natural oscillation period will determine the period of the periodic motion visualization task during use. After the virtual force is updated, based on the system's original state s milliseconds ago, various numerical methods can be used in the numerical solution process. In this embodiment, the Euler method is used for numerical integration to complete the system state estimation.

[0059] S6. System status triggers switch control.

[0060] When the state of the virtual dynamics system reaches a set threshold, it will trigger the output command of the brain-computer interface switch, thereby realizing the control of the brain-computer interface switch. In this embodiment, the brain switch is used to control the pausing and starting of video playback software.

[0061] A threshold is a threshold value for a certain state in a physical system. When the system reaches or exceeds the threshold at a certain moment, the system will output a switch switching command and clear the current state of the system. In this embodiment, the swing amplitude of the system is selected as the control quantity. When the swing amplitude is 1m, detection is triggered (the corresponding size on the screen is calculated according to the screen size), and the swing amplitude and speed are cleared after the trigger.

[0062] S7, System Status Feedback.

[0063] The virtual dynamics system provides feedback to the user through screen displays or sound, allowing the user to grasp the current state of the virtual dynamics system during operation. This facilitates the completion of periodic motion visualization tasks based on the system's rhythm and cycle. In this embodiment, circles on the screen represent the positions of the system's mass blocks, and rectangles represent the thresholds for switch triggering. The circles update their positions every milliseconds (S) to maintain consistency with the positions of the mass blocks in the virtual physical system.

[0064] The state of a virtual dynamic system includes multiple states. Preferably, for a spring-mass-damped system, these include the swing position of the mass, the velocity of the mass, and the acceleration of the mass.

[0065] Through the above steps, this embodiment achieves on / off control of the video playback process. By using a periodic motor imagery task, it provides motor imagery features that are balanced in duration for both hands and can be continuously repeated. By using a virtual dynamics system, it achieves the fusion and attenuation of multiple periodic features and constructs a trigger threshold, thereby improving the feature separability of the brain-computer interface switch. This significantly reduces false triggers while the speed of active triggering remains at an acceptable level. At the same time, other irrelevant actions or images that do not conform to the rhythm will not cause false triggers of the system. This makes the brain-computer interface switch closer to the needs of practical applications, reduces the requirements for users to use the brain-computer interface system, and expands the application scope of brain-computer interfaces.

[0066] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A switching control method for an asynchronous brain-computer interface based on a periodic motor imagery task, characterized in that, The method includes the following steps: Step 1: The user performs an operation. When it is not necessary to trigger the brain-computer interface switch, the user remains idle or performs irrelevant tasks. When it is necessary to trigger the brain-computer interface switch, the user performs the periodic motor imagery task, generating periodically changing EEG signals. Step 2: Collect the user's scalp EEG signals using a non-invasive scalp EEG device and perform preprocessing. Step 3: Extract online features from the EEG signal to obtain the feature signal; Step 4: Map the feature signal to a virtual force; convert the feature signal into the value of the virtual force through filtering, function mapping, and baseline removal; Feature 5: Applying the virtual force to the virtual dynamics system to drive the state changes and oscillations of the virtual dynamics system; Step 6: When the state of the virtual dynamics system reaches a set threshold, the output command of the brain-computer interface switch is triggered to control the brain-computer interface switch. Step 7: Feedback the state of the virtual dynamics system to the user, and the user completes the periodic motion visualization task according to the rhythm and period of the virtual dynamics system.

2. The asynchronous brain-computer interface switching control method based on a periodic motor imagery task as described in claim 1, characterized in that, The idle state refers to a state in which the user remains awake or performs activities unrelated to the brain-computer interface's on / off control without moving.

3. The asynchronous brain-computer interface switching control method based on a periodic motor imagery task as described in claim 1, characterized in that, The periodic motion visualization task refers to the user periodically switching between two limbs performing motion visualization according to a certain rhythm; the rhythm of the periodic motion visualization task is determined according to the state of the virtual dynamics system.

4. The asynchronous brain-computer interface switching control method based on a periodic motor imagery task as described in claim 3, characterized in that, Step 3 includes: extracting the imagined limb movement state through a decoding algorithm; the imagined limb movement state is obtained by extracting time-domain or frequency-domain features.

5. The asynchronous brain-computer interface switching control method based on a periodic motor imagery task as described in claim 4, characterized in that, Step 3 further includes: extracting a time window of N milliseconds before the current moment for the electrode channel related to motion imagery, and performing spatial filtering and power spectrum estimation to extract features; the features represent the limbs currently being executed by the user in motion imagery, and change after the user switches limbs.

6. The asynchronous brain-computer interface switching control method based on a periodic motor imagery task as described in claim 5, characterized in that, The electrode channels related to motor imagery include multiple channels in the left and right hemispheres.

7. The asynchronous brain-computer interface switching control method based on a periodic motor imagery task as described in claim 6, characterized in that, The extraction of time-domain or frequency-domain features includes: combining one of the multiple channels and its surrounding electrodes in a linear combination to form a single left-side electrode channel, representing the neural activity of the left sensorimotor area; combining another of the multiple channels and its surrounding electrodes in a linear combination to form a single right-side electrode channel, representing the neural activity of the right sensorimotor area; and performing spectral analysis on the single left-side electrode channel and the single right-side electrode channel respectively to calculate the frequency-domain features of the sensorimotor area.

8. The asynchronous brain-computer interface switching control method based on a periodic motor imagery task as described in claim 7, characterized in that, Step 4 also includes: Step 4.1: Filter the left and right frequency domain features separately using filters to reduce interference; Step 4.2: Extract the amplitude difference of the frequency domain features of the sensory motor zone between the left and right electrode channels using the function mapping method, obtain the decoding result of the user's current motion imagination state, and map the estimation of the motion imagination state into the direction and amplitude of the virtual force. Step 4.3: Subtract the average value of the virtual force over a period of time from the current virtual force using the baseline removal method, so that the virtual force has the statistical characteristic of zero mean, in order to facilitate the driving of the virtual dynamic system.

9. The asynchronous brain-computer interface switching control method based on a periodic motor imagery task as described in claim 1, characterized in that, Step 5 further includes: every S milliseconds, after the value of the virtual force is updated, the current virtual force is input into the virtual dynamics system, and numerical solutions are performed based on the original state of the system S milliseconds ago to calculate the changes in the system and obtain the current state of the system.

10. The asynchronous brain-computer interface switching control method based on a periodic motor imagery task as described in claim 9, characterized in that, The virtual dynamic system consists of a physical or electrical system with oscillation and decay characteristics. The state changes of the virtual dynamic system over time are virtually calculated on a computer or mobile electronic device, and the response of the virtual dynamic system to the input signal is solved by numerical integration method.

Images