Local SSMVEP stimulation and side MI linkage triggering method based on fixation area driving, brain-computer interface system and application

Abstract

The invention provides a local SSMVEP stimulation and side MI linkage triggering method based on fixation area driving, a brain-computer interface system and application, and the method comprises the steps: determining a current effective fixation area of a user in at least three different fixation areas in real time, and synchronously triggering local SSMVEP stimulation activation and side MI task prompt uniquely corresponding to the area, and a cooperative trigger link of fixation area-local stimulation-collateral task is formed. According to the system, linkage is achieved through cooperative work of the eye movement tracking and area judging unit, the linkage triggering control unit, the stimulation presenting unit and the task prompting unit. The method serves as a basic front-end technology of the multi-mode BCI, accurate alignment and synchronization of stimulation, tasks and user intentions are achieved, and key underlying support is provided for improving system performance and user experience.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface technology, and particularly to a method for triggering local SSMVEP stimulation and lateral MI linkage based on gaze region-driven local SSMVEP stimulation, a brain-computer interface system, and its applications. Background Technology In brain-computer interface research that integrates steady-state visual evoked potentials (SSVEP / SSMVEP) and motor imagery (MI), effectively presenting visual stimuli and guiding MI tasks is a fundamental and crucial issue. Traditional methods often simply combine the two: for example, continuously presenting multiple flashing SSVEP targets at fixed positions on the screen (such as the left and right sides), while instructing the user to perform motor imagery on a specific side through central text or icons (such as "imagine left hand movement").

[0002] This method has significant drawbacks: First, the stimulus and task are spatially and cognitively disconnected. A user might be focusing on a flashing target on the right (for SSVEP recognition) but need to imagine a left-hand movement. This inconsistency makes it difficult for the brain's visual attention network and sensorimotor cortex to coordinate their activities, weakening the specificity of the MI EEG signal. Second, visual interference is severe. The continuous flashing of multiple targets itself constitutes visual noise, and the superimposed static or dynamic task cues further increase the user's cognitive load, easily leading to visual fatigue and distraction. Essentially, current technology lacks a fundamental triggering mechanism capable of precisely binding specific visual stimuli to specific cognitive tasks one-to-one, millisecond-level according to the user's real-time intentions.

[0003] Therefore, developing a co-stimulus-task triggering method that enables "gaze-to-selection and selection-to-trigger" has become an urgent need to improve the performance of multimodal BCI and enhance user experience. Summary of the Invention

[0004] This invention aims to overcome the aforementioned shortcomings of existing technologies and provide a method, brain-computer interface system, and application for triggering local SSMVEP stimulation and lateral MI based on gaze region-driven linkage. Its core inventive concept lies in using the user's real-time gaze region as a unified control information source to simultaneously trigger two operations: selectively activating local SSMVEP stimulation of the corresponding region and generating lateral MI task prompts semantically bound to that region, thereby constructing an efficient and precise collaborative triggering link of "gaze region - local stimulation - lateral task".

[0005] The technical problems solved by this invention include at least the following: 1. How to improve the accuracy of initial motor imagery (MI) recognition: In traditional brain-computer interfaces (MIs), the initial brainwave signal characteristics of users are not obvious, and the classification accuracy is usually only around 50% to 60%, leading to unreliable operation and affecting the rehabilitation training effect. Therefore, the problem that this invention aims to solve is how to use visual attention guidance and local stimulation strategies to improve the separability and recognition accuracy of MI signals in the early stages of training, and realize a plug-and-play brain-computer interface.

[0006] 2. How to reduce visual stimulation-induced interference and fatigue: Traditional SSVEP full-screen or multi-sided target flicker can easily lead to cross-target interference and visual fatigue, and may even pose a risk of epilepsy to some patients. The problem this invention aims to solve is how to adaptively activate local SSVEP stimulation based on the user's gaze side, thereby reducing the number of stimuli and interference, and improving comfort and signal stability.

[0007] 3. How to shorten the MI training sample collection time and efficiently acquire labeled data: Traditional motion imagery (MI) data acquisition phases typically last 15 to 20 minutes or longer, requiring users to remain still and repeatedly visualize actions, a time-consuming and tedious process. The problem this invention aims to solve is how to combine SSMVEP classification results with the fixation side to achieve synchronized motion imagery cues and data acquisition, thereby efficiently generating labeled MI samples and reducing the training burden.

[0008] The technical solution of the present invention is as follows: According to one aspect of the present invention, a method for triggering local SSMVEP stimulation and lateral MI based on gaze region-driven local SSMVEP stimulation is disclosed, comprising the following steps: The coordinates of the user's gaze point are acquired in real time and mapped to at least three preset different gaze regions to determine the current effective gaze region; Based on the current effective gaze area, execute the linkage trigger control: A. Stimulus control: Only one local SSMVEP visual stimulus target corresponding to the current effective fixation area is activated, while other SSMVEP visual stimulus targets in non-corresponding areas are inactive. B. Task prompt control: Synchronously generate lateral motion imagery prompts associated with the current effective gaze area to guide the user to perform specific motion imagery tasks bound to the current effective gaze area; In this process, steps A and B are triggered within the same processing cycle based on the same currently effective gaze region information, forming a collaborative triggering chain of "gaze region - local stimulus - lateral task".

[0009] In some implementations, at least three different fixation areas include a left region, a central region, and a right region; the co-triggered link of "fixation area-local stimulus-lateral task" is as follows: when the current effective fixation area is the left region, trigger activation of left local SSMVEP stimulation and prompt left limb motor imagery; when it is the central region, trigger activation of central local SSMVEP stimulation and prompt bilateral limb or trunk motor imagery; when it is the right region, trigger activation of right local SSMVEP stimulation and prompt right limb motor imagery.

[0010] In some implementations, the local SSMVEP visual stimulus target and the lateral motor imagery cue information are presented through a composite state change of the same visual element: when a certain area is gazed upon, the visual element representing the task content in that area is activated as the SSMVEP stimulus target, and its visual salience is simultaneously enhanced as a task cue; while the visual elements in the non-gaze area are weakened in display.

[0011] In some implementations, the local SSMVEP visual stimulus target and the lateral motor imagery cue information are presented separately in the display space; the stimulus target is concentrated in the core visual area, and the cue information is placed in the peripheral annotation area.

[0012] In some implementations, the lateral motor imagery cue information includes at least one of visual cues, auditory cues, or tactile cues.

[0013] According to another aspect of the present invention, a brain-computer interface system is also disclosed for implementing the above-described method of triggering local SSMVEP stimulation and lateral MI based on gaze region driving, the system comprising: The eye-tracking and region determination unit is used to obtain the user's gaze coordinates in real time and map them to at least three preset gaze regions to determine the current effective gaze region. The linkage trigger control unit is connected to the eye tracking and region determination unit and is used to generate linkage control signals based on the current effective gaze region; The stimulus presentation unit, connected to the linkage trigger control unit, is used to activate only one local SSMVEP visual stimulus target corresponding to the current effective gaze area in response to the stimulus control command in the linkage control signal. The task prompting unit, connected to the linkage triggering control unit, is used to respond to the task prompting instructions in the linkage control signal and synchronously generate lateral motion imagery prompting information associated with the current effective gaze area.

[0014] In some implementations, the linkage trigger control unit pre-stores a region-stimulus-task mapping relationship, wherein the at least three different gaze regions include a left region, a central region, and a right region, and are respectively mapped to different local SSMVEP stimulus parameters and different motor imagery task prompts.

[0015] In some implementations, the functions of the stimulus presentation unit and the task prompting unit are implemented by the same graphics rendering unit, and stimulus activation and task prompting are completed synchronously by changing the state of the same set of visual elements.

[0016] In some implementations, the visual stimulus target display area controlled by the stimulus presentation unit and the prompt information display area controlled by the task prompt unit do not overlap on the physical screen.

[0017] According to another aspect of the present invention, an application of the fixation region-driven local SSMVEP stimulation and lateral MI linkage triggering method described above is also disclosed, comprising the following steps: The coordinates of the user's gaze point are obtained in real time and mapped to at least three preset gaze regions to determine the current effective gaze region; In response to the current effective gaze area, local stimulus control is performed: only one local SSMVEP visual stimulus target corresponding to the current effective gaze area is activated, while generating specific motion imagery cue information associated with that area; wherein, SSMVEP visual stimulus targets outside the current effective gaze area are inactive. Simultaneously collect the user's electroencephalogram (EEG) signals; From the EEG signals, SSMVEP features related to the local SSMVEP visual stimulus target and MI features related to the motor imagery cue information were extracted respectively. Determine whether the number of valid MI samples that have been collected and labeled with motion imagery prompts has reached a preset threshold; Perform adaptive decoding based on the judgment result: If the number of valid MI samples does not reach the preset threshold, the first control command is generated based on the SSMVEP feature. If the number of valid MI samples reaches a preset threshold, the second control command is generated based on the fusion result of SSMVEP features and MI features. Output control signals to drive associated external devices or update the interactive interface according to a first control command or a second control command.

[0018] The beneficial effects of the fixation region-driven local SSMVEP stimulation and lateral MI linkage triggering method of the present invention are as follows: 1. Achieving neural activation consistency: Aligning visual attention, SSMVEP evoked sources, and MI cognitive tasks in space and intention enhances the lateralization activation of the motor cortex, providing a higher quality signal foundation for subsequent EEG decoding.

[0019] 2. Significantly reduce visual interference: The "local activation" strategy eliminates multi-target flickering interference; the "integrated" or "spatial separation" cue design reduces cognitive confusion and improves visual comfort.

[0020] 3. Provides intuitive interaction logic: The "look-to-trigger" mechanism conforms to natural interaction habits and reduces the learning and operation burden for users.

[0021] 4. Foundational Patent: This invention protects the most advanced and critical stimulus-task triggering mechanism in multimodal BCI. This mechanism is a prerequisite for the effective operation of any subsequent advanced signal processing and fusion algorithms, and has important foundational significance and wide application value.

[0022] The brain-computer interface system of the present invention and the application of a local SSMVEP stimulation and lateral MI linkage triggering method based on gaze region driving have the following beneficial effects: 1. Significantly improves initial MI recognition accuracy and training efficiency: By linking eye-tracking-guided local stimulation with associated MI tasks, the specific activation of the brain's motor cortex is enhanced, improving the separability of MI signals. Simultaneously, the real-time high-precision recognition results of SSMVEP are used to automatically and accurately label the synchronously generated MI EEG data, greatly shortening the supervised data acquisition time required for MI model training, achieving "plug-and-play" functionality and rapid performance improvement.

[0023] 2. Effectively reduces visual interference and fatigue: It abandons the traditional full-screen or multi-target continuous flashing mode and adopts the strategy of "gaze-driven, local activation" and "stimulus and cue space partitioning". It significantly reduces visual confusion and cognitive load from both the stimulus source and cue information levels, reduces user visual fatigue, and improves the comfort and long-term usability of the system. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a local SSMVEP stimulation and lateral MI linkage triggering method based on gaze region-driven approach according to the present invention.

[0025] Figure 2 This is a system architecture block diagram of a brain-computer interface system according to the present invention.

[0026] Figure 3 This is a diagram of the stimulation interface before activation, using an integrated state indicator layout, according to Embodiment 1 of the present invention. Figure 4 for Figure 3 The diagram shows the interface after the central area of ​​the stimulation interface is activated. Figure 5 for Figure 4 A simplified diagram of the stimulation interface; Figure 6 This is a simplified schematic diagram of the stimulation interface with a spatially separated layout according to Embodiment 2 of the present invention.

[0027] Figure 7 This is a framework diagram of the brain-computer interface system according to Embodiment 3 of the present invention; Figure 8 This is a flowchart illustrating Embodiment 3 of the present invention; Figure 9 This is a detailed flowchart of the adaptive fusion decoding unit in Embodiment 3 of the present invention.

[0028] Reference numerals: 200-Display interface; 201-Eye tracking and region determination unit; 202-Linkage trigger control unit; 203-Stimulus presentation unit; 204-Task prompt unit; 205-Central region; 206-Left side region; 207-Right side region. Detailed Implementation

[0029] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0030] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0031] like Figure 1-9 As shown, this invention provides a method and system for triggering local SSMVEP stimulation and lateral MI linkage based on gaze region-driven approach. The core of this method is to use the user's real-time gaze region as a unified control information source to simultaneously trigger two operations: selectively activating local SSMVEP stimulation in the corresponding region and generating a lateral MI task prompt that is semantically bound to that region. This constructs an efficient and accurate collaborative triggering link of "gaze region - local stimulation - lateral task" to solve the pain points in the prior art.

[0032] like Figure 1 As shown, it illustrates a flowchart of a local SSMVEP stimulus and lateral MI linkage triggering method based on gaze region-driven approach according to one embodiment of the present invention. The process begins with the continuous perception of the user's gaze region (S101), the core of which is the linkage triggering decision and execution based on region information (S102), and finally achieves the coordinated presentation of stimulus and cue (S103).

[0033] Specifically, the process begins with real-time perception of the user's gaze area (S101). By acquiring the coordinates of the gaze point and mapping them to a preset area, the current effective gaze area (such as left, center, or right) is determined. Subsequently, the system makes a linkage triggering decision based on this area information (S102), generating synchronous control commands within the same processing cycle. Finally, the system collaboratively executes the presentation of stimuli and cues (S103), activating the corresponding local SSMVEP stimulus target in the central area of ​​the display interface, while simultaneously activating the corresponding lateral MI cues in the side areas, providing the user with consistent visual and auditory feedback and completing one triggering cycle.

[0034] like Figure 2 As shown, it illustrates the system architecture of a brain-computer interface system for implementing the gaze-region-driven local SSMVEP stimulation and lateral MI linkage triggering method of the present invention, comprising four core units: an eye-tracking and region determination unit 201, a linkage triggering control unit 202, a stimulus presentation unit 203, and a task prompting unit 204. The display interface 200 is logically divided into a central region 205, a left side region 206, and a right side region 207.

[0035] in: The eye-tracking and region determination unit 201 is responsible for sensing the user's gaze and outputting the current effective gaze region; The linkage trigger control unit 202, as the central hub, receives regional information and synchronously generates stimulation control commands and task prompt commands. The stimulus presentation unit 203 and the task prompting unit 204 receive instructions and control the content presented in different areas of the display interface.

[0036] The display interface 200 is logically divided into a central area 205, a left side area 206, and a right side area 207, which are used to present the SSMVEP stimulus target and MI prompt information, respectively, thus achieving the separation of function and space.

[0037] Preferably, the eye-tracking and region determination unit 201 includes: The eye-tracking module is configured to acquire the user's gaze coordinates in real time. It is used to acquire binocular images in real time and calculate the gaze point, gaze stability, and gaze area. The eye-tracking module includes a camera, infrared illumination, pupil detection algorithm, and gaze mapping algorithm. The eye-tracking module can be referenced from the Tobii device in CN113419628A. The gaze point refers to the focal position of the eye's gaze, commonly used in eye tracking or visual analysis, representing the specific coordinates of the line of sight on a screen or in space. In eye-tracking technology, the gaze point is calculated by capturing eye movements through sensors or cameras to determine the projection point of the gaze onto a target surface (such as a screen). The gaze mapping algorithm is mainly used to estimate the direction of human eye gaze or the gaze point; common methods include geometric methods, appearance methods, and hybrid methods.

[0038] The gaze region determination module is configured to map the gaze point coordinates to at least three preset gaze regions and output the currently valid gaze regions. For example, the gaze region determination module is used to map the user's gaze point to three predefined regions on the left, center, and right of the screen and output parameters such as gaze confidence and gaze stabilization time.

[0039] The present invention will be further described below through specific embodiments.

[0040] like Figure 3-5 As shown, it demonstrates the linkage triggering based on the left, center, and right regions. The system divides the screen into three gaze regions: left, center, and right. This is a concrete manifestation of achieving "at least three different gaze regions" and supporting bilateral and central tasks.

[0041] Regarding the two specific implementation methods for interface presentation: Example 1: Integrated Status Indicator like Figure 3 As shown, the interface directly displays icons representing tasks: a left-hand icon on the left, a double-leg icon in the center, and a right-hand icon on the right. Initially, all three icons share the same color and brightness.

[0042] like Figure 4-5 As shown, when the user gazes at and locks onto the central area, the system triggers the following: Stimulus control: Activate the central double-leg icon as the SSMVEP stimulus target, for example by applying a colored halo or dynamic border that flashes at a frequency f1 (e.g., 15Hz).

[0043] Task prompt control: Simultaneously highlight the two-leg icon (e.g., change to a bright color, increase size), while visually weakening the left-hand icon on the left and the right-hand icon on the right (e.g., make them semi-transparent, reduce brightness).

[0044] Effect: By focusing on the center, users directly and synchronously receive clear information: "There is a flashing stimulus here (to induce SSMVEP)" and "Imagine leg movement at this moment." The status changes of the icons on the left and right clearly indicate that they are not selected.

[0045] Example 2: Spatial Separation Presentation: like Figure 6 As shown, this diagram illustrates a spatially separated layout. The display interface is strictly divided into spatial zones: Central area: Three different abstract figures are arranged horizontally as the SSMVEP stimulus target. When focusing on the center, only the central stimulus figure is activated (e.g., it begins to move or flash at a specific frequency), while the figures on the left and right sides remain inactive.

[0046] Side Area: Set up a task prompt area at the top (or bottom / left / right sides) of the screen. When looking at the center, the "Imagine Both Legs" prompt is activated and highlighted. The "Imagine Left Hand" and "Imagine Right Hand" prompts remain inactive.

[0047] This layout achieves physical spatial separation between the SSMVEP stimulus target and the MI task cue information, ensuring clear functionality and preventing interference between them.

[0048] Example 3: A specific application example of a fixation-region-driven local SSMVEP stimulation and lateral MI linkage triggering method. Figure 7-9 This paper demonstrates a complete application embodiment of the present invention, which uses a gaze region-driven local SSMVEP stimulation and lateral MI linkage triggering method. This application runs on the aforementioned brain-computer interface system, such as... Figure 7 As shown, in addition to the units and modules mentioned above, the brain-computer interface system also includes: The EEG acquisition unit is configured to acquire the user's EEG signals; preferably, it is an EEG acquisition device capable of acquiring EEG signals from the user's occipital lobe, central region, motor cortex, etc., and supporting 8 to 32 channels of acquisition.

[0049] The feature extraction unit is configured to extract SSMVEP features and MI features from the EEG signal, respectively; The adaptive fusion decoding unit is configured as follows: Record and determine whether the number of valid MI samples labeled with motion imagery prompts has reached a preset threshold. When the number of valid MI samples does not reach the preset threshold, the first control command is generated based on the SSMVEP feature. When the number of valid MI samples reaches a preset threshold, the second control command is generated by decoding based on the fusion result of SSMVEP features and MI features. The control output unit is configured to output control signals to drive associated external devices or update the interactive interface according to a first control command or a second control command.

[0050] Preferably, the adaptive fusion decoding unit includes a quality assessment module and a fusion decision module; The quality assessment module is configured to assess the first quality indicator corresponding to the SSMVEP feature and the second quality indicator corresponding to the MI feature. The fusion decision module is configured to: when the number of valid MI samples reaches a preset threshold, acquire the first identification result based on SSMVEP features and the second identification result based on MI features, and perform weighted fusion of the two based on the first quality index and the second quality index.

[0051] Preferably, the adaptive fusion decoding unit is also configured to perform adaptive mode switching: When the first quality index continues to be lower than the first switching threshold, an instruction is sent to the stimulus presentation unit to pause the SSMVEP visual stimulus and control the fusion decision module to switch to the mode of decoding based on MI features. When the second quality index continues to be lower than the second switching threshold, the control fusion decision module switches to the mode of decoding based on SSMVEP features.

[0052] Preferably, the adaptive fusion decoding unit is configured to: when the local SSMVEP visual stimulus target is activated and the EEG signal acquisition is effective, associate and store the synchronously acquired EEG signal segment with the motor imagery task label determined by the current effective gaze area, and accumulate it as a valid MI sample.

[0053] like Figure 8-9 The diagram illustrates a specific flowchart of one application of the fixation region-driven local SSMVEP stimulation and lateral MI linkage triggering method of the present invention, which specifically includes the following steps: The coordinates of the user's gaze point are obtained in real time and mapped to at least three preset gaze regions to determine the current effective gaze region; In response to the current effective gaze area, local stimulus control is performed: only one local SSMVEP visual stimulus target corresponding to the current effective gaze area is activated, while generating specific motion imagery cue information associated with that area; wherein, SSMVEP visual stimulus targets outside the current effective gaze area are inactive. Simultaneously collect the user's electroencephalogram (EEG) signals; From the EEG signals, SSMVEP features related to the local SSMVEP visual stimulus target and MI features related to the motor imagery cue information were extracted respectively. Determine whether the number of valid MI samples that have been collected and labeled with motion imagery prompts has reached a preset threshold; Perform adaptive decoding based on the judgment result: If the number of valid MI samples does not reach the preset threshold, the first control command is generated based on the SSMVEP feature. If the number of valid MI samples reaches a preset threshold, the second control command is generated based on the fusion result of SSMVEP features and MI features. Output control signals to drive associated external devices or update the interactive interface according to a first control command or a second control command.

[0054] Preferably, decoding is performed based on the fusion result of SSMVEP features and MI features, including: The first identification result based on SSMVEP features and its first quality index, and the second identification result based on MI features and its second quality index are obtained respectively. Based on the first and second quality indicators, the weights of the first and second identification results in the final decision are dynamically adjusted and then weighted and fused.

[0055] As a preferred embodiment, the eye-tracking-based adaptive SSMVEP–MI fusion brain-computer interface method further includes an adaptive mode switching step: Continuously monitor the primary and secondary quality indicators; When the first quality index remains below the first switching threshold, the activation of the local SSMVEP visual stimulus target is paused, and the mode of decoding based on MI features is switched. When the second quality index remains below the second switching threshold, switch to the mode that decodes based on SSMVEP features.

[0056] Preferably, the first quality metric includes the signal-to-noise ratio or classification confidence of the SSMVEP feature; the second quality metric includes the classification confidence of the MI feature.

[0057] Preferably, the specific motion imagery prompt is at least one of visual, auditory, or tactile prompts, used to guide the user to perform a specific motion imagery task associated with the currently effective gaze area.

[0058] Combination Figure 8 and Figure 9 A detailed explanation of the application method of a local SSMVEP stimulus and lateral MI linkage triggering method based on fixation region driving is provided: Step S801, System Initialization: The user wears an integrated or separate eye tracker and EEG acquisition device, facing the stimulation screen. After the system starts, eye track calibration (such as 9-point calibration) and EEG electrode impedance detection are performed first to ensure signal acquisition quality.

[0059] Step S802, Real-time Eye Tracking and Region Determination: The eye tracking module continuously acquires images of the user's eyes and calculates the coordinates of the gaze point in the screen coordinate system in real time using algorithms such as pupil-corneal reflection. The gaze region determination module divides the screen into at least three predefined regions (this embodiment uses left, center, and right regions as an example, e.g.) Figure 3 As shown in the figure, the eye-tracking module maps the real-time gaze coordinates to the corresponding region. The eye-tracking module can also calculate gaze stability (such as the variance of the gaze point within a certain time window) and confidence. Only when the gaze is stable for more than a certain time (such as 200ms) and the confidence is higher than the threshold is it determined to be the "current effective gaze region".

[0060] Step S803, Local Stimulus Activation and MI Cue Generation: This is one of the key steps in this embodiment. The linkage trigger control unit receives information about the current effective gaze area. Three potential stimulus positions (left, center, and right) are preset on the screen. The core logic executed by the stimulus presentation unit is: only the stimulus target corresponding to the current effective gaze area starts to flash or move according to a preset specific frequency and encoding pattern to induce SSMVEP; simultaneously, it immediately turns off or keeps the stimulus targets in other non-gaze areas static or dark. Synchronously, the task prompting unit generates clear and associated MI task prompts based on the gaze area.

[0061] Step S804, Synchronous EEG Acquisition: During stimulus activation and MI cue presentation, the EEG acquisition unit synchronously acquires continuous EEG signals.

[0062] Step S805, Feature Extraction: After preprocessing the acquired EEG signal (such as filtering and artifact removal), the feature extraction unit processes two modes in parallel: SSMVEP Feature Extraction: For EEG data within a short time window (e.g., 1-4 seconds) after the stimulus begins, methods such as filter bank canonical correlation analysis (FBCCA) are used to calculate the correlation between the data and each preset stimulus frequency and its harmonics, and to obtain feature values ​​or confidence levels.

[0063] MI Feature Extraction: For EEG data within a relatively long time window (e.g., 2-6 seconds) after the occurrence of the MI cue, extract time-domain, frequency-domain, or time-frequency-domain features related to motion imagery, such as the energy variations of event-related desynchronization / synchronization (ERD / ERS) in the μ (8-13Hz) and β (13-30Hz) frequency bands. These features can be input into a lightweight classifier (e.g., EEGNet) for preliminary classification.

[0064] The classification confidence of SSMVEP features can be characterized by the maximum correlation coefficient with the target frequency calculated by the FBCCA algorithm or by the normalized score; the classification confidence of MI features can be characterized by the probability of the corresponding imagined category output by classifiers such as EEGNet or the decision function value. These confidence values ​​are directly used for subsequent quality assessment and fusion decisions.

[0065] Step S806, Sample Management and Quantity Judgment: This function is performed by the adaptive fusion decoding unit. Each time steps S803-S805 are executed, if the current effective gaze region is reliably determined and the quality of the synchronously acquired EEG signal is acceptable, the system automatically generates a labeled MI sample. The label represents the motion imagery task category determined by the current effective gaze region (e.g., "left-hand imagery"). The adaptive fusion decoding unit continuously counts the number N of such effective samples.

[0066] Step S807, Adaptive Fusion Decoding and Output: This is the core step of this embodiment, and its detailed logic is as follows: Figure 9 As shown.

[0067] The adaptive fusion decoding unit first determines whether the accumulated number of valid MI samples N has reached the preset threshold N_th (e.g., N_th = 30 for each task class).

[0068] Path A (N < N_th, MI model not ready): At this stage, the MI classifier is unreliable or does not exist. The adaptive fusion decoding unit will ignore MI features and decode solely based on SSMVEP features. For example, the region corresponding to the stimulus frequency with the largest feature value is selected as the recognition result. This result is used to generate the first control command. The system can still provide stable BCI control at this stage, while continuously accumulating high-quality labeled MI samples in the background.

[0069] Path B (N≥N_th, MI model ready): The system switches to fusion mode at this point. The adaptive fusion decoding unit simultaneously receives the SSMVEP decoding results (e.g., target class C_ssvep and its confidence Conf_s) and the MI decoding results (e.g., target class C_mi and its confidence Conf_m). It executes the adaptive fusion mechanism: Consistency check and direct output: If C_ssvep and C_mi are consistent, then the consistency result is directly output as the final instruction.

[0070] Weighted Fusion in Case of Inconsistency: If the two metrics are inconsistent, a signal quality metric is introduced for arbitration. The signal-to-noise ratio (SNR_s) of the SSMVEP signal is calculated as its quality metric Q_s, and the MI classification confidence score Conf_m is taken as its quality metric Q_m. The final decision D can be obtained through weighted voting, with the weights being functions of Q_s and Q_m. This ensures that the higher-quality mode dominates the decision-making process.

[0071] Adaptive Mode Switching: The adaptive fusion decoding unit continuously monitors Q_s and Q_m. If Q_s remains below a low threshold (e.g., low SNR, possibly due to frequent blinking or eye movement by the user), while Q_m remains high, the module can automatically trigger mode switching, sending a command to the stimulus presentation and control module 103 to temporarily pause SSMVEP stimulation. The system enters "MI-only" mode, relying solely on MI for control. Conversely, if Q_m remains too low, it switches to "SSMVEP-dominated" mode. Once signal quality recovers, it automatically switches back to fusion mode.

[0072] A second control instruction is generated based on the fusion or arbitration result.

[0073] Step S808: Output control signal: The control output unit outputs a corresponding control signal according to the generated control command. This signal can be used to drive external devices such as rehabilitation robots and prostheses, and can also be used to update the content and status of interactive interfaces such as virtual reality scenes and rehabilitation software interfaces to realize the user's intentions.

[0074] Step S809, Loop Judgment: The system determines whether the current training or control task has ended. If it has not ended, the process returns to step S802 and continues to the next trial; if it has ended, the current run is terminated.

[0075] Through the above embodiments, this invention achieves a seamless transition from stable control relying on SSMVEP in the initial stage to high-precision dual-modal fusion control in the later stage. Local stimulus activation and spatial partitioning cue logic significantly reduce visual interference and enhance MI signals; adaptive decoding path selection based on sample number ensures the availability and performance growth of the system throughout the entire process; and the fusion and switching mechanism based on signal quality ensures the robustness of the system in the face of real-world interference.

[0076] The advantages of the fixation region-driven local SSMVEP stimulation and lateral MI linkage triggering method, system and application of the present invention compared with the prior art are as follows: 1. Significantly improves MI recognition accuracy in the early stages of training: The eye-tracking-SSMVEP-MI three-channel consistency enhances the MI signal characteristics, enabling the system to achieve accuracy far exceeding that of traditional MI BCI with very few training samples.

[0077] 2. Effectively reduces visual stimuli, fatigue, and risks: Local activation of SSMVEP stimulation reduces irrelevant flicker, improving comfort and long-term usability.

[0078] 3. Significantly reduce MI annotation data collection time: Through automated and synchronized tag generation, reduce MI data collection time by 50%–80%.

[0079] 4. Stable and reliable fusion decoding: Supports mechanisms such as eye movement loss compensation and SSMVEP signal failure switching, improving the reliability of practical applications.

[0080] 5. Suitable for multiple application scenarios such as rehabilitation, interaction, and enhanced control: It can significantly improve the performance of intent recognition in complex control tasks.

[0081] This invention discloses a method, system, and application for triggering local SSMVEP stimulation and lateral MI (Missing Task) linkage in a brain-computer interface based on gaze region-driven approaches. This method determines the user's current effective gaze region among at least three different gaze regions in real time, and simultaneously triggers the activation of a uniquely corresponding local SSMVEP stimulus and a lateral MI task prompt, forming a collaborative triggering link of "gaze region - local stimulus - lateral task." The system achieves this linkage through the collaborative work of an eye-tracking and region determination unit, a linkage triggering control unit, a stimulus presentation unit, and a task prompting unit. As a foundational front-end technology for multimodal BCI, this invention achieves precise alignment and synchronization of stimuli, tasks, and user intentions, providing crucial underlying support for improving system performance and user experience.

[0082] It is understood that the fixation-region-driven local SSMVEP stimulation and lateral MI linkage triggering method of the present invention is independent of the subsequent EEG signal processing flow. It protects the front-end "trigger" that generates specific pattern EEG signals. Regardless of whether simple SSMVEP classification, complex MI feature extraction, or adaptive fusion of the two is subsequently used, as long as the system adopts the triggering mechanism of "fixation-region-driven local stimulation and lateral task cues" as defined in this invention, it may fall within the protection scope of this invention.

[0083] It is understood that the above specific embodiments are merely examples, and the scope of protection of this invention is not limited to the above embodiments. For example, the gaze area can be divided into upper, middle, and lower quadrants, or a finer grid (four quadrants, etc.), as long as the number is at least three; MI cues can also take other forms such as tactile vibration. Any modifications, equivalent substitutions, and improvements made within the principles and design concepts disclosed in this invention should be included within the scope of protection of the claims of this invention.

Claims

1. A method for triggering local SSMVEP stimulation and lateral MI based on fixation region-driven local stimuli, characterized in that, Includes the following steps: The coordinates of the user's gaze point are acquired in real time and mapped to at least three preset different gaze regions to determine the current effective gaze region; Based on the currently effective gaze area, execute the linkage trigger control: A. Stimulus control: Only one local SSMVEP visual stimulus target corresponding to the current effective fixation area is activated, while SSMVEP visual stimulus targets in other non-corresponding areas are deactivated. B. Task prompt control: Synchronously generate lateral motion imagery prompts associated with the current effective gaze area to guide the user to perform a specific motion imagery task bound to the current effective gaze area; In this process, steps A and B are triggered within the same processing cycle based on the same current effective gaze region information, forming a collaborative triggering chain of "gaze region - local stimulus - lateral task".

2. The method for triggering local SSMVEP stimulation and lateral MI based on gaze region-driven approach according to claim 1, characterized in that, The at least three different fixation areas include the left region, the central region, and the right region; the coordinated triggering link of the "fixation area-local stimulus-lateral task" is as follows: when the current effective fixation area is the left region, the left local SSMVEP stimulus is triggered and prompts left limb motor imagery; when it is the central region, the central local SSMVEP stimulus is triggered and prompts bilateral limb or trunk motor imagery; when it is the right region, the right local SSMVEP stimulus is triggered and prompts right limb motor imagery.

3. The method for triggering local SSMVEP stimulation and lateral MI based on gaze region driving according to claim 1, characterized in that, The local SSMVEP visual stimulus target and the lateral motor imagery cue information are presented through the composite state changes of the same visual element: when a certain area is gazed upon, the visual element representing the task content in that area is activated as the SSMVEP stimulus target, and its visual salience is simultaneously enhanced as a task cue; the visual elements in the non-gaze area are weakened in display.

4. The method for triggering local SSMVEP stimulation and lateral MI based on fixation region driving according to claim 1, characterized in that, The local SSMVEP visual stimulus target and the lateral motor imagery cue information are presented separately in the display space; the stimulus target is concentrated in the core visual area, and the cue information is placed in the peripheral annotation area.

5. The method for triggering local SSMVEP stimulation and lateral MI based on gaze region driving according to any one of claims 1-4, characterized in that, The lateral motor imagery cue information includes at least one of visual cues, auditory cues, or tactile cues.

6. A brain-computer interface system for implementing the local SSMVEP stimulation and lateral MI linkage triggering method based on gaze region driving as described in any one of claims 1-5, characterized in that, include: The eye-tracking and region determination unit is used to obtain the user's gaze coordinates in real time and map them to at least three preset gaze regions to determine the current effective gaze region. A linkage trigger control unit, connected to the eye-tracking and region determination unit, is used to generate a linkage control signal based on the current effective gaze region; The stimulus presentation unit, connected to the linkage trigger control unit, is used to activate only one local SSMVEP visual stimulus target corresponding to the current effective gaze area in response to the stimulus control command in the linkage control signal. The task prompting unit, connected to the linkage triggering control unit, is used to respond to the task prompting instruction in the linkage control signal and synchronously generate lateral motion imagery prompting information associated with the current effective gaze area.

7. The brain-computer interface system according to claim 6, characterized in that, The linkage trigger control unit has a pre-stored region-stimulus-task mapping relationship. The at least three different gaze regions include the left region, the central region, and the right region, and are respectively mapped to different local SSMVEP stimulus parameters and different motor imagery task prompts.

8. The brain-computer interface system according to claim 6, characterized in that, The functions of the stimulus presentation unit and the task prompting unit are implemented by the same graphics rendering unit, and the stimulus activation and task prompting are completed synchronously by changing the state of the same set of visual elements.

9. The brain-computer interface system according to claim 6, characterized in that, The visual stimulus target display area controlled by the stimulus presentation unit and the prompt information display area controlled by the task prompt unit do not overlap on the physical screen.

10. An application of the local SSMVEP stimulation and lateral MI linkage triggering method based on gaze region driving as described in any one of claims 1-5, characterized in that, Includes the following steps: The coordinates of the user's gaze point are obtained in real time and mapped to at least three preset gaze regions to determine the current effective gaze region; In response to the current effective gaze area, local stimulus control is performed: only one local SSMVEP visual stimulus target corresponding to the current effective gaze area is activated, and specific motion imagery cue information associated with that area is generated; wherein, SSMVEP visual stimulus targets outside the current effective gaze area are inactive. Simultaneously collect the user's electroencephalogram (EEG) signals; From the EEG signals, SSMVEP features related to the local SSMVEP visual stimulus target and MI features related to the motor imagery cue information are extracted respectively. Determine whether the number of valid MI samples that have been collected and labeled with the aforementioned motion imagery prompt information has reached a preset threshold; Perform adaptive decoding based on the judgment result: If the number of valid MI samples does not reach the preset threshold, then the first control command is generated based on the SSMVEP feature. If the number of valid MI samples reaches a preset threshold, then the second control command is generated based on the fusion result of the SSMVEP feature and the MI feature. Output control signals to drive associated external devices or update the interactive interface according to the first control command or the second control command.

Images