A cross-platform brain-computer interface paradigm system suitable for natural state monitoring

Abstract

The application provides a cross-platform brain-computer interface paradigm system suitable for natural state monitoring, comprising: a cross-platform development module, which realizes running of a set of codes on multiple platforms through a self-drawing engine; a task integration and expansion module, which integrates multiple brain-computer interface paradigm tasks; a local material library management mechanism is established, and task types and stimulation materials are associated and configured through a data configuration file; a multimedia precise synchronization control module, which adopts an asynchronous programming mechanism and a timer function of Flutter, and when audio and video materials are played or ended, labels are marked in the hardware circuit or streaming data of an electroencephalogram acquisition device, a physiological polygraph or a network camera, alignment of multiple devices and multiple modal data is realized, and a task log containing a timestamp mark is generated; and a natural stimulation task module, which incorporates tasks simulating natural scenes into the brain-computer interface paradigm. The application provides a brain-computer interface system which takes into account cross-platform consistency and flexible configuration of natural scene tasks.

Description

Technical Field

[0001] This application relates to the field of brain-computer interface technology, and in particular to a cross-platform brain-computer interface paradigm system suitable for natural state monitoring. Background Technology

[0002] Brain-computer interface (BCI) technology enables human-computer interaction by collecting and analyzing electroencephalogram (EEG) signals, and has significant application value in fields such as medicine and neuroscience research. Traditional BCI systems are mostly designed for laboratory environments and suffer from the following technical bottlenecks:

[0003] 1. Insufficient cross-platform compatibility: Existing systems often rely on native components of various platforms for development, resulting in differences in interface display and function implementation across different operating systems (such as Android, iOS, and Windows). This increases development costs and maintenance difficulty, making it difficult to meet the unified operation requirements of multi-device collaborative experiments.

[0004] 2. Limited task paradigms and poor scalability: Most mainstream brain-computer interface tasks involve laboratory-customized stimuli (such as flashing lights, pure sounds, etc.), lacking simulation of natural life scenarios (such as eating, reading aloud, watching videos, etc.); moreover, material management and task configuration depend on code modification, and users cannot flexibly adjust instructions, multimedia materials, etc. according to experimental needs, which limits the diversity of research scenarios.

[0005] 3. Low synchronization accuracy of multimodal data: When synchronizing multimedia stimulation (audio, video) and EEG signal acquisition, traditional systems often suffer from time deviations due to differences in the underlying scheduling mechanism of the platform, and lack a unified time alignment scheme for multiple devices (EEG acquisition device, physiological polyphony analyzer, camera), which affects the analysis of data correlation.

[0006] 4. Lack of coverage of natural scenarios: Existing systems rarely include complex tasks in daily life (such as the acquisition of EEG signals accompanying chewing and swallowing), making it difficult to obtain real neural electrical activity data of the brain processing multi-sensory information (audiovisual integration, motor coordination) in natural states, which limits the transformation of brain-computer interface technology into practical application scenarios.

[0007] Currently, there is no brain-computer interface system that can simultaneously achieve cross-platform consistency, flexible configuration of natural scene tasks, and high-precision multimodal data synchronization. Summary of the Invention

[0008] This application provides a cross-platform brain-computer interface paradigm system suitable for natural state monitoring, aiming to solve the problem that there is no existing brain-computer interface system that can take into account cross-platform consistency, flexible configuration of natural scene tasks, and high-precision multimodal data synchronization.

[0009] In a first aspect, this application provides a cross-platform brain-computer interface paradigm system suitable for natural state monitoring, comprising:

[0010] The cross-platform development module is developed using the Flutter framework. It uses a custom drawing engine to enable a single codebase to run on multiple platforms, avoiding differences in native components across different platforms and ensuring consistency of interface and functionality across all platforms. These platforms include Android, iOS, and Windows.

[0011] The task integration and expansion module integrates multiple brain-computer interface paradigm tasks, including tasks simulating natural scenes; it establishes a local material library management mechanism to add, delete, and modify instructions, images, audio, or video according to experimental needs, and configures task types and stimulus materials through data configuration files.

[0012] The multimedia precision synchronization control module adopts Flutter's asynchronous programming mechanism and timer function to achieve precise synchronization between multimedia stimulus playback and EEG signal acquisition. When playing or ending audio and video materials, it tags the hardware circuits or streaming data of the EEG acquisition device, physiological polyphonic transducer, or webcam to achieve alignment of multi-device and multi-modal data and generate a task log containing timestamps. The task log records the stimulus start and end times, the corresponding stimulus materials presented, and the user's key presses and screen operation events during the task.

[0013] The natural stimulus task module incorporates the simulated natural scene task into the brain-computer interface paradigm to collect corresponding EEG data according to the task type.

[0014] In the existing technology, there is no brain-computer interface system that can simultaneously achieve cross-platform consistency, flexible configuration of natural scene tasks, and high-precision multimodal data synchronization. This invention, through innovative system architecture design, combines the Flutter framework's self-drawing engine, configurable material management mechanism, and asynchronous precise synchronization technology with the natural scene task paradigm, effectively solving the aforementioned technical problems and providing a novel technical solution for the study of EEG signals in natural states.

[0015] This invention utilizes the Flutter framework's self-drawing engine (based on the Skia graphics library) to achieve a single codebase running on multiple platforms, eliminating the differences between native components on Android, iOS, and Windows platforms. This ensures complete consistency in interface layout and interaction logic across different devices, significantly reducing cross-platform development costs and improving system adaptation efficiency. It integrates various natural scene tasks such as resting with eyes closed, listening to music, watching videos, reading aloud, and eating. Through a local material library management mechanism and data configuration files (such as JSON format), users can dynamically add, delete, or change instructions and multimedia materials without modifying the code, meeting the personalized needs of different experimental designs and expanding the application scenarios of brain-computer interface tasks. It leverages Flutter asynchronous programming and timers to achieve millisecond-level synchronization between multimedia stimulation and EEG acquisition. By tagging EEG devices, physiological multi-channel analyzers, and cameras (including timestamps), it ensures precise alignment of EEG signals, physiological data, and video footage on the timeline, generating a traceable multi-task log and providing a high-quality data source for multimodal data correlation analysis. By incorporating complex tasks in daily life (such as chewing and audiovisual integration) into the brain-computer interface paradigm, the first brain-computer interface has achieved the acquisition of EEG signals for multi-sensory stimulation and motor coordination processes in a natural state. This provides key technical support for studying the neural processing mechanisms of the brain in real environments and promotes the brain-computer interface technology from the laboratory to practical application scenarios.

[0016] In summary, this invention, through systematic architectural innovation, breaks through the technical bottlenecks of traditional brain-computer interfaces in terms of platform compatibility, task flexibility, data synchronization accuracy, and natural scene coverage.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic block diagram of a cross-platform brain-computer interface paradigm system for natural state monitoring provided in one embodiment of this application;

[0020] Figure 2 This is a schematic diagram illustrating an example experimental task provided in one embodiment of this application;

[0021] Figure 3 This is a comparison graph of variables under different experimental conditions provided in an embodiment of this application;

[0022] Figure 4 This is a flowchart of an example experimental task provided in an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of the pre-task instructions for the example experiment of listening to music with eyes open, provided in one embodiment of this application;

[0024] Figure 6 A screenshot of a video subtask provided in one embodiment of this application.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0028] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0029] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0030] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0031] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0032] In recent years, non-invasive brain-computer interfaces (BCIs) have developed various experimental paradigms. In addition to BCIs based on sensorimotor rhythms such as motor imagery (MI), significant progress has also been made in the direction of BCIs based on non-sensorymotor rhythms. For example, methods such as steady-state visual evoked potentials (SSVEP), auditory evoked potentials (AEP), and event-related potentials (ERP) have all achieved high information transmission rates. These paradigms present different specially encoded stimulus targets in visual and auditory forms, and then use computer algorithms to decode the acquired neural signals, ultimately identifying the single target selected by the user from multiple targets.

[0033] In the next 10-20 years, transforming brain-computer interfaces from laboratory prototypes to commercial consumer products will be a key task. Most of the brain-computer interface systems mentioned above that have been well validated in academia use simple stimuli, such as visual flashes at a fixed frequency or pure tones of a specific pitch. These are standardized, artificially designed materials that differ significantly from natural stimuli in real life. Therefore, further exploration is needed to optimize testing for different life scenarios and the naturalness of the stimuli themselves.

[0034] Some researchers have conducted experiments along this line of thought, selecting specific behaviors and psychological states to explore the cognitive neural mechanisms and brain-computer interface decoding performance under corresponding tasks. Researchers focused on open-eye resting-state and closed-eye resting-state tasks, discovering differences in EEG energy across different frequency bands, particularly alpha wave energy, and differences in brain functional connectivity; differences in EEG response patterns when listening to different music or speech; neural electrical activity during vocal preparation and vocalization; and the electrophysiological signal patterns accompanying chewing and swallowing movements during natural eating, indirectly inferring eating behavior. These studies are beneficial for developing more "natural" brain-computer interface paradigms, thus better serving real-world applications in education, daily life, and health.

[0035] The development of these brain-computer interface paradigms relies heavily on suitable development tools. Currently, in both research and industry, most paradigm developers use corresponding tools for specific systems (such as MATLAB, Python, and E-prime) for programming. The advantages of these software programs are that they ensure the presentation of stimulus materials at a relatively stable rate and that the stimuli and event labels are synchronized relatively accurately (with millisecond-level latency). However, a significant portion of these software programs are commercially closed-source and quite expensive. Furthermore, these paradigms often adopt a single framework for development, resulting in significant functional limitations and hindering the construction of complex and varied user interface interactions, as well as adapting to personalized experimental needs.

[0036] The shortcomings of existing technologies include at least the following:

[0037] Poor cross-platform compatibility: Native development languages ​​need to be developed separately for different platforms (Android, iOS, Windows, etc.), which results in long development cycles, high costs, and high maintenance difficulty, making it difficult to meet the needs of rapid iteration and multi-platform deployment.

[0038] Difficulties in task integration and expansion: The high degree of coupling between task modules makes it difficult to flexibly expand the types of tasks and stimulus materials, which cannot meet the diverse and personalized research needs in brain-computer interface experiments.

[0039] Poor user experience: The interface design and interaction methods are relatively simple, lacking friendliness and flexibility, and the ecosystem effectiveness is low.

[0040] The lack of research on natural stimuli: Over-reliance on scientific research stimuli makes it impossible to simulate complex natural stimulus scenarios in real life, such as browsing the web, operating customized games, and multi-tasking with multiple sub-windows on the same screen. This limits the development of brain-computer interface technology in practical applications and makes it difficult to obtain EEG data that is closer to real life.

[0041] To resolve the above issues, please refer to [link / reference]. Figure 1 This application provides a cross-platform brain-computer interface paradigm system suitable for natural state monitoring, comprising: a cross-platform development module, developed using the Flutter framework, which uses a self-drawing engine to enable a single codebase to run on multiple platforms, avoiding differences in native components across different platforms and ensuring consistency of interface and functionality on each platform; the platforms include Android, iOS, and Windows; a task integration and expansion module, integrating multiple brain-computer interface paradigm tasks, including tasks simulating natural scenes; and a local material library management mechanism to add, delete, and modify instructions, images, audio, or video according to experimental needs, and to manage task types and stimulus materials through data configuration files. The system includes a row-related configuration; a multimedia precision synchronization control module, which uses Flutter's asynchronous programming mechanism and timer function to achieve precise synchronization between multimedia stimulus playback and EEG signal acquisition. When playing or ending audio and video materials, tags are added to the hardware circuitry or streaming data of the EEG acquisition device, physiological polyphonic transducer, or webcam to achieve alignment of multi-device and multi-modal data. A task log containing timestamps is generated, recording the stimulus start and end times, the corresponding presented stimulus materials, and user events such as key presses and screen operations during the task; and a natural stimulus task module, which incorporates the simulated natural scene tasks into the brain-computer interface paradigm to acquire corresponding EEG data based on the task type.

[0042] Specifically, the cross-platform brain-computer interface paradigm system for natural state monitoring provided by this invention focuses on three core technical objectives: "cross-platform compatibility," "natural scene task integration," and "precise synchronization of multimodal data." Through modular architecture design, it realizes the efficient application of brain-computer interface systems in natural state monitoring scenarios.

[0043] The cross-platform development module solves the compatibility issues caused by differences in native components when developing traditional brain-computer interface systems across platforms.

[0044] Using the Flutter framework as the development foundation, leveraging its "write once, run on multiple platforms" feature, it adapts to the three major platforms of Android, iOS, and Windows through a single codebase.

[0045] The UI rendering is achieved using Flutter's custom drawing engine (Skia graphics library), which abandons the native UI components of each platform (such as Android's View and iOS's UIKit). Through unified graphics rendering logic (such as layout calculation, animation drawing, and event response), it ensures that the interface layout (such as button position, font display, and window size) and interaction effects (such as click feedback and swipe animation) are completely consistent across different platforms.

[0046] It supports dynamic adaptation to the hardware characteristics of various platforms (such as screen resolution and input devices), but the core business logic (such as task scheduling and data synchronization) remains consistent across platforms to avoid functional fragmentation caused by platform differences.

[0047] The task integration and expansion module breaks through the limitations of traditional brain-computer interface task paradigms, which are limited by their single nature and poor scalability, and supports flexible configuration of tasks in natural scenarios.

[0048] The Natural Scene Task Integration integrates eight types of tasks that simulate natural states (resting quietly with eyes closed, listening to music with eyes closed, resting quietly with a black screen, listening to music with a black screen, watching a silent video, watching a video with sound, reading aloud, and eating), covering multiple dimensions of natural scenes such as resting state, auditory stimulation, visual stimulation, and motor coordination, and collecting real neural electrical activity of the brain in daily activities.

[0049] The local material library management mechanism, through the construction of an independent stimulus material management module, allows users to customize and add instruction text (such as task description pop-ups), images (such as natural scene images), audio (such as environmental sound effects and music), and videos (such as silent / sound natural videos) through the interactive interface provided by the system (such as file import windows and material category lists). It supports version control of materials (such as marking "experimental version" and "official version") and category management (grouping by task type and stimulus modality), and provides deletion and modification functions to meet the personalized needs of different experiments for stimulus materials.

[0050] The data configuration file association mechanism defines the mapping relationship between tasks and stimulus materials using a lightweight JSON-formatted configuration file. The configuration file contains three core fields: a task type field (e.g., "reading aloud task," "eating task"); a stimulus material path field (pointing to the storage path of specific files in the local media library, such as " / audio / music1.wav" or " / video / nature_video.mp4"); and task parameter fields (e.g., video playback duration, audio volume, and instruction display time). By modifying the configuration file, tasks can be added or stimulus materials changed without secondary development, significantly improving system scalability.

[0051] The multimedia precision synchronization control module solves the problems of asynchronous multimedia playback and EEG acquisition time, and difficulty in aligning data from multiple devices in traditional systems.

[0052] Asynchronous programming and timer collaboration leverage Flutter's asynchronous task scheduling mechanism (such as Future, async / await) to trigger audio / video playback or stop operations, ensuring parallel execution of the multimedia module and the EEG acquisition module. Combined with a high-precision timer (accuracy up to 1ms) to record operation execution time in real time, synchronous trigger signals are generated at the start / end time of stimulation, controlling the time deviation between the EEG acquisition device and the multimedia playback module within the millisecond range (traditional systems typically have a deviation of hundreds of milliseconds).

[0053] Multi-device tagging and data alignment involves sending a trigger command containing a precise timestamp (formatted as "YYYY-MM-DD HH:mm:ss.SSS") to the EEG acquisition device via wired (e.g., USB, HDMI) or wireless (e.g., Bluetooth, Wi-Fi) communication when audio / video playback or termination occurs. This inserts a task event marker into the acquired EEG signal data stream. Simultaneously, a task status marker with the same timestamp is inserted into the data stream of a physiological multi-channel instrument (e.g., heart rate, respiratory sensor). The corresponding task type at the time point is recorded in the video stream metadata of the webcam (e.g., "Watching video with sound - 00:15:23"). Finally, a timeline calibration algorithm precisely aligns the three types of data—EEG signals, physiological data, and video footage—to generate a task log containing timestamps. The log content includes: stimulus material name, start / end time, user key operation records (e.g., pause button during task), and screen touch events (e.g., swipe to switch tasks).

[0054] The natural stimulation task module incorporates natural life scenarios into the brain-computer interface paradigm, enabling EEG signal acquisition in complex scenarios. Dedicated acquisition logic is designed for different natural tasks, including: resting-state tasks (eyes closed / screen off): acquiring baseline EEG signals (such as alpha and beta waves) without strong external stimuli to establish an individual's resting-state EEG baseline; sensory stimulation tasks (listening to music, watching videos): simultaneously acquiring response signals from the auditory and visual cortices to analyze neural electrical activity (such as P300 and N100 components) during multi-sensory integration; and motor coordination tasks (reading aloud, eating): capturing pre-activation signals of the motor cortex during vocal preparation (such as μ rhythm inhibition) and somatosensory cortical feedback signals accompanying chewing and swallowing movements, filling the gap in traditional brain-computer interfaces for dynamic action scenarios. The task process is standardized, including automatically loading the corresponding configuration file before each task begins, synchronously triggering multimedia stimulation and EEG acquisition; and automatically generating raw data files with timestamps and task logs after the task ends, facilitating subsequent offline analysis.

[0055] The environment setup involves writing the core business logic in the Dart language and compiling it using the Flutter SDK to generate installation packages for each platform (.apk for Android, .ipa for iOS, .exe for Windows), eliminating the need to rewrite code for different platforms.

[0056] The interface rendering includes all UI components (buttons, text boxes, sliders) which are custom-drawn based on the Skia graphics library. For example, the CustomPaint component is used to achieve a unified button click ripple effect across platforms, avoiding reliance on the native styles of iOS's UIButton or Android's AppCompatButton.

[0057] Event handling uses Flutter's event dispatch mechanism (such as GestureDetector) to uniformly handle input events (touch screen clicks, mouse clicks, keyboard key presses) on different platforms, ensuring that the same operation triggers the same functional logic on each platform.

[0058] The media library operation includes users clicking the "Import" button to select local files (supporting formats such as txt, jpg, mp3, mp4, etc.) through the "Media Management" interface provided by the system. The system automatically generates a media index and stores it in a local database (such as SQLite). It also supports batch deletion of expired media or rollback to historical versions through the version management function (such as restoring instruction text that was accidentally deleted in an experiment).

[0059] Example JSON configuration snippets include:

[0060] {;

[0061] "task_name": "Task to watch audio videos";

[0062] "stimulus_path": {;

[0063] "video": ;" / storage / emulated / 0 / stimulus / video / nature_scene.mp4",;

[0064] "instruction": ;" / storage / emulated / 0 / stimulus / text / guide_video.txt";

[0065] },;

[0066] "parameters": {;

[0067] "play_duration": 60,;

[0068] "volume": 0.8;

[0069] };

[0070] };

[0071] The system automatically loads the configuration file upon startup and associates tasks with stimulus materials by parsing JSON fields.

[0072] The synchronous triggering mechanism encapsulates asynchronous task scheduling logic (such as Future) by creating a separate sync_controller class in Flutter. <void>The `playVideo(String path)` function internally generates a high-precision timer using `Timer.periodic`, which checks the multimedia playback status every 1ms. When the video playback starts, it sends a trigger command to the EEG acquisition device via the Flutter Platform Channel (e.g., sending the string "TRIGGER_START_001" with the current timestamp via serial communication).

[0073] After receiving instructions from the EEG acquisition device, the multi-device tagging process adds event markers to the currently acquired EEG data frames (e.g., writing "VIDEO_START@18:05:20.345" to the annotation channel of the EEG signal); the physiological polyacryloscope receives timestamps through the API interface and inserts task events of the same format into the data stream; the webcam embeds task status information into the video stream metadata through the RTSP protocol, and finally aligns the three types of data according to the timestamps using the data analysis tools provided by the system (such as the accompanying PC software).

[0074] The implementation process of the natural stimulation task module includes: Task selection and loading: When the user selects the "eating task" in the UI interface, the system reads the corresponding configuration file, loads the "chewing action guidance" text and timing parameters; Data acquisition: When the user starts chewing, the system synchronously plays a prompt sound (triggers the audio module) and sends a start acquisition command to the EEG device, while simultaneously recording the chewing action video through the camera; Task completion processing: After the preset duration ends, the system stops audio playback, saves the EEG data file (.edf format), and records events such as "chewing start time 18:07:12" and "swallowing action triggered 3 times" in the task log.

[0075] Traditional cross-platform development requires writing UI and some logic separately for each platform. This invention achieves three-platform coverage through a single Flutter codebase, reducing the amount of repetitive development work. The self-drawing engine completely eliminates the display differences of native components (such as jagged font rendering on Android and deviations in button rounded corners on iOS), ensuring that the interface is completely consistent on different devices and improving the consistency of user operation experience.

[0076] Researchers can update stimulus materials independently through an interactive interface without relying on programmers, reducing the response time from "hours of code modification" to "minutes of interface operation," making it particularly suitable for neuroscience research that requires frequent adjustments to experimental protocols. Incorporating everyday actions such as "eating" and "reading aloud" into brain-computer interface tasks fills the gap in traditional systems that only support simple laboratory stimuli, providing a key tool for studying the neural mechanisms of the brain in real life (such as language production and the neural regulation of eating behavior).

[0077] In traditional systems, data alignment across multiple devices requires manual calibration (which takes several hours). This invention achieves "instant alignment" of data by automatically tagging and timestamping, significantly reducing preprocessing time.

[0078] Millisecond-level synchronization precision ensures a strict correspondence between EEG signals and stimulus events (such as precise matching between changes in video footage and visual evoked potentials), avoiding misinterpretation of neural signals due to time deviations and improving the credibility of experimental conclusions.

[0079] Clinical diagnostic assistance involves collecting brainwave signals from patients during natural eating to aid in the diagnosis of neurological diseases related to swallowing disorders (such as Parkinson's disease and post-stroke sequelae); Human-computer interaction innovation involves decoding brainwave signals based on natural scenarios to develop brain-controlled devices that are closer to daily life (such as controlling smart homes through "reading intention" brainwave signals), thus promoting brain-computer interface technology from the laboratory to consumer applications.

[0080] This invention, through the deep integration of Flutter cross-platform architecture, configurable natural task paradigm, and multi-device precise synchronization technology, constructs the first brain-computer interface system suitable for natural state monitoring. It breaks through the multiple bottlenecks of traditional technologies in terms of platform compatibility, task flexibility, and data synchronization accuracy, and provides a brand-new technical path for the research and application of EEG signals in real-world scenarios.

[0081] In some embodiments, the step of using a self-drawing engine to run a single codebase on multiple platforms, avoiding differences in native components across different platforms and ensuring consistency of interface and functionality on each platform, includes: the self-drawing engine is based on the Skia graphics library for rendering, and eliminates differences in the display of native components across multiple platforms through unified graphics rendering logic, so that the same codebase, after compilation, maintains consistency in interface layout, interactive effects, and functional implementation on different platforms.

[0082] Cross-platform UI rendering is achieved through the Flutter self-drawing engine (based on the Skia graphics library). By using a unified graphics rendering logic (layout calculation, animation drawing, event response), the display differences of native components on Android, iOS, and Windows platforms are eliminated, ensuring that the UI layout, interaction effects, and functional implementation are completely consistent across different platforms after the same code is compiled.

[0083] Skia graphics library rendering: All UI components (buttons, text, input fields, etc.) are custom-drawn using Flutter's CustomPaint, Canvas, and other drawing interfaces, rather than calling native UI components from each platform. For example, button styles are set with rounded corners, shadows, and click ripple effects using the Paint class, resulting in the same visual effect on Android and iOS; text rendering uses Flutter's built-in font rendering engine, avoiding reliance on Android's Typeface or iOS's UIFont native font rendering differences. Unified interaction logic: Touch, mouse, and keyboard events are handled uniformly through Flutter's event handling mechanism (such as GestureDetector), ensuring that clicks, swipes, and other operations trigger the same functional logic across platforms.

[0084] Cross-platform layout adaptation utilizes Flutter's LayoutBuilder, MediaQuery and other components to dynamically obtain screen size and resolution, and uses a unified layout algorithm (such as elastic layout Row / Column, constrained layout ConstrainedBox) to achieve interface adaptation on different devices (phones, tablets, PCs) while maintaining consistent element proportions and spacing.

[0085] To avoid UI inconsistencies caused by native component characteristics (such as rounded corners for iOS buttons and Material Design on Android), ensuring users have the same user experience across different devices. No need to debug the UI separately for each platform; a single codebase is adapted to all three platforms simultaneously, reducing cross-platform compatibility debugging time. Subsequent UI feature iterations (such as adding interactive animations) only require modification to a single codebase, avoiding version conflicts caused by omissions in multi-platform code synchronization.

[0086] In some embodiments, the tasks simulating natural scenes include at least the following tasks: resting quietly with eyes closed, listening to music with eyes closed, resting quietly with a black screen, listening to music with a black screen, watching silent videos, watching videos with sound, reading aloud, and eating. The collection of corresponding EEG data based on the task type includes: collecting EEG signals in the resting quietly with eyes closed; collecting EEG responses while attentively listening to music in the listening to music with eyes closed; collecting EEG signals in the resting quietly with a black screen open; collecting EEG signals while listening to music with a black screen open; collecting EEG signals while watching silent natural videos in the watching silent video task; collecting audiovisual integration EEG signals while watching natural videos with sound in the watching videos with sound task; collecting neural electrical activity signals during vocal preparation and vocalization in the reading aloud task; and collecting EEG signals accompanying natural eating actions in the eating task, where natural eating actions include at least chewing and swallowing.

[0087] By predefining eight types of brain-computer interface tasks that simulate natural scenarios, and designing exclusive EEG signal acquisition logic for different task types, covering natural states such as resting state, sensory stimulation, and motor coordination, the brain's neural electrical activity in real-life scenarios is collected.

[0088] Task type and data collection logic: Close your eyes and rest quietly. Task: Turn off the screen display, prompt the user to close their eyes and sit quietly, and collect basic resting EEG (mainly alpha waves) without visual stimulation.

[0089] Reading aloud task: Display text content and prompt the user to read it aloud, while simultaneously collecting electrical activity of the motor cortex and language cortex (such as β-wave enhancement) during the vocalization preparation stage (neural pre-activation signals before lip and tongue muscle movement) and during the vocalization process.

[0090] Eating task: Provide solid / liquid food (to be prepared by the user), prompt the user to chew and swallow naturally, and collect somatosensory cortical feedback signals (such as μ rhythm inhibition) accompanying masticatory muscle movement and brainstem electrical activity related to swallowing reflex.

[0091] Task triggering mechanism: After the user selects a task in the UI, the system automatically loads the corresponding acquisition parameters (such as task duration and signal sampling rate) and guides the task execution through voice / text prompts (e.g., "Reading will begin in 3 seconds, please get ready").

[0092] Filling the gap in traditional brain-computer interfaces that only support simple laboratory stimuli (such as flashing light), this invention achieves, for the first time, the acquisition of EEG signals for complex daily actions (eating, language production), providing data support for studying neural mechanisms in real-world environments. Resting-state tasks (eyes closed / screen off) are used to establish an individual's EEG baseline; sensory stimulation tasks (listening to music, watching videos) analyze multi-sensory integration mechanisms; and motor coordination tasks (reading aloud, eating) capture motor-neural coupling signals, forming a complete natural-state EEG dataset.

[0093] In some embodiments, the development using the Flutter framework includes: leveraging the cross-platform capabilities of the Flutter framework, writing a single codebase in the Dart language, and adapting it to the compilation environments of Android, iOS, and Windows platforms to achieve one-time development and multi-platform deployment.

[0094] Leveraging the cross-platform capabilities of the Flutter framework, a single codebase can be written in the Dart language and adapted to compilation environments (such as Android Studio, Xcode, and Visual Studio) on Android, iOS, and Windows platforms, enabling "develop once, deploy across multiple platforms".

[0095] Single codebase architecture: Core business logic (task scheduling, data synchronization, material management) is all written in Dart language and does not depend on platform-specific APIs; only the underlying hardware communication (such as serial port connection to EEG devices) achieves cross-platform adaptation through the Flutter platform channel, ensuring that more than 95% of the code is reusable.

[0096] The multi-platform compilation process includes: Android: APK file is generated by compiling with the Flutter command flutter build apk, which automatically adapts to ARM / x86 architecture and different Android versions (API 21+); iOS: IPA package is generated using flutter build ipa, which supports iPhone / iPad devices and simulators; Windows: EXE file is compiled with flutter build windows, which supports 32-bit / 64-bit systems.

[0097] Compared to traditional cross-platform solutions (such as React Native, which requires maintaining platform-specific code), Flutter's single codebase reduces development workload, making it especially suitable for research teams with limited resources to quickly deploy systems. The entire process, from code writing to compilation and deployment, is unified, avoiding functional deficiencies caused by platform differences (such as platform-specific crash issues) and improving system stability.

[0098] In some embodiments, the establishment of a local material library management mechanism to add, delete, and modify instructions, images, audio, or video according to experimental needs includes: building a stimulus material management module in local storage to import custom instruction texts, image files, audio files, and video files as stimulus materials through an interactive interface, and classifying and managing the imported materials for version control, and flexibly replacing or updating the material library content according to the experimental design.

[0099] By building a stimulus material management module in local storage, the system supports users to import custom stimulus materials such as instruction text, images, audio, and video through an interactive interface. It also provides categorized management (by task type / modality) and version control functions to enable flexible updates to the material library.

[0100] The media management module architecture includes: Interactive Interface: A dedicated "Media Library" page with a file import button, a media list (displaying name, type, and update time), category filters (e.g., "Audio Media" and "Visual Media"), and a version rollback button; Storage Mechanism: Media files are stored in a fixed local directory (e.g., / sdcard / stimulus / on Android, C: / StimulusLibrary / on Windows), and index information (path, type, and version number) is stored in an SQLite database, supporting fuzzy search (e.g., searching all audio media by the keyword "music"); Version Control Implementation: A version number (e.g., "v1.0" or "v1.1") is automatically generated each time media is modified. Users can restore to any version through history, preventing media loss due to accidental deletion or modification.

[0101] Researchers can manage materials independently without programming, reducing the response time to experimental design changes from "hours of code development" to "minutes of interface operation," significantly improving experimental flexibility. Categorization and version control functions ensure the standardization of material usage in complex experiments (such as distinguishing between "training materials" and "test materials"), reducing human error.

[0102] In some embodiments, the configuration of task types and stimulus materials through a data configuration file includes: defining the correspondence between tasks and stimulus materials using a JSON-formatted data configuration file; setting task type fields, stimulus material path fields, and task parameter fields in the configuration file; and modifying the corresponding field content in the configuration file when adding new tasks or changing stimulus materials.

[0103] The mapping relationship between tasks and stimulus materials is defined by using a JSON-formatted configuration file. The configuration file contains fields such as task type, stimulus material path, and task parameters (such as playback duration and volume). Tasks can be added or materials can be changed by modifying the configuration file without modifying the code.

[0104] The JSON configuration file structure includes:

[0105] {;

[0106] "tasks": [;

[0107] {;

[0108] "task_id": 1,;

[0109] "task_name": "Task to watch audio videos";

[0110] "stimulus": {;

[0111] "video_path": " / stimulus / video / nature_video.mp4",;

[0112] "audio_path": " / stimulus / audio / bird_sound.mp3",;

[0113] "instruction_text": "Please focus on watching the video below";

[0114] },;

[0115] "parameters": {;

[0116] "duration": 60;

[0117] "volume": 0.7,;

[0118] "signal_sampling_rate": 1000;

[0119] };

[0120] }; ];

[0122] };

[0123] Upon system startup, the jsonDecode function parses the configuration file. During runtime, it reads corresponding fields based on the user-selected task; for example, when playing video, it retrieves the file path from the video_path field and controls the playback duration using the duration field. This allows non-technical personnel to quickly adjust experimental plans (such as changing video materials or extending task duration) by modifying JSON fields, breaking down technical barriers and improving research efficiency. Adding a new task only requires adding fields to the configuration file, without modifying the core code logic, laying the foundation for future expansion to more natural scene tasks (such as writing and walking).

[0124] In some embodiments, the use of Flutter's asynchronous programming mechanism and timer function to achieve precise synchronization between multimedia stimulus playback and EEG signal acquisition includes: triggering the playback or stop operation of multimedia files through Flutter's asynchronous task scheduling mechanism, recording the operation execution time in combination with a high-precision timer module, and synchronously generating trigger signals at the start or end time nodes of multimedia stimulation to ensure that the time deviation between the EEG signal acquisition device and the multimedia playback module is controlled within the millisecond range.

[0125] By utilizing Flutter's asynchronous programming (Future, async / await) to trigger multimedia playback / stop, and combining it with a high-precision timer (1ms accuracy) to record operation time, a synchronous trigger signal is generated at the start / end of stimulation, controlling the time deviation between EEG acquisition and the multimedia module to the millisecond level.

[0126] Asynchronous task scheduling is achieved by defining a MultimediaController class, which encapsulates playback functions (such as Future). <void>The `playAudio(String path)` function internally uses the `await` keyword to ensure that subsequent logic (such as sending a trigger signal) is executed only after the playback operation is complete. It uses `Timer.periodic(Duration(milliseconds: 1), (timer) =>checkPlayState())` to monitor the audio / video playback status in real time and capture the precise start time. When multimedia playback is detected to have started, a trigger signal containing the current timestamp (accurate to milliseconds) is immediately generated and sent to the EEG acquisition device driver via the Flutter platform channel to trigger synchronous acquisition.

[0127] The asynchronous programming mechanism ensures that multimedia playback and EEG acquisition run independently without blocking each other, improving the system's real-time performance (such as playing video and displaying EEG waveforms in real time).

[0128] In the Flutter system, trigger signals are exchanged with the EEG acquisition device driver through the Platform Channel. The specific communication method and protocol implementation are as follows:

[0129] The Flutter platform channel mechanism enables communication between the Dart layer and the native platform (Android / iOS / Windows) through MethodChannel (for bidirectional method calls) or EventChannel (for data stream transmission). The core process is as follows: Dart side: When a multimedia playback event (such as start / end) is detected, trigger signal data containing a timestamp is generated and sent to the native side through the platform channel. Native side: Receives the message from the Dart side, parses it, and sends a physical trigger signal to the EEG device through a device-specific communication interface (such as USB serial port, Bluetooth, TCP / IP), and records the response time.

[0130] The trigger signal data format uses JSON to encapsulate key information, ensuring consistent parsing across platforms. An example is shown below:

[0131] {"event_type": "stimulus_start", / / Event type ("stimulus_start" / "stimulus_end");

[0132] "timestamp": "2025-10-20T05:30:40.474Z", / / ISO 8601 timestamp accurate to milliseconds;

[0133] "task_id": "video_task_001", / / Unique identifier for the task;

[0134] "stimulus_type": "video", / / Stimulus type (audio / video / image, etc.);

[0135] "stimulus_path": " / video / nature_scene.mp4" / / Stimulus material path;};

[0136] The hardware communication protocol implementation (taking USB serial port as an example) includes: 1. Windows platform (USB serial port communication): Native end (C++ / C#): Use the SerialPort class (C#) or CreateFile API (C++) to open the serial port and configure parameters such as baud rate (e.g., 115200), data bits, and stop bits. Convert JSON data to a byte stream (e.g., UTF-8 encoding), and append a fixed-format frame header and trailer (e.g., 0xAA and 0xBB) to avoid packet fragmentation issues: [Frame header 0xAA][Data length][JSON byte stream][Checksum][Frame trailer 0xBB] Send the byte stream through the serial port. After receiving it, the EEG device parses the timestamp and inserts event markers into the data stream.

[0137] Android platform (USB Host mode): Native side (Java / Kotlin): Uses UsbManager to obtain device permissions and sends byte data via UsbDeviceConnection. The protocol is similar to Windows, using a binary format with frame headers and trailers to ensure reliable reception on the device.

[0138] Example code snippet (Kotlin):

[0139] val bytes = jsonString.toByteArray(Charsets.UTF_8);

[0140] val packet = byteArrayOf(0xAA) + bytes.size.toByte() + bytes +checksum(bytes) + byteArrayOf(0xBB);

[0141] usbDeviceConnection.bulkTransfer(writeEndpoint, packet, packet.size,TIMEOUT);

[0142] iOS platform (Bluetooth / BLE communication): Native client (Swift): Connects to the EEG device via the Core Bluetooth framework and sends data using Characteristic. Due to BLE packet length limitations (typically ≤20 bytes), JSON data needs to be split into multiple packets or compressed before transmission to ensure timestamp accuracy is not lost. The device receives the data, reassembles it, parses the timestamp, and triggers a synchronization flag.

[0143] Network communication protocol (for TCP / IP devices): If the EEG device supports network connectivity (e.g., via Wi-Fi or Ethernet), the TCP socket protocol is used: Dart side: Directly create a socket connection using the dart:io library to connect to the device's IP address and port. Data format: Transmit directly using JSON strings, appending newline characters (\n) as message separators: {"event_type":"stimulus_start","timestamp":"2025-10-20T05:30:40.474Z",...}\n; Device side: Listen on the port, parse the JSON data line by line, and update the event markers in real time.

[0144] Timestamp calibration and error control: Hardware time synchronization: Both the EEG device and the system host (the device running the Flutter program) are connected to the NTP server to ensure clock synchronization (error ≤ 1ms). Before sending the trigger signal from the native device, a high-precision timestamp is obtained through System.currentTimeMillis() (Android), CACurrentMediaTime() (iOS), and DateTime.now().millisecondsSinceEpoch (Dart) to ensure alignment with the device's acquisition clock.

[0145] Error compensation is achieved by recording the "signal transmission time" and "device reception time" in the task log and compensating for transmission delay through offline calibration algorithms (such as linear regression), ultimately controlling the time deviation between the EEG signal and the stimulus event to ≤5ms.

[0146] By unifying the interaction logic between Dart and the native device through a platform channel, the underlying hardware communication is implemented independently by each platform, shielding the differences between protocols such as USB / BLE / TCP. When adding a new device type, only the communication protocol needs to be extended on the native device (such as Modbus or CAN bus), without modifying the Dart layer logic. Combined with JSON timestamp format and hardware clock calibration, the system ensures that the time correspondence between EEG signals and stimulus events reaches experimental-grade requirements (millisecond level). Through this approach, the system achieves precise transmission of trigger signals from the Flutter application layer to the EEG device hardware layer, providing reliable technical support for multimodal data alignment.

[0147] In some embodiments, the step of tagging the hardware circuitry or streaming data of the EEG acquisition device, physiological polygraph, or webcam when playing or ending audio and video materials to achieve alignment of multi-device and multimodal data includes: sending a trigger command containing a timestamp to the EEG acquisition device via wired or wireless communication; inserting a task event marker into the data stream of the physiological polygraph; and recording the task status at the corresponding time point in the video stream metadata of the webcam, so that the EEG signal, physiological data, and video image are precisely aligned on the timeline to form a traceable multimodal data association record.

[0148] At the start / end of multimedia stimulation, a time-stamped trigger command is sent to the EEG acquisition device, physiological polyphony analyzer, and webcam via wired (USB / serial port) or wireless (Bluetooth / Wi-Fi) communication. Task event markers are inserted into the data streams of each device to achieve timeline alignment of multimodal data.

[0149] Multi-device communication protocol: EEG acquisition device: Sends the command "TRIGGER|TASK_START|2025-07-23 18:08:56.093" via USB serial port, adding annotation channel markers to the acquired EEG data; Physiological multichannel analyzer: Inserts events into the heart rate and respiratory data streams by calling the insertEventMarker(timestamp, taskName) method via Bluetooth API; Webcam: Adds the metadata field x-task-state: {"time":"18:08:56.093","type":"Watch video with sound"} to the SDP description of the RTSP video stream. Data alignment algorithm: During subsequent data analysis, based on the timestamps in the data from each device, small time differences are calibrated using linear interpolation or Dynamic Time Warping (DTW) algorithms to generate an aligned multimodal dataset.

[0150] The time reference system design (core alignment foundation) includes: The global time reference definition adopts UTC time (high-precision atomic clock synchronization) as a unified time reference across devices. All devices must record a UTC timestamp in ISO8601 format (accurate to milliseconds, e.g., 2025-10-20T14:30:45.123Z) during data acquisition to ensure: Hardware layer: Professional equipment such as EEG acquisition devices and physiological polygraphs have built-in high-precision clocks (error ≤ 1ppm, i.e., 1-day error ≤ 8.64ms), supporting calibration with the host clock via NTP (Network Time Protocol) or hardware synchronization lines (e.g., IEEE 1588 PTP). Software layer: Flutter applications, cameras, and the host system are all synchronized to a laboratory-grade time server (e.g., Stratum 1 level, synchronization accuracy ≤ 1ms) via an NTP client.

[0151] Device timestamp acquisition mechanism: EEG devices: When the stimulus trigger signal arrives, a timestamp (marked as T_eeg) is generated by the device's hardware clock, ensuring strict alignment with the EEG sampling clock (error ≤ 1 / 2 of the sampling period, e.g., error ≤ 2ms for 250Hz sampling). Physiological polyphonic transducers: The hardware timestamp (T_physio) is obtained by calling the device API, or the host timestamp is injected by the host during Bluetooth / USB communication (communication latency compensation is required). Webcams: The host's real-time UTC timestamp (T_cam) is embedded in the RTSP stream metadata. The host obtains the high-precision time (approximately 1ms precision at the Dart layer) via DateTime.now().toIso8601String().

[0152] The time difference quantification standard (error assessment and threshold setting) includes: 1. Communication delay measurement method: Before the experiment, the time difference between devices is quantified by testing the round trip using a trigger signal. The steps are as follows: The host sends an empty trigger command (containing only the timestamp T0) to the target device; after receiving the signal, the device immediately sends back an acknowledgment packet (carrying the device reception time T1); the host calculates the one-way delay Delay = (T1 - T0) / 2, repeats the test 100 times, and takes the average value ± standard deviation as the inherent delay parameter of the device (e.g., the average delay of a USB device is 5ms, and the standard deviation is 1ms; the average delay of a Bluetooth device is 15ms, and the standard deviation is 5ms). Acceptable time difference threshold: Based on the device type and data modal characteristics, a three-level error tolerance is set, such as:

[0153] ;

[0154] Alignment requirements are highest (millisecond level) through electrical neural signals (EEG), but can be relaxed to sub-frame level (≤33ms) due to frame rate limitations in video streams.

[0155] The multimodal alignment algorithm implementation (scenario-specific selection) includes: 1. Trigger event anchor point alignment (basic layer algorithm): Applicable conditions: All devices generate timestamps at the same trigger event (e.g., stimulus start / end), and there are at least 3 non-collinear trigger points (to avoid single trigger point from failing to calibrate drift). Steps: Collect the timestamps of all devices in the same trigger event: {T_eeg, T_physio, T_cam, T_host} (T_host is the host transmission time); Calculate the time offset of the device relative to the host: \Delta T_{eeg} = T_eeg - T_host - \text{USB delay compensation value}; \Delta T_{physio} = T_physio - T_host - \text{Bluetooth delay compensation value}; Perform linear regression on the offsets of multiple triggers to fit the device clock drift model (e.g., \Delta T = a \cdot t + b, where a is the drift rate and b is the initial offset). By eliminating fixed delay and linear drift, it is suitable for wired connected devices (delay stable) with alignment accuracy up to ±2ms.

[0156] Dynamic Time Warping (DTW) Algorithm (Adapted for Complex Scenarios): Applicable Conditions: Non-linear time deviation exists between devices (e.g., Bluetooth devices experience delay jitter due to signal interference); data streams have periodic characteristics (e.g., respiratory and heart rate waveforms of physiological signals, time series of video frames). Implementation Steps: Extract feature point sequences of each modality of data (e.g., event-related potential peaks in EEG, R-wave peaks in physiological signals, keyframe times in video frames); Construct a time distance matrix D(i,j), representing the absolute value of the time difference between the i-th sample of EEG and the j-th sample of physiological signal; Find the optimal path W = (w_1,w_2, ..., w_k) through dynamic programming, satisfying path constraints (e.g., step size ≤ 2, avoiding excessively large spans), minimizing the total distance: W* = \arg\min \sum_{k=1}^KD(w_k); Generate a time mapping function based on the optimal path, resampling the physiological / video data onto the EEG time axis. It can adapt to irregular delays and supports offline high-precision alignment. Its disadvantage is high computational complexity (O(NM), where N / M is the data length), making it suitable for offline data analysis.

[0157] Linear interpolation alignment based on a common time axis (real-time processing) includes: applicable conditions: the clock drift between devices is linear (the residual drift rate is ≤10ppm after triggering anchor calibration); real-time data stream alignment is required (such as online brain-computer interface scenarios).

[0158] Implementation steps: Define the time axis T_master of the master device (e.g., EEG device) as the baseline; for the time series T_slave of the slave device (physiological / video), calculate the mapping relationship using the calibrated offset \Delta T and drift rate a: T_slave' = T_master - (\Delta T + a \cdot T_master); fill missing time points with linear interpolation or nearest neighbor sampling to ensure data points are aligned on the common time axis. This method is computationally efficient (O(N)) and suitable for real-time systems. Accuracy depends on the initial calibration parameters (it is recommended to insert a trigger anchor update parameter every 10 seconds).

[0159] Anomaly handling and robustness optimization include: Outlier detection: Outliers with trigger time differences exceeding 3 times the standard deviation are repaired through median filtering or nearest-neighbor interpolation; Multi-anchor redundancy: Each stimulus event sends 3 trigger signals (1ms interval), and the median value is used as the effective timestamp to reduce the impact of single communication errors; Clock drift monitoring: The initial calibration time and drift rate parameters of the device are recorded in the header of the data file for dynamic compensation during subsequent offline alignment. Through the above time reference system, quantization standard, and hierarchical alignment algorithm, the system can achieve alignment accuracy of ≤5ms in wired scenarios and ≤20ms in wireless scenarios, meeting the strict time synchronization requirements of multimodal data analysis such as neuroscience experiments and psychological research. In practical applications, it is recommended to select the optimal algorithm combination based on the device communication characteristics (latency stability, data frame rate) and the analysis scenario (real-time / offline) to ensure the traceability and correlation of multimodal data on the timeline.

[0160] EEG signals, physiological indicators (heart rate variability), and video footage (facial expressions / movements) are precisely aligned on the timeline, supporting complex correlation analysis (such as changes in theta wave power during chewing movements). The task log records the time and type of each stimulus event in detail, facilitating the reproduction of the experimental process and the screening of data anomalies (such as whether a sudden change in a certain segment of EEG signal corresponds to a sudden scene in the video).

[0161] In some embodiments, the system further includes an intelligent algorithm analysis module, which is configured to: construct an EEG signal decoding model based on the collected EEG data and corresponding task logs using machine learning algorithms, wherein the machine learning algorithms include any one of support vector machines, random forests, and convolutional neural networks; identify neural activity patterns corresponding to preset tasks by training on EEG signal features under different task scenarios; the preset tasks include identifying closed-eye rest, listening to music, watching videos, reading aloud, and eating; output task type classification results or user intent recognition results; and support personalized model updates based on user historical data to improve the decoding performance and natural scene adaptation capabilities of the brain-computer interface paradigm.

[0162] By integrating an intelligent algorithm analysis module, based on the collected EEG data and task logs, the EEG signal decoding model is trained using machine learning algorithms such as Support Vector Machine (SVM), Random Forest (RF), and Convolutional Neural Network (CNN) to identify neural activity patterns corresponding to different tasks and support personalized model updates.

[0163] Algorithm Flow: Data Preprocessing: Filtering the EEG signal (removing power frequency interference), segmenting (extracting signal segments according to task timestamps), and feature extraction (calculating power spectra of each frequency band, event correlation desynchronization / synchronization EDR / ERS); Model Training: Inputting the feature vectors into the SVM / CNN model, and performing supervised learning with task type as the label. For example, the CNN model extracts time series features through one-dimensional convolutional layers, and the fully connected layers output classification probabilities; Personalized Updates: After each experiment, new data is added to the training set, and the model parameters are updated through incremental learning algorithms to adapt to the differences in EEG signals of different users (such as individual differences in alpha wave frequency).

[0164] In the filtering process of EEG signals, after 50Hz notch filtering (to remove power frequency interference), band-pass filtering is usually required. These two methods have different but complementary functions, for the following reasons: Power frequency notch filter: Specifically designed for power grid frequency interference (50Hz in China, 60Hz in some countries). It attenuates narrowband noise centered at 50Hz using a notch filter (such as an IIR band-stop filter) (typically set with a bandwidth of ±1Hz, i.e., 49-51Hz), avoiding contamination of the EEG signal by power frequency harmonics. Band-pass filter: Extracts the signal components of interest based on the target EEG frequency band (such as alpha and beta waves), eliminating irrelevant frequency noise. For example, when studying resting-state alpha waves, it is necessary to retain the 8-12Hz band and filter out low-frequency noise such as delta (1-4Hz) and theta (4-8Hz), as well as high-frequency electromyographic interference above beta (12-30Hz). When studying event-related potentials (ERPs), broadband filtering (such as 0.1-30Hz) may be needed to retain slow potential changes and high-frequency components.

[0165] Example of filtering process: Raw EEG signal → 50Hz notch filter (removing power frequency) → Bandpass filter (preserving target frequency band) → subsequent processing; Key parameter configuration: Notch filter: Use an IIR filter (such as Butterworth bandstop), Q value set to 50 (narrowband high selectivity) to avoid excessive phase distortion; Bandpass filter: Select the frequency band according to the task type, for example: Phase compensation: Perform zero-phase filtering on the filtered signal (such as using scipy.signal.filtfilt) to eliminate linear phase delay and ensure the time accuracy of event-related signals.

[0166] In the feature extraction process, each frequency band can be divided into 5 basic frequency bands according to the international 10-20 electrode lead system standard:

[0167]

[0168] Output Interface: Provides a real-time decoding API, supporting the output of model classification results to external devices (such as a brain-controlled wheelchair executing actions based on the decoding result of the "turn left" task). The model is dynamically optimized based on user historical data, addressing the pain point of "large individual differences" in brain-computer interfaces, upgrading the system from "general-purpose" to "personalized," and improving reliability in practical applications.

[0169] Feature extraction can include: temporal segmentation (windowing) strategy: fixed window length method: extracting signal segments according to task timestamps (e.g., -0.5s to +4s after stimulus start), with the window length set according to task type: short events (e.g., image stimulation): 200-500ms; long tasks (e.g., video viewing): sliding window (1s window length, 50% overlap) to extract dynamic features. Baseline correction: using 0.5s before stimulus as the baseline period, calculating relative energy changes (e.g., ERD / ERS requires subtracting baseline power).

[0170] Power spectrum feature extraction includes: FFT (Fast Fourier Transform): suitable for stationary signals, calculating the absolute power (μV²) or relative power (percentage of total power) of each frequency band; Welch power spectrum estimation: frame-by-frame windowing (such as Hanning window) to reduce spectral leakage, suitable for non-stationary signals.

[0171] ERD / ERS calculation (event-related desynchronization / synchronization): Formula definition: ERD / ERS(%) = (Ptask) Pbaseline / P baseline) × 100%; Ptask: target frequency band power within the task window (e.g., 0-2s after stimulation); Pbaseline: same frequency band power within the baseline window (e.g., -1s to 0s before stimulation). Reference time period selection: the baseline period is usually 1s before stimulation (to avoid trigger artifacts), and the task window is set according to the event type (e.g., 1-3s for motor imagery tasks).

[0172] Nonlinear dynamic characteristics include: Approximate Entropy (ApEn): measures signal complexity; a higher value indicates stronger signal irregularity. Calculation steps include: reconstructing the phase space (delay time τ, embedding dimension m); calculating the pattern similarity probability, reflecting long-term signal correlation. Lyapunov exponent: quantifies the degree of signal chaos; a positive value indicates a chaotic system (suitable for nonlinear dynamic analysis such as epilepsy EEG).

[0173] Time-frequency domain features include: Wavelet Transform: obtains time-frequency energy distribution, suitable for non-stationary signals (such as γ-band transient response); Hilbert-Huang Transform (HHT): adaptively decomposes the signal into intrinsic mode functions (IMFs) and extracts instantaneous frequency and amplitude.

[0174] The splicing method corresponding to feature splicing and dimensionality reduction is to splice the power spectrum of each frequency band (e.g., 5 basic frequency bands × 32 conduction electrodes = 160 dimensions), ERD / ERS values ​​(same dimension), and nonlinear features (e.g., ApEn × 32 conduction = 32 dimensions) into a high-dimensional feature vector: Feature vector = [δ power_Ch1, θ power_Ch1, ..., γ power_Ch1, δ power_Ch2, ..., ERD_α_Ch1, ..., ApEn_Ch1, ...].

[0175] Feature selection algorithms include: variance thresholding: removing invalid features with variance close to zero (such as a certain electrode that never sends a signal); recursive feature elimination (RFE): combining model weights such as SVM, recursively deleting the least important features and retaining the top 50% of key features; mutual information method: calculating the mutual information value between features and task labels, and screening highly correlated features (threshold ≥ 0.1).

[0176] An example of dimensionality optimization is that if the original feature dimension is 500, after RFE filtering, only 150 dimensions are retained, which can reduce the risk of overfitting and improve the model training efficiency.

[0177] The filtering process includes: power frequency notch filtering (mandatory) → bandpass filtering (optional depending on the task frequency band), with zero-phase filtering preferred to avoid time shift; feature engineering includes: short-time tasks (such as button response): using fixed window length + ERD / ERS features to focus on the energy changes of the event-related frequency band; long-time tasks (such as video viewing): using a sliding window to extract the dynamic power spectrum + approximate entropy to capture the time-varying complexity of the signal.

[0178] Model adaptation includes: shallow models (SVM / RF): suitable for manually designed statistical features (such as power spectrum, ERD / ERS); deep learning (CNN): directly input the original signal or time-frequency graph (such as wavelet energy graph), automatically extract hierarchical features, and has a stronger fitting ability for nonlinear features (such as chaotic parameters).

[0179] Through the above-mentioned detailed optimizations, the separability of EEG features can be significantly improved. In particular, for the problem of individual differences, the model parameters can be dynamically updated by incremental learning (such as adaptive adjustment of user-specific alpha wave frequency), achieving a key breakthrough from a general model to personalized decoding.

[0180] In some embodiments, it further includes: a user interface module that uses Flutter's UI components and animation effects to design the user interface, the user interface adding transition animations when switching tasks, and providing operation prompts and instructions for each task.

[0181] Design the user interface using Flutter's UI components and animation library, add transition animations (such as gradient transitions and swipe transitions) when switching tasks, and provide real-time operation prompts (text / voice) for each task to improve the user experience.

[0182] The UI / UX design includes: Transition animations: Flutter's AnimatedSwitcher component is used to implement task interface switching animations, such as the old interface fading out while the new interface slides in from the right. The animation duration is uniformly set to 300ms to enhance visual continuity; Operation prompts: A countdown pop-up is displayed 3 seconds before each task starts (with a voice announcement "Task is about to start, 3, 2, 1..."). During the task, the real-time status is displayed in the corner of the screen (e.g., "Current task: Reading, 45 seconds remaining"). Cross-platform UI consistency: All UI elements (font size, button size, animation curves) are uniformly defined through Flutter code to ensure consistent display ratios on mobile phones (small screens) and PCs (large screens).

[0183] Transition animations reduce the abruptness of interface switching, and operation prompts lower the learning cost for users (especially non-professional users), thus improving task completion rates. A unified prompt mechanism ensures that different users receive task instructions at the same time, reducing experimental data deviations caused by inconsistent operations (such as starting data collection before users are ready).

[0184] In some embodiments, the task difficulty (such as video complexity and audio duration) is dynamically adjusted based on the real-time EEG signal decoding results, forming a "collection-analysis-feedback" closed loop: when the user's neural activity is detected to be fatigued (such as the theta wave power continuously increasing), the task is automatically simplified; if the decoding accuracy is higher than the threshold, task interference (such as background noise) is increased to improve the efficiency of brain-computer interface training.

[0185] Real-time fatigue monitoring uses the power ratio (θ / α) of theta waves (4-8Hz) to alpha waves (8-12Hz) in EEG signals as a fatigue indicator. The average value is calculated through a real-time sliding window (30 seconds long). When the value exceeds a threshold (e.g., 0.8), the difficulty level is reduced.

[0186] Parameterized design of task difficulty: Define 3 levels of difficulty parameters for each task (e.g., the difficulty of "watching video with sound" is composed of "video frame rate", "background noise decibels" and "number of dynamic elements in the picture"), and store them in the difficulty_levels field of the JSON configuration file.

[0187] Closed-loop control logic: After each task trial is completed, the real-time decoding model is used to predict the user's current state (focus / fatigue / distraction), and the task scheduling module is called asynchronously through Future.delayed to load the stimulus material of corresponding difficulty in the next trial (such as switching from video_high.mp4 to video_low.mp4).

[0188] In some embodiments, a graph neural network (GNN) fusion model is constructed for the multimodal data of EEG, physiological (heart rate, respiration), and video (facial expression, chewing action) collected by the system: different modal data are mapped to graph nodes (EEG frequency band features are node attributes, physiological indicators are edge weights, and video frame features are graph structures), and the correlation between modalities is captured through graph convolution operations to improve the decoding accuracy of complex tasks (such as eating and reading aloud).

[0189] Multimodal data graph construction: Node definition: EEG signals are divided into 64 nodes (corresponding to 64 electrodes) according to channels. Each node attribute includes power values ​​of the δ / θ / α / β / γ frequency bands; physiological indicators (heart rate, respiratory rate) are used as global nodes.

[0190] Edge weight calculation: The Pearson correlation coefficient between EEG nodes and physiological nodes is calculated. Connections with a value greater than 0.3 are retained, and the weight is the correlation coefficient value. Video frames are used to extract motion features (such as the jaw movement trajectory during chewing) as dynamic attributes of the edges through ResNet.

[0191] GNN model architecture: It adopts a graph attention network (GAT) as the base layer and learns the importance weights of different modalities through a multi-head attention mechanism (such as the automatic increase of the association weight between the language cortex channel and the lip movement video in the reading task); the output layer combines the task log timestamps to perform temporal classification of the fused features (using the Transformer temporal coding module to handle cross-frame dependencies).

[0192] In some embodiments, for small sample scenarios (such as rare task state data missing), an EEG signal generative adversarial network is designed: the generator learns the distribution characteristics of real EEG data and generates virtual signals that conform to the laws of natural scenes; the discriminator distinguishes between real data and generated data, and improves the authenticity of generated data through adversarial training, thereby solving the model overfitting problem caused by data imbalance.

[0193] Conditional GAN ​​design: Task type labels (such as one-hot encoded "eating task" labels) are added to the input layer to guide the generator to generate EEG signals for specific tasks only, avoiding cross-task data confusion; the generator adopts a 3D convolutional structure (adapting to the time-channel-frequency three-dimensional features of EEG signals), and the discriminator uses PatchGAN to judge the authenticity of local signal segments.

[0194] Data augmentation process: Preprocess real data for rare tasks (such as "listening to rock music with eyes closed") and input it into the GAN model; after training for 500 rounds, generate augmented samples that are 10 times larger than the original data, mix them with the real data at a 1:1 ratio, and then input them into the decoding model for training.

[0195] In some embodiments, for frequent task switching in natural scenes (such as the rapid transition from "watching videos" to "eating"), a meta-learning model is designed: the system is taught to quickly adapt to new task switching patterns through "task sequence training", and the dynamic changes of neural activity are captured at the moment of switching, reducing the "switching delay" of traditional models (i.e., the problem of high data decoding error rate in the first 3 seconds after switching).

[0196] Meta-learning task design: Construct a training set containing 200 task switching sequences (such as "close your eyes and rest → listen to music → read aloud", "listen to music with black screen → eat → watch silent video", etc.), with each sequence containing 100 switching trials.

[0197] Model architecture: The MAML (Model-Agnostic Meta-Learning) algorithm is used to simulate the "task switching → rapid adaptation" process in each training task: the initial model learns general switching features on the basic task set (such as the β-wave surge in the prefrontal cortex during task switching); for new switching sequences, the classification layer parameters are quickly adjusted through gradient updates to adapt to the neural response of specific switching modes.

[0198] Real-time application logic: When a task switching event is detected (via the multi-device tag alignment signal in Example 7), the rapid adaptation module of the meta-learning model is immediately triggered to complete parameter adjustment within 500ms, ensuring that the decoding accuracy of the first second of data after the switch is ≥80%.

[0199] In some embodiments, such as Figures 2 to 6 As shown, by leveraging the advantages of the Flutter framework, a single codebase can run simultaneously on multiple platforms, including Android and iOS. Flutter employs a custom drawing engine and uses the Skia graphics library for rendering, avoiding the differences between native components on different platforms. This ensures consistency in interface and functionality across platforms, significantly shortening the development cycle and reducing development and maintenance costs. This type of program is convenient for installation on different peripherals, such as laptops, tablets, and mobile phones on different platforms. When used in conjunction with wearable EEG or multimodal physiological devices, it can complete experimental testing in natural scenarios.

[0200] The rich variety of tasks includes multiple brain-computer interface paradigms such as resting quietly with eyes closed, listening to music with eyes closed, resting quietly with a black screen, listening to music with a black screen, watching silent videos, watching videos with sound, reading aloud, and eating. These include numerous tasks simulating natural scenarios to meet the needs of different experimental scenarios and research objectives. It can be expanded to accommodate more experimental tasks, supports fixed task time countdown reminders, and allows for free control of task duration modes via response buttons. Compared to conventional brain-computer interface paradigms that rely solely on scientific research stimuli, this technology can more realistically recreate stimuli from everyday life.

[0201] The local resource library expansion allows users to add, delete, and modify stimuli such as instructions, images, audio, or video according to their experimental needs through a local resource library management mechanism. Task types and stimuli are associated and configured via data configuration files (such as JSON format). When adding new task types or changing stimuli, only the configuration file needs to be modified, without altering the code, greatly improving the flexibility and convenience of task expansion. Users can import natural stimuli from real life, such as audio of daily conversations and videos of street scenes, into the resource library for EEG signal acquisition experiments, filling a gap in natural stimulus research.

[0202] Employing Flutter's asynchronous programming mechanism and timer functionality, precise synchronization between multimedia stimulus playback and EEG signal acquisition is achieved. When audio, video, or other materials are played or stopped, the program automatically tags the hardware circuitry or streaming data of the EEG acquisition device, physiological polygraph, and webcam. It supports various wireless or wired trigger transmission methods, including Bluetooth, Wi-Fi, and parallel port, enabling alignment of multi-device and multimodal data. Each record provides a detailed task log, including the start and end times of the stimulus marked with timestamps, the stimulus material presented for each time period, user key presses during the task, and screen operation events, providing reliable data for research analysis and subsequent feature development.

[0203] Leveraging Flutter's rich UI components and animation effects, we designed a clean, user-friendly, and highly interactive interface. For example, we added transition animations when switching tasks to make interface transitions smoother; and we provided detailed operation prompts and instructions for each task to help users quickly understand and execute tasks, improving the user experience during the experiment. In tasks simulating natural stimuli, we optimized the interface design to make users feel as if they are in a real-world scenario, enhancing the immersion in the experiment and the validity of the data.

[0204] Breaking away from the limitations of traditional brain-computer interface (BCI) paradigms that overly rely on scientific research stimuli, this technology focuses on natural stimulus research, incorporating tasks from various everyday scenarios into BCI research and product development. Taking the "eating" task as an example, EEG signals are collected during natural actions such as chewing and swallowing to study the brain's response to real, natural behaviors. In the "watching audio-visual videos" task, real film clips, documentaries, and other natural video materials are used, rather than artificially designed standardized videos, to obtain EEG data on the brain's processing of complex natural visual and auditory information. By studying EEG signals under natural stimuli, we can gain a deeper understanding of the working mechanisms of the brain and other physiological data in real life, providing more valuable data support for the practical application of BCI technology and propelling it from the laboratory to real-life scenarios.

[0205] Attached are instructions for 8 sub-tasks:

[0206] Close your eyes and rest: During the following task, you need to keep your eyes closed, breathe naturally, relax your body, and not think about any specific things until you hear the voice prompt. If you are ready, place your index finger on the space bar, close your eyes, and then press the space bar to start the task.

[0207] Close your eyes and listen to music: You will need to keep your eyes closed throughout the following task. This application will play a piece of music; please listen attentively. A voice prompt will appear when the music ends, instructing you to open your eyes. If you are ready, place your index finger on the spacebar, close your eyes, and then press the spacebar to begin the task.

[0208] Black Screen and Quiet Retreat: During the following mission, you will need to keep your eyes open, breathe naturally, relax your body, and avoid thinking about any specific things. The screen will turn black during the mission. If you are ready, please press the "space bar" with your index finger to start the mission.

[0209] Black screen with music: You will need to keep your eyes open throughout the following mission. The screen will turn black while music plays; please listen attentively. If you are ready, press the spacebar with your index finger to start the mission.

[0210] Watch the silent video: You will need to keep your eyes open throughout the following task. This application will play a silent video. Please focus on the images in the video. If you are ready, please press the space bar with your index finger to start the task.

[0211] Watch the video with sound: You will need to keep your eyes open throughout the following task. This application will play a video with music. Please pay attention to both the visuals and the sound. If you are ready, please press the space bar with your index finger to start the task.

[0212] Read aloud: Text will appear on the screen during the following task. Please read the text aloud. Try to maintain a natural speaking speed and clear pronunciation. If you are ready, please press the "space bar" with your index finger to start the task.

[0213] Eating: Next, this application will provide some food. Please eat naturally; there is no need to speed up or deliberately slow down. You can chew and swallow according to your own habits. If you are ready, please press the "space bar" with your index finger to start the task.

[0214] By leveraging Flutter, a single codebase can run on multiple platforms. A self-drawing engine avoids platform differences, ensuring consistent interface and functionality, shortening development cycles, and reducing costs. The system offers a rich variety of tasks, including multiple brain-computer interface paradigms and numerous simulated natural scenarios, meeting diverse experimental needs and allowing for expansion of stimulus materials. It achieves precise synchronization between multimedia stimulation and EEG signal acquisition, aligning data across multiple devices and modalities through tag transmission. Utilizing Flutter's rich UI components and animation effects, it designs a clean, user-friendly, and highly interactive interface, such as adding transition animations and providing operation prompts, enhancing immersion in simulated natural stimulus tasks. Breaking through the limitations of traditional brain-computer interface paradigms, it focuses on natural stimulus research, incorporating real-life scenarios into EEG paradigm tasks, collecting EEG signals under real-world natural behavior, obtaining EEG data on the brain's processing of complex natural information, gaining a deeper understanding of the brain's working mechanisms, providing valuable data support for brain-computer interface technology applications, and driving the technology into real-life scenarios.

[0215] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.< / void> < / void>

Claims

1. A cross-platform brain-computer interface paradigm system suitable for natural state monitoring, characterized in that, Comprise: A cross-platform development module developed with the Flutter framework, which realizes the running of a set of codes on multiple platforms through a self-drawing engine, avoids the differences of native components of different platforms, and ensures the consistency of interfaces and functions on each platform; the platforms include Android, iOS and Windows; A task integration and expansion module that integrates multiple brain-computer interface paradigm tasks, including tasks that simulate natural scenes; a local material library management mechanism is established to add, delete and modify instructions, pictures, audio or video according to experimental requirements, and task types and stimulus materials are associated and configured through data configuration files; A multimedia precise synchronization control module that uses the asynchronous programming mechanism and timer function of Flutter to realize the precise synchronization of multimedia stimulus playing and electroencephalogram signal acquisition, and labels the hardware circuit or streaming data of the electroencephalogram acquisition device, physiological polygraph or network camera when playing or ending audio and video materials, realizes the alignment of multiple devices and multiple modal data, and generates a task log containing timestamp markers, which records the start and end time of the stimulus, the corresponding presented stimulus material, the user's key and operation screen events in the task; A natural stimulus task module that includes the task that simulates natural scenes into the brain-computer interface paradigm to collect corresponding electroencephalogram data according to the task type corresponding to the task.

2. The system of claim 1, wherein, The self-drawing engine is based on the Skia graphics library for rendering, and through unified graphics rendering logic, the display differences of native components of multiple platforms are eliminated, so that the interface layout, interaction effect and function implementation of the same set of codes after compilation on different platforms remain consistent. The task that simulates natural scenes includes at least the tasks of closing eyes and resting quietly, closing eyes and listening to music, black screen and resting quietly, black screen and listening to music, watching silent video, watching audio video, reading aloud and eating; collecting corresponding electroencephalogram data according to the task type corresponding to the task, including:

3. The system of claim 1, wherein, In the task of closing eyes and resting quietly, the electroencephalogram signal in the closed-eye resting state is collected, in the task of closing eyes and listening to music, the electroencephalogram response when focusing on listening to music is collected, in the task of black screen and resting quietly, the electroencephalogram signal in the open-eye black screen resting state is collected, in the task of black screen and listening to music, the electroencephalogram signal of listening to music in the open-eye black screen state is collected, in the task of watching silent video, the electroencephalogram signal of watching silent natural video is collected, in the task of watching audio video, the audio-visual integrated electroencephalogram signal when watching natural video with sound is collected, in the task of reading aloud, the neural electrical activity signal in the process of voice preparation and voice is collected, in the task of eating, the electroencephalogram signal accompanied by natural eating action is collected, and the natural eating action includes at least chewing and swallowing. The development with the Flutter framework includes:

4. The system of claim 1, wherein, ​ The Flutter framework is used to develop a single code base in Dart language, which is compatible with Android, iOS and Windows platforms, and realize multi-platform deployment with one development.

5. The system of claim 1, wherein, The local material library management mechanism is established to add, delete and modify instructions, pictures, audio or video according to experimental requirements, including: A stimulus material management module is constructed in the local storage to import custom instruction text, picture files, audio files and video files as stimulus materials through an interactive interface, and the imported materials are classified and managed and version controlled, and the content of the material library can be flexibly replaced or updated according to experimental design.

6. The system of claim 1, wherein, The task type and stimulus material are associated and configured through a data configuration file, including: A JSON format data configuration file is used to define the correspondence between tasks and stimulus materials, and a task type field, a stimulus material path field and a task parameter field are set in the configuration file. When a new task needs to be added or the stimulus material needs to be replaced, the corresponding field content in the configuration file is modified.

7. The system of claim 1, wherein, The asynchronous programming mechanism and timer function of Flutter are used to realize precise synchronization of multimedia stimulus playing and EEG signal acquisition, including: The asynchronous task scheduling mechanism of Flutter is used to trigger the playing or stopping operation of multimedia files, and a high-precision timer module is used to record the operation execution time. Trigger signals are generated at the time nodes when multimedia stimuli start or end to ensure that the time deviation between the EEG signal acquisition device and the multimedia playing module is controlled within the millisecond level.

8. The system of claim 1, wherein, When playing or ending audio and video materials, synchronization marking is realized in the streaming data of the EEG acquisition device, physiological polygraph or network camera to realize the alignment of multi-device and multi-modal data, including: The tagging operation sends a trigger instruction containing a timestamp to the EEG acquisition device through wired or wireless communication, inserts a task event marker in the data stream of the physiological polygraph, and records the task state of the corresponding time point in the video stream metadata of the network camera, so that the EEG signal, physiological data and video picture are accurately aligned on the time axis to form a traceable multi-modal data association record.

9. The system of claim 1, wherein, Further comprising: An intelligent algorithm analysis module is configured to: based on the collected EEG data and the corresponding task log, use a machine learning algorithm to build an EEG signal decoding model, the machine learning algorithm including any one of support vector machine, random forest and convolutional neural network, by training the EEG signal features in different task scenarios, identify the neural activity pattern corresponding to the preset task, the preset task including identifying closed eyes, listening to music, watching video, reading aloud and eating, output the task type classification result or the user intention recognition result, and support personalized model updating according to user historical data to improve the decoding performance and natural scene adaptation ability of the brain-computer interface paradigm.

10. The system of claim 1, wherein, Further comprising: A user interface module uses the UI components and animation effects of Flutter to design a user interface, which adds a transition animation when switching tasks and provides operation prompts and instructions for each task.

Images