Man-machine interaction method and device based on mixed brain-computer interface
Through hybrid brain-computer interface technology, SSVEP and motion imagination signals are collected and processed, and adaptive scripts are generated using adaptive learning algorithms, which solves the control accuracy and response delay problems of the existing system, and realizes efficient and complex human-computer interactions among people with limb movement disorders.
Patent Information
- Application Number
- CN202510682787.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-26
- Publication Date
- 2025-08-19
AI Technical Summary
The existing EEG-based human-computer interaction system has shortcomings in control accuracy, response delay, information transmission rate and user personalized adaptation, and it is difficult to meet the computer interaction needs of people with physical movement disorders in daily life.
Using hybrid brain-computer interface technology, by collecting and preprocessing visually evoked potential SSVEP signals and motion imagination signals, using adaptive learning algorithms for classification and identification, generating adaptive scripts and simulating key and mouse operations, realizing the interaction of complex tasks.
It realizes more flexible, personalized and efficient human-computer interaction, supports people with physical movement disorders to perform complex computer operations, and improves the naturalness and efficiency of interaction.
Smart Images
Figure CN120508211A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of human-computer interaction technology, and in particular relates to a human-computer interaction method and device based on a hybrid brain-computer interface. Background Art
[0002] Human-computer interaction is a crucial foundation of the modern information society, and its development directly impacts how efficiently people access information, process transactions, and create value. With the increasing prevalence of computer technology, human-computer interaction methods are constantly evolving—from the initial command-line interface to today's graphical user interface, each transformation has significantly enhanced the user experience. Currently, mainstream human-computer interaction methods still rely on traditional input devices such as keyboards and mice. However, these traditional methods are gradually revealing their inherent limitations in the face of increasingly complex and diverse application scenarios. For people with motor impairments, standard keyboard and mouse operations pose a significant challenge, severely limiting their ability to access and use computer resources, which in turn affects their quality of life and social participation. Therefore, developing more natural, efficient, and inclusive human-computer interaction technologies has significant social and economic significance.
[0003] Brain-computer interface (BCI) technology, as an emerging human-computer interaction paradigm, offers a promising path to addressing these challenges. It directly decodes brain activity, translating user intent into computer-recognizable control signals, enabling control of external devices while bypassing traditional neuromuscular pathways. Electroencephalography (EEG)-based BCI systems, due to their non-invasive nature and relatively low cost, have become one of the most widely studied BCI types. However, existing EEG-based human-computer interaction systems still face numerous technical bottlenecks, such as insufficient control accuracy, long response delays, low information transmission rates, and a lack of support for complex task flows and user-specific adaptation capabilities. These issues mean that current BCI systems are typically limited to basic pointer control and simple text input functions, making them unable to meet the practical needs of people with motor impairments for computer interaction in their daily lives. This severely restricts their widespread adoption as a practical human-computer interaction method. Summary of the Invention
[0004] The purpose of the present invention is to address the deficiencies in the above-mentioned prior art and propose a human-computer interaction method and device based on a hybrid brain-computer interface to bridge the gap between the limitations of current brain-computer interface technology and actual application needs, and to provide a powerful and universal solution for human-computer interaction for people with limb movement disorders.
[0005] To achieve the above object, the present invention adopts the following technical solutions:
[0006] A human-computer interaction method based on a hybrid brain-computer interface comprises the following steps:
[0007] S1, collects and preprocesses EEG signals, including visual evoked potential (SSVEP) signals and motor imagery signals;
[0008] S2, using overlapping sliding windows to divide the pre-processed SSVEP signal and motor imagery signal into multiple segments X k , where k∈K, K represents the number of data partitions;
[0009] S3, classify and identify SSVEP signals and motor imagery signals respectively;
[0010] S4, determining the user's intention based on the SSVEP signal classification results and the motor imagery signal classification results, and converting this intention into a keyboard and mouse operation instruction that can be recognized by the computer; and mapping the intention into a keyboard and mouse operation event, and simulating it on the system interface through the API function provided by the computer operating system;
[0011] S5, based on the computer's event monitoring mechanism, captures in real time the events of the user executing keyboard and mouse operation instructions within the specified time period during the simulation process, and associates the timestamps of each event in chronological order to generate an event sequence; at the same time, the SSVEP signal features and motor imagery signal features corresponding to each event trigger are extracted and integrated into an EEG feature vector, and then the timestamp is used as the key to associate the event information with the EEG features, and a relational database structure is used to realize data storage.
[0012] S6, based on the event sequence and relational database obtained in step S5, the user's intended behavior is realized through event playback and adaptive script generation, thereby completing human-computer interaction; wherein:
[0013] Event playback is to extract the user-specified event sequence from the database and replay the event information script that executes the keyboard and mouse operation instructions;
[0014] Adaptive script generation analyzes the user's EEG signal characteristics and matches them with the EEG characteristics of the corresponding time in the database, adaptively generates a script file for keyboard and mouse operations, and then simulates the corresponding keyboard and mouse interaction operations through the script file.
[0015] Furthermore, the evoked potential SSVEP signal is generated by the subject performing an evoked potential SSVEP paradigm, and the motor imagery signal is generated by the subject performing a motor imagery paradigm.
[0016] A human-computer interaction device based on a hybrid brain-computer interface includes a signal induction module, a data acquisition and preprocessing module, an interaction control module, a behavior capture module, and a macro instruction generation module; these modules are used to implement the above-mentioned human-computer interaction method based on a hybrid brain-computer interface.
[0017] The present invention proposes a human-computer interaction method and device based on a hybrid brain-computer interface. It uses the idea of adaptive learning algorithm, that is, it realizes adaptive script generation by integrating feature matching and context parameter perception optimization to improve the implementation idea of computer keyboard and mouse recording and playback tools and applies it to hybrid brain-computer interface technology. It records the user's EEG signals and corresponding keyboard and mouse interaction operations and adaptively generates automatic operation scripts to realize the user's interactive intention to perform complex tasks. After collecting EEG signals, the corresponding features are extracted according to different brain-computer interface paradigms to obtain the user's initial behavioral intention for keyboard and mouse interaction, execute the interaction content and record the corresponding keyboard and mouse operations and EEG features. When the user chooses to perform complex tasks, the recorded script is reproduced or a new operation script is adaptively generated according to the EEG signals to complete the complex interactive tasks between the user and the computer.
[0018] Compared with the existing technology, the human-computer interaction method based on hybrid brain-computer interface provided by the present invention uses the hybrid brain-computer interface to extract user operation intentions and adaptively generate keyboard and mouse operation scripts, realizing more flexible, personalized and efficient human-computer interaction, and providing a method tool for complex human-computer interaction for people with limb motor dysfunction. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 This is a flow chart of a human-computer interaction method based on a hybrid brain-computer interface in an embodiment;
[0020] Figure 2 is a diagram of an embodiment of a steady-state visual evoked potential elicitation paradigm;
[0021] Figure 3 This is a diagram of the embodiment motor imagery elicitation paradigm;
[0022] Figure 4 This is a flowchart of a keyboard and mouse operation simulation according to the classification results of the embodiment;
[0023] Figure 5 is a flow chart of behavior capture in an embodiment;
[0024] Figure 6 It is a flowchart of event playback and adaptive script generation in an embodiment. DETAILED DESCRIPTION
[0025] The modules of the present invention are described in further detail below with reference to the figures, but the embodiments of the present invention are not limited thereto.
[0026] like Figure 1As shown, this embodiment provides a human-computer interaction method based on a hybrid brain-computer interface, comprising the following steps:
[0027] S1, data acquisition and preprocessing, collects EEG signals and preprocesses them.
[0028] In this embodiment, the EEG signal is provided by a signal induction module. This module is used to present a specific EEG signal induction paradigm to the user, guiding the patient to execute the paradigm to induce an EEG signal representing their intention. In this embodiment, two paradigms are set: one for inducing steady-state visual evoked potentials (SSVEPs) and the other for inducing motor imagery signals.
[0029] The steady-state visual evoked potential (SSVEP) signal paradigm shown is to set multiple color blocks with different flashing frequencies, guide the user to focus on the corresponding color blocks flashing according to the task requirements to induce EEG response, that is, to generate steady-state visual evoked potential. The steady-state visual evoked potential (SSVEP) signal paradigm used in this embodiment is to realize keyboard interaction through a virtual keyboard on the computer screen, such as Figure 2 The virtual keyboard consists of 26 alphabetical keys, 10 numeric keys, and 4 special keys, for a total of 40 keys. Each key is assigned a frequency ranging from 4Hz to 14Hz at 0.25Hz intervals, and flashes continuously. The stimulus uses a sinusoidal signal-modulated flashing pattern, and the color is white, which is visually more perceptible.
[0030] The motor imagery paradigm is to induce the user to imagine the limbs moving in a specific direction through language prompts, image guidance, etc., thereby inducing neural activity in the motor cortex of the brain, that is, generating motor imagery signals. This embodiment realizes mouse interaction through the inductive language and images on the computer screen, such as Figure 3 Different mouse interaction actions correspond to different types of inductive language and images: left and right movement corresponds to the movement imagination of the left and right arms, up and down movement corresponds to the movement imagination of the lower limbs standing and sitting, and left and right button clicks correspond to the movement imagination of the left and right hands grasping.
[0031] In the data collection part, this embodiment uses eego TM The mylab 32-lead EEG acquisition device consists of a 32-channel EEG cap, a 32-channel 16kHz amplifier, and a laptop computer equipped with the EEG signal acquisition software eego64. The EEG cap connects to the laptop via the amplifier, amplifying the collected EEG signals before transmitting them to the laptop for storage. The EEG signal acquisition software eego64 displays them in real time, performs signal preprocessing, and includes event marking.
[0032] In the data preprocessing part, this embodiment performs preprocessing operations on the collected 2s EEG signal, including channel selection, re-referencing, filtering, downsampling, etc., where:
[0033] The channel selection operation was to select the following electrodes covering the occipital lobe and central brain regions in the international 10-20 standard lead system among the 32 channels for SSVEP signals and motor imagery signals: Oz, O1, O2 and C3, C4, Cz.
[0034] The re-reference operation selects the A1 and A2 bilateral mastoid electrode channels as reference channels, and uses the average value of the bilateral mastoid channel signals minus each EEG electrode channel signal to achieve EEG signal re-reference; the re-reference operation is used to avoid the interference of multi-channel EEG signal activity on single channel signals caused by volume conduction effects, and to avoid the loss of single-channel EEG signal information.
[0035] The filtering operation uses a third-order Butterworth bandpass filter with an upper limit frequency of 30 Hz and a lower limit frequency of 1 Hz. The filtering operation is used to filter out noise interference such as DC offset, eye movement, and heartbeat in the signal, and retain EEG signal data in a specific frequency band.
[0036] The downsampling operation is to reduce the sampling rate from 1000 Hz to 250 Hz. The downsampling operation is used to reduce the length of the collected EEG signal.
[0037] S2, data segmentation, using overlapping sliding windows to segment the pre-processed SSVEP signal and motor imagery signal into multiple segments. In this embodiment, the segmentation is performed using a step time window with a time window of 500ms and a window overlap rate of 50%.
[0038] S3, interactive control, obtains the user's intention to control the keyboard or mouse for interaction by classifying and identifying SSVEP signals and motor imagery signals, and maps the intention into keyboard and mouse operation events, which are simulated on the system interface through the API functions provided by the computer operating system.
[0039] The specific process of classifying and identifying SSVEP signals in this embodiment includes the following steps:
[0040] a1, using the canonical correlation analysis (CCA) method, measure the SSVEP signal segment X in each time window k The correlation ρ between the SSVEP signal and the reference signal Y at the target frequency f is used to find the correlation ρ between the SSVEP signal and the reference signal corresponding to each target frequency. j ; Select the target frequency f with the largest correlation pred As a result of classification;
[0041] The calculation formula of correlation ρ is as follows:
[0042] in, and ω y They are X k The linear projection vector formed by the linear combination of and Y; With Y T They are X k and the transposed vector of Y;
[0043] a2, all SSVEP signal segments X k Calculate f pred Voting is performed to select the target frequency with the largest number as the classification result of the SSVEP signal; the target frequency f with the largest correlation is calculated. pred The formula is as follows:
[0044]
[0045] in, Represents ρ j The value of ρ that appears most times in .
[0046] The specific process of classifying and identifying motor imagery signals in this embodiment includes the following steps:
[0047] b1, spatial filter generation, using the improved common spatial pattern CSP algorithm to incorporate channel prior information to extract each motion imagery signal segment The spatial features of , and generate a spatial filter based on the spatial features; the specific calculation formula is as follows:
[0048]
[0049] y k =max([M i N j ]) (4);
[0050] W CN =B T ·P (5);
[0051] Among them S i Represents two classification tasks, S m represents the source signal common to both tasks, C i Stands for S i The co-spatial pattern, C m Stands for S m The co-spatial mode, y k Represents the maximum prior information of the spatial multidimensional channel,
[0052] [M i Nj ] is the subspace combination of spatial channels, W CN The obtained spatial filter, B is the eigenvector matrix, and P is the whitening feature matrix;
[0053] b2, construct a support vector machine classifier for each motion imagery signal segment The spatial filter W obtained by b1 CN Perform projection to obtain the projection signal; calculate the variance of the projection signal to obtain the characteristic scalar value; all signal fragments The characteristic scalar values of are combined into a feature vector F, and the feature vector F is used as input to construct a support vector machine classifier based on Gaussian kernel;
[0054] b4, training phase: by solving the convex optimization problem of the support vector machine classifier (SVM), minimizing the hinge loss function and maximizing the classification interval of the decision hyperplane, the model parameters W and B are obtained s ;
[0055] b5, application stage: After obtaining the support vector machine parameters, for the next received motion imagery signal segment First calculate the feature F extracted by the common space pattern CSP algorithm n , by comparing the decision function F of the support vector machine n W+B s The size of and 0 is used to determine the binary classification results of the user's motion imagination;
[0056] b6, execute the above process for each two categories of the six actions to be classified, and finally determine the motor imagery classification result through round-by-round binary classification. The six actions to be classified in this embodiment are left and right movement, up and down movement, and left and right key clicks.
[0057] The user's intention is determined based on the classification results of the SSVEP signal and the motor imagery signal, and this intention is converted into keyboard and mouse operation instructions that can be recognized by the computer. The intention is then mapped into keyboard and mouse operation events and simulated on the system interface through the API functions provided by the computer operating system. The specific process includes:
[0058] First, various input events are generated based on the classification results, including simulated keyboard keystrokes, mouse movements, or clicks. In this embodiment, the SSVEP signal classification results correspond to the keyboard, and the motor imagery classification results correspond to the mouse. Keyboard keystroke simulation is achieved by generating a sequence of virtual keydown and keyup events. Mouse movements rely on generating a mousemove event containing the target coordinates. Mouse clicks, on the other hand, require a sequence of mousedown and mouseup events.
[0059] Secondly, these events are invoked using APIs provided by the operating system (such as the Windows API and input management libraries like Linux X11), simulating actual user input behavior. The operating system's input management layer acts as the event dispatch center, receiving virtual input events and delivering them to the currently active application. Upon receiving the event, the application can respond appropriately and execute the corresponding instructions.
[0060] For example, if a motor imagery triggers a "left mouse click," the system generates a sequence of left-button press and release events and injects them into the operating system through an API. The operating system recognizes this sequence as a real mouse operation and drives the application to execute the corresponding event.
[0061] S4, behavior capture, based on the computer's time monitoring mechanism, records the keyboard and mouse interaction operations and EEG signal characteristics made by the user within a specified time period and builds a database.
[0062] The underlying principle of behavior capture in this embodiment is based on the event monitoring mechanism of the operating system. Under this mechanism, when a user presses a key on the keyboard or clicks the mouse, an event will be generated by the operating system. As long as a corresponding event interception process is created in the operating system, certain information of the designated window can be monitored and intercepted before the information enters the target window processing function, and processed, interrupted, or not processed at all. Behavior capture only copies and stores information (user input operation events). The detailed process is as follows Figure 5 As shown:
[0063] The event monitoring mechanism based on the operating system monitors the input event stream at the bottom of the operating system and captures the user's keyboard and mouse interaction operations in real time within a specified time period. Based on the API provided by the operating system, the library functions encapsulated in the programming language are used to call and edit the API, dynamically recording the keyboard key status, mouse coordinates and click events. Each event is associated with a timestamp t i , the generated event sequence E is shown in Formula 6:
[0064] E={e1(t1),e2(t2),…,e n (t n )} (6)
[0065] Among them, e i Indicates the event types included, such as button presses and mouse movements, and parameters such as key codes and coordinate values.
[0066] While capturing the event, extract multiple EEG signal segments X corresponding to each event trigger, divided into 2s time windows. k The corresponding EEG signal feature, that is, the SSVEP correlation vector ρ j and motor imagery CSP feature Fn , integrated to form the eigenvector F i . Then, the event information and EEG features are associated with the timestamp as the key, and the data is stored using a relational database structure.
[0067] S5, based on the event sequence and relational database obtained in step S4, the user's intended behavior is realized through event playback and adaptive script generation, thereby completing human-computer interaction. The implementation method is as follows Figure 6 As shown:
[0068] The event playback is to extract the user-specified event sequence from the database and replay the event information script that executes the keyboard and mouse operation instructions. The implementation method is as follows:
[0069] Parse the event sequence according to the interactive operation selected by the user: extract the event sequence E from the database and sort it by timestamp t i Sorting, generating a time-event mapping table. Then perform time interpolation and smoothing: For continuous events, such as mouse movement trajectories, use the cubic spline interpolation algorithm to calculate the coordinates of the intermediate points.
[0070] Avoid mechanical jumps, as shown in Formula 7:
[0071] x(t)=a(tt i ) 3 +b(tt i ) 2 +c(tt i )+d (7);
[0072] Among them, the coefficients a, b, c, and d are determined by the coordinates of the adjacent event points and the velocity boundary conditions, t i ≤t≤t i+1 ;
[0073] Script generation and execution: Generate standardized script files and execute them by timestamp t through a multi-threaded scheduler i Call the operating system API to execute events one by one.
[0074] The adaptive script generation analyzes the EEG signal characteristics of the current user and matches them with the EEG characteristics of the corresponding event in the database to adaptively generate a script file for keyboard and mouse operations. The corresponding keyboard and mouse interaction operations are then simulated using the script file. The specific process includes the following steps:
[0075] First, perform feature matching and template retrieval: Calculate the user's current EEG feature F based on the normalized Euclidean distance c With all historical features in the database {F i}, and based on the similarity, the top k most matching template event sequences are selected as candidate sets; the similarity calculation formula is shown in Formula 8:
[0076]
[0077] Among them, S(F c ,F i ) is F c With F i The similarity of ‖F c -F i ‖2 is F c With F i Normalized Euclidean distance;
[0078] Perform context-aware parameter optimization: Natural language processing (NLP) technology is then used to parse the user's intended target location. Combined with database semantic reasoning and the current window layout, the absolute coordinates in the script are dynamically converted into relative position parameters for context-aware automated parameter optimization.
[0079] Finally, the script is dynamically generated: according to the real-time environment variables (such as the control type of the current focus window), conditional judgment is inserted. For different control types (such as file managers, browsers, etc.), the corresponding candidate set event sequence is called and context-aware parameter optimization is repeatedly performed. The coordinate parameters in the event sequence are converted and the logical order of script event execution in the event playback mode is matched. The script file is generated and executed by the multi-threaded scheduler according to the timestamp t i Call the operating system API to execute events one by one.
[0080] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.
Claims
1. A human-computer interaction method based on a hybrid brain-computer interface, characterized in that: The following steps are involved: S1, collects and preprocesses EEG signals, including visual evoked potential (SSVEP) signals and motor imagery signals; S2, using overlapping sliding windows to divide the preprocessed SSVEP signal and motor imagery signal into multiple segments; S3, classify and identify SSVEP signals and motor imagery signals respectively; S4, determining the user's intention based on the SSVEP signal classification results and the motor imagery signal classification results, and converting this intention into a keyboard and mouse operation instruction that can be recognized by the computer; and mapping the intention into a keyboard and mouse operation event, and simulating it on the system interface through the API function provided by the computer operating system; S5, based on the computer's event monitoring mechanism, captures in real time the events of the user executing keyboard and mouse operation instructions within the specified time period during the simulation process, and associates the timestamps of each event in chronological order to generate an event sequence; at the same time, the SSVEP signal features and motor imagery signal features corresponding to each event trigger are extracted and integrated into an EEG feature vector, and then the timestamp is used as the key to associate the event information with the EEG features, and a relational database structure is used to realize data storage. S6, based on the event sequence and relational database obtained in step S5, the user's intended behavior is realized through event playback and adaptive script generation, thereby completing human-computer interaction; wherein: Event playback is to extract the user-specified event sequence from the database and replay the event information script that executes the keyboard and mouse operation instructions; Adaptive script generation analyzes the user's EEG signal characteristics and matches them with the EEG characteristics of the corresponding time in the database, adaptively generates a script file for keyboard and mouse operations, and then simulates the corresponding keyboard and mouse interaction operations through the script file.
2. A human-computer interaction method based on a hybrid brain-computer interface according to claim 1, characterized in that: The evoked potential SSVEP signal is generated by the subject performing the evoked potential SSVEP paradigm, and the motor imagery signal is generated by the subject performing the motor imagery paradigm.
3. The human-computer interaction method based on hybrid brain-computer interface according to claim 1, characterized in that: The specific process of classifying and identifying the SSVEP signal in step S3 includes the following steps: a1, using CCA method, measure the SSVEP signal segment X in each time window k The correlation ρ between the SSVEP signal and the reference signal Y at the target frequency f is used to find the correlation ρ between the SSVEP signal and the reference signal corresponding to each target frequency. j ; Select the target frequency f with the largest correlation pred As a result of classification; The calculation formula of correlation ρ is as follows: in, and ω y They are X k The linear projection vector formed by the linear combination of and Y; With Y T They are X k and the transposed vector of Y, k∈K, K represents the number of data partitions; a2, all SSVEP signal segments X k Calculate f pred Voting is performed to select the target frequency with the largest number as the classification result of the SSVEP signal; the target frequency f with the largest correlation is calculated. pred The formula is as follows: in, Represents ρ j The value of ρ that appears most times in .
4. The human-computer interaction method based on hybrid brain-computer interface according to claim 3, characterized in that: The specific process of classifying and identifying the motor imagery signal in step S3 includes the following steps: b1, spatial filter generation, using the improved common spatial pattern CSP algorithm to incorporate channel prior information to extract each motion imagery signal segment and generating a spatial filter based on the spatial feature; The specific calculation formula is as follows: y k =max([M i N j ]); W CN =B T ·P; Among them S i Represents two classification tasks, S m represents the source signal common to both tasks, C i Stands for S i The co-spatial pattern, C m Stands for S m The co-spatial mode, y k Represents the maximum prior information of the spatial multidimensional channel, [M i N j ] is the subspace combination of spatial channels, W CN The obtained spatial filter, B is the eigenvector matrix, and P is the whitening feature matrix; b2, construct a support vector machine classifier for each motion imagery signal segment The spatial filter W obtained by b1 CN Perform projection to obtain a projection signal; calculate the variance of the projection signal to obtain a characteristic scalar value; All signal fragments The characteristic scalar values of are combined into a feature vector F, and the feature vector F is used as input to construct a support vector machine classifier based on Gaussian kernel; b4, training phase: by solving the convex optimization problem of the support vector machine classifier, minimizing the hinge loss function and maximizing the classification interval of the decision hyperplane, the model parameters W and B are obtained. s ; b5, application stage: After obtaining the support vector machine parameters, for the next received motion imagery signal segment First calculate the feature F extracted by the common space pattern CSP algorithm n , by comparing the decision function F of the support vector machine n W+B s The size of and 0 is used to determine the binary classification results of the user's motion imagination; b6. Execute the above process for every two categories of the six actions to be classified, and finally determine the motor imagery classification result through round-by-round binary classification.
5. The human-computer interaction method based on hybrid brain-computer interface according to claim 4, characterized in that: The specific implementation process of the event playback in step S6 includes: a1, extract the event sequence E from the database, by timestamp t i Sorting, generating a time-event mapping table; a2. Perform time interpolation and smoothing: For continuous events, such as mouse movement trajectories, use the cubic spline interpolation algorithm to calculate the coordinates of the intermediate points to avoid mechanical jumps, as shown below: x(t)=a(t-t i ) 3 +b(t-t i ) 2 +c(t-t i )+d; Among them, the coefficients a, b, c, and d are determined by the coordinates of the adjacent event points and the velocity boundary conditions, t i ≤t≤t i+1 ; a3, script generation and execution: Generate standardized script files and execute them by multi-threaded scheduler according to timestamp t i Call the operating system API to execute events one by one.
6. The human-computer interaction method based on hybrid brain-computer interface according to claim 5, characterized in that: The specific implementation process of the adaptive script generation in step S6 includes: b1, calculate the user's current EEG feature F based on the normalized Euclidean distance c With all historical features in the database {F i }, based on the similarity, the top k most matching template event sequences are selected as candidate sets; the similarity calculation formula is as follows: Among them, S(F c ,F i ) is F c With F i The similarity of ‖F c -F i ‖2 is F c With F i Normalized Euclidean distance. b2. Context-aware parameter optimization: This uses natural language processing technology to analyze the user's intended target location. Combining database semantic reasoning and the current window layout, the absolute coordinates in the script are dynamically converted into relative position parameters for context-aware automated parameter optimization. b3, insert conditional judgment based on real-time environment variables, call the corresponding candidate set event sequence for different control types and repeatedly optimize context-aware parameters, transform the coordinate parameters in the event sequence and match the logical order of script event execution in the event playback mode, generate a script file and execute it by timestamp t through the multi-threaded scheduler. i Call the operating system API to execute events one by one.
7. A human-computer interaction device based on a hybrid brain-computer interface, characterized in that: It includes a signal induction module, a data acquisition and preprocessing module, an interactive control module, a behavior capture module and a macro instruction generation module; these modules are used to implement the human-computer interaction method based on a hybrid brain-computer interface as described in any one of claims 1 to 6.