Electroencephalogram-signal-based control method, and medium, device and product
By employing a dual safety verification process based on electroencephalogram (EEG) signals, combined with steady-state visual evoked potential signals and motor imagery signals, the problem of balancing safety and efficiency in existing vehicle control schemes has been solved, achieving efficient and safe vehicle control.
Patent Information
- Application Number
- PCT/CN2026/086291
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-11-25
- Filing Date
- 2026-03-26
- Publication Date
- 2026-11-19
AI Technical Summary
Existing vehicle control solutions cannot guarantee both driving safety and control efficiency. Physical buttons and remote controls have limited range, while touch screen control and voice control can easily distract the driver and frequently result in recognition errors. Remote controls are also prone to being lost or running out of power.
A dual safety verification process based on EEG signals is adopted. By combining steady-state visual evoked potential signals and motor imagery signals, a mechanism of "SSVEP intention selection + MI action confirmation" is constructed to generate control commands and ensure that the operation intention is consistent with the preset imagery mode before execution.
It significantly improves the safety and efficiency of the control process, avoids the operational delays and distractions of traditional control methods, and achieves efficient human-vehicle interaction under the premise of driving safety, thereby enhancing the user experience.
Smart Images

Figure CN2026086291_19112026_PF_FP_ABST
Abstract
Description
A control method, medium, device, and product based on electroencephalogram (EEG) signals
[0001] This application claims priority to Chinese Patent Application No. 202511741193.0, filed on November 25, 2025, entitled "A Control Method, Medium, Device and Product Based on Electroencephalogram Signals", the entire contents of which are incorporated herein by reference. Technical Field
[0002] This application belongs to the field of control, specifically relating to a control method, storage medium, electronic device, and computer program product based on electroencephalogram (EEG) signals. Background Technology
[0003] Current vehicle control solutions mainly include physical button control, central control screen touch control, voice control, gesture control, and remote control control, in order to achieve comprehensive control over vehicle settings, body control, vehicle driving, navigation and entertainment, and other dimensions.
[0004] However, physical button control and remote control have limited control range and cannot cover all in-vehicle functions; touch screen control requires the driver to look at the screen, which can easily distract the driver and pose a safety hazard; touch screen control, voice control, and gesture control often experience recognition problems or errors; and remote controls are additional control devices that can be lost or run out of power, rendering them unusable. Therefore, current vehicle control solutions cannot simultaneously ensure driving safety and control efficiency. Summary of the Invention
[0005] The purpose of this application is to provide a control method, medium, device, and product based on electroencephalogram (EEG) signals, which can solve the problem that current vehicle control schemes cannot ensure both driving safety and control efficiency.
[0006] In a first aspect, embodiments of this application provide a control method based on electroencephalogram (EEG) signals, the method comprising:
[0007] Acquire electroencephalogram (EEG) signals; the EEG signals include steady-state visual evoked potential signals and motor imagery signals;
[0008] The steady-state visual evoked potential signal is identified to obtain the object to be controlled and the operational intention for the object to be controlled; and the motion imagination signal is classified to obtain the target imagination pattern; the operational intention corresponds to a preset imagination pattern.
[0009] If the preset imagination mode corresponding to the operation intention is consistent with the target imagination mode, then a control command is generated based on the operation intention;
[0010] Control the object to be controlled to execute the control command.
[0011] Optionally, the step of identifying the steady-state visual evoked potential signal to obtain the object to be controlled and the operational intent for the object to be controlled includes:
[0012] Based on the steady-state visual evoked potential signal, a target reference signal matrix is determined in a preset reference signal library, and the target reference signal matrix corresponds to a target frequency;
[0013] Based on the target frequency corresponding to the target reference signal matrix, a target visual stimulus control that flashes at the target frequency is determined;
[0014] The functional attributes of the target visual stimulus control are analyzed to obtain the object to be controlled and the operational intent for the object to be controlled.
[0015] Optionally, the reference signal library includes multiple reference signal matrices; determining the target reference signal matrix from the preset reference signal library based on the steady-state visual evoked potential signal includes:
[0016] Based on the projection relationship between the steady-state visual evoked potential signal and each reference signal matrix, the linear correlation coefficient between the steady-state visual evoked potential signal and each reference signal matrix is calculated;
[0017] Based on the magnitude relationship of multiple linear correlation coefficients, a candidate reference signal matrix is determined from the multiple reference signal matrices;
[0018] The target reference signal matrix is determined based on the multiple linear correlation coefficients, the candidate reference signal matrix, and the preset threshold.
[0019] Optionally, calculating the linear correlation coefficient between the steady-state visual evoked potential signal and each reference signal matrix based on the projection relationship between the steady-state visual evoked potential signal and each reference signal matrix includes:
[0020] A canonical correlation analysis is performed on the steady-state visual evoked potential signal and each reference signal matrix to obtain a first projection vector and a second projection vector corresponding to each reference signal matrix. The first projection vector and the second projection vector together define a canonical correlation space. The first projection vector is used to define the projection direction of the steady-state visual evoked potential signal in the canonical correlation space, and the second projection vector is used to define the projection direction of the reference signal matrix in the canonical correlation space.
[0021] Based on the steady-state visual evoked potential signal, each reference signal matrix, and the first projection vector and second projection vector corresponding to each reference signal matrix, the first projection signal and second projection signal corresponding to each reference signal matrix are obtained.
[0022] Based on the first and second projection signals corresponding to each reference signal matrix, the linear correlation coefficient between the steady-state visual evoked potential signal and each reference signal matrix is calculated.
[0023] Optionally, the preset threshold includes a correlation coefficient threshold and a confidence threshold; determining the target reference signal matrix based on the plurality of linear correlation coefficients, the candidate reference signal matrix, and the preset threshold includes:
[0024] Based on the magnitude relationship of the plurality of linear correlation coefficients, a first correlation coefficient and a second correlation coefficient are determined among the plurality of linear correlation coefficients; the first correlation coefficient is greater than the second correlation coefficient, and the first correlation coefficient has a corresponding relationship with the candidate reference signal matrix;
[0025] Calculate the signal-to-noise ratio confidence level based on the first correlation coefficient and the second correlation coefficient;
[0026] If the first correlation coefficient is greater than the correlation coefficient threshold and the signal-to-noise ratio confidence level is greater than the confidence threshold, the candidate reference signal matrix is determined as the target reference signal matrix.
[0027] Optionally, classifying the motion imagery signal to obtain the target imagery pattern includes:
[0028] Based on the power variation of the motion imagination signal in a preset frequency band, motion imagination features related to motion imagination are extracted from the motion imagination signal.
[0029] The motion imagery features are input into a pre-trained classifier so that the classifier outputs the target imagery pattern.
[0030] Optionally, the method further includes:
[0031] In response to a mechanical control command for the object to be controlled, an interrupt command is generated to interrupt the control command, and the object to be controlled is controlled to execute the mechanical control command.
[0032] Secondly, embodiments of this application provide a storage medium that stores computer instructions, which, when executed by a computer, are used to perform the steps of the control method based on electroencephalogram (EEG) signals as described in the first aspect.
[0033] Thirdly, embodiments of this application provide an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the control method based on electroencephalogram signals as described in the first aspect.
[0034] Fourthly, embodiments of this application provide a computer program product, including a computer program / instructions, characterized in that, when the computer program / instructions are executed by a processor, they implement the steps of the control method based on electroencephalogram signals as described in the first aspect.
[0035] In this embodiment, an electroencephalogram (EEG) signal is acquired; the EEG signal includes steady-state visual evoked potential (SVP) signals and motor imagery signals; the SVP signals are identified to obtain a controllable object and an operational intention for the controllable object; and the motor imagery signals are classified to obtain a target imagery pattern; the operational intention corresponds to a preset imagery pattern; if the preset imagery pattern corresponding to the operational intention is consistent with the target imagery pattern, a control command is generated based on the operational intention; and the controllable object is controlled to execute the control command. This embodiment constructs a dual-safety verification process based on EEG signals by using SVP signals as the selection mechanism for operational intentions and motor imagery signals as the confirmation mechanism for operational intentions. This not only significantly improves the safety of the control process but also effectively avoids the common operational delays and attention distractions in traditional physical control, voice control, and gesture control. This improves control efficiency while ensuring driving safety, achieving efficient human-vehicle interaction of "what you think is what you control," thereby enhancing the user's control experience and avoiding potential safety hazards. Attached Figure Description
[0036] Figure 1 is a flowchart of a control method based on electroencephalogram (EEG) signals provided in an embodiment of this application.
[0037] Figure 2 is a schematic flowchart of a control method based on electroencephalogram (EEG) signals provided in an embodiment of this application.
[0038] Figure 3 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0040] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0041] The control method based on electroencephalogram (EEG) signals provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.
[0042] The current vehicle control technology solutions mainly include the following five types: physical button control, central control screen touch control, voice control, gesture control, and remote control control, which can realize all-round control of vehicle settings, body control, vehicle driving, navigation and entertainment.
[0043] Physical button control generates an electrical signal by the driver triggering mechanical or capacitive buttons / knobs. This signal is sent directly or through the Body Control Module (BCM) to the corresponding actuator (such as a motor or relay), enabling instant control. The operation is direct, reliable, and requires no visual distraction.
[0044] The central control screen touch control (soft switch) calls the corresponding application by clicking the screen, and sends corresponding commands to the relevant electronic control unit (ECU) of the vehicle via CAN bus (Controller Area Network) or Ethernet, which can control almost all vehicle functions;
[0045] Voice commands are collected via an in-vehicle microphone array and then processed by a local speech recognition engine (or cloud server) using natural language processing (NLP) to convert the speech into text and understand its intent. Control commands are then generated and sent to the corresponding ECU for execution via the bus.
[0046] Gesture control typically relies on 3D cameras or Time of Flight (ToF) sensors installed inside the vehicle (usually on the roof or in front of the center console) to capture the hand movement trajectory and posture of the driver or passenger. These are then analyzed by specialized algorithms to identify predefined gesture patterns (such as waving, swiping, rotating, etc.) and mapped to specific control commands, which are then sent for execution via the vehicle bus.
[0047] Remote controls typically use radio frequency (RF) technology to transmit encrypted radio frequency signals to the vehicle. After the receiver on the vehicle receives and verifies the legitimacy of the signal, the relevant controller executes the corresponding command.
[0048] However, physical button and remote control controls have limited scope and cannot cover all in-vehicle functions; touchscreen controls require the driver to look at the screen, which can easily distract the driver and pose a safety hazard; touchscreen, voice, and gesture controls often experience recognition problems or errors; and remote controls, being additional control devices, can be lost or run out of power, rendering them unusable. Clearly, current vehicle control solutions fail to balance driving safety with control efficiency, resulting in a poor overall control experience and potential safety risks.
[0049] This application provides a control method based on electroencephalogram (EEG) signals, applied to a vehicle, which includes an EEG signal acquisition component, an EEG signal analysis component, and a control component.
[0050] Referring to Figure 1, which is a flowchart of a control method based on electroencephalogram (EEG) signals according to an embodiment of this application, the method specifically includes the following steps:
[0051] Step 101: Acquire EEG signals; EEG signals include steady-state visual evoked potential signals and motor imagery signals;
[0052] The embodiments of this application aim to achieve an interactive mode of "what you think is what you control" by analyzing the driver's neural activity intentions, thereby freeing the user's limbs and enabling them to focus more on driving conditions. To achieve this goal, it is necessary to first obtain the user's electroencephalogram (EEG), specifically including SSVEP signals (Steady-State Visually Evoked Potential) and MI signals (Motor Imagery).
[0053] This application embodiment uses non-invasive brain-computer interface (BCI) technology to collect the electroencephalogram (EEG) signals of the driver or passenger. The basic principle of non-invasive brain-computer interface technology is to collect EEG signals from the scalp and analyze the signals. After the wearer wears the EEG acquisition device (such as a head cap, headband, or helmet), the corresponding control commands are identified and issued through specific paradigms (such as steady-state visual evoked potentials (SSVEP), motor imagery (MI), etc.) after the EEG signals are collected. Finally, the execution system completes the vehicle control operation.
[0054] Specifically, flexible dry electrodes are used as the EEG signal acquisition component, eliminating the need for conductive gel. While ensuring signal quality, wearing comfort and ease of use are prioritized. It is equipped with a lithium battery that can meet 24-hour battery life in continuous acquisition mode, supports a wireless charging module with a maximum power of 10W, and adopts a low-power Bluetooth communication solution to continuously and stably transmit the acquired data to the vehicle system.
[0055] In terms of electrode placement, dry electrodes are placed on the wearer's forehead and back of the head respectively. The forehead electrodes are mainly used to capture MI signals related to cognitive state, while the back of the head electrodes are specifically used to collect SSVEP signals induced by visual stimuli.
[0056] Regarding the signal induction mechanism, for SSVEP signals, the vehicle's central control screen displays visual stimulus controls (such as windows, air conditioning, lights, etc.) that flash at different frequencies (such as 12Hz, 15Hz, 20Hz) to induce the driver to produce a specific SSVEP response corresponding to the icon; for MI signals, the system will guide the user to perform a specific motor imagery task (such as imagining the movement of the left hand, right hand or foot), thereby inducing motor imagery EEG patterns that can be used for classification and recognition.
[0057] It should be noted that the EEG signal acquisition component simultaneously acquires two types of signals, SSVEP and MI, during the signal acquisition process, providing a data foundation for the subsequent implementation of a dual verification mechanism for intent selection and security confirmation.
[0058] Step 102: Identify the steady-state visual evoked potential signal to obtain the object to be controlled and the operational intention for the object to be controlled; and classify the motion imagination signal to obtain the target imagination mode; the operational intention corresponds to a preset imagination mode.
[0059] In this embodiment, after acquiring the raw EEG signal through the EEG signal acquisition component, the EEG signal analysis component performs lightweight preprocessing on the signal, including filtering, noise reduction, and effective frequency band extraction, followed by feature compression. Based on this, the EEG signal analysis component processes the SSVEP signal and MI signal in parallel: on one hand, it identifies the SSVEP signal to determine the object to be controlled and its corresponding operational intent; on the other hand, it simultaneously performs motor imagery classification on the MI signal to obtain the target imagery pattern.
[0060] Specifically, filtering refers to signal filtering, employing a high-precision bioelectrical acquisition chip with high common-mode rejection ratio (>110dB) and low noise performance, integrating analog filtering circuits to initially suppress power frequency interference and DC offset; denoising refers to combining hardware filtering with software algorithms, using wavelet transform-based or adaptive filtering methods to further eliminate physiological noise such as electrooculogram (EOG) and electromyogram (EMG) noise, as well as environmental noise interference, thereby improving the signal-to-noise ratio; effective frequency band extraction is carried out using a low-power microcontroller, utilizing its built-in DSP (Digital Signal Processing) instruction set for real-time signal preprocessing, performing 5-35Hz digital bandpass filtering to extract the effective frequency bands of SSVEP and MI; feature compression refers to calculating simple feature values of each channel data (such as the power spectral density of each key frequency band), rather than transmitting the raw massive EEG data, reducing the Bluetooth transmission bandwidth requirements.
[0061] To implement a secure verification mechanism between operational intent and confirmation action, in this embodiment, a specific MI confirmation action is pre-bound for each SSVEP command. Specifically, the system assigns different motion visualization tasks to different operational intents based on the control function to construct an SSVEP-MI mapping table. Simply put, each operational intent has a corresponding preset visualization mode; for example, the preset visualization mode corresponding to the "turn left" operational intent is "imagine your left hand making a fist."
[0062] Step 103: If the preset imagination mode corresponding to the operation intention is consistent with the target imagination mode, then generate control instructions based on the operation intention.
[0063] In traditional brain-controlled or single-modal brain-computer interfaces, control commands are usually generated directly based on the frequency recognition results of SSVEP signals or the classification results of MI signals. However, this can lead to a high rate of misoperation due to the susceptibility of signals to interference or unintentional triggering by the user, making it difficult to apply directly in safety-sensitive vehicle control scenarios.
[0064] Therefore, in this embodiment of the application, in order to fundamentally reduce the risk of misoperation and construct a safe brain-controlled interaction paradigm suitable for actual driving environment, the SSVEP signal and MI signal are organically combined. Instead of using them in parallel or serial mode, a dual safety confirmation mechanism of "SSVEP intention selection + MI action confirmation" is constructed. Based on the aforementioned constructed SSVEP-MI mapping table, the EEG signal analysis component finds the preset imagination mode that uniquely corresponds to the operation intention. Only when the target imagination mode obtained based on the MI signal is consistent with the preset imagination mode will the dual matching be determined to be successful.
[0065] Specifically, the combination of "SSVEP intent selection" and "MI action confirmation" is used as a dual matching mechanism. This dual matching acts as a trigger switch for generating control commands, systematically reducing the possibility of misoperation from the following three levels:
[0066] 1. Intent Verification: Introducing the AND logic of "SSVEP selection + MI confirmation" requires users to "both look and think", effectively filtering unconscious staring or random divergent thinking, reducing the probability of accidental single misoperation from the theoretical level of 5% to the level of 0.25%.
[0067] 2. Signal Redundancy: The SSVEP (exogenous) and MI (intrinsic) signals are independent of each other, with different generation mechanisms and susceptibility to interference. This mechanism ensures that a single type of interference (such as ambient light or body jolts) is unlikely to damage both channels simultaneously, significantly improving the overall robustness of the system in complex in-vehicle environments.
[0068] 3. Decision buffer: The "select-confirm" process provides a brief decision window, allowing users to change their minds before final execution, increasing the controllability and security of the operation, and avoiding errors caused by impulsiveness or hesitation.
[0069] After the EEG signal analysis component determines that the preset imagination mode and the target imagination mode corresponding to the operation intention are consistent, the EEG signal analysis component will send the operation intention to the control component to match the control command corresponding to the operation intention.
[0070] In this embodiment, instead of analyzing the SSVEP signal and the MI signal independently, when the corresponding operation intention is identified based on the SSVEP signal, the confirmation and verification process of the MI signal is started simultaneously. Only when the target imagination mode obtained by classifying the MI signal is consistent with the preset imagination mode corresponding to the operation intention will the final control command be triggered and generated.
[0071] Step 104: Control the object to be controlled to execute control instructions.
[0072] In this embodiment, after the control component generates a control command, it sends the control command to the controller corresponding to the object to be controlled. The controller then executes the control command to complete the corresponding action, thereby realizing vehicle control operation. Specifically, the control component sends the decoded control command to the corresponding controller (ECU) via the vehicle CAN bus / Ethernet to drive the execution of the corresponding function.
[0073] The controlled objects in this application embodiment include, but are not limited to: door locking mechanisms, electric window motors, air conditioning compressors and damper actuators, seat adjustment motors, and vehicle lighting controllers. The vehicle control operations that can be implemented include, but are not limited to: vehicle unlocking and locking; door opening and closing (prerequisite: all four doors and two hoods are electric); raising and lowering of all four windows; temperature and airflow adjustment for each air conditioning zone; seat fore-and-aft, height and backrest angle adjustment, and leg rest height adjustment; and exterior lighting adjustment: low beam on / off, high beam on / off, fog lights on / off, and turn signals on / off.
[0074] It should be noted that, in order to ensure the highest level of safety in vehicle operation and meet the functional safety design requirements, the controllable objects based on EEG signals in this application embodiment need to be strictly limited and do not involve core objects related to driving safety such as driving, steering and braking.
[0075] This application embodiment constructs a dual safety verification process based on EEG signals by using steady-state visual evoked potential signals as the selection mechanism for operational intentions and motor imagery signals as the confirmation mechanism for operational intentions. This process transforms EEG signals into highly secure control commands, significantly improving the safety of the control process and effectively avoiding common operational delays and attention distractions in traditional physical, voice, and gesture controls. Thus, while ensuring driving safety, it achieves safe, accurate, fast, seamless, and highly efficient human-vehicle interaction that allows drivers to focus on road conditions and provides faster reaction times in emergencies, comprehensively improving control efficiency and user experience.
[0076] In one embodiment of this application, a layered computing architecture of "lightweight front-end data collection and analysis + in-vehicle deep analysis" is adopted, which takes into account both the portability of wearable devices and the powerful computing power of vehicle systems.
[0077] In one embodiment of this application, a target reference signal matrix is determined in a preset reference signal library based on a steady-state visual evoked potential signal, and the target reference signal matrix corresponds to a target frequency; a target visual stimulus control that flashes at the target frequency is determined according to the target frequency corresponding to the target reference signal matrix; the functional attributes of the target visual stimulus control are analyzed to obtain the object to be controlled and the operation intention for the object to be controlled.
[0078] In this embodiment of the application, in order to accurately map the SSVEP response generated when a user gazes at a specific flashing stimulus to the specific object to be controlled and the operation intention, during the SSVEP identification process, it is necessary to calculate the matching degree between the preprocessed EEG signal and each candidate reference signal matrix in the reference signal library, and select the target reference signal matrix with the highest correlation. The target reference signal matrix is essentially a mathematical representation of a specific stimulus frequency and its harmonics. That is, each reference signal matrix has its corresponding frequency. Then, the corresponding object to be controlled and the operation intention for the object to be controlled are determined according to the target frequency corresponding to the target reference signal matrix.
[0079] Specifically, based on the target frequency corresponding to the target reference signal matrix, the target visual stimulus control that the user is looking at can be determined. The target visual stimulus control flashes at the target frequency. Each visual stimulus control is pre-configured with functional attributes that are bound to its functional semantics. By parsing the functional attributes of the target visual stimulus control, the object to be controlled bound to the control and its corresponding operation intention can be determined. For example, when the target frequency is 12Hz, it corresponds to the "left window down" control that flashes at 12Hz on the central control screen. By parsing the functional attributes of the "left window down" control, it can be determined that the object to be controlled is the "left window" and the operation intention is "down".
[0080] The embodiments of this application can achieve fast and accurate decoding of user visual intent through a standardized reference signal library, providing a reliable basis for the generation of subsequent control commands.
[0081] In one embodiment of this application, the reference signal library includes multiple reference signal matrices. Based on the projection relationship between the steady-state visual evoked potential signal and each reference signal matrix, a linear correlation coefficient between the steady-state visual evoked potential signal and each reference signal matrix is calculated. Based on the magnitude relationship of the multiple linear correlation coefficients, candidate reference signal matrices are determined from the multiple reference signal matrices. Based on the multiple linear correlation coefficients, the candidate reference signal matrices, and a preset threshold, a target reference signal matrix is determined.
[0082] In this embodiment, the reference signal library includes multiple reference signal matrices. Specifically, a set of reference signal matrices Y is constructed for each flicker frequency (e.g., 12Hz, 15Hz). The reference signal library can be viewed as a mapping set with flicker frequency as the key and the corresponding reference signal matrix as the value. Each reference signal matrix is a two-dimensional matrix, with its column vectors composed of multiple reference signals from the same source (i.e., sine and cosine components) and its row vectors composed of discretized time points. It can be understood that a single reference signal is the basic time-domain component constituting the matrix. The reference signals typically include the fundamental frequency of the stimulus frequency (i.e., the flicker frequency), the sine and cosine signals of the second harmonic, and sometimes even the sine and cosine signals of the third harmonic, to better simulate the nonlinear harmonic response of the brain to periodic visual stimuli.
[0083] As an example, the reference signal matrix Y_15hz constructed for a 15Hz flicker control can be represented at time points t1, t2, ..., tm as:
[0084] Among them, [sin(2π·15·t1),sin(2π·15·t2),…sin(2π·15·tm)] includes multiple reference signals from the same source.
[0085] It is understandable that each reference signal matrix is constructed based on a flashing frequency. That is, the target frequency corresponding to the target reference signal matrix refers to the flashing frequency used to construct the target reference signal matrix, and it is also the frequency at which the target visual stimulus control flashes on the central control screen.
[0086] Specifically, the projection relationship between the steady-state visual evoked potential signal and each reference signal matrix can reflect the intrinsic structural similarity between the two in multidimensional space. Therefore, the linear correlation coefficient ρ between the steady-state visual evoked potential signal and each reference signal matrix can be calculated through this projection relationship to quantify the degree of matching between the steady-state visual evoked potential signal and each reference signal matrix. Subsequently, the reference signal matrix with the highest linear correlation coefficient is selected as the candidate reference signal matrix.
[0087] In traditional SSVEP signal identification and classification processes, the final identification result is often determined solely from a reference signal library based on correlation metrics. However, this can lead to a higher false recognition rate when the signal is subject to noise interference or the user is not paying attention. Therefore, to improve the reliability and accuracy of the identification results, this application embodiment, in addition to selecting the candidate reference signal matrix corresponding to the maximum correlation coefficient, further verifies and confirms the target reference signal matrix through a preset threshold. The preset threshold refers to a threshold value optimized based on a large amount of experimental data, used to filter out low-quality or unreliable matching results.
[0088] The embodiments of this application can effectively filter out false recognitions caused by poor signal quality or accidental high matching degree by setting a preset threshold, which significantly improves the robustness and practicality of SSVEP recognition in complex vehicle environments.
[0089] In one embodiment of this application, canonical correlation analysis is performed on the steady-state visual evoked potential signal and each reference signal matrix to obtain a first projection vector and a second projection vector corresponding to each reference signal matrix. The first projection vector and the second projection vector jointly define the canonical correlation space. The first projection vector is used to define the projection direction of the steady-state visual evoked potential signal in the canonical correlation space, and the second projection vector is used to define the projection direction of the reference signal matrix in the canonical correlation space. Based on the steady-state visual evoked potential signal, each reference signal matrix, and the first projection vector and the second projection vector corresponding to each reference signal matrix, a first projection signal and a second projection signal corresponding to each reference signal matrix are obtained. According to the first projection signal and the second projection signal corresponding to each reference signal matrix, the linear correlation coefficient between the steady-state visual evoked potential signal and each reference signal matrix is calculated.
[0090] In this embodiment, the projection relationship between the steady-state visual evoked potential signal and each reference signal matrix is mainly analyzed by CCA (Canonical Component Analysis) to obtain the linear correlation coefficient ρ between the steady-state visual evoked potential signal and each reference signal matrix.
[0091] Specifically, canonical correlation analysis (CCA) is performed on the SSVEP signal X and the reference signal matrix Y corresponding to each flicker frequency in the reference signal library. The purpose of this analysis is to find a pair of optimal projection vectors, namely the first projection vector w_x and the second projection vector w_y, to project the original SSVEP signal X and the reference signal matrix Y onto the canonical correlation space, obtaining the projected signals, namely the first projected signal X·w_x and the second projected signal Y·w_y. An optimization algorithm is used to maximize the linear correlation coefficient ρ between these two projected signals, and the maximum linear correlation coefficient ρ_max is used as the correlation measure between the SSVEP signal X and the reference signal matrix Y. The first and second projection vectors together define a canonical correlation space. The first projection vector defines the projection direction of the SSVEP signal X in this space, and the second projection vector defines the projection direction of the reference signal matrix Y in this space. The optimization algorithm typically maximizes the linear combination correlation of two sets of variables through singular value decomposition or eigenvalue decomposition. This application does not limit the optimization algorithm; those skilled in the art can adopt the corresponding optimization algorithm according to actual needs.
[0092] The embodiments of this application use the CCA method to deeply explore the intrinsic relationship between the SSVEP signal and the reference signal matrix in the projection space, which can effectively improve the accuracy and robustness of signal matching, overcome the problem of decreased recognition rate in low signal-to-noise ratio or interference environments of traditional methods, and lay a solid foundation for achieving highly reliable visual intent decoding.
[0093] In one embodiment of this application, the preset threshold includes a correlation coefficient threshold and a confidence threshold. Based on the magnitude relationship of multiple linear correlation coefficients, a first correlation coefficient and a second correlation coefficient are determined among the multiple linear correlation coefficients; the first correlation coefficient is greater than the second correlation coefficient, and the first correlation coefficient has a corresponding relationship with the candidate reference signal matrix; the signal-to-noise ratio confidence score is calculated based on the first correlation coefficient and the second correlation coefficient; if the first correlation coefficient is greater than the correlation coefficient threshold and the signal-to-noise ratio confidence score is greater than the confidence threshold, the candidate reference signal matrix is determined as the target reference signal matrix.
[0094] In this embodiment, after identifying the candidate reference signal matrix, to eliminate false identifications caused by noise interference or accidental matching and improve the reliability of the identification results, it is necessary to further determine whether the current candidate reference signal matrix is the final target reference signal matrix based on preset thresholds. The preset thresholds include a correlation coefficient threshold ρ_threshold and a confidence threshold α_threshold. The correlation coefficient threshold is used to ensure that the SSVEP response signal has sufficient strength, and the confidence threshold is used to ensure that the target frequency has sufficient distinguishability relative to the interference frequency. Both the correlation coefficient threshold and the confidence threshold are empirical values set based on statistical analysis of a large amount of experimental data.
[0095] Specifically, for each reference signal matrix and the projection relationship between them and SSVEP, a corresponding linear correlation coefficient can be calculated.
[0096] In some embodiments, based on the magnitude relationship of multiple linear correlation coefficients, the linear correlation coefficient with the largest value is selected as the first correlation coefficient, and the linear correlation coefficient with the second largest value is selected as the second correlation coefficient.
[0097] Specifically, among multiple linear correlation coefficients, the largest and second largest linear correlation coefficients are determined, namely the first correlation coefficient ρ_max and the second correlation coefficient ρ_second. The first correlation coefficient ρ_max is the linear correlation coefficient calculated based on the projection relationship between the candidate reference signal matrix and the SSVEP signal, and the first correlation coefficient ρ_max is greater than the second correlation coefficient ρ_second.
[0098] The confidence level is directly related to the linear correlation coefficient. A larger linear correlation coefficient indicates a high degree of synchronization between the SSVEP signal and the flicker frequency corresponding to the reference signal matrix, representing a clear and definite SSVEP response. Therefore, to quantify this discriminative power, this application requires examining the difference between the first and second correlation coefficients. Specifically, this is done using a signal-to-noise ratio (SNR) metric. The SNR confidence level α can be calculated using the following formula: α = (ρ_max - ρ_second) / ρ_second
[0099] To improve the reliability of the target reference signal matrix, this application embodiment uses a dual verification method with a correlation coefficient threshold and a confidence threshold. That is, only when ρ_max > ρ_threshold and α > α_threshold will the candidate reference signal matrix be determined as the target reference signal matrix, so that it can be further verified based on the target reference signal matrix and the identification results based on the MI signal.
[0100] This application embodiment uses a dual-threshold decision mechanism to significantly reduce the false recognition rate in complex noisy environments or when the user's attention is not focused, ensuring that only high-confidence SSVEP recognition results can enter the subsequent process, thereby improving the robustness and security of the entire brain control system from the source.
[0101] In one embodiment of this application, when the first correlation coefficient is less than or equal to the correlation coefficient threshold and / or the signal-to-noise ratio confidence level is less than or equal to the confidence level threshold, an abnormal situation is considered to have occurred. At this time, the control mode based on EEG signals will be immediately exited, and the driver will be reminded to take over.
[0102] In one embodiment of this application, motion imagination features related to motion imagination are extracted from the motion imagination signal based on the power variation of the motion imagination signal in a preset frequency band; the motion imagination features are input into a pre-trained classifier so that the classifier outputs the target imagination pattern.
[0103] In this embodiment of the application, it is necessary to extract motor imagery features related to motor imagery from the MI signal. Specifically, the power change of event-related desynchronization (ERD) in a preset frequency band (μ rhythm, 8-13Hz) can be used as the core feature (motor imagery feature). Event-related desynchronization refers to the phenomenon of power decrease of the oscillation rhythm of a group of neurons in a specific area of the cerebral cortex when an individual performs a specific cognitive or motor task (such as motor imagery).
[0104] After extracting multiple motion visualization features, these motion visualization features will form a feature vector. This feature vector is then input into a pre-trained classifier, which will output a classification label, such as "left-hand visualization", "right-hand visualization", or "no visualization". It can be understood that this classification label is the target visualization pattern.
[0105] Specifically, the power variation curve of the motion imagery signal in the μ-rhythm (8-13Hz) frequency band is calculated by time-frequency analysis (such as short-time Fourier transform or wavelet transform). The power in the resting state before the start of the motion imagery task is used as the baseline to calculate the intensity value of event-related desynchronization (ERD). By extracting the statistics of ERD intensity (such as the mean, variance or minimum value within a specific time window) as features, these statistics are finally combined to form a feature vector representing the user's motion imagery pattern.
[0106] It should be noted that the classifier used to output the target visualization pattern is developed during the user calibration phase. This is achieved by having the user execute different preset visualization patterns multiple times, and obtaining feature vectors composed of motion visualization features during these executions. These feature vectors are then used as paired training sets. The preset visualization patterns serve as training labels, and the corresponding feature vectors are used as training inputs. The classifier's output labels are compared with the corresponding training labels. By reducing the error between the output and training labels, the classifier is trained. Furthermore, along with the output labels, the classifier also outputs a probability or confidence index. The probability refers to the likelihood that the classification label is correct, typically represented by the value output from the last layer of the classifier (such as the Softmax layer), and their sum is 1. The confidence index is a comprehensive reliability assessment, possibly combining probability values, feature vector clarity, and other statistics to measure the credibility of the classification result.
[0107] This application embodiment extracts motion imagery features directly related to motion imagery and uses a personalized calibrated classifier for pattern recognition, which can significantly improve the accuracy and reliability of motion imagery classification, providing a stable and reliable basis for subsequent security confirmation mechanisms.
[0108] In one embodiment of this application, in response to a mechanical manipulation command for a controlled object, an interrupt command for interrupting the control command is generated, and the controlled object is controlled to execute the mechanical manipulation command.
[0109] In this embodiment of the application, in order to ensure the absolute safety and controllable priority of vehicle control, the traditional mechanical control commands are set as commands with the highest interruption authority, which can interrupt the control commands generated based on EEG signals at any time.
[0110] Specifically, the control component prioritizes and arbitrates command sources at the logic level. Mechanical control commands (such as buttons on the steering wheel and switch signals associated with the brake pedal) are set as non-maskable interrupt sources. Once such a command is detected, the currently processed or executed brainwave control command will be unconditionally suspended, and the mechanical operation will be responded to immediately to ensure that the driver has final control in case of emergency or system misjudgment.
[0111] As an example, suppose the driver controls the window to rise via EEG signals, but the passenger's hand is still on the window frame. The driver can immediately press the physical window lower button on the door. This mechanical command will directly interrupt the EEG command to raise the window and immediately execute the lowering action, thus avoiding potential danger.
[0112] This application embodiment uses mechanical control commands as the highest priority commands, which can fully leverage the convenience of EEG control while ensuring that the commands always obey the driver's direct physical operation under any circumstances, thus fundamentally guaranteeing the safety redundancy and operational reliability of human-computer interaction.
[0113] Referring to Figure 2, which is a schematic flowchart of a control method based on electroencephalogram (EEG) signals provided in an embodiment of this application, the method specifically includes the following steps:
[0114] Step 201: The vehicle screen provides a visual stimulus: The vehicle screen displays control icons that flash at a specific frequency to provide a visual stimulus to the user in order to induce steady-state visual evoked potential signals.
[0115] Step 202 involves real-time monitoring and acquisition of the user's SSVEP response signal and motion imagery signal, followed by preprocessing of both types of signals. During this process, the system also performs sensor and signal quality status monitoring and fault detection.
[0116] Specifically, step 202 includes step 2021, real-time monitoring of the driver's SSVEP response signal induced by the stimulus and preprocessing it; step 202 also includes step 2022, real-time monitoring of the driver's MI signal and preprocessing it. It should be noted that steps 2021 and 2022 are not sequential and are two steps performed simultaneously.
[0117] Step 203: The preprocessed EEG signals are sent to the vehicle's control unit for decoding and judgment: The preprocessed SSVEP signal and MI signal are sent to the vehicle's control unit for decoding and judgment.
[0118] Step 204, EEG signal recognition: Match the preprocessed SSVEP signal with a preset reference signal library to obtain the matching command. When the confidence level of the SSVEP signal exceeds the internally set judgment threshold, the corresponding vehicle control command is output. Perform pattern matching analysis on the preprocessed MI signal. When the MI signal successfully matches the preset motor imagery pattern, output a "pattern matching" confirmation signal.
[0119] Step 205, Dual verification judgment: Determine whether the vehicle control command candidate from the SSVEP recognition stage and the pattern matching confirmation signal from the MI recognition stage are received simultaneously.
[0120] Step 206, Command Confirmation and Sending: When the judgment result of step 205 is yes, the vehicle system confirms and generates the final vehicle control command, and sends the command to the specific vehicle controller.
[0121] Step 207, Instruction Execution: After receiving the instruction, the specific vehicle controller executes the corresponding vehicle control operation.
[0122] This application employs a dual verification mechanism using SSVEP and MI signals, and introduces real-time monitoring and fault detection into the signal processing flow to ensure high safety and reliability of vehicle control based on EEG signals, and effectively avoids the occurrence of misoperation.
[0123] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.
[0124] This application also provides a storage medium that stores computer instructions. When the computer executes the computer instructions, it is used to execute the various processes of the above-described control method embodiment based on EEG signals and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0125] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0126] This application also provides an electronic device, including a processor 3010, a memory 309, and a program or instructions stored in the memory 309 and executable on the processor 3010. When the program or instructions are executed by the processor 3010, they implement the various processes of the above-described control method embodiment based on EEG signals and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0127] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.
[0128] Figure 3 is a schematic diagram of the hardware structure of an electronic device that implements an embodiment of this application.
[0129] The electronic device 300 includes, but is not limited to, components such as: radio frequency unit 301, network module 302, audio output unit 303, input unit 304, sensor 305, display unit 306, user input unit 307, interface unit 308, memory 309, and processor 3010.
[0130] Those skilled in the art will understand that the electronic device 300 may also include a power supply (such as a battery) for powering various components. The power supply can be logically connected to the processor 3010 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The electronic device structure shown in Figure 3 does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.
[0131] This application also provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, it implements the various processes of the above-described control method embodiment based on EEG signals and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0132] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0133] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0134] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A control method based on electroencephalogram (EEG) signals, the method being executed by a computer device, the method comprising: Acquiring EEG signals; The EEG signals include steady-state visual evoked potential signals and motor imagery signals; The steady-state visual evoked potential signal is identified to obtain the object to be controlled and the operational intention for the object to be controlled; and the motion imagination signal is classified to obtain the target imagination pattern; the operational intention corresponds to a preset imagination pattern. If the preset imagination mode corresponding to the operation intention is consistent with the target imagination mode, then a control command is generated based on the operation intention; Control the object to be controlled to execute the control command.
2. The method of claim 1, wherein, The step of identifying the steady-state visual evoked potential signal to obtain the object to be controlled and the operational intent for the object to be controlled includes: Based on the steady-state visual evoked potential signal, a target reference signal matrix is determined in a preset reference signal library, and the target reference signal matrix corresponds to a target frequency; Based on the target frequency corresponding to the target reference signal matrix, a target visual stimulus control that flashes at the target frequency is determined; The functional attributes of the target visual stimulus control are analyzed to obtain the object to be controlled and the operational intent for the object to be controlled.
3. The method of claim 2, wherein, The reference signal library includes multiple reference signal matrices; The step of determining the target reference signal matrix in a preset reference signal library based on the steady-state visual evoked potential signal includes: Based on the projection relationship between the steady-state visual evoked potential signal and each reference signal matrix, the linear correlation coefficient between the steady-state visual evoked potential signal and each reference signal matrix is calculated; Based on the magnitude relationship of multiple linear correlation coefficients, a candidate reference signal matrix is determined from the multiple reference signal matrices; The target reference signal matrix is determined based on the multiple linear correlation coefficients, the candidate reference signal matrix, and the preset threshold.
4. The method of claim 3, wherein, The calculation of the linear correlation coefficient between the steady-state visual evoked potential signal and each reference signal matrix based on the projection relationship between the steady-state visual evoked potential signal and each reference signal matrix includes: A canonical correlation analysis is performed on the steady-state visual evoked potential signal and each reference signal matrix to obtain a first projection vector and a second projection vector corresponding to each reference signal matrix. The first projection vector and the second projection vector together define a canonical correlation space. The first projection vector is used to define the projection direction of the steady-state visual evoked potential signal in the canonical correlation space, and the second projection vector is used to define the projection direction of the reference signal matrix in the canonical correlation space. Based on the steady-state visual evoked potential signal, each reference signal matrix, and the first projection vector and second projection vector corresponding to each reference signal matrix, the first projection signal and second projection signal corresponding to each reference signal matrix are obtained. Based on the first and second projection signals corresponding to each reference signal matrix, the linear correlation coefficient between the steady-state visual evoked potential signal and each reference signal matrix is calculated.
5. The method of claim 3 or 4, wherein, The preset thresholds include a correlation coefficient threshold and a confidence threshold; The step of determining the target reference signal matrix based on the plurality of linear correlation coefficients, the candidate reference signal matrix, and a preset threshold includes: Based on the magnitude relationship of the plurality of linear correlation coefficients, a first correlation coefficient and a second correlation coefficient are determined among the plurality of linear correlation coefficients; the first correlation coefficient is greater than the second correlation coefficient, and the first correlation coefficient has a corresponding relationship with the candidate reference signal matrix; Calculate the signal-to-noise ratio confidence level based on the first correlation coefficient and the second correlation coefficient; If the first correlation coefficient is greater than the correlation coefficient threshold and the signal-to-noise ratio confidence level is greater than the confidence threshold, the candidate reference signal matrix is determined as the target reference signal matrix.
6. The method according to any one of claims 1 to 5, wherein, The process of classifying the motion imagery signal to obtain the target imagery pattern includes: Based on the power variation of the motion imagination signal in a preset frequency band, motion imagination features related to motion imagination are extracted from the motion imagination signal. The motion imagery features are input into a pre-trained classifier so that the classifier outputs the target imagery pattern.
7. The method according to any one of claims 1 to 6, wherein, The method further includes: In response to a mechanical control command for the object to be controlled, an interrupt command is generated to interrupt the control command, and the object to be controlled is controlled to execute the mechanical control command.
8. A storage medium storing computer instructions, which, when executed by a computer, are used to perform a control method based on electroencephalogram (EEG) signals as described in any one of claims 1 to 7.
9. An electronic device comprising at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a control method based on electroencephalogram (EEG) signals as described in any one of claims 1 to 7.
10. A computer program product comprising a computer program / instructions that, when executed by a processor, implement a control method based on electroencephalogram (EEG) signals as described in any one of claims 1 to 7.
