Multi-target control system for brain control interaction

Abstract

The invention discloses a multi-target control system for brain control interaction, which adopts non-invasive EEG (electroencephalogram) acquisition equipment, electroencephalogram signals are transmitted to a data processing computer in a wireless communication mode, and the computer processes the signals in real time and transmits control instructions to external equipment. And accurate selection and control of the target by the user are realized. The multi-target control device is constructed based on the SSVEP, efficient and accurate recognition of multiple targets is achieved through the non-invasive brain-computer interface technology, the limitation of a traditional remote control or manual input mode is overcome, and the naturalness and real-time performance of interaction are improved. According to the method, a training mode is not introduced, so that a user can quickly use the system without pre-training, the use threshold of a new user is reduced, and meanwhile, the accuracy of signal classification is improved and the stability and generalization ability of the system are ensured through optimized canonical correlation analysis (CCA) and multi-band CCA (FBCCA algorithms.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction, and in particular to a multi-target control system for brain-controlled interaction. Background Art

[0002] With the continuous development of human-computer interaction technology, brain-computer interface, as an emerging interaction method, has shown broad prospects in applications such as medical rehabilitation, intelligent device control and augmented reality. SSVEP is a common visually evoked electroencephalogram signal, which produces a corresponding steady-state response in the human brain based on the periodic flashing of visual stimulus targets. This signal can be collected by non-invasive electroencephalogram (EEG) equipment and classified by signal processing and pattern recognition methods to achieve the selection and control of specific targets. Existing SSVEP brain-computer interface systems mostly focus on single target recognition. When multiple stimulus targets exist at the same time, it may cause mutual interference between targets and reduce the recognition accuracy.

[0003] In addition, traditional methods often only use single frequency features for classification, the information transmission rate is low, and factors such as external noise and individual differences can easily affect the quality of SSVEP signals, resulting in unstable control. Therefore, in order to solve the above problems, an efficient and stable SSVEP multi-target control device is urgently needed to improve recognition accuracy, response speed and system stability. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and propose a multi-target control system for brain-controlled interaction. The present invention aims to solve the following technical problems: (1) Limited multi-target control capability: Existing SSVEP systems usually only support a small number of targets (e.g., 4-8), which makes it difficult to meet the demand for high-capacity control in complex scenarios (e.g., collaborative control of multiple devices in smart homes and virtual reality interaction).

[0005] (2) Insufficient classification accuracy and real-time performance: Traditional frequency classification algorithms are susceptible to noise interference and signal attenuation. The classification accuracy drops significantly in dynamic environments, and the signal processing delay is high (>200ms), which cannot meet the real-time interaction requirements.

[0006] (3) Poor user adaptability and high calibration cost: The existing system lacks a personalized adaptation mechanism. Users need to calibrate for a long time (>30 minutes), and the fixed threshold design is difficult to adapt to the SSVEP response differences of different users. In addition, the uniform visual stimulation can easily cause user fatigue and affect the long-term usage experience.

[0007] (4) Single training paradigm: Most systems rely on a single classification algorithm (such as CCA) and lack the ability to flexibly switch training modes. They are unable to meet the dual requirements of “zero calibration instant response” and “high-precision complex control”.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions: A multi-target control system for brain-controlled interaction, which uses non-invasive EEG acquisition equipment to transmit EEG signals to a data processing computer through wireless communication. The computer processes the signals in real time and transmits control instructions to external devices to enable users to accurately select and control targets. The computer processes the collected EEG signals according to the current experimental mode through EEG signal processing software; The experimental modes include no training mode and training mode.

[0009] Preferably, in the no-training mode, an interaction mode is first selected, including a free gaze mode and a guided gaze mode; In the free gaze mode, the user directly gazes at the target area, and the system automatically detects the SSVEP signal for classification and decoding without additional prompts or guidance; In guided gaze mode, the system will give users clear gaze instructions, guiding them to look at specific targets in a set order or with real-time feedback to enhance signal stability and recognition accuracy.

[0010] Preferably, in the non-training mode, after entering the experimental stage, the EEG device collects the user's EEG signal, and the pre-processed signal is input into the signal classification module, and the multi-band CCA method is used to extract SSVEP features, match the target frequency, and perform target discrimination based on the maximum correlation coefficient; If the classification confidence is high, the system immediately converts the classification result into a control instruction and transmits it to the external device through the communication protocol to execute the target operation. At the same time, it updates the user interface and provides visual and audio feedback to ensure the real-time and operability of the interaction. If the classification confidence is low, the system will prompt the user to adjust the gaze pattern and re-collect the signal to ensure the reliability and stability of the recognition; After the entire no-training mode process is completed, the user can choose whether to continue the experiment. If they choose to continue, they will enter the next round of recognition and interaction process; If you select End, the system will stop running and enter standby mode.

[0011] Preferably, in the above experimental stage, the system monitors the changes in the user's EEG signals in real time to adapt to different user states and environmental interference factors, optimize the target selection effect, and reduce the misrecognition rate.

[0012] Preferably, in the training mode, when the user selects the training mode, if there is an existing personalized model, it can be used directly, otherwise the system will create a new model and enter the training process; During the training data collection phase, users need to follow the system instructions and look at different SSVEP stimulation targets in turn. Each target will flash at a set frequency, and the EEG device will synchronously collect EEG signals and perform preprocessing.

[0013] Preferably, in the training data processing stage, the system uses canonical correlation analysis CCA, multi-band CCA and convolutional neural network CNN methods to extract EEG features and construct feature vectors; Subsequently, the system uses support vector machines (SVMs) and deep neural network (DNN) classifiers for training and adjusts model parameters to adapt them to the individual characteristics of users. To improve recognition accuracy, the system introduces a transfer learning mechanism to fine-tune the existing model so that the newly trained data maintains high consistency with the original model, thereby improving generalization capabilities; During the training process, the system evaluates the classification confidence in real time and optimizes the feature extraction method based on the individual differences of users to ensure the accuracy and robustness of signal classification.

[0014] Preferably, after the training mode is completed, the user enters the real-time control stage and can choose between the free gaze mode and the guided mode. The system collects the user's EEG signals and classifies them in real time, identifies the target by the maximum confidence, outputs control instructions, and synchronously updates the user interface. If the system determines that the confidence is high, it will directly output the control command; if the confidence is low, the system will optimize the classification strategy based on historical data, or prompt the user to adjust the gaze mode and re-collect data. The user can also choose to re-establish a personal model. After the experiment, if the current training model performs well, the system can store the user's personalized features for subsequent use, making the interaction more accurate and efficient; If the user chooses to continue the experiment, the system will enter the next round of signal acquisition and classification process, otherwise it will exit the training mode and return to the main interface standby state.

[0015] Preferably, the preprocessing method is to use bandpass filtering and independent component analysis (ICA) method to perform signal preprocessing to remove environmental noise, electromyographic and electrooculographic artifacts.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention constructs a multi-target control device based on steady-state visual evoked potential (SSVEP), realizes efficient and accurate recognition of multiple targets through non-invasive brain-computer interface technology, overcomes the limitations of traditional remote control or manual input methods, and improves the naturalness and real-time nature of interaction. The introduction of the no-training mode enables users to quickly use the system without pre-training, which reduces the threshold for new users. At the same time, the optimized canonical correlation analysis (CCA) and multi-band CCA (FBCCA) algorithms improve the accuracy of signal classification and ensure the stability and generalization ability of the system. The training mode optimizes the user model through personalized modeling and transfer learning, reduces the initial training time of the user, and reduces the influence of individual differences on the classification accuracy of the system, so that users can adapt to the system faster and improve the interaction efficiency. In addition, the present invention adopts a multi-mode interaction strategy, including a free gaze mode and a guided mode, which flexibly adapts to different usage scenarios, and further improves the stability and accuracy of target selection. Through the optimization of the signal processing module, efficient filtering and artifact removal of EEG signals are achieved, the signal-to-noise ratio of EEG signals is improved, and the system can still maintain a high classification reliability in a complex environment. The optimized transmission mechanism of control instructions ensures that the recognition results can be fed back to external devices in real time, achieving efficient interaction with low latency.

[0017] At the application level, the present invention can be applied to a variety of scenarios such as brain-controlled spelling systems, drone control, and smart home interaction, allowing users to complete complex interactive tasks through natural visual gaze, greatly improving the breadth of application and ease of use of brain-computer interfaces. Compared with traditional methods, this system can decode user intentions more quickly and accurately, significantly reduce the operating burden, and improve the human-computer interaction efficiency of the system, providing a better solution for the practical application of non-invasive brain-computer interfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more specifically and intuitively illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0019] Figure 1 is a schematic diagram of the overall system architecture of an embodiment of the present invention; Figure 2 is a schematic diagram of multi-target stimulation coding according to an embodiment of the present invention; Figure 3 is an online training mode selection diagram of an embodiment of the present invention; Figure 4 is a schematic diagram of the instruction area in the instruction training mode of an embodiment of the present invention; Figure 5 is an experimental flow chart of an embodiment of the present invention; Figure 6is a flow chart of an embodiment of the present invention in a non-training mode; Figure 7 It is a flow chart of the training mode of an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0021] Reference Figure 1-7 , a multi-target control system for brain-controlled interaction, which uses non-invasive EEG acquisition equipment to transmit EEG signals to a data processing computer through wireless communication. The computer processes the signals in real time and transmits control instructions to external devices to enable users to accurately select and control targets; The computer processes the collected EEG signals according to the current experimental mode through EEG signal processing software; The experimental modes include no training mode and training mode.

[0022] In this implementation, in the no-training mode, the interaction mode is first selected, including a free gaze mode and a guided gaze mode; In the free gaze mode, the user directly gazes at the target area, and the system automatically detects the SSVEP signal for classification and decoding without additional prompts or guidance; In guided gaze mode, the system will give users clear gaze instructions, guiding them to look at specific targets in a set order or with real-time feedback to enhance signal stability and recognition accuracy.

[0023] In this implementation, in the no-training mode, after entering the experimental phase, the EEG device collects the user's EEG signals, and the pre-processed signals are input into the signal classification module, and the multi-band CCA method is used to extract SSVEP features, match the target frequency, and perform target discrimination based on the maximum correlation coefficient; If the classification confidence is high, the system immediately converts the classification result into a control instruction and transmits it to the external device through the communication protocol to execute the target operation. At the same time, it updates the user interface and provides visual and audio feedback to ensure the real-time and operability of the interaction. If the classification confidence is low, the system will prompt the user to adjust the gaze pattern and re-collect the signal to ensure the reliability and stability of the recognition; After the entire no-training mode process is completed, the user can choose whether to continue the experiment. If they choose to continue, they will enter the next round of recognition and interaction process; If you select End, the system will stop running and enter standby mode.

[0024] In this implementation scheme, during the above experimental phase, the system monitors the user's EEG signal changes in real time to adapt to different user states and environmental interference factors, optimize the target selection effect, and reduce the misrecognition rate.

[0025] In this implementation, in the training mode, when the user selects the training mode, if there is an existing personalized model, it can be used directly, otherwise the system will create a new model and enter the training process; During the training data collection phase, users need to follow the system instructions and look at different SSVEP stimulation targets in turn. Each target will flash at a set frequency, and the EEG device will synchronously collect EEG signals and perform preprocessing.

[0026] In this implementation, during the training data processing phase, the system uses canonical correlation analysis (CCA), multi-band CCA, and convolutional neural network (CNN) methods to extract EEG features and construct feature vectors; Subsequently, the system uses support vector machines (SVMs) and deep neural network (DNN) classifiers for training and adjusts model parameters to adapt them to the individual characteristics of users. To improve recognition accuracy, the system introduces a transfer learning mechanism to fine-tune the existing model so that the newly trained data maintains high consistency with the original model, thereby improving generalization capabilities; During the training process, the system evaluates the classification confidence in real time and optimizes the feature extraction method based on the individual differences of users to ensure the accuracy and robustness of signal classification.

[0027] In this implementation scheme, after the training mode is completed, the user enters the real-time control stage and can choose between the free gaze mode and the guided mode. The system collects the user's EEG signals and classifies them in real time, identifies the target by the maximum confidence, outputs control instructions, and synchronously updates the user interface; If the system determines that the confidence is high, it will directly output the control command; if the confidence is low, the system will optimize the classification strategy based on historical data, or prompt the user to adjust the gaze mode and re-collect data. The user can also choose to re-establish a personal model. After the experiment, if the current training model performs well, the system can store the user's personalized features for subsequent use, making the interaction more accurate and efficient; If the user chooses to continue the experiment, the system will enter the next round of signal acquisition and classification process, otherwise it will exit the training mode and return to the main interface standby state.

[0028] In this embodiment, the preprocessing method is to use bandpass filtering and independent component analysis (ICA) method to perform signal preprocessing to remove environmental noise, electromyographic and electrooculographic artifacts.

[0029] Taking smart home control as an example, the application process of the present invention is described: System initialization: The user wears the EEG cap and faces the stimulation interface (including 16 device icons, such as lights, air conditioners, curtains, etc.).

[0030] Personalized calibration: The user looks at each icon in turn, and the system records its SSVEP response characteristics and establishes a threshold library.

[0031] Real-time control: When the user looks at the target icon (such as "light"), the system identifies the frequency-phase combination within 100ms and sends a switch command to the smart light via WiFi.

[0032] In addition, the present invention proposes the following alternatives: 1. Software Alternatives Spatial encoding: Combines the position of the target on the screen (such as upper left, lower right) with the frequency combination to further expand the target capacity.

[0033] Hybrid modulation coding: Amplitude modulation (AM) is superimposed on the frequency-phase basis to enhance feature differentiation.

[0034] 2. Hardware Alternatives Flexible electrode array: Wearable flexible electrodes are used to replace traditional dry electrodes to improve comfort.

[0035] AR glasses integration: Integrate the stimulation interface into AR glasses to achieve an interactive experience that combines virtual and real elements.

[0036] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A multi-objective control system for brain-controlled interaction, characterized in that: The control system uses non-invasive EEG acquisition equipment to transmit EEG signals to a data processing computer through wireless communication. The computer processes the signals in real time and transmits control instructions to external devices, enabling users to accurately select and control targets. The computer processes the collected EEG signals according to the current experimental mode through EEG signal processing software; The experimental modes include no training mode and training mode.

2. A multi-objective control system for brain-controlled interaction according to claim 1, characterized in that: In the no-training mode, the interaction mode is first selected, including free gaze mode and guided gaze mode; In the free gaze mode, the user directly gazes at the target area, and the system automatically detects the SSVEP signal for classification and decoding without additional prompts or guidance; In guided gaze mode, the system will give users clear gaze instructions, guiding them to look at specific targets in a set order or with real-time feedback to enhance signal stability and recognition accuracy.

3. A multi-objective control system for brain-controlled interaction according to claim 2, characterized in that: In the no-training mode, after entering the experimental phase, the EEG device collects the user's EEG signals, and the pre-processed signals are input into the signal classification module, where the multi-band CCA method is used to extract SSVEP features, match the target frequency, and perform target discrimination based on the maximum correlation coefficient; If the classification confidence is high, the system immediately converts the classification result into a control instruction and transmits it to the external device through the communication protocol to execute the target operation. At the same time, it updates the user interface and provides visual and audio feedback to ensure the real-time and operability of the interaction. If the classification confidence is low, the system will prompt the user to adjust the gaze pattern and re-collect the signal to ensure the reliability and stability of the recognition; After the entire no-training mode process is completed, the user can choose whether to continue the experiment. If they choose to continue, they will enter the next round of recognition and interaction process; If you select End, the system will stop running and enter standby mode.

4. A multi-objective control system for brain-controlled interaction according to claim 3, characterized in that: During the above experimental stage, the system monitors the changes in the user's EEG signals in real time to adapt to different user states and environmental interference factors, optimize the target selection effect, and reduce the misrecognition rate.

5. A multi-objective control system for brain-controlled interaction according to claim 4, characterized in that: In training mode, when the user selects training mode, if there is an existing personalized model, it can be used directly, otherwise the system will create a new model and enter the training process; During the training data collection phase, users need to follow the system instructions and look at different SSVEP stimulation targets in turn. Each target will flash at a set frequency, and the EEG device will synchronously collect EEG signals and perform preprocessing.

6. A multi-objective control system for brain-controlled interaction according to claim 5, characterized in that: In the training data processing stage, the system uses canonical correlation analysis (CCA), multi-band CCA and convolutional neural network (CNN) methods to extract EEG features and construct feature vectors. Subsequently, the system uses support vector machines (SVMs) and deep neural network (DNN) classifiers for training and adjusts model parameters to adapt them to the individual characteristics of users. To improve recognition accuracy, the system introduces a transfer learning mechanism to fine-tune the existing model so that the newly trained data maintains high consistency with the original model, thereby improving generalization capabilities; During the training process, the system evaluates the classification confidence in real time and optimizes the feature extraction method based on the individual differences of users to ensure the accuracy and robustness of signal classification.

7. A multi-objective control system for brain-controlled interaction according to claim 6, characterized in that: After the training mode is completed, the user enters the real-time control stage and can choose between the free gaze mode and the guided mode. The system collects the user's EEG signals and classifies them in real time, identifies the target by the maximum confidence, outputs control instructions, and synchronously updates the user interface. If the system determines that the confidence is high, it will directly output the control command; if the confidence is low, the system will optimize the classification strategy based on historical data, or prompt the user to adjust the gaze mode and re-collect data. The user can also choose to re-establish a personal model. After the experiment, if the current training model performs well, the system can store the user's personalized features for subsequent use, making the interaction more accurate and efficient; If the user chooses to continue the experiment, the system will enter the next round of signal acquisition and classification process, otherwise it will exit the training mode and return to the main interface standby state.

8. The multi-objective control system for brain-controlled interaction according to claim 3, characterized in that: The preprocessing method is to use bandpass filtering and independent component analysis (ICA) to preprocess the signal to remove environmental noise, electromyographic and electrooculographic artifacts.

Images