Attention detection method and system based on brain-computer interface

Abstract

The invention provides an attention detection method and system based on a brain-computer interface, and particularly relates to the technical field of brain-computer interfaces, and the system integrates a hardware unit (including TGAM electroencephalogram acquisition, Bluetooth connection and LED illumination control), a software unit (including login, detection, display and data export) and an algorithm unit. According to the method, electroencephalogram collection is carried out, wavelet filtering is utilized to decompose EEG signals, an Attention value and a Meditation value are extracted after calculation, a fusion value AM of the Attention value and the Meditation value is further provided, and control over the LED illumination brightness is achieved according to the fusion value. Besides, the functions of user verification, signal processing, result display and data recording are achieved through the software unit, the problems of extraction and fusion of attention features and relaxation features of electroencephalogram signals in the man-machine interaction process are solved through the algorithm unit, the accuracy and reliability of collection and processing of electroencephalogram data are improved, and the system is suitable for popularization and application. Effective attention feature extraction and feature fusion based on the electroencephalogram signals and adjustment of the LED illumination brightness are achieved, and development of the human-computer interaction technology is promoted.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface technology, and in particular to an attention detection method and system based on brain-computer interface. Background Art

[0002] In recent years, with the rapid development of the industry driven by scientific and technological innovation, the development of new quality productivity in the field of brain-computer interface has gradually received attention. Brain-computer interface is a system that uses brain signals to initiate communication between a person's mental state and a signal processing system. In the process of human-computer interaction, the acquisition of attention features of electroencephalogram (EEG) signals is a very challenging task. First of all, the EEG signal itself has the characteristics of small amplitude, low signal-to-noise ratio, non-stationary and nonlinear, which makes it extremely difficult to accurately extract attention features from the original EEG signal. Secondly, external environmental interference, individual physiological differences and the limitations of the acquisition equipment itself, such as electrode sliding and impedance changes, will introduce noise and further reduce the signal quality. In addition, during the human-computer interaction process, the user's attention state may be dynamically adjusted as the task changes. How to capture this change in real time and accurately is also a major problem. Finally, there may be significant differences in the attention features between different users. How to establish a universal and effective attention feature acquisition method to meet the needs of different users is also an important issue currently faced.

[0003] In response to the above problems, the prior art has taken a series of measures to improve the accuracy and real-time performance of EEG signal attention feature acquisition. First, by optimizing electrode design (such as using flexible micro-column electrodes) and improving signal acquisition technology (such as using nanoimprinting and 3D printing technology), the interference of external signals is reduced and the signal quality is improved. Secondly, in terms of signal processing, advanced filtering technology (such as wavelet filtering) and feature extraction algorithms (such as independent component analysis, common space mode, etc.) are introduced to effectively remove noise and extract features related to attention. In addition, combined with machine learning technology (such as support vector machine, neural network, etc.), the extracted features are classified and identified to improve the accuracy of attention detection. Finally, in view of the differences between different users, the prior art also attempts to optimize the effect of attention feature acquisition through adaptive algorithms or personalized models to meet the needs of different users. However, although the prior art has made certain progress in solving the problem of EEG signal attention feature acquisition in the process of human-computer interaction, there are still some unresolved problems. First, the signal-to-noise ratio problem of EEG signals has not been fundamentally solved, especially in complex environments and dynamic tasks, noise interference is still a factor that cannot be ignored. Secondly, there is often a trade-off between complexity and accuracy in existing feature extraction and classification algorithms. How to improve classification accuracy while ensuring real-time performance remains a challenge. In addition, although personalized models have adapted to the needs of different users to a certain extent, their construction process is complex and costly, making them difficult to promote and apply on a large scale. Finally, the user's attention state during human-computer interaction may be affected by a variety of factors (such as emotions, fatigue, etc.). Existing technologies have not yet fully considered the impact of these factors on the collection of attention features, so there may be certain limitations in practical applications.

[0004] To sum up, how to solve the problem of EEG signal attention detection method and system construction during human-computer interaction is a difficult problem that needs to be overcome urgently in the field of brain-computer interface technology. Summary of the invention

[0005] The main purpose of the present invention is to provide an attention detection method and system based on brain-computer interface to solve the problem of feature extraction and feature fusion of EEG signal attention and relaxation during human-computer interaction, thereby improving the accuracy and reliability of data acquisition, realizing the effective extraction, fusion and application of attention and relaxation features, and promoting the development of human-computer interaction technology.

[0006] In order to achieve the above objectives, the present invention provides an attention detection method and system based on brain-computer interface, thereby improving the accuracy and reliability of EEG data acquisition, realizing the effective extraction, fusion and application of attention and relaxation features, and promoting the development of human-computer interaction technology.

[0007] In a first aspect, the present invention provides an attention detection system based on a brain-computer interface, the system comprising: a hardware unit, a software unit and an algorithm unit. The system detects attention and relaxation of the original EEG signal and controls the brightness of the lighting hardware according to the detection results. The system specifically comprises:

[0008] The hardware unit includes a TGAM EEG acquisition module, a Bluetooth connection module and an LED lighting control module; the TGAM EEG acquisition module is used to collect the original EEG signal, the Bluetooth connection module is used to realize wireless data transmission between the computer and the TGAM module, and the LED lighting control module controls the brightness of the LED lighting through the ESP32 chip and PWM technology.

[0009] The software unit includes a login module, a detection module, a display module and an xls file generation module; the login module is used for user identity authentication and identity registration, and the detection module is used to process the received original EEG signal and calculate the Attention value and Meditation value and the fusion value A corresponding to the Attention value and Meditation value in real time M , the display module is used to display the original EEG signal waveform and the real-time line graph of the energy of each frequency band after the original EEG signal is processed by wavelet filtering and the real-time line graph of the attention detection result on the Web end in real time, and the xls file generation module is used to export the original EEG signal and the original EEG signal detection result xls file in real time, wherein the detection result xls file is used to indicate the real-time energy of eight different frequency bands and the real-time detection result of the original EEG signal after filtering;

[0010] The algorithm unit includes a single-channel EEG subject attention detection algorithm, wherein the single-channel EEG subject attention detection algorithm is used to divide the collected original EEG signal into 8 independent frequency bands through wavelet filtering, and calculate the Attention value, Meditation value and fusion value A corresponding to the Attention value and Meditation value of the original EEG signal per second in real time. M .

[0011] Optionally, the Bluetooth connection module includes a Bluetooth adapter Dongle serial port module, and the system further includes:

[0012] The original data packet with a transmission frequency of 512 Hz sent by the TGAM module is received through the Bluetooth adapter Dongle serial port module, and the original data packet is parsed to obtain the original EEG signal.

[0013] Use MQTT protocol to transfer the fusion value AM Push to the MQTT device cloud, the ESP32 chip connects via WiFi and subscribes to the MQTT device cloud to receive the fusion value A M , and convert it into an int type integer, control the brightness of the LED lighting through map mapping and PWM technology, and feed back the real-time results to the MQTT device cloud.

[0014] Optionally, the xls file generation module can export the original EEG signal xls file containing the signal acquisition time, the original EEG signal value, and the signal acquisition time, the energy of 8 independent frequency bands, the Attention value and the Meditation value, and the fusion value A corresponding to the Attention value and the Meditation value. M The test results xls file.

[0015] Optionally, the system also includes a MySQL database for storing original EEG signals and their detection results to achieve persistent storage and management of data.

[0016] Optionally, the Web terminal realizes real-time processing and display of data through a servlet container, and provides a user-friendly interactive interface for real-time viewing of the original EEG signal waveform, frequency band energy line graph, attention detection result line graph, and export of xls files.

[0017] In a second aspect, the present invention provides an attention detection method based on a brain-computer interface, the method being applied to the attention and relaxation detection system based on a brain-computer interface according to any one of claims 1 to 5, comprising:

[0018] Collecting raw EEG signals, and transmitting the raw EEG signals to a computer via Bluetooth wireless;

[0019] The original EEG signal is processed by wavelet filtering and single-channel EEG subject attention detection algorithm to calculate the Attention value and Meditation value;

[0020] The Attention value and the Meditation value are fused to obtain the fusion value A M ;

[0021] Push the fusion value A via the MQTT protocol M To the LED lighting control module, the brightness of the LED lighting control module is controlled by PWM technology, and the processing results and exported data files are displayed in real time on the Web end.

[0022] Optionally, performing wavelet filtering and single-channel EEG subject attention detection algorithm processing on the original EEG signal to calculate the Attention value and the Meditation value includes:

[0023] The original EEG signal is subjected to wavelet filtering to obtain 8 different frequency bands, and the 8 different frequency bands are stored in a MySQL database;

[0024] The eight different frequency bands are detected by the subject's electroencephalogram signal based on single-channel EEG to obtain the Attention value and Meditation value.

[0025] Optionally, the fusion value A is pushed via the MQTT protocol M To the LED lighting control module, the brightness control of the LED lighting control module is realized by PWM technology, including:

[0026] The fusion value A M Convert to int type, and map the fusion value A from 0 to 100 M The LED corresponding PWM value mapped to 0-255 completes the brightness control of the LED lighting hardware from 0 to 255, where 0 represents off and 255 represents the brightest.

[0027] In a third aspect, the present invention provides an electroencephalogram information acquisition device based on a brain-computer interface, the device comprising:

[0028] Memory;

[0029] processor;

[0030] Wherein, the memory stores computer-executable instructions;

[0031] The processor executes the computer-executable instructions stored in the memory to implement the above second aspect and / or various possible implementations of the second aspect.

[0032] In a fourth aspect, the present application provides a computer storage medium, wherein the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement the above second aspect and / or various possible implementations of the second aspect.

[0033] The present application provides a method and system for attention detection based on brain-computer interface, which integrates hardware units (including TGAM EEG acquisition, Bluetooth connection and LED lighting control), software units (including login, detection, display and data export) and algorithm units. The system collects EEG signals and calculates the fusion value A by the Attention value and Meditation value in the EEG signal attention detection result.M , and adjust the LED lighting brightness accordingly. The software implements user verification, signal processing, result display and data recording functions. The algorithm uses wavelet filtering to decompose the EEG signal, calculate the Attention and Meditation values, and the fusion value A corresponding to the Attention and Meditation values M The system solves the problem of collecting attention features of EEG signals during human-computer interaction, thereby improving the accuracy and reliability of data collection, achieving effective extraction of attention features, and promoting the development of human-computer interaction technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0035] Figure 1 A schematic diagram of the attention detection system based on the brain-computer interface provided for this application;

[0036] Figure 2 A flowchart of the attention detection method based on brain-computer interface provided in this application;

[0037] Figure 3 A schematic diagram of an attention detection device based on a brain-computer interface provided for this application;

[0038] Figure 4 The overall execution flow chart of the hardware unit of the attention detection system based on brain-computer interface provided in this application;

[0039] Figure 5 The present application provides a flowchart of the overall execution of software units of a brain-computer interface-based attention detection system.

[0040] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein.

[0043] In the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0044] The present application provides a method and system for attention detection based on brain-computer interface, which integrates hardware units (including TGAM EEG acquisition, Bluetooth connection and LED lighting control), software units (including login, detection, display and data export) and algorithm units. The system detects attention in real time by collecting EEG signals and adjusts the brightness of LED lighting accordingly. The software implements user verification, signal processing, result display and data recording functions, and the algorithm uses wavelet filtering to decompose EEG signals, calculate Attention and Meditation values ​​and the fusion value A corresponding to Attention and Meditation values. M The system solves the problem of collecting attention features of EEG signals during human-computer interaction, thereby improving the accuracy and reliability of data collection, achieving effective extraction of attention features, and promoting the development of human-computer interaction technology.

[0045] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0046] Figure 1 A flow chart of the attention detection system based on brain-computer interface provided in this application, such as Figure 1 As shown, the attention detection system based on brain-computer interface provided in this embodiment includes: a hardware unit, a software unit and an algorithm unit. The system detects attention and relaxation of the original EEG signal and controls the brightness of the lighting hardware according to the detection results. The system specifically includes:

[0047] The hardware unit includes a TGAM EEG acquisition module, a Bluetooth connection module and an LED lighting control module; the TGAM EEG acquisition module is used to collect the original EEG signal, the Bluetooth connection module is used to realize wireless data transmission between the computer and the TGAM module, and the LED lighting control module controls the brightness of the LED lighting through the ESP32 chip and PWM technology.

[0048] The software unit includes a login module, a detection module, a display module and an xls file generation module; the login module is used for user identity authentication and identity registration, and the detection module is used to process the received original EEG signal and calculate the Attention value and Meditation value and the fusion value A corresponding to the Attention value and Meditation value in real time M The display module is used to display the original EEG signal waveform and the real-time line graph of the energy of each frequency band after the original EEG signal is processed by wavelet filtering and the real-time line graph of the attention detection result on the Web side in real time. The xls file generation module is used to export the original EEG signal and the original EEG signal detection result xls file in real time, wherein the detection result xls file is used to indicate the real-time energy of eight different frequency bands and the real-time detection result of the original EEG signal after filtering.

[0049] The algorithm unit includes a single-channel EEG subject attention detection algorithm, wherein the single-channel EEG subject attention detection algorithm is used to divide the collected original EEG signal into 8 independent frequency bands through wavelet filtering, and calculate the Attention value, Meditation value and fusion value A corresponding to the original EEG signal per second in real time. M .

[0050] The hardware unit includes: TGAM module connected with Bluetooth module to realize single-channel EEG signal acquisition and storage, ESP32 chip design to realize hardware lighting circuit, ESP32 connected with MQTT to realize fusion value A MThe hardware and software are interconnected; the algorithm unit includes: wavelet filtering of the original EEG signal, using wavelet packet transform (WPT) to decompose the signal into 8 independent frequency bands such as delta (0.5-2.75Hz), theat (3.5-6.75Hz), low_alpha (7.5-9.25Hz), high_alpha (10-11.75Hz), and then performing attention detection calculation processing on the filtered signal to obtain attention and relaxation (that is, Attention value and Meditation value); the software unit includes: real-time output of single-channel EEG signal, real-time display of line graph within 60s of the energy of 8 frequency bands after wavelet filtering, real-time display of line graph within 60s of Attention value and Meditation value and instrument visualization output, and export of original EEG signal and detection result xls file during operation.

[0051] It can be understood that the overall design of the system is mainly divided into TGAM EEG acquisition module, Bluetooth connection module, display module, detection module, and LED lighting control module. The detection module, LED lighting control module, and display module are interconnected through the MQTT protocol to realize the single-channel EEG signal. After the attention detection, the Attention value and Meditation value in the detection result are calculated and fused to obtain the fusion value A M , so according to the fusion value A M Real-time judgment and pulse width modulation (PWM technology) of LED lighting hardware, brightness control of LED lighting hardware is achieved by adjusting duty cycle and frequency.

[0052] The corresponding brain-computer interface principle of this system mainly realizes the mental control of LED lighting hardware through raw EEG signal acquisition, raw EEG signal analysis, attention status judgment, LED lighting hardware response and attention extraction of single-channel raw EEG signals, thereby realizing direct control of the outside world by the human brain.

[0053] Before attention detection, the collected raw EEG signal is subjected to wavelet filtering. The signal is decomposed into 8 independent frequency bands, including delta (0.5-2.75Hz), theat (3.5-6.75Hz), low_alpha (7.5-9.25Hz), and high_alpha (10-11.75Hz), using wavelet packet transform (WPT) to evaluate the impact of different frequency bands on the task. For the specific implementation process, please refer to Figure 2 An attention detection method based on brain-computer interface is provided in an embodiment.

[0054] The system hardware unit includes the following parts: TGAM module connected with Bluetooth module to realize the acquisition and storage of single-channel original EEG signal; ESP32 chip LED lighting control module design to realize hardware lighting circuit; and ESP32 connected with MQTT protocol to realize fusion value A M The hardware and software interconnection. The hardware operation progress is generally divided into the following four steps:

[0055] Step 1: ESP32 is connected to WiFi and then to the MQTT device cloud. The TGAM module is connected to the Bluetooth adapter Dongle serial port module. The original EEG signal small packet data and the 513th large packet data sent by the TGAM module at a frequency of 512Hz per second are parsed from the PC COM port, and the original EEG raw signal is stored in the MySQL database. When it is determined that the current sending packet is a small packet, data reception begins.

[0056] Step 2: The results of attention detection in the eight frequency bands after wavelet filtering of the original EEG signal are pushed to the corresponding MQTT server by subscribing to the corresponding topic through the MQTT device cloud.

[0057] Step 3: ESP32 connects to the MQTT device cloud and subscribes to the corresponding topic. It determines in real time whether there is any information pushed to the MQTT server. If there is any information pushed to the server, it receives the pushed information.

[0058] Step 4: The received fusion value A with a threshold of 0-100 M Map it to a length of 0-255, and then use pulse width modulation (PWM technology) to control the brightness of the LED lighting hardware.

[0059] The system software unit includes the following functions: real-time output of single-channel EEG signals, real-time display of the energy of 8 frequency bands after wavelet filtering within 60 seconds, real-time display and instrument visualization output of Attention and Meditation values ​​within 60 seconds, and export of original EEG signals and test results to xls files.

[0060] The specific execution process of the system hardware unit is as follows: Figure 4 As shown, Figure 4 This is a flowchart of the overall execution of the hardware units of the brain-computer interface-based attention detection system provided in this application.

[0061] The software running progress is generally divided into the following four steps:

[0062] Step 1: Check whether the user is a new user. If yes, register an account. Otherwise, log in. Check whether the account and password are correct. If yes, enter the EEG Attention and Meditation interface to start receiving hardware transmission signals. If not, an error window will pop up and re-enter the corresponding login information.

[0063] Step 2: After entering the EEG Attention and Meditation interface, the interface displays two graphs, 8-EEG-power and EEG Attention and Meditation, in real time. Click 8-EEG-power to enter the 8EEG Power interface, and click EEG Attention and Meditation to enter the Attention and Relaxation interface.

[0064] Step 3: If you enter the 8EEG Power interface, you can choose to observe the energy rectangle comparison chart of 8 frequency bands.

[0065] Step 4: If you enter Attention and Relaxation, you can display the real-time output of the original EEG signal, the real-time output of the Attention and Relaxation curve graph, and the real-time change graph of the Attention and Relaxation instrument. At this time, click Attention and Relaxation to generate the EEGpower.txt file, and click EEG to generate the EEGdata.txt file.

[0066] The specific execution process of the system software unit is as follows: Figure 5 As shown, Figure 5 The present application provides a flowchart of the overall execution of software units of a brain-computer interface-based attention detection system.

[0067] Optionally, the Bluetooth connection module includes a Bluetooth adapter Dongle serial port module, and the attention detection system based on the brain-computer interface further includes: receiving an original data packet with a transmission frequency of 512 Hz sent by the TGAM module through the Bluetooth adapter Dongle serial port module, and parsing the original data packet to obtain an original EEG signal; using the MQTT protocol to transfer the fusion value A M Push to the MQTT device cloud, the ESP32 chip connects via WiFi and subscribes to the MQTT device cloud to receive the fusion value A M , and convert it into an int type integer, control the brightness of the LED lighting through map mapping and PWM technology, and feed back the real-time results to the MQTT device cloud.

[0068] Furthermore, the xls file generation module can export the original EEG signal xls file containing the signal acquisition time, the original EEG signal value, and the signal acquisition time, the energy of 8 independent frequency bands, the Attention value and the Meditation value, and the fusion value A corresponding to the Attention value and the Meditation value. M The test results xls file.

[0069] Optionally, the brain-computer interface-based attention detection system also includes a MySQL database for storing original EEG signals and their detection results to achieve persistent storage and management of data.

[0070] Furthermore, the Web end realizes real-time processing and display of data through a servlet container, and provides a user-friendly interactive interface for real-time viewing of the original EEG signal waveform, frequency band energy line graph, attention detection result line graph, and export of xls files.

[0071] Figure 2 The flowchart of the attention detection method based on the brain-computer interface provided in this application is a detailed description of the attention detection method based on the brain-computer interface, such as Figure 2 As shown, the attention detection method based on brain-computer interface provided in this embodiment includes:

[0072] S201: Collecting original EEG signals, and transmitting the original EEG signals to a computer via Bluetooth wireless.

[0073] Among them, the original EEG signal is an electrical signal generated by placing electrodes on the scalp to record the activity of brain neurons. The way to transmit the original EEG signal to the computer via Bluetooth wireless is through an EEG signal acquisition device with integrated Bluetooth function. Such devices usually have built-in Bluetooth chips that can send the collected original EEG signals wirelessly. For example, EEGM102 is a high-precision, low-power EEG signal sensor module launched by Xinsen Microelectronics, which integrates a low-power Bluetooth 5.0 processor MCU and supports Bluetooth wireless transmission.

[0074] S202: Perform wavelet filtering and single-channel EEG subject attention detection algorithm processing on the original EEG signal to calculate the Attention value and Meditation value.

[0075] Among them, the single-channel EEG subject attention detection algorithm is mainly achieved by wavelet filtering the original EEG signal. Wavelet filtering divides the original EEG signal into 8 independent frequency bands. Table 1-1 is the EEG interpretation of the cognitive state of the 8 independent frequency bands. According to the EEG interpretation of the cognitive state of the 8 independent frequency bands, we can know that the concentration or alertness is mainly strongly correlated with the Beta (β) frequency band, and the relaxation is strongly correlated with the Alpha (α) frequency band. The changes in the above bands are closely related to a series of brain activities such as attention, relaxation, meditation, and brain dysfunction.

[0076] The original EEG signal is subjected to wavelet filtering and single-channel EEG subject attention detection algorithm processing to calculate the Attention value and Meditation value, including:

[0077] The original EEG signal is subjected to wavelet filtering to obtain 8 different frequency bands, and the 8 different frequency bands are stored in a MySQL database;

[0078] The eight different frequency bands are subjected to the subject's attention detection based on the single-channel EEG to obtain the Attention value and the Meditation value, wherein the subject's attention detection process of the single-channel EEG is implemented by the subject's attention detection algorithm of the single-channel EEG.

[0079] Furthermore, the original EEG signal is filtered by wavelet to obtain 8 different frequency bands, including:

[0080] The original EEG signal is decomposed into 8 independent frequency bands including delta (1-3Hz), theat (4-7Hz), low_alpha (8-9Hz), and high_alpha (10-12Hz) using wavelet packet transform (WPT). The wavelet filtered signal is then processed for attention detection and calculation to obtain the Attention value and Meditation value.

[0081] Among them, wavelet transform can be expressed by equations (1)-(2):

[0082]

[0083] Here, φ(t) is called the mother wavelet function. The mother wavelet is a transformation function that is scaled and shifted to match the target waveform to find the wavelet coefficients. The mother wavelet can form a basis set as shown in Equation (3) based on its expansibility and transition properties.

[0084]

[0085] Among them, φ in formula (3) s,u(t) where t is the independent variable, u is the translation parameter, indicating the region of interest, and s is the positive scaling parameter, since negative scaling is not defined. s,u (t)} is orthonormal according to the multiresolution property. Conceptually, the CWT is a basis φ s,u (t). It can be expressed by equations (4)-(6):

[0086]

[0087] Using this transformation, a one-dimensional signal f(t) can be mapped into a two-dimensional signal Wf(s,u). The two variables can be used for time-frequency analysis, and a specific frequency can be located at a given time, represented by the parameters s and u respectively.

[0088] Furthermore, the 8 different frequency bands are subjected to the subject's attention detection based on a single-channel EEG to obtain the Attention value and the Meditation value, including:

[0089] After the above wavelet filtering of the original signal, we can quantify the attention (i.e., Attention value) of the original EEG signal in real time based on the attention detection of the subject based on single-channel EEG. At the same time, many studies have found that the alpha wave component increases when relaxed and decreases when stressed. Therefore, the alpha wave content is considered to be an indicator of calm relaxation. In the process of studying relaxation, we focus on the energy of the three frequency bands of alpha, beta, and theta waves, and use the ratio of the alpha wave band energy to the total energy of the three waves Med Ratioα As a representation of stress / relaxation, Med Ratioα The bigger the number, the more relaxed the person is. Ratioα The smaller it is, the more pressure a person is under.

[0090] After wavelet filtering, the EEG data is subjected to FFT fast Fourier transform to obtain the energy E of α wave, θ wave and β wave respectively. α 、E β 、E θ Finally, the ratio of the α wave band energy to the total energy of the three waves is calculated to obtain the calm relaxation value (i.e., Meditation value). Formulas (7) and (8) are used here to calculate the attention and relaxation values. The attention detection of the subjects based on single-channel EEG is mainly achieved by wavelet filtering the original EEG signal. The wavelet filter divides the original EEG signal into 8 independent frequency bands. Table 1-1 is the EEG interpretation of the cognitive state of the 8 independent frequency bands.

[0091] Table 1-1 Interpretation of EEG frequency band after filtering

[0092]

[0093] According to the EEG interpretation of the eight independent frequency bands of cognitive states shown in the table above, we can know that the concentration or alertness is mainly related to the Beta (β) frequency band, and the relaxation is mainly related to the Alpha (α) frequency band. The changes in the above bands are closely related to a series of brain activities such as attention, relaxation, meditation, and brain dysfunction. [1] .

[0094] At present, the most widely used method for evaluating attention is the formula proposed by Pope et al. in 1995 to calculate attention based on the energy of α, θ, and β wave bands (Pope et al., 1995). [2] That is, the ratio of the energy in the beta band to the sum of the energy in the alpha and theta bands. According to the current understanding of EEG signals, this is because when people are concentrating or alert, their EEG signals are mainly in the beta band, while when people are resting or sleeping, they are mainly in the alpha or theta or even lower frequency bands. Therefore, this ratio can represent the degree of people's concentration. [3] This formula has been used by many scholars to analyze and calculate attention when performing cognitive tasks (Brookhuis & Waard, 1993) [4] The frontal lobe of the cerebral cortex is responsible for the body's attention and motor planning (Gentili et al., 2010; Girouard et al., 2013) [5][6] , so the EEG signal here is taken from the user's frontal lobe. When collecting signals, we place the electrodes for collecting signals near the temples of the forehead, which is close to the premotor area of ​​the frontal lobe of the brain.

[0095] The article Transfer Learning of Visual Attention and Working Memory in the Motor Cortex of Brain-Computer Interfaces published in 2022 pointed out that the classification of deep learning algorithms in EEG signal analysis showed differences between resting states and cognitive activities. In addition, the levels of these activities in different frequency bands also showed differences.

[0096] The literature shows that the maximum accuracy in the beta band was 89.74% and 91.02% in the two conditions of rest and attention and working memory, respectively. At the same time, the accuracy of the alpha band was significantly lower for the same activities, indicating that activities such as visual attention and working memory appear more in the beta and theta bands than the alpha band. However, when the attention and working memory signals were compared with each other, the accuracy of the beta band was 74.35%, while the accuracy of all three categories was 76.06%, indicating that these two activities are prominent in both frequency bands.

[0097] Studies in the literature have established the separability of neural responses to visual attention and visuospatial working memory at different frequency bands, and the results show that cognitive activity in the motor cortex is most prominent in the beta band, while the theta band also has cognitive properties.

[11] In summary, formula (7) is used to calculate attention during the detection process.

[0098]

[0099] The amount of alpha waves is closely related to the degree of relaxation of the brain. When a person closes his eyes or is in a state of physical and mental relaxation, the proportion of alpha brain waves will increase significantly. [7] .

[0100] Many studies have found that alpha waves increase when you are relaxed and decrease when you are stressed. [8][9] Therefore, the content of alpha waves is considered to be an indicator of calm and relaxation. In the process of studying relaxation, we focus on the energy of alpha waves, theta waves, and beta waves, and use the ratio of alpha wave energy to the total energy of the three waves as Med Ratioα As a representation of stress / relaxation, Med Ratioα The bigger the number, the more relaxed the person is. Ratioα The smaller the number, the more pressure people have.

[10] .

[0101] After filtering, the EEG data is subjected to FFT fast Fourier transform to obtain the energy E of α wave, θ wave and β wave respectively. α 、E β 、E θ Finally, the ratio of the α wave band energy to the total energy of the three waves is calculated to obtain the calm relaxation value. Therefore, formula (8) is used here to calculate the calm relaxation value.

[0102]

[0103] As shown in equations (7) and (8), we can perform attention detection calculation on the energy of the eight independent frequency bands after wavelet filtering to obtain the corresponding attention and relaxation values ​​(i.e., Attention value and Meditation value).

[0104] S203: Fusing the Attention value and the Meditation value to obtain a fusion value A M ;

[0105] The way to fuse the Attention value and the Meditation value is to fuse the two calculated values ​​(Attention and Meditation) into a value that can reflect attention, and propose and push the fusion value A MThe value is sent to the LED lighting control module, and the brightness control of the LED lighting control module is realized through PWM technology, and the fusion value A M The specific calculation method is as follows:

[0106] (1) Standardization: First, the two values ​​are standardized (i.e., normalized) to values ​​between 0 and 1 as shown in equation (9). This allows them to be compared on the same scale.

[0107]

[0108] (2) Weighted fusion: The standardized values ​​of Attention and Meditation are weighted fused. According to the single-channel EEG subject attention detection algorithm, the attention of the original EEG signal in different frequency bands after filtering is calculated through FFT fast Fourier transform energy in the process of Attention and Meditation. Attention is basically proportional to attention, while Meditation is basically inversely proportional to attention. Therefore, the fusion value can be calculated using the following formula (10):

[0109] A M (Attention, Meditation)=ω1×Atten norm +ω2×(1-Med norm ) (10)

[0110] Among them, ω1 and ω2 are two weight coefficients, representing the influence of Attention and Meditation on attention. The present invention believes that Attention has a greater influence on attention, and Meditation has a smaller influence (highly concentrated tasks pay more attention to attention), so the weight coefficients are set as: ω1 = 0.7, ω2 = 0.3

[0111] (3) Return the fusion value: the obtained fusion value A M (Attention, Meditation) is the fused attention value, which ranges from 0 to 1. Finally, the fused value A M Multiply (Attention, Meditation) by 100 and convert it to an integer value between 0 and 100.

[0112] The final formula is as follows (11):

[0113]

[0114] The fusion value A M (Attention, Meditation) is converted to int type, and the fusion value A of 0-100 is mapped through mapM (Attention, Meditation) is mapped to the LED corresponding PWM value of 0-255, completing the brightness control of the LED lighting hardware from 0 to 255, where 0 represents off and 255 represents the brightest.

[0115] S204: Push fusion value A via MQTT protocol M To the LED lighting control module, the brightness of the LED lighting hardware is controlled through PWM technology, and the processing results and exported data files are displayed in real time on the Web.

[0116] Among them, PWM (Pulse Width Modulation) is a digital technology for analog circuits. It modulates the width of a series of pulses to equivalently obtain the required waveform (including shape and amplitude). PWM signals are usually digital signals, but they are widely used in power electronics technology, automatic control systems and other fields to simulate analog signals.

[0117] When this step is implemented, the fusion value A obtained in step S203 can be M Subscribe to the corresponding topic through the MQTT device cloud and push the message to the corresponding MQTT server; ESP32 connects to the MQTT device cloud and subscribes to the corresponding topic, and determines in real time whether there is information pushed to the MQTT server. If there is information pushed to the server, receive the pushed information; the fusion value A in the received information M Convert to int type, and map the fusion value A from 0 to 100 M The LED corresponding to the PWM value mapped to 0-255 completes the brightness control of the LED lighting hardware from 0 to 255, where 0 represents off and 255 represents the brightest.

[0118] It can be understood that the present invention completes the brightness control of lighting hardware through the original EEG signal attention detection. The present invention implements the hardware and software and algorithm units and implementation. The hardware unit consists of three parts: TGAM EEG acquisition module, Bluetooth connection module, and LED lighting control module; the software unit consists of four parts: login module, detection module, display module, and xls file generation module; the algorithm unit selects the subject attention detection based on single-channel EEG for the attention detection method of this system after comparison. The computer is connected to the TGAM module through the Bluetooth adapter Dongle serial port module to receive the data packet sent by the TGAM module, and the original data packet with a transmission frequency of 512Hz is analyzed by large packet judgment to obtain the original EEG signal, and the original EEG signal waveform is displayed in real time on the Web end. The collected original EEG signal is filtered by wavelet filtering and divided into 8 independent frequency bands, and the subject attention detection algorithm based on single-channel EEG is used for real-time attention detection to determine the attention value and meditation value corresponding to the original EEG signal per second. At the same time, the real-time line graph of the energy of the 8 independent frequency bands after filtering and the real-time line graph corresponding to the attention detection result are output in real time on the Web end. Push the fusion value A in the attention detection result through the MQTT protocol M To the corresponding MQTT device cloud, the MQTT device cloud receives the test results, the ESP32 chip connects to WiFi and subscribes to the corresponding MQTT device cloud, receives the test result information pushed by the corresponding MQTT device cloud and converts the corresponding value into an int type integer, realizes the brightness control of the LED lighting hardware through map mapping and pulse width modulation (PWM), and feeds back the real-time results to the corresponding topic of the MQTT device cloud. Through the servlet container, the original EEG signal can be exported in real time on the Web side during the acquisition process. The original EEG signal xls file (including signal acquisition time, signal value) can also be exported in real time after the attention detection of the collected signal. The xls file (including signal acquisition time, 8 8 independent frequency band energy, attention value (ie Attention value), relaxation value (ie Meditation value)).

[0119] In the process of implementing the system designed by the present invention, the original EEG signal and the detection results are stored in the MySQL database, and the MQTT device cloud realizes the interconnection between the Web terminal and the ESP32 chip. The overall system design realizes the functions of EEG signal acquisition, Bluetooth data wireless transmission, brain wave signal processing, attention feature extraction and output, real-time line chart display of corresponding information on the Web terminal, and brightness control of LED lighting hardware.

[0120] The advantages of the present invention are: (1) EEG signal acquisition and preprocessing: The computer connects to the TGAM module through the Bluetooth adapter Dongle serial port module to receive the data packet sent by the TGAM module, obtains the original EEG signal after judging and parsing the received data packet, and stores the original EEG signal in the MySQL database. The collected original EEG signal is subjected to wavelet filtering, and the signal is decomposed into 8 independent frequency bands using wavelet packet transform (WPT). The filtered signal is subjected to attention detection processing to obtain the detection result, and the detection result is received by connecting to MQTT through the ESP32 chip to realize the mind control of the brightness of the lighting hardware, which can provide data support for brain cognitive modeling. (2) Design and implementation of attention detection method: Compare the four-classification task based on 22-channel EEG under the attention mechanism and the subject attention detection based on single-channel EEG to select the attention detection method suitable for this system. At the same time, compare the two EEG acquisition devices, Neuroscan40 portable EEG instrument and TGAM EEG module produced by NeuroSky, and select the EEG acquisition device suitable for this system. (3) Visual design and implementation: Adopt B / S architecture, Tomcat+JAVA+MySQL development mode to design web page (Web) to realize real-time waveform display of original EEG signal; line graph display of 8 different frequency bands energy within 60s after filtering; attention detection to obtain the attention value and relaxation value of the test results within 60s, and visual display of the test result instrument; export the original EEG signal xls file; export the energy of 8 different frequency bands and attention detection results xls file after filtering. (4) The present invention fuses the two test result values ​​(Attention and Meditation) after the subject's attention detection based on single-channel EEG into a value that can reflect attention, and proposes and pushes the fusion value A M The value is sent to the LED lighting control module, and the brightness of the LED lighting control module is controlled by PWM technology.

[0121] Figure 3 A schematic diagram of an attention acquisition device based on a brain-computer interface provided for this application, such as Figure 3 As shown, the attention collection device 300 based on the brain-computer interface provided in this embodiment includes:

[0122] Memory 302;

[0123] Processor 301;

[0124] Wherein, the memory 302 stores computer-executable instructions;

[0125] The processor 301 executes the computer-executable instructions stored in the memory to implement the attention detection method based on brain-computer interface as shown in the above embodiment.

[0126] The present application also provides a computer storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the attention detection method based on brain-computer interface shown in the above embodiment.

[0127] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0128] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0129] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0130] Related Literature:

[0131] [1]MERINO M, I,MOLINA A J.EEG feature variations under stresssituations[C] / / Annual International Conference of the IEEE EngineeringinMedicine andBiology Society(EMBC).2015:6700-6703.

[0132] [2]Pope,AT,Bogart,EH,&Bartolome,DS(1995).Biocybernetic systemevaluates indices of operator engagement in automated tasks.BiologicalPsychology,40(1),187-195.

[0133] [3] Li Chong. Research on patient training participation and concentration based on robot-assisted neurorehabilitation[D]. Tsinghua University, 2019.

[0135] [4]Brookhuis,KA,&De Waard,D.(1993).The use ofpsychophysiology toassess driver status.ERGONOMICS,36(9),1099-1110.

[0136] [5] Gentili, RJ, Hadavi, C., Ayaz, H., Shewokis, PA, & Contreras-Vidal, JL (2010). Hemodynamic correlates of visuomotor motor adaptation by functionalnear infrared spectroscopy. In IEEE ENG MED BIO. IEEE, 2918-2921.

[0139] [6] Girouard, A., Solovey, ET, & Jacob, RJ (2013). Designing a passivebrain computer interface using real time classification of functional near–infrared spectroscopy. International Journal of Autonomous and Adaptive Communications Systems, 6 (1), 26-44.

[0140] [7]NIEDERMEYER E,et al.The normal EEG of the waking adult[J].Electroencephalography:Basic Principles,Clinical Applications,and RelatedFields,2005,167:155-164.

[0142] [8] PARK KS, CHOI H, LEE KJ, et al. Patterns of electroencephalography (EEG) change against stress through noise and memorization test [J]. International Journal of Medicine and Medical Sciences, 2011, 3(14): 381-389.

[0146] [9]ALONSO J, ROMERO S, BALLESTER M, et al.Stress assessment based on EEGunivariate features and functional connectivity measures[J].PhysiologicalMeasurement,2015,36(7):1351.

[0148]

[10] Wu Yi. Implementation and research of multimodal emotional relaxation system based on EEG[D]. Shanghai Jiaotong University, 2022. DOI: 10.27307 / d.cnki.gsjtu.2020.003432.

[0149]

[11] Kant,P.,Laskar,S.H.&Hazarika,J.Transfer learning-based EEGanalysis of visual Attention and working memory on motor cortex forBCI.Neural Comput&Applic34,20179-20190(2022).https: / / doi.org / 10.1007 / s00521-022-07580-z.

Claims

1. An attention detection system based on brain-computer interface, characterized in that: The system includes: a hardware unit, a software unit and an algorithm unit. The system detects the characteristics of attention and relaxation by collecting the original EEG signal, and controls the brightness of the lighting hardware according to the calculation result of the fusion value of the detection characteristics. The system specifically includes: The hardware unit includes a TGAM EEG acquisition module, a Bluetooth connection module and an LED lighting control module; the TGAM EEG acquisition module is used to collect the original EEG signal, the Bluetooth connection module is used to realize wireless data transmission between the computer and the TGAM module, and the LED lighting control module controls the brightness of the LED lighting through the ESP32 chip and PWM technology; The software unit includes a login module, a detection module, a display module and an xls file generation module; the login module is used for user identity authentication and identity registration, and the detection module is used to process the received original EEG signal and calculate the Attention value and Meditation value and the fusion value A corresponding to the Attention value and Meditation value in real time M , the display module is used to display the original EEG signal waveform and the real-time line graph of the energy of each frequency band after the original EEG signal is processed by wavelet filtering and the real-time line graph of the attention detection result on the Web end in real time, and the xls file generation module is used to export the original EEG signal and the original EEG signal detection result xls file in real time, wherein the detection result xls file is used to indicate the real-time energy of eight different frequency bands and the real-time detection result of the original EEG signal after filtering; The algorithm unit includes a single-channel EEG subject attention detection algorithm, wherein the single-channel EEG subject attention detection algorithm is used to divide the collected original EEG signal into 8 independent frequency bands through wavelet filtering, and calculate the Attention value, Meditation value and fusion value A corresponding to the Attention value and Meditation value of the original EEG signal per second in real time. M .

2. The system according to claim 1, characterized in that The Bluetooth connection module includes a Bluetooth adapter Dongle serial port module, and the system also includes: Receiving the original data packet with a transmission frequency of 512 Hz sent by the TGAM module through the Bluetooth adapter Dongle serial port module, and parsing the original data packet to obtain the original EEG signal; The fusion value A is transmitted using the MQTT protocol M Push to the MQTT device cloud, the ESP32 chip connects via WiFi and subscribes to the MQTT device cloud to receive the fusion value A M , and convert it into an int type integer, control the brightness of the LED lighting through map mapping and PWM technology, and feed back the real-time results to the MQTT device cloud.

3. The system according to claim 1, characterized in that The xls file generation module can export the original EEG signal xls file containing the signal acquisition time, the original EEG signal value, and the signal acquisition time, the energy of 8 independent frequency bands, the Attention value and the Meditation value, and the fusion value A corresponding to the Attention value and the Meditation value. M The test results xls file.

4. The system according to claim 1, characterized in that The system also includes a MySQL database for storing original EEG signals and detection results thereof to achieve persistent storage and management of data.

5. The system according to claim 1, characterized in that The Web end realizes real-time processing and display of data through a servlet container, and provides a user-friendly interactive interface for real-time viewing of the original EEG signal waveform, frequency band energy line graph, attention detection result line graph, and export of xls files.

6. A method for detecting attention based on a brain-computer interface, characterized in that: The method is applied to the attention detection system based on brain-computer interface according to any one of claims 1 to 5, comprising: Collecting raw EEG signals, and transmitting the raw EEG signals to a computer via Bluetooth wireless; The original EEG signal is processed by wavelet filtering and single-channel EEG subject attention detection algorithm to calculate the Attention value and Meditation value; The Attention value and the Meditation value are fused to obtain the fusion value A M ; Push the fusion value A via the MQTT protocol M To the LED lighting control module, the brightness of the LED lighting control module is controlled by PWM technology, and the processing results and exported data files are displayed in real time on the Web end.

7. The method according to claim 6, characterized in that The processing of the original EEG signal by wavelet filtering and single-channel EEG subject attention detection algorithm to calculate the Attention value and the Meditation value includes: The original EEG signal is subjected to wavelet filtering to obtain 8 different frequency bands, and the 8 different frequency bands are stored in a MySQL database; The eight different frequency bands are subjected to the subject's attention detection based on single-channel EEG to obtain the Attention value and Meditation value.

8. The method according to claim 6, characterized in that The fusion value A is pushed through the MQTT protocol M To the LED lighting control module, the brightness control of the LED lighting control module is realized by PWM technology, including: The fusion value A M Convert to int type, and map the fusion value A from 0 to 100 M The LED corresponding PWM value mapped to 0-255 completes the brightness control of the LED lighting hardware from 0 to 255, where 0 represents off and 255 represents the brightest.

9. An attention acquisition device based on a brain-computer interface, characterized in that: include: Memory; processor; Wherein, the memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the attention detection method based on brain-computer interface as described in any one of claims 6-8.

10. A computer storage medium, characterized in that: The computer storage medium stores computer execution instructions, which, when executed by a processor, are used to implement the attention detection method based on a brain-computer interface as described in any one of claims 6 to 8.

Images