Attention recognition system based on joint spectral and brain network features

Abstract

The application discloses an attention recognition system based on joint spectrum and brain network features, comprising: an electroencephalogram data acquisition module, which acquires electroencephalogram data of a user; a data preprocessing module, which preprocesses the electroencephalogram data; a feature extraction module, which extracts features from the preprocessed data, obtains spectrum features, and uses a phase-locked value as a brain network feature; a feature fusion module, which fuses the spectrum features and the brain network features to obtain fused features; a feature selection module, which respectively selects features from the brain network features and the fused features to obtain features most suitable for attention classification; an attention model training module, which trains a model based on the selected features to obtain a classification model most suitable for attention recognition; and an attention model prediction module, which identifies and judges the attention state of a user to be tested. The application can more comprehensively capture information in electroencephalogram signals, find the most relevant features, brain regions and channels, and further improve the accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning and biomedical, and particularly discloses an attention recognition system based on joint frequency spectrum and brain network features. BACKGROUND

[0002] Attention is a high-level cognitive function of the human brain, which refers to the description of the concentration ability of the inner activity of a person on a certain thing, and it contains a series of complex processing processes such as information acquisition, decoding and adjustment control, and is a prerequisite for human learning activities. Early attention recognition methods include crossing-out test, paced auditory serial addition test (PASAT), symbol digit modalities test (SDMT), trail making test, Stroop color-word interference test and the like. At present, many studies show that physiological signals are an effective indicator of attention test. Electroencephalogram is a very promising neurophysiological indicator, which can directly reflect the activity of the human brain, so as to effectively detect the attention change of the subject. In recent years, there are two kinds of brain electrical signal collection methods, which are embedded collection and non-embedded collection. The embedded collection method refers to craniotomy and implanting a sensor inside the brain. However, this method is not suitable for daily use of users. The non-embedded brain electrical signal collection method is to place an electrode sensor outside the cerebral cortex to collect brain electrical signals, and this method is used more in research. Although the attention research based on brain electrical signals has been for many years, and many studies have achieved some good results, however, the research on the attention recognition method combining joint frequency spectrum and brain network features is very few.

[0003] Current brain-computer interface-based attention systems fall into two categories: 1) systems based on statistical features and classifiers: a series of features need to be extracted from the raw electroencephalogram (EEG) signals first, which may include information on frequency, amplitude, time, and space. Then, these features are input into traditional classifiers such as Support Vector Machine (SVM), Logistic Regression, or Random Forest, etc. to train a model to identify the user's intention or thought. The advantage of this traditional method is relatively simple and easy to understand, and can achieve good results in some cases. 2) Attention recognition systems based on deep learning: deep learning systems do not require manual design of feature extractors, providing a new approach to processing complex EEG signals, and are expected to more accurately capture the user's cognitive state and intention. In summary, existing attention recognition systems require a large amount of data features and a large number of channels for training, which will cause high computational complexity and overfitting if all channels and all features are used. Moreover, most studies are offline recognition of user's attention level, which is less real-time. The representation of the features is also relatively single. Therefore, designing an attention recognition system based on joint spectral and brain network features is the key to attention research. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide an attention recognition system based on joint spectral and brain network features, which breaks through the high computational complexity caused by too much data or too many data channels in the training process of existing research data. The spectral features based on brain activation and the brain network features based on information interaction are fused to construct a feature representation that can comprehensively reflect the attention state, called fused features. These features can capture information in different frequency bands and waveforms of the EEG signal, thus more comprehensively displaying the changes in the EEG data. Combined with classical feature selection methods, the feature dimension is reduced and the most relevant features to attention recognition are selected to provide a more accurate and comprehensive information basis for attention recognition. There are unique differences between the EEG signals of different subjects. Therefore, the present application will select the most suitable features for attention analysis, combine multiple classifiers for training and analysis, and aim to find the best model for classification to improve the accuracy of cross-subject attention recognition.

[0005] To achieve the above purpose, the technical scheme provided by the present application is: an attention recognition system based on joint spectral and brain network features, comprising:

[0006] An EEG data acquisition module guides the user to perform the experiment according to the designed attention experiment paradigm, uses an EEG device to acquire the user's attention EEG data, and reads the EEG data.

[0007] a data preprocessing module, configured to perform preprocessing operations on electroencephalogram data read by the electroencephalogram data acquisition module, including filtering, artifact removal, and sampling frequency adjustment;

[0008] a feature extraction module, configured to extract features from the preprocessed data using a spectral feature extraction method, to obtain spectral features, and to use a phase locking value as a brain network feature;

[0009] a feature fusion module, configured to fuse the spectral features and the brain network feature, to capture information of different frequency bands and waveforms in the electroencephalogram data, and to obtain fused features;

[0010] a feature selection module, configured to select features from the brain network feature and the fused features using an improved adaptive particle swarm optimization algorithm, to obtain features most suitable for attention classification, wherein the improved adaptive particle swarm optimization algorithm introduces a dynamic adaptive neighborhood search and a dynamic mutation direction strategy;

[0011] an attention model training module, configured to train a classification model most suitable for attention recognition using five classifiers, including a support vector machine, a random forest, a decision tree, a K-nearest neighbor, and a Bayesian;

[0012] an attention model prediction module, configured to input electroencephalogram data of a user to be tested into the classification model most suitable for attention recognition obtained by the attention model training module, to identify and judge the attention state of the user, and to feed back a result to the user.

[0013] Further, the electroencephalogram data acquisition module includes an attention experiment paradigm module and a data loading module, wherein:

[0014] the attention experiment paradigm module guides a user to enter different states according to instructions on a computer screen using vision and hearing, in the process, an electroencephalogram acquisition device is used to collect, amplify, and filter electroencephalogram data, and then read the electroencephalogram data;

[0015] the data loading module is configured to read electroencephalogram data in mat, cnt, and eeg formats from a local, and to receive electroencephalogram data transmitted by the attention experiment paradigm module online.

[0016] Further, the data preprocessing module is configured to remove noise in the electroencephalogram data, to downsample the electroencephalogram data to reduce data volume and computational complexity, and to divide the electroencephalogram data into a plurality of time windows to facilitate subsequent module operations.

[0017] Further, the feature extraction module is configured to extract features from the preprocessed data using a spectral feature extraction method and to calculate a phase locking value between two channels as a brain network feature, including the following steps:

[0018] 1) Use the spectral feature extraction method to perform spectral estimation and extract features on the preprocessed data in each channel, the steps are as follows:

[0019] a. Extract PSD features:

[0020] PSD feature is the power spectral density, which describes the energy distribution of the signal at different frequencies, and is a tool for analyzing the spectral characteristics of the signal. The periodogram method is used to calculate the power spectral density, and the formula is as follows:

[0021]

[0022] In the formula, n represents the discrete time index, n = 0, 1, 2,..., N-1, N represents the total number of signal sampling points, x(n) represents the value of the signal at time index n, ω(n) represents the value of the window function at time index n, P xx represents the power spectral density PSD of the signal, f represents the frequency, is the core part of the Fourier transform, which is used to convert the signal from the time domain to the frequency domain;

[0023] b. Extract DE features:

[0024] DE feature is the differential entropy, which is used to measure the entropy of continuous random variables, reflecting the complexity and uncertainty of the signal. Its application in brain-computer interface is feature extraction to capture the dynamic changes of EEG signal. The steps to calculate DE feature are as follows:

[0025] i. Segmentation processing: segment the preprocessed electroencephalogram data according to the fixed length;

[0026] ii. Window function design: apply Hanning window to reduce the influence of spectral leakage;

[0027] iii. Calculate the power spectral density: apply the window function to the signal data in each time window, then perform fast Fourier transform calculation to obtain the power spectral density of the signal, the formula is as follows:

[0028] P xx = |FFT(section_data x window, NFFT)

[0029] In the formula, P xxPower Spectral Density (PSD) of the signal, which represents the energy distribution of the signal in the frequency domain; FFT represents Fast Fourier Transform, which is a method to convert time-domain signals to frequency-domain; section_data represents the signal data in the current time window, which is a fixed-length data segment extracted from the original signal for segmented processing of the signal; NFFT represents the length of FFT, which determines the frequency resolution of the FFT calculation; window represents the window function, which is used to reduce the impact of spectral leakage and make the signal boundary smoother, and the length of the window function is the same as section_data; the formula of the window function is:

[0030]

[0031] where n' is the sample point of the window function, and N' is the total length of the window function;

[0032] iv. Calculate the frequency band energy: calculate the energy in each frequency band by calculating the sum of the squares of the power spectral density, and the frequency bands are delta wave, theta wave, beta wave, alpha wave, and gamma wave, respectively, and the formula is as follows:

[0033]

[0034] where psd(i) represents the energy in the i-th frequency band, f∈band i represents that the frequency f belongs to the i-th frequency band, p xx (f) represents the power spectral density value at frequency f, which is obtained by performing fast Fourier transform on the signal;

[0035] 2) Calculate the brain network feature: calculate the phase locking value (PLV) of the electroencephalogram data as the brain network feature, which is used to estimate the instantaneous phase relationship between two neural electrical or biomagnetic signals, and the formula is as follows:

[0036]

[0037] where PLV t represents the phase locking value at time t; N" represents the number of trials, i.e. trial[1,2,....,N"] ; θ(t,n") represents the instantaneous phase difference between different leads in the same trial; exp(jθ(t,n")) represents the complex signal obtained by using the phase through Euler's formula; represents the superposition of all trial complex signals; represents the amplitude of the superimposed complex signal and the average.

[0038] Further, the feature fusion module fuses the frequency spectrum features based on brain region activation and the brain network features based on information interaction to construct a feature representation capable of comprehensively reflecting the attention state, referred to as fusion features, and finally four different features are used for experiments: fusion features, brain network features PLV, and two frequency spectrum features: PSD features and DE features; wherein, the PSD features and the DE features are represented as: x*y, x represents the number of samples, and y represents the feature dimension of each sample; the PLV is represented as: x*g, g represents the feature dimension of each sample, and the fusion features are represented as: x*(y+g), (y+g) represents the feature dimension of each sample after fusion.

[0039] Further, the feature selection module uses an improved adaptive particle swarm optimization algorithm to select the most suitable features for attention recognition, and the improvements of the improved adaptive particle swarm optimization algorithm include:

[0040] a. Dynamic adaptive neighborhood search is introduced:

[0041] In the adaptive particle swarm optimization algorithm PSO, each particle learns its own best position and the best position of the entire population. Here, the dynamic adaptive neighborhood search is introduced to enhance the local search ability of the particle and avoid falling into local optimum. The specific steps are as follows:

[0042] i. Define neighborhood: each particle dynamically adjusts its neighborhood range at each generation, rather than a fixed global optimum;

[0043] ii. Dynamic neighborhood adjustment: set a neighborhood radius R, adjust the radius according to the distance between the particle and other particles, set the initial neighborhood radius R0 to be large, and then gradually reduce it with the number of iterations, the formula is as follows:

[0044]

[0045] wherein, R(t') is the neighborhood radius of the current iteration t', and T is the total number of iterations;

[0046] iii. Neighborhood best position: calculate the best position lBest of each particle within its neighborhood radius i , the formula is as follows:

[0047]

[0048] wherein, is the particle set within the neighborhood of the particle at iteration t, j represents other particles in the particle swarm that are different from the particle , f(x j ) represents the objective function value of the particle j at position x j .

[0049] iv. Velocity update formula: Introducing the neighborhood best position of the velocity update formula:

[0050]

[0051] where ω is the inertia weight, used to balance the influence of the historical velocity and the current velocity on the position update; c1, c2, c3 and r1, r2, r3 are learning factors, denotes the historical best position of the particle , gBest denotes the historical best position of the entire population; denotes the current position of the particle at time t, denotes the updated velocity of the particle at time t+1, denotes the velocity of the particle at time t;

[0052] v. The position update formula of the particle is as follows:

[0053]

[0054] where t represents time, is the updated position according to ;

[0055] b. Introducing a dynamic mutation direction strategy:

[0056] In the adaptive particle swarm optimization algorithm PSO, mutation operations are often static, using a fixed mutation strategy in each iteration of a generation. However, when dealing with complex optimization problems, this static mutation strategy may not fully utilize the information in the population, resulting in limited search performance. Therefore, a new mutation strategy, dynamic mutation direction, is introduced, with the following specific steps:

[0057] i. Initialization: Initialize the position and velocity of the particle;

[0058] ii. Velocity and position update: Update the velocity and position according to the PSO formula:

[0059]

[0060]

[0061] iii. Dynamic mutation direction calculation: Dynamic mutation direction is calculated by the following formula:

[0062]

[0063] where β is a scaling factor, N''' is the number of particles, represents a particle the current position at time t;

[0064] iv. Mutation operation: in each iteration of the generation, with a set probability P m Mutate the particle, the mutation formula is:

[0065]

[0066] where γ is a mutation strength factor, used to control the amplitude of mutation; is the mutation direction calculated according to the dynamic distribution information of the particle group, which ensures that the mutation can be adaptively adjusted according to the population distribution;

[0067] v. Update global and local optimum: update the historical optimal position of each particle and the historical optimal position gBest of the entire population;

[0068] vi. Termination condition: when the termination condition is met, that is, when the maximum number of iterations or the convergence standard is reached, the optimal solution is output, that is, the feature most suitable for attention classification.

[0069] Further, the attention model training module uses five classifiers: random forest, decision tree, support vector machine, K nearest neighbor and Bayesian for classification, and the training set and test set of each user for testing are divided using five-fold cross-validation to train the classifier. The above five classifiers all have superior performance in different scenarios, so as to comprehensively evaluate the performance of the extracted and selected features in attention recognition.

[0070] Further, the attention model prediction module includes the following steps:

[0071] 1) Collect the electroencephalogram data of the user to be tested using the designed attention experiment paradigm module and amplify and filter, use the data loading module to receive the electroencephalogram data transmitted by the attention experiment paradigm module online, and use the data preprocessing module to process the electroencephalogram data;

[0072] 2) The electroencephalogram data processed in step 1) is input into the feature extraction module to extract spectral features and brain network features;

[0073] 3) Fuse the spectral features and brain network features extracted in step 2) to obtain fused features;

[0074] 4) Use the improved adaptive particle swarm algorithm to select the features suitable for attention recognition, and eliminate the redundant features, so as to improve the performance of subsequent data analysis;

[0075] 5) Input the selected features into the attention model training module to obtain the classification model that is most suitable for attention recognition, output the user's attention level status, and provide timely feedback to the user.

[0076] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0077] 1) By introducing dynamic adaptive neighborhood search on top of the adaptive particle swarm optimization (PSO) algorithm, particles can explore the local space more effectively and avoid getting trapped in local optima. This improves the algorithm's optimization capability in complex search spaces. Simultaneously, dynamically adjusting the neighborhood range allows the particle swarm to flexibly expand or shrink the search range at different stages, thus better balancing global and local search.

[0078] 2) Based on the adaptive particle swarm optimization (PSO) algorithm, a dynamic mutation direction is introduced, allowing the mutation operation to be dynamically adjusted according to the population state. This maintains population diversity across different iterations and avoids premature convergence. Furthermore, the dynamic mutation strategy can adjust the mutation direction based on the current optimization state, improving the algorithm's ability to handle complex optimization problems.

[0079] 3) Compared to most existing studies that use spectral features for experimental analysis, this invention introduces brain network features, fusing spectral features based on brain region activation with brain network features based on information interaction. This method can more comprehensively and accurately characterize attentional states and reduce information loss. The fused features can better reflect the multidimensional information of attention, improve the representativeness and robustness of the features, and thus enhance recognition performance.

[0080] 4) Feature selection combined with multiple classification models: By combining the fused features after feature selection with different classifiers, the classification model most suitable for attention recognition can be found, which can improve classification accuracy and stability, and has high practical value and promotion prospects. Attached Figure Description

[0081] Figure 1 This is a schematic diagram showing the relationship between the various modules of the system of the present invention.

[0082] Figure 2 This is a schematic diagram of a 32-channel electrode used to acquire brainwave signals in an EEG data acquisition module.

[0083] Figure 3 This is a schematic diagram of the attention experiment paradigm designed for the system of this invention. Detailed Implementation

[0084] The present invention will be further described below with reference to specific embodiments.

[0085] The embodiment provides an attention recognition system based on joint spectrum and brain network features, is an attention recognition system developed by using a matlab language and capable of running on a Windows device, relations among modules of the system are as shown in Figure 1 The system comprises the following modules.

[0086] An electroencephalogram data acquisition module is used for guiding a user to perform an experiment according to a designed attention experiment paradigm, acquiring attention electroencephalogram data of the user by using an electroencephalogram device, and reading the electroencephalogram data.

[0087] A data preprocessing module is used for performing a preprocessing operation on the electroencephalogram data read by the electroencephalogram data acquisition module (because the electroencephalogram data is easily affected by various artifacts such as electrooculogram and electromyogram during acquisition, therefore, relevant preprocessing is performed before data analysis), including filtering, artifact removal and sampling frequency adjustment.

[0088] A feature extraction module is used for performing feature extraction on the preprocessed data by using a spectrum feature extraction method, obtaining spectrum features, and using a phase-locked value as a brain network feature.

[0089] A feature fusion module is used for fusing the spectrum features and the brain network features, capturing information of different frequency bands and waveforms in the electroencephalogram signal, and obtaining fused features.

[0090] A feature selection module is used for performing feature selection on the brain network features and the fused features respectively by using an improved adaptive particle swarm algorithm, and obtaining features most suitable for attention classification.

[0091] An attention model training module is used for performing model training on the selected features by using five kinds of classifiers including a support vector machine, a random forest and a decision tree, a K nearest neighbor and a Bayesian, and obtaining a classification model most suitable for attention recognition.

[0092] An attention model prediction module is used for inputting electroencephalogram data of a user to be tested into the classification model most suitable for attention recognition obtained by the attention model training module, recognizing and judging an attention state of the user, and feeding back a result to the user.

[0093] Specifically, the electroencephalogram data acquisition module comprises an attention experiment paradigm module and a data loading module.

[0094] An evoked paradigm of the attention experiment paradigm module is as shown in Figure 3 The user is guided to perform an experiment in different states according to indications by using vision and hearing, and a 32-electrode electroencephalogram cap (an electrode map thereof is as shown in Figure 2The EEG data is collected, amplified and filtered, and then read. In this experiment, 85 subjects with healthy vision and cognitive ability were recruited from universities in Guangzhou. The average age of the subjects was 25.3±2.4 years, including 45 males and 40 females. The experimental process is as follows: the subjects sit in a quiet room, and each subject contains 20 experiments. Each experiment includes a 3-second prompt, a 60-second task, and a 10-second rest period. During the prompt, the subject is required to prepare to enter an attention or relaxation state (non-attention state) according to the screen prompt. The attention prompt is shown in FIG. 6(b), and the relaxation prompt is shown in FIG. 6(c). In the attention state, the subject is required to perform mental calculation according to the rhythm of the screen, while a random number (may be positive or negative) appears on the screen, which is constantly reduced by 3 over time, as shown in FIG. 6(d). In the relaxation state, a fixed plus sign appears on the screen, as shown in FIG. 6(e); at the same time, the subject is required to keep his eyes open and rest quietly. At the end of each experiment, the subject will have a 10-second rest period, called the rest period. Each experimental stage contains 10 attention states and 10 relaxation states (non-attention), and the order is randomized. Figure 3 Figure 3 Figure 3 Figure 3

[0095] The data loading module can read EEG data in local formats such as mat, cnt, and eeg, and can also receive EEG data transmitted by the attention experiment paradigm module online.

[0096] Specifically, the data preprocessing module is used to remove the influence of noise in the EEG signal and downsample the EEG signal to reduce data volume and computational complexity, and to segment the EEG signal into several time windows to facilitate subsequent module operation.

[0097] Specifically, the feature extraction module uses a spectral feature extraction method to extract features from the preprocessed data and calculates the phase locking value between each pair of channels as a brain network feature, including the following steps:

[0098] 1) Use the spectral feature extraction method to perform spectral estimation and extract features from the data in each channel after preprocessing, as follows:

[0099] a. Extract PSD features:

[0100] PSD features, i.e. power spectral density, describe the energy distribution of a signal at different frequencies, and are a tool for analyzing the spectral characteristics of a signal. The periodogram method is used to calculate the power spectral density, and the formula is as follows:

[0101]

[0102] ​​​​In the formula, n represents the discrete time index, n = 0, 1, 2,..., N-1, N represents the total number of sampling points of the signal, x(n) represents the value of the signal at time index n, ω(n) represents the value of the window function at time index n, P xx represents the power spectral density PSD of the signal, f represents the frequency, is the core part of the Fourier transform, used to convert the signal from the time domain to the frequency domain;

[0103] b. Extract DE features:

[0104] DE features, namely differential entropy, are used to measure the entropy of continuous random variables, reflecting the complexity and uncertainty of the signal. Its application in brain-computer interface is feature extraction to capture the dynamic changes of EEG signals. The steps for calculating DE features are as follows:

[0105] i. Segment processing: segment the preprocessed EEG data according to a fixed length;

[0106] ii. Window function design: apply the Hanning window to reduce the impact of spectral leakage;

[0107] iii. Calculate the power spectral density: apply the window function to the signal data in each time window, then perform fast Fourier transform calculation to obtain the power spectral density of the signal, the formula is as follows:

[0108] P xx = |FFT(section_data x window, NFFT)|

[0109] In the formula, P xx represents the power spectral density PSD of the signal, which represents the energy distribution of the signal in the frequency domain; FFT represents the fast Fourier transform, which is a method for converting time domain signals to frequency domain; section_data represents the signal data in the current time window, which is a fixed length of data extracted from the original signal for segmenting the signal; NFFT represents the length of FFT, which determines the frequency resolution of FFT calculation; window represents the window function, which is used to reduce the impact of spectral leakage and make the signal boundary smoother, the length of the window function is the same as section_data; the formula of the window function is:

[0110]

[0111] In the formula, n' is the sample point of the window function, and N' is the total length of the window function;

[0112] iv. Calculate the energy of the frequency band: calculate the sum of the squares of the power spectral density to obtain the energy in each frequency band, the frequency bands are delta wave, theta wave, beta wave, alpha wave and gamma wave, the formula is as follows:

[0113]

[0114] where psd(i) represents the energy in the ith frequency band, f ∈ band i denotes that the frequency f belongs to the ith frequency band, p xx (f) represents the power spectral density value at frequency f, obtained by performing a fast Fourier transform on the signal;

[0115] 2) Calculate the brain network feature: calculate the phase locking value (PLV) of the electroencephalogram data as the brain network feature, which is used to estimate the instantaneous phase relationship between two neuroelectric or biomagnetic signals, and the formula is as follows:

[0116]

[0117] where PLV t denotes the phase locking value at time t; N" represents the number of trials, i.e. trial[1, 2,..., N"]; θ(t, n") represents the instantaneous phase difference between different leads in the same trial; exp(jθ(t, n")) represents a complex signal obtained by using the phase through the Euler formula; denotes that the complex signals of all trials are superimposed; denotes the amplitude of the superimposed complex signal and the average.

[0118] Specifically, the feature fusion module fuses the frequency spectrum features based on brain activation and the brain network features based on information interaction to construct a feature representation that can comprehensively reflect the attention state, referred to as a fusion feature. Finally, four different features are used for experiments: the fusion feature, the brain network feature PLV, and two frequency spectrum features: the PSD feature and the DE feature. Among them, the PSD feature and the DE feature are represented as: x*y, x represents the number of samples, and y represents the feature dimension of each sample; PLV is represented as: x*g, g represents the feature dimension of each sample, and the fusion feature is represented as: x*(y+g), (y+g) represents the feature dimension of each sample after fusion.

[0119] Specifically, the feature selection module uses an improved adaptive particle swarm optimization (PSO) algorithm to select the most suitable features for attention recognition. This improved PSO algorithm has two main innovations: First, dynamic adaptive neighborhood search. Compared to traditional PSO algorithms, it dynamically adjusts the neighborhood range of each particle, making each particle more focused on its local search space, thus helping to avoid getting trapped in local optima. Furthermore, the dynamic adjustment of the neighborhood range can be flexibly adjusted according to the current search situation and problem characteristics, giving the algorithm better adaptability and robustness. Second, the introduction of dynamic mutation direction makes the mutation directions in each generation more diverse and adaptive, helping to increase the algorithm's global search capability and better explore the solution space. The specific steps of the dynamic adaptive neighborhood search and dynamic mutation direction strategies are as follows:

[0120] a. Introducing dynamic adaptive neighborhood search:

[0121] In the adaptive particle swarm optimization (PSO) algorithm, each particle learns its own optimal position and the optimal position of the entire population. Here, dynamic adaptive neighborhood search is introduced to enhance the local search ability of particles and avoid getting trapped in local optima. The specific steps are as follows:

[0122] i. Define neighborhood: Each particle dynamically adjusts its neighborhood range in each generation, rather than fixing the global optimum;

[0123] ii. Dynamic Neighborhood Adjustment: Set a neighborhood radius R, and adjust this radius according to the distance between the particle and other particles. Initially, the neighborhood radius R0 is set large, and then gradually decreases with the number of iterations, as shown in the following formula:

[0124]

[0125] In the formula, R(t′) is the neighborhood radius of the current iteration number t′, and T is the total number of iterations;

[0126] iii. Optimal neighborhood position: Calculate the optimal position for each particle. Best location within its radius i The formula is as follows:

[0127]

[0128] In the formula, It is a particle The set of particles in the neighborhood at iteration t, where j represents the particle swarm that is distinct from particle t. Other particles, f(x) j ) indicates that particle j is at position x j The objective function value;

[0129] iv. Velocity update formula: Introducing the neighborhood best position of the velocity update formula:

[0130]

[0131] where ω is the inertia weight, used to balance the influence of the historical velocity and the current velocity on the position update; c1, c2, c3 and r1, r2, r3 are learning factors, denotes the historical best position of the particle , gBest denotes the historical best position of the entire population; denotes the current position of the particle at time t, denotes the updated velocity of the particle at time t+1, denotes the velocity of the particle at time t;

[0132] v. The position update formula of the particle is as follows:

[0133]

[0134] where t represents time, is the updated position according to ;

[0135] b. Introducing a dynamic mutation direction strategy:

[0136] In the adaptive particle swarm optimization PSO, mutation operations are often static, and they use a fixed mutation strategy in each generation iteration. However, when dealing with complex optimization problems, this static mutation strategy may not be able to fully utilize the information in the population, resulting in limited search performance. Therefore, a new mutation strategy, dynamic mutation direction, is introduced, and the specific steps are as follows:

[0137] i. Initialization: Initialize the position and velocity of the particle;

[0138] ii. Velocity and position update: Update the velocity and position according to the PSO formula:

[0139]

[0140]

[0141] iii. Dynamic mutation direction calculation: Dynamic mutation direction is calculated by the following formula:

[0142]

[0143] where β is a scaling factor, N''' is the number of particles, representing particles current position at time t;

[0144] iv. Mutation operation: in each iteration of the generation, with a set probability P m Mutate the particles, the mutation formula is:

[0145]

[0146] where γ is a mutation strength factor, used to control the amplitude of mutation; is the mutation direction calculated according to the dynamic distribution information of the particle group, which ensures that the mutation can be adaptively adjusted according to the population distribution;

[0147] v. Update global and local optimum: update the historical optimal position of each particle and the historical optimal position gBest of the entire population;

[0148] vi. Termination condition: when the termination condition is met, that is, when the maximum number of iterations or the convergence criterion is reached, the optimal solution is output, that is, the most suitable feature for attention classification.

[0149] Specifically, the attention model training module uses five classifiers: random forest, decision tree, support vector machine, K nearest neighbor and Bayesian for classification, and the training set and test set of each user for testing are divided using five-fold cross-validation to train the classifier. The above five classifiers all have superior performance in different scenarios, so as to comprehensively evaluate the performance of the extracted and selected features in attention recognition.

[0150] Specifically, the attention model prediction module includes the following steps:

[0151] 1) Use the designed attention experiment paradigm module to collect the electroencephalogram data of the user to be tested and amplify and filter, use the data loading module to receive the electroencephalogram data transmitted by the attention experiment paradigm module online, and use the data preprocessing module to process the electroencephalogram data;

[0152] 2) The electroencephalogram data processed in step 1) is input into the feature extraction module to extract spectral features and brain network features;

[0153] 3) The spectral features and brain network features extracted in step 2) are fused to obtain fused features;

[0154] 4) Use the improved adaptive particle swarm algorithm to select the features suitable for attention recognition, and eliminate the redundant features, so as to improve the performance of subsequent data analysis;

[0155] 5) The selected features are input into the attention model training module to obtain a classification model most suitable for attention recognition, and the attention level state of the user is output, and feedback is given to the user in a timely manner.

[0156] The above embodiments are preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications, etc. made without departing from the spirit and principles of the present application should be equivalent replacement methods and are included in the protection scope of the present application.

Claims

1. An attention recognition system based on joint spectrum and brain network features, characterized in that, include: The EEG data acquisition module guides users to conduct experiments according to the designed attention experiment paradigm, uses EEG devices to collect users' attention EEG data, and reads the EEG data. The data preprocessing module is used to perform preprocessing operations on the EEG data read by the EEG data acquisition module, including filtering, artifact removal, and sampling frequency adjustment. The feature extraction module uses a spectral feature extraction method to extract features from the preprocessed data, obtains spectral features, and uses phase-locked values ​​as brain network features. The feature fusion module is used to fuse spectral features and brain network features to capture information from different frequency bands and waveforms in EEG data and obtain fused features. The feature selection module uses an improved adaptive particle swarm optimization algorithm to select features from brain network features and fused features respectively, to obtain the features most suitable for attention classification; the improved adaptive particle swarm optimization algorithm introduces dynamic adaptive neighborhood search and dynamic mutation direction strategies. The attention model training module uses five classifiers—support vector machine, random forest, decision tree, K-nearest neighbors, and Bayesian—to train the model on selected features and obtain the classification model most suitable for attention recognition. The attention model prediction module inputs the EEG data of the user to be tested into the classification model most suitable for attention recognition obtained by the attention model training module, identifies and judges the user's attention state, and feeds back the results to the user.

2. The attention recognition system based on joint spectrum and brain network features according to claim 1, characterized in that, The EEG data acquisition module includes an attention experiment paradigm module and a data loading module, wherein: The attention experiment paradigm module uses visual and auditory guidance to guide users into different states according to the instructions on the computer screen. During this process, EEG data is collected, amplified, and filtered using an EEG acquisition device, and then read. The data loading module is used to read EEG data from local storage, including MAT, CNT, and EEG formats, and to receive EEG data transmitted online from the attention experiment paradigm module.

3. The attention recognition system based on joint spectrum and brain network features according to claim 2, characterized in that, The data preprocessing module is used to remove noise from the EEG data and downsample the EEG data to reduce the amount of data and computational complexity. At the same time, the EEG data is divided into several time windows to facilitate subsequent module operations.

4. The attention recognition system based on joint spectrum and brain network features according to claim 3, characterized in that, The feature extraction module uses spectral feature extraction methods to extract features from the preprocessed data and calculates the phase-locked values ​​between pairs of channels as brain network features, including the following steps: 1) Use spectral feature extraction methods to estimate the spectrum and extract features from the preprocessed data in each channel. The steps are as follows: a. Extract PSD features: PSD, or Power Spectral Density, describes the energy distribution of a signal at different frequencies. It is a tool used to analyze the spectral characteristics of a signal. The Power Spectral Density is calculated using the periodogram method, as shown in the following formula: In the formula, n represents the discrete-time index, n = 0, 1, 2, ..., N-1, N represents the total number of sampling points of the signal, x(n) represents the value of the signal at time index n, ω(n) represents the value of the window function at time index n, and P xx The power spectral density (PSD) of the signal is represented by f, where f represents the frequency. It is the core part of the Fourier transform, used to convert signals from the time domain to the frequency domain; b. Extract DE features: DE features, or differential entropy, are used to measure the entropy of a continuous random variable, reflecting the complexity and uncertainty of a signal. In brain-computer interfaces, their application is feature extraction to capture the dynamic changes in EEG signals. The steps for calculating DE features are as follows: i. Segmentation processing: The preprocessed EEG data is segmented into segments of fixed length; ii. Window function design: Apply the Hanning window to reduce the impact of spectral leakage; iii. Calculate the power spectral density: Apply a window function to the signal data within each time window, and then perform a fast Fourier transform to calculate the power spectral density of the signal, as shown in the following formula: P xx =|FFT(section_data×window,NFFT)| In the formula, P xx The power spectral density (PSD) represents the signal's energy distribution in the frequency domain; FFT stands for Fast Fourier Transform, a method for converting time-domain signals to the frequency domain; section_data represents the signal data within the current time window, a fixed-length data segment extracted from the original signal for segmented signal processing; NFFT represents the length of the FFT, which determines the frequency resolution of the FFT calculation; window represents the window function, used to reduce spectral leakage and smooth signal boundaries; the window function has the same length as section_data; the formula for the window function is: In the formula, n′ represents the sample points of the window function, and N′ represents the total length of the window function; iv. Calculate the energy of each frequency band: The energy in each band is obtained by calculating the sum of squares of the power spectral density. The frequency bands are: delta wave, theta wave, beta wave, alpha wave, and gamma wave. The formula is as follows: In the formula, psd(i) represents the energy in the i-th frequency band, f∈band i This indicates that frequency f belongs to the i-th frequency band, p xx (f) represents the power spectral density value at frequency f, which is obtained by performing a fast Fourier transform on the signal; 2) Calculate brain network features: Calculate the phase lock value (PLV) of the EEG data as a brain network feature. PLV is used to estimate the instantaneous phase relationship between two neural electrical or biomagnetic signals. The formula is as follows: In the formula, PLV t denoted by t; N″ represents the number of trials, i.e., trial[1,2,....,N″]; θ(t,n″) represents the instantaneous phase difference between identical trials in different leads; exp(jθ(t,n″)) represents obtaining complex signals using the phase through Euler's formula; This indicates that the complex signals of all trials are superimposed; This indicates obtaining the amplitude of the superimposed complex signal and calculating its average.

5. The attention recognition system based on joint spectrum and brain network features according to claim 4, characterized in that, The feature fusion module fuses spectral features based on brain region activation and brain network features based on information interaction to construct a feature representation that can comprehensively reflect the attention state, called the fusion feature. Finally, four different features were tested: the fusion feature, the brain network feature PLV, and two spectral features: PSD feature and DE feature. The PSD feature and DE feature are both represented as x*y, where x represents the number of samples and y represents the feature dimension of each sample; PLV is represented as x*g, where g represents the feature dimension of each sample; and the fusion feature is represented as x*(y+g), where (y+g) represents the feature dimension of each sample after fusion.

6. The attention recognition system based on joint spectrum and brain network features according to claim 5, characterized in that, The feature selection module uses an improved adaptive particle swarm optimization algorithm to select the features most suitable for attention recognition. The improvements to this algorithm include: a. Introducing dynamic adaptive neighborhood search: In the adaptive particle swarm optimization (PSO) algorithm, each particle learns its own optimal position and the optimal position of the entire population. Here, dynamic adaptive neighborhood search is introduced to enhance the local search ability of particles and avoid getting trapped in local optima. The specific steps are as follows: i. Define neighborhood: Each particle dynamically adjusts its neighborhood range in each generation, rather than fixing the global optimum; ii. Dynamic Neighborhood Adjustment: Set a neighborhood radius R, and adjust this radius according to the distance between the particle and other particles. Initially, the neighborhood radius R0 is set large, and then gradually decreases with the number of iterations, as shown in the following formula: In the formula, R(t′) is the neighborhood radius of the current iteration number t′, and T is the total number of iterations; iii. Optimal neighborhood position: Calculate the optimal position for each particle. Best location within its radius i The formula is as follows: In the formula, It is a particle The set of particles in the neighborhood at iteration t, where j represents the particle swarm that is distinct from particle t. Other particles, f(x) j ) indicates that particle j is at position x j The objective function value; iv. Speed ​​update formula: Introducing the optimal neighborhood position Speed ​​update formula: In the formula, ω is the inertial weight, used to balance the influence of the particle's historical velocity and current velocity on position updates; c1, c2, c3 and r1, r2, r3 are learning factors. Represents particles The historical best position, gBest represents the historical best position of the entire population; Represents particles At the current position at time t, Represents particles The speed after the update at time t+1, Represents particles The velocity at time t; v. The particle position update formula is as follows: In the formula, t represents time. It is based on Updated location; b. Introduce a dynamic mutation direction strategy: In the adaptive particle swarm optimization (PSO) algorithm, mutation operations are often static, using a fixed mutation strategy in each iteration. However, when dealing with complex optimization problems, this static mutation strategy may not fully utilize the information in the population, leading to limited search performance. Therefore, a novel mutation strategy is introduced here: dynamic mutation direction, with the following specific steps: i. Initialization: Initialize the position and velocity of the particles; ii. Velocity and Position Update: Update velocity and position according to the PSO formula: iii. Calculation of dynamic mutation direction: Dynamic mutation direction Calculated using the following formula: In the formula, β is a scaling factor, and N″′ is the number of particles. Represents particles At the current position at time t; iv. Mutation operation: In each iteration, a mutation operation is performed with a set probability P. m The particle is mutated using the following formula: In the formula, γ is a variation intensity factor used to control the magnitude of the variation; The mutation direction is calculated based on the dynamic distribution information of the particle swarm, ensuring that the mutation can be adaptively adjusted according to the population distribution. v. Update global and local optima: Update the historical best position of each particle. and the historical best position of the entire population, gBest; vi. Termination condition: When the termination condition is met, that is, when the maximum number of iterations or the convergence criterion is reached, the optimal solution is output, that is, the feature most suitable for attention classification.

7. The attention recognition system based on joint spectrum and brain network features according to claim 6, characterized in that, The attention model training module uses five classifiers: random forest, decision tree, support vector machine, K-nearest neighbors, and Bayesian for classification. The training set and test set for each user's experiment are divided using five-fold cross-validation to train the classifier. All five classifiers have the characteristic of superior performance in different scenarios, thus enabling a comprehensive evaluation of the performance of extracted and selected features in attention recognition.

8. The attention recognition system based on joint spectrum and brain network features according to claim 7, characterized in that, The attention model prediction module includes the following steps: 1) Use the designed attention experiment paradigm module to collect EEG data from the test users and amplify and filter it; use the data loading module to receive the EEG data transmitted by the attention experiment paradigm module online; and use the data preprocessing module to process the EEG data. 2) The EEG data processed in step 1) is input into the feature extraction module to extract spectral features and brain network features; 3) The spectral features and brain network features extracted in step 2) are fused to obtain the fused features; 4) An improved adaptive particle swarm optimization algorithm is used to select features from brain network features and fusion features respectively, select features suitable for attention recognition, and remove redundant features to improve the performance of subsequent data analysis. 5) Input the selected features into the attention model training module to obtain the classification model that is most suitable for attention recognition, output the user's attention level status, and provide timely feedback to the user.

Images