Real-time artifact processing and feature extraction method and system for electroencephalogram signal

Abstract

Provided in the present application are a real-time artifact processing and feature extraction method and system for an electroencephalogram signal. The method comprises: receiving in real time an electroencephalogram signal data stream collected by means of an electroencephalogram device; performing real-time segmentation on the electroencephalogram signal data stream by means of a sliding window; acquiring parameter information including the number of channels and the sampling rate of the electroencephalogram device, and on the basis of the parameter information, performing a suitable filtering processing step and artifact removal step on electroencephalogram signal data stream segments; and matching a feature extraction strategy for one or more pre-selected output indexes, and according to a matched feature extraction strategy, performing, from perspectives including a time domain, a frequency domain, a time frequency domain and / or nonlinear analysis, real-time extraction on the electroencephalogram signal data stream segments which have been subjected to the filtering processing step and the artifact removal step, so as to obtain a feature value meeting the output index. In the present application, corresponding feature values can be extracted by means of integration and automation for different types of electroencephalogram devices and different output indexes, thereby facilitating subsequent analysis processing.

Description

A method and system for real-time artifact processing and feature extraction of EEG signals

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the Patent Office of China on December 21, 2023, with application number 202311775102.6 and invention name “A method and system for real-time artifact processing and feature extraction of EEG signals”, the Chinese patent application filed with the Patent Office of China on December 28, 2023, with application number 202311844300.3 and invention name “EEG signal processing method and device based on human intelligence”, and the Chinese patent application filed with the Patent Office of China on December 27, 2023, with application number 202311823752.3 and invention name “EEG signal correction method and device based on human intelligence”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present invention relates to the technical field of electroencephalogram (EEG) signal processing, and in particular to a method and system for real-time artifact processing and feature extraction of EEG signals. Background Art

[0004] EEG signals capture electrical activity in the cerebral cortex with millisecond-level temporal resolution, enabling the extraction of a rich variety of feature values ​​and high responsiveness. Furthermore, with the advancement of sensor technology, the types of portable EEG devices have become increasingly diverse, with widespread applications in fields such as biofeedback, emotion detection, state recognition, and brain-computer interfaces. EEG signals originate from the synchronized firing of postsynaptic membranes across a large number of neurons in the cerebral cortex. Depending on whether or not they are receiving external stimuli, they can be divided into evoked EEG and spontaneous EEG. The former, induced by external stimuli, exhibits significant time-locked and phase-locked properties. By superimposing them, event-related potentials (ERPs) can be generated to reflect cognitive processing in the brain. Spontaneous EEG has a specific physiological rhythm and can be divided into five frequency bands: delta, theta, alpha, beta, and gamma. Each frequency band has a specific scalp distribution and physiological significance. It is modulated by stimuli, resulting in increases and decreases in energy within that specific frequency band, reflecting an individual's physical and mental state.

[0005] However, EEG data is significantly affected by noise and artifacts, and its eigenvalues ​​are complex and diverse, reflecting diverse aspects of the EEG signal. Therefore, correctly analyzing the data and extracting valid eigenvalues ​​are crucial for application sensitivity. This requires researchers to have extensive EEG data processing experience and a strong code base, which hinders the widespread application of EEG devices across multiple fields. Furthermore, existing applications often rely on a single type of eigenvalue and lack the ability to integrate data from multiple dimensions. This results in a lack of information representation of brain activity, reduced data volume, and reduced accuracy in applications such as emotion detection, state recognition, and brain-computer interfaces. EEG signals are highly random and have weak amplitudes, making them easily contaminated by irrelevant noise, resulting in various artifacts such as electrooculography, electromyography, sweat, and mains interference. Therefore, EEG signals recorded directly from scalp electrodes often fail to accurately represent brain neural signals. Preprocessing and noise reduction of the raw EEG data are necessary to minimize or eliminate the impact of these artifacts. Artifacts in EEG signals can be broadly divided into two categories: physiological and non-physiological. Physiological artifacts are often caused by movements of body parts close to the head. The most common are electrophysiological signals generated by blinking, eye movement, tongue movement, heartbeat, breathing, muscle movement, and sweat gland activation. For example, electrooculography (EOG) is caused by the potential difference between the dipoles between the cornea and retina shifting during eye movement. This potential difference alters the electric field around the eye, thereby affecting the scalp field. Electrocardiographic artifacts are often caused by using the ipsilateral ear, which is farther away from the electrode, as a reference. Frontalis muscle artifacts are primarily caused by forceful eye closure. Non-physiological artifacts often arise from external environmental interference, most commonly mains interference. They can also occur when the electrode contacts are poorly contacted with the scalp or when the EEG recording system malfunctions. Movement of the electrodes can disrupt the double layer, generating direct current. Loose wires or circuit boards are also significant causes of non-physiological artifacts, potentially leading to partial signal loss and intermittent failures.

[0006] In the existing technology, there is a lack of technical solutions for real-time processing of EEG signals. At the same time, there is a lack of integrated and automated solutions for EEG signals from different EEG devices. Summary of the Invention

[0007] In order to solve the above problems in the prior art, the present application proposes a method and system for real-time artifact processing and feature extraction of EEG signals to eliminate or improve one or more defects in the prior art.

[0008] One aspect of the present application provides a method for real-time artifact processing and feature extraction of EEG signals, the method comprising the following steps:

[0009] Receive the EEG signal data stream collected by EEG equipment in real time;

[0010] The sliding window method is used to segment the EEG signal data stream in real time;

[0011] Obtaining parameter information of the EEG device, including the number of channels and sampling rate, and performing appropriate filtering and artifact removal steps on the EEG signal data stream segments based on the parameter information;

[0012] For one or more pre-selected output indicator matching feature extraction strategies, characteristic values ​​that meet the output indicators are extracted in real time from the EEG signal data stream segments that have undergone filtering processing steps and artifact removal steps according to the matching feature extraction strategies from perspectives including time domain, frequency domain, time-frequency domain and / or nonlinear analysis.

[0013] Another aspect of the present application provides a system for real-time artifact processing and feature extraction of EEG signals, the system comprising:

[0014] A user configuration module is used to receive one or more pre-selected output indicator matching feature extraction strategies and set output indicators, and obtain parameter information of the EEG device including the number of channels and sampling rate;

[0015] A built-in processing module is used to perform a filtering process and an artifact removal process on the EEG signal data stream segments based on the parameter information, match a feature extraction strategy to one or more pre-selected output indicators, and extract feature values ​​that meet the output indicators in real time from the EEG signal data stream segments that have undergone the filtering process and the artifact removal process according to the matched feature extraction strategy from perspectives including time domain, frequency domain, time-frequency domain and / or nonlinear analysis;

[0016] The data transmission module is used to receive the EEG signal data stream collected by the EEG device in real time, and transmit the characteristic values ​​​​that meet the output indicators extracted in real time.

[0017] Another aspect of the present application provides a device for real-time artifact processing and feature extraction of EEG signals, comprising a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the method described in any one of the above embodiments.

[0018] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in any one of the above embodiments when the program is executed by a processor.

[0019] The method and system for real-time artifact processing and feature extraction of EEG signals proposed in this application can perform adaptive filtering and artifact removal on the EEG signal data stream based on parameter information such as the number of channels and sampling rate contained in the EEG device, and extract feature values ​​that meet the output indicators in real time based on the pre-selected output indicator matching feature extraction strategy. On the one hand, it can perform real-time artifact removal and feature extraction on EEG signals, and on the other hand, it can extract corresponding feature values ​​for different types of EEG devices and different output indicators in an integrated and automated manner, facilitating subsequent analysis and processing of EEG data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. In the drawings:

[0021] FIG1 is a flow chart of a method for real-time artifact processing and feature extraction in one embodiment of the present application;

[0022] FIG2 is a flowchart of a system workflow for real-time artifact processing and feature extraction according to an embodiment of the present application;

[0023] FIG3 is a schematic diagram of a system built-in algorithm module for real-time artifact processing and feature extraction in one embodiment of the present application;

[0024] FIG4 is a flow chart of a method for removing artifacts using the FastICA+GFP combination technology in one embodiment of the present application;

[0025] FIG5 is a schematic diagram of the structure of a convolutional neural network model in an embodiment of the present application;

[0026] FIG6 is a flowchart of a method for processing EEG signals based on human intelligence in an embodiment of the present application;

[0027] FIG7 is a schematic diagram of a specific architecture of a convolutional neural network model in an embodiment of the present application;

[0028] FIG8 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail in conjunction with the embodiments and drawings. Here, the illustrative embodiments of this application and their descriptions are used to explain this application, but are not intended to limit this application.

[0030] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, the accompanying drawings only show structures and / or processing steps that are closely related to the scheme according to the present application, while other details that are not closely related to the present application are omitted.

[0031] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0032] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0033] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0034] Example 1

[0035] In order to solve the problems existing in the prior art for real-time artifact processing and feature extraction methods of EEG signals, the present application provides a method for real-time artifact processing and feature extraction of EEG signals.

[0036] Problems with existing technologies include: (1) Inability to perform real-time data analysis: Existing EEG data analysis software such as EEGLAB and MNE cannot receive and process data in real time. They can only perform post-analysis of EEG data, usually in units of several minutes, and cannot output data of several seconds. This makes it difficult to meet the application requirements of real-time state recognition, emotion detection and other fields. (2) Difficulty in data analysis and export: The structure of EEG data is complex and multidimensional. When extracting features, the extraction of each type of feature requires a large amount of code base, and it is difficult to export feature values ​​for each subject and channel, making data application extremely inconvenient. (3) Low degree of automation: Existing artifact removal methods mostly use independent component analysis (ICA). On the one hand, this method requires a high level of experience from the researcher and requires the researcher to be able to identify and remove artifact components such as electrooculograms. It cannot be automated and is time-consuming and labor-intensive. (4) Unable to meet the processing requirements of diverse devices: When performing ICA processing, the number of EEG channels must be sufficient and the duration of EEG data must be long enough. Therefore, it is impossible to remove artifacts from EEG data with a single channel or fewer channels, nor can it be performed on data segments of several milliseconds. Existing research on automatic artifact detection and removal can only be performed on multiple channels, while others can only be performed on a single channel, lacking an integrated method. (5) High application development requirements: When the analyzed eigenvalues ​​are applied in various fields, it is necessary to develop real-time APIs or TCP / UDP and other data transmission functions, which makes the application complex.

[0037] FIG1 is a flow chart of a method for real-time artifact processing and feature extraction according to an embodiment of the present application. The method comprises the following steps:

[0038] Step S110: receiving the EEG signal data stream collected by the EEG device in real time.

[0039] During the specific implementation process, the system for real-time artifact processing and feature extraction of EEG signals supports multiple protocol transmission methods such as Wireless Blue and LSL (Lab streaming layer). It can receive data streams from EEG devices in real time, read and cache real-time collected data according to user configuration, and use an adaptive sliding window paradigm to perform real-time noise reduction, artifact removal and feature extraction.

[0040] In practice, an EEG device is used to record brain activity and includes electrodes placed on the scalp, an amplifier for amplifying the weak signals captured by the electrodes to a level that can be processed by a computer, and a computer for converting the amplifier's output signal into a graphical or digital format. Furthermore, the collected EEG signal data stream contains a preset number of channels, and EEG signal data streams from different channels can be distinguished using the electrode placement area as a classification label.

[0041] Step S120: Segment the EEG signal data stream in real time using a sliding window approach.

[0042] In practice, a sliding window approach is used to segment the EEG signal data stream in real time. This involves dividing the EEG signal data stream into multiple continuous, fixed-length segments, each of which is called a window. First, the EEG signal data stream is arranged in chronological order. Then, the window length is set. Starting from the beginning, segments of the EEG signal data stream with a continuous window length are sequentially extracted, forming a window. This step is repeated until the EEG signal data stream is completely segmented into multiple windows.

[0043] Step S130: Obtain parameter information of the EEG device including the number of channels and sampling rate, and perform appropriate filtering and artifact removal steps on the EEG signal data stream segments based on the parameter information.

[0044] During implementation, the system investigated EEG devices used in various scenarios, including standard laboratories, biofeedback, emotional brain-computer interfaces, and medical brain-computer interfaces. It designed personalized filtering solutions for high-precision, high-resolution, and high-density gel electrode EEG devices, portable wearable high-density saline (semi-dry) electrode EEG devices, and portable wearable dry electrode EEG devices of different forms (different numbers of channels). The system combined traditional frequency-domain filtering and wavelet noise reduction with innovative hybrid methods such as deep learning, blind source separation, and machine learning to achieve fully automated signal noise reduction and artifact removal. Users only need to perform simple configuration, and the entire artifact removal process is automatically completed without the need for extensive code writing.

[0045] The filtering step includes one or more of low-pass filtering, high-pass filtering, band-pass filtering, and notch filtering. The artifact removal step includes one or more of independent component analysis (FastICA), grid-based natural language structure analysis (GPF), blind source signal separation, and support vector machine (SVM) artifact removal.

[0046] Step S140: For one or more pre-selected output indicator matching feature extraction strategies, characteristic values ​​that meet the output indicators are extracted in real time from the EEG signal data stream segments that have undergone the filtering processing step and the artifact removal step according to the matching feature extraction strategies from perspectives including time domain, frequency domain, time-frequency domain and / or nonlinear analysis.

[0047] In practice, the system integrates EEG signal feature extraction algorithms from multiple sources, including time-domain, frequency-domain, time-frequency-domain, nonlinear, and brain functional connectivity eigenvalue calculation algorithms, resulting in diverse eigenvalues ​​and high sensitivity. The system also receives user-configured output metrics, feature extraction strategies, and filtering steps to adapt to different scenarios.

[0048] Among them, the supported protocols for transmitting the EEG signal data stream collected by the EEG device and the extracted eigenvalues ​​include one or more of the TCP protocol, UDP protocol, wireless Bluetooth protocol, MQTT protocol, RS-232 / RS-485 protocol and LSL protocol.

[0049] The method and system for real-time artifact processing and feature extraction of EEG signals proposed in this application can perform adaptive filtering and artifact removal on the EEG signal data stream based on parameter information such as the number of channels and sampling rate contained in the EEG device, and extract feature values ​​that meet the output indicators in real time based on the pre-selected output indicator matching feature extraction strategy. On the one hand, it can perform real-time artifact removal and feature extraction on EEG signals, and on the other hand, it can extract corresponding feature values ​​for different types of EEG devices and different output indicators in an integrated and automated manner, facilitating subsequent analysis and processing of EEG data.

[0050] In some embodiments of the present application, before receiving the EEG signal data stream and performing artifact processing and feature extraction, the method further includes: obtaining device information of the EEG device of each input EEG signal data stream, wherein the device information includes the number of channels and sampling rate.

[0051] Furthermore, in a specific embodiment of the present application, the method also includes a step of obtaining user configuration for real-time artifact processing and feature extraction, wherein the user configuration includes the time window size of the sliding window, the filtering processing strategy, the artifact removal strategy, the eigenvalue output index, and the eigenvalue transmission support protocol selection.

[0052] In some embodiments of the present application, in scenarios where EEG signals are used to identify anxiety states, the energy values ​​of the alpha, theta, and gamma frequency bands are used as output indicators, and the artifact removal step includes independent component analysis and grid-based natural language structure analysis. In scenarios where EEG signals are collected to control prosthetic limbs, characteristic values ​​of event-related synchronization and desynchronization output indicators for the μ and β rhythms are extracted in real time from EEG signal data stream segments that have undergone filtering and artifact removal steps, using a matching feature extraction strategy.

[0053] In some embodiments of the present application, the step of extracting in real time from the EEG signal data stream segments that have undergone the filtering processing step and the artifact removal step the characteristic values ​​that meet the output indicators in accordance with the matching feature extraction strategy in the perspectives including time domain, frequency domain, time-frequency domain and / or nonlinear analysis includes: (1) in the perspective of time domain analysis, extracting in real time one or more of the mean, variance, standard deviation, kurtosis, skewness and autocorrelation coefficient from the EEG signal data stream segments that have undergone the filtering processing step and the artifact removal step based on a statistical algorithm or a Hjorth algorithm; (2) in the perspective of frequency domain analysis, extracting in real time one or more of the mean, variance, standard deviation, kurtosis, skewness and autocorrelation coefficient based on a fast Fourier transform algorithm, a periodicity algorithm and a time domain algorithm; Any one of the graph method, Welch method, multi-window method and autoregressive model is used to extract energy values ​​and / or power values ​​in real time from the EEG signal data stream segments that have undergone the filtering processing step and the artifact removal step; (3) from the perspective of time-frequency domain analysis, based on the short-time Fourier transform method or the continuous wavelet transform method, characteristic values ​​are extracted in real time from the EEG signal data stream segments that have undergone the filtering processing step and the artifact removal step; (4) from the perspective of nonlinear analysis, based on recursive variable analysis and complexity, one or more of Shannon entropy, approximate entropy, sample entropy and sorting entropy are extracted in real time from the EEG signal data stream segments that have undergone the filtering processing step and the artifact removal step.

[0054] By adopting the embodiment of this application, a variety of feature extraction methods can be integrated to extract corresponding feature values ​​according to the EEG feature extraction requirements in different scenarios.

[0055] In some embodiments of the present application, the steps of performing adaptive filtering and artifact removal on the EEG signal data stream segments based on parameter information include: when the number of EEG signal channels collected by the EEG device is less than a preset threshold, using a pre-trained neural network model to perform adaptive filtering and artifact removal on the EEG signal data stream segments, wherein the neural network model is obtained by supervised learning training using a large amount of EEG signals containing different types of artifacts and / or different signal-to-noise ratios. For example, the channel number threshold can be any one of 3, 5, or 10, and the target result is obtained based on the extracted feature values ​​using the pre-trained model.

[0056] By adopting the embodiment of the application, training can be performed on different EEG signals based on historical data, so that artifact removal and feature extraction of EEG signals are more representative, and purer EEG data can be obtained.

[0057] Another aspect of the present application provides a system for real-time artifact processing and feature extraction of EEG signals, which includes: (1) a user configuration module for receiving one or more pre-selected output indicator matching feature extraction strategies and set output indicators, and obtaining parameter information of the EEG device including the number of channels and sampling rate; (2) a built-in processing module for performing corresponding filtering processing steps and artifact removal steps on the EEG signal data stream segments based on the parameter information, and for one or more pre-selected output indicator matching feature extraction strategies, extracting in real time from the EEG signal data stream segments that have undergone the filtering processing steps and artifact removal steps characteristic values ​​that meet the output indicators from the matched feature extraction strategies in the time domain, frequency domain, time-frequency domain and / or nonlinear analysis; (3) a data transmission module for receiving in real time the EEG signal data stream collected by the EEG device, and transmitting the characteristic values ​​that meet the output indicators extracted in real time.

[0058] The system for real-time EEG artifact processing and feature extraction proposed in this application is based on an integrated EEG data analysis platform, ErgoLAB. It aims to address many of the difficulties and shortcomings in existing EEG data analysis, making real-time EEG data processing and feature extraction more efficient, automated, and convenient. ErgoLAB refers to the ErgoLAB human-machine-environment synchronization cloud platform. Based on cloud architecture and synchronization technology, this cloud platform specializes in "human-centric" multimodal data synchronization and quantitative analysis, highlighting its intelligent and wearable features. Combining VR virtual reality and simulation technologies, light environment simulation technologies, brain cognitive neuroscience and electrophysiology technologies, visual tracking technologies, motion capture technologies, behavioral analysis technologies, facial expression and state recognition technologies, etc., it objectively and quantitatively analyzes the interaction between human, machine, and environment, enhancing the depth of longitudinal research and the extension of horizontal research. Compatible with many scientific research equipment in the fields of cognitive neuroscience, human factors engineering, and artificial intelligence, it provides complete solutions in fields such as human factors engineering and ergonomics, architectural and environmental behavior, human-computer interaction and artificial intelligence, psychology and cognitive science, and traffic driving behavior research.

[0059] The system mainly consists of three parts: user configuration module, built-in algorithm module and data transmission module. Among them, (1) the user configuration module allows users to select the noise reduction method, artifact removal algorithm, eigenvalue type and data transmission method according to their needs. They can also input the sampling rate, number of channels and other information of the device used. The system automatically recommends the algorithm (information such as the number of channels and sampling rate affects the artifact removal algorithm and the type of eigenvalues ​​that can be extracted) to achieve personalized data processing settings. (2) The built-in algorithm module is the core part of this toolbox. First, it integrates a variety of classic and advanced noise and artifact removal methods, such as filtering, wavelet denoising, FastICA+GFP, SOBI+SVM, as well as complex neural networks and convolutional neural networks, aiming to automatically remove artifacts such as electrooculogram. Secondly, the built-in algorithm module also contains various eigenvalue extraction algorithms, covering multi-dimensional eigenvalue calculations such as time domain, frequency domain, time-frequency and nonlinearity, making the feature extraction of EEG data simple and efficient. At the same time, the built-in algorithm module also designs different feature fusion schemes for different combinations of eigenvalues, such as using CNN to fuse EEG features in the spatial domain and EEG features in the frequency domain. (3) The data transmission module provides multiple transmission methods, including TCP transmission, UDP transmission and API transmission, providing convenient output and application interfaces for the analyzed eigenvalues. Appropriate filtering can effectively reduce the noise in EEG data. Common filtering methods include low-pass filtering, high-pass filtering, band-pass filtering and notch filtering. Low (high) pass filters allow signals lower (higher) than a certain cutoff frequency to pass through, and weaken or eliminate signals higher (lower) than this frequency. Band-pass filters only allow specific frequencies to pass through, while effectively suppressing signals of other frequencies. Notch filters weaken or eliminate signal components of a specific frequency. Notch filters weaken or eliminate signal components of a specific frequency. Filters are intuitive and easy to understand, and can be processed quickly and in real time, but they cannot fully adapt to complex signal structures and spectral characteristics. For example, they cannot remove artifacts such as electrooculograms that are confused in real signals. Wavelet denoising is a commonly used signal denoising method based on the principle of wavelet transform. It reduces the noise component in the signal by decomposing the signal into frequency bands of different scales and performing noise estimation and elimination based on the statistical characteristics of the signal. Wavelet denoising can simultaneously capture local and global signal characteristics, adaptively select thresholds for noise estimation and elimination, and effectively preserve signal detail. However, its computational complexity is high, and the selection of wavelet basis functions and thresholds significantly impacts the noise removal effect. Empirical Mode Decomposition (EMD) is an adaptive signal decomposition method that decomposes nonlinear and non-stationary signals into a set of intrinsic mode functions (IMFs). This method constructs local extreme points of the signal, interpolates to obtain upper and lower envelopes, calculates the local mean, and subtracts the local mean from the signal to obtain the detail curve.This process is repeated until the detail curve meets the IMF conditions, that is, the number of maximum and minimum points on the curve is equal or differs by at most one. After the iterative process, the final IMF is added to approximately reconstruct the original signal. Empirical mode decomposition has the advantages of adaptability, multi-scale decomposition and no need for preset parameters, but it is highly dependent on data, may have modal aliasing problems, and is sensitive to the selection of initial points and interpolation methods. Independent Component Analysis (ICA) is used to decompose mixed signals into independent components. It separates the source signals by assuming that the mixed signal is a linear combination of the source signals and using the statistical properties of independence.

[0060] This system for real-time EEG artifact processing and feature extraction frees researchers and application developers from the constraints of traditional EEG data analysis software. It enables rapid and accurate processing of real-time data and extraction of effective eigenvalues, providing a new solution for the widespread application of EEG devices in areas such as biofeedback, emotion detection, state recognition, and brain-computer interfaces. The development of this system will significantly improve the efficiency and automation of EEG data analysis, bringing broader development prospects to EEG research and applications.

[0061] In another embodiment of the present application, the system for real-time artifact processing and feature extraction of EEG signals includes a real-time EEG data acquisition and analysis module, an intelligent noise and artifact removal module, an automated multi-dimensional feature extraction module, and a data transmission module. Specifically: (1) The real-time EEG data acquisition and analysis module of the system can receive data streams from EEG devices in real time, supports multiple transmission protocols such as wireless Bluetooth and LSL (Lab streaming layer), reads and caches the real-time collected data according to user configuration, and adopts an adaptive sliding window paradigm to perform real-time noise reduction, artifact removal, and feature extraction. (2) The system's intelligent noise and artifact removal module investigates the types of EEG devices used in different scenarios, such as standard laboratories, biofeedback, emotional brain-computer interfaces, and medical brain-computer interfaces. It designs personalized filtering solutions for high-precision, high-resolution, and high-density gel electrode EEG devices, portable wearable high-density saline (semi-dry) electrode EEG devices, and portable wearable dry electrode EEG devices of different forms (different number of channels). It integrates traditional frequency domain filtering, wavelet noise reduction, and innovative deep learning, blind source separation, and machine learning hybrid methods to achieve fully automated signal noise reduction and artifact removal. Users only need to perform simple configuration, and the entire artifact removal process can be completed automatically without the need for extensive code writing. (3) After researching a large number of papers and algorithms in related fields, the system integrates eigenvalue calculation algorithms corresponding to time domain, frequency domain, time-frequency domain, nonlinearity, and brain functional connectivity, with diverse eigenvalues ​​and high sensitivity. (4) The data transmission module of this system is compatible with multiple data transmission protocols such as TCP, UDP, wireless Bluetooth, MQTT, RS-232 / RS-485, and can realize convenient data transmission between different systems, different devices and different applications. Researchers can choose according to their needs without writing code.

[0062] Furthermore, FastICA+GFP can be used to remove electrooculogram artifacts from EEGs with 8 channels or more. The general approach is as follows: use the FastICA algorithm to calculate the independent components of each lead separately, and at the same time calculate the GFP values ​​of the selected N leads, calculate the correlation coefficient between the independent components and GFP, output the independent component corresponding to the maximum correlation coefficient, and reorganize the other components with this maximum component to obtain a clean signal. This method is not suitable for EEG devices with fewer channels in the frontal lobe. In this case, the system will automatically recommend a pre-trained neural network model to achieve end-to-end EEG signal noise reduction. When training this type of model, first input a large number of EEG signals containing different artifacts (such as electrooculogram and electromyography) and different signal-to-noise ratios. Use the EEG signals for supervised learning to obtain the optimal parameters, which are then deployed in the system for real-time or offline noise recognition and removal.

[0063] In some embodiments of the present application, the automated multi-dimensional feature extraction module includes feature extraction from multiple angles. Specific examples are as follows:

[0064] (1) From the perspective of time domain analysis, time domain analysis reveals the change of amplitude over time, and various eigenvalues ​​are usually calculated through statistical methods.

[0065] Table 1 Examples of time domain characteristic indicators of EEG signals

[0066] (2) From the perspective of frequency domain analysis, frequency domain analysis describes the distribution of EEG energy, phase and other information at frequency points, which can reflect the individual's cognitive state. Common methods of frequency domain analysis include Fourier transform, periodogram method, Welch method, multi-window method, autoregressive model, etc.

[0067] Among them, the Fourier transform is a basic frequency domain analysis method used to convert time domain signals into the frequency domain. It can decompose the signal into the amplitude and phase information of different frequency components. The Fourier transform is generally applicable to stationary signals. The periodogram method is a method used to study the periodic components of a signal. It includes tools such as autocorrelation function, cross-correlation function, and power spectral density, which can be used to identify periodicity in the signal. The Welch method is a signal processing method that divides the signal into multiple overlapping windows and calculates the power spectral density over each window. These power spectral density estimates are then averaged to obtain a final estimate to reduce the variance of the estimate. Multi-window methods involve using different types of window functions, such as rectangular windows, Hamming windows, and Hanning windows, to analyze the spectral characteristics of the signal. Different window functions are suitable for different applications and signal types. The autoregressive model is a method used to model time series data, typically used to estimate the frequency components of the signal. Autoregressive models include autoregressive (AR) models and autoregressive moving average (ARMA) models.

[0068] From a spectrum analysis perspective, EEG spectrum analysis includes the classic direct method (periodogram method) and the most commonly used improved direct method (Welch method). The Welch method works as follows: first, divide a signal of length N into L segments, each of length M, where N = LM; then, apply a window function w to each segment to determine the power spectrum of each segment; finally, average the power spectra of each segment to obtain the power spectrum of the entire signal. The individual segments can overlap, and the window function w can be any window type, such as Hanning or Hamming.

[0069] Table 2 Examples of frequency domain characteristic indicators of EEG signals

[0070] (4) From the perspective of time-frequency analysis, the power value of the signal at each specific time and frequency point in the time-frequency domain is estimated. The time information is retained when performing spectrum analysis. It is usually performed using short-time Fourier transform or continuous wavelet transform. The results of time-frequency analysis are used to indicate whether the power of a specific frequency band in the corresponding ROI area increases (ERS) or decreases (ERD) after a stimulus occurs.

[0071] (5) From the perspective of nonlinear dynamics analysis, the results of nonlinear dynamics analysis are used to reflect changes in the brain's dynamic characteristics, including indicators such as complexity, entropy, and Lyapunov exponent. Complexity measures the information capacity of EEG segments, thereby reflecting the potential activity characteristics of neurons and indicating the speed at which new patterns appear in time series as the length of the sequence increases. Complexity indicators include Lempel-Ziv complexity and Lempel-Ziv sorting complexity. The entropy value indicates the degree of confusion of the EEG signal. Different entropy algorithms describe the information capacity from different perspectives. A decrease in entropy means a decrease in the ability of information to interact within the brain. Entropy indicators include Shannon entropy, approximate entropy, sample entropy, and sorting entropy based on the time domain. Entropy based on time and frequency include wavelet entropy and Hilbert-Huang spectrum entropy. The maximum Lyapunov exponent can quantitatively characterize the average divergence rate of adjacent orbits in the phase space, and how small initial perturbations in the system gradually increase over time.

[0072] (6) Brain functional connectivity indicators are used to evaluate the information connectivity between brain regions. Each part of the brain has its own unique function in human behavior. Even the simplest tasks require the collaboration of multiple brain regions. Commonly used indicators include coherence-based indicators, phase synchronization-based indicators, generalized synchronization-based indicators, and Granger causality-based indicators.

[0073] On this basis, the system for real-time artifact processing and feature extraction of EEG signals proposed in this application will limit the types of features that can be selected based on the channel, sampling rate and other information of the device set by the user. At the same time, the system can also receive further screening instructions from researchers in the feature values ​​according to their own experimental needs, and the extracted feature values ​​support one-click export.

[0074] Figure 2 is a flowchart of the system workflow for real-time artifact processing and feature extraction in one embodiment of the present application. The system first receives user configuration, which includes the size of the real-time processing time window from the perspective of timeliness, selects the filtering algorithm, noise reduction algorithm and artifact removal method from the perspective of device characteristics, selects the appropriate feature value to be extracted from the perspective of feature usage, and selects the application layer interface type from the perspective of transmission protocol. After determining the user configuration, real-time EEG data transmission is performed. Electrodes are arranged at different positions on the subject's head. The acquisition device transmits the collected EEG signals to the system proposed in this application. The collected raw EEG signals are pre-processed by the built-in algorithm, including data segmentation, noise removal and artifact removal. Feature values ​​are then extracted from one or more of the time domain, frequency domain, time-frequency and nonlinear dynamics perspectives. The extraction is performed according to the pre-selected appropriate feature values. The extracted feature values ​​are then transmitted in real time via TCP, UDP or API. Finally, at the application layer, based on the different EEG signal application scenarios faced, emotion recognition, biofeedback, brain-computer interface, state monitoring and other behaviors are performed.

[0075] FIG3 is a schematic diagram of the system built-in algorithm module for real-time artifact processing and feature extraction in an embodiment of the present application. FIG3 lists different built-in algorithms that can be used for data preprocessing and feature extraction based on FIG2, for example: (1) for data segmentation, sliding window and real-time segmentation are used; (2) for noise removal, filtering and / or wavelet noise reduction algorithms are used; (3) for artifact removal, algorithms such as Fastica+GPF, BSS+SVM, neural network, etc. are used; (4) for time domain feature extraction, statistics or Hjorth algorithm are used; (5) for frequency domain feature extraction, fast Fourier transform is used; (6) for time-frequency feature extraction, short-time Fourier transform or wavelet transform algorithm is used; (7) for feature extraction of nonlinear angles, entropy, complexity or recursive variable analysis algorithm is used. After removing physiological and non-physiological artifacts such as electrooculogram, electromyography, mains power, noise, drift, etc. through noise reduction and other methods to obtain a clean EEG signal, the characteristics of the EEG signal in multiple dimensions such as time domain, frequency domain, time-frequency domain and nonlinearity can be extracted through fast Fourier transform, short-time Fourier transform, wavelet transform, etc.

[0076] A specific embodiment of the present application is applied in the field of emotion recognition using EEG signals. By collecting EEG signals to identify anxiety states, meditation and relaxation are guided. Anxiety can affect an individual's life and work status. In the case of persistent anxiety, the human immune system can also be affected, causing pain and disease. In the case of anxiety, the energy values ​​of the brain's alpha and theta bands will decrease. After mindfulness training such as meditation and relaxation, the gamma band will be enhanced. Real-time visualization of the energy values ​​of the alpha, theta, and gamma bands can guide anxious individuals to meditate and relax.

[0077] Specifically, the detailed process in this specific embodiment includes:

[0078] ① Step 1: User Configuration. On the configuration interface, select wireless network transmission (which can be Bluetooth, with efficiency being the priority; this is not a specific requirement) for real-time data transmission. Enter device parameters, set the number of channels to 12 (EEG is called leads, so 12 channels is sufficient), and the sampling rate to 250Hz. Based on the device information, the ErgoLAB real-time artifact removal and feature extraction system recommends high-pass and low-pass filtering with a range of 0.1-70Hz and a notch filter at 50Hz. For artifact removal, it recommends using multi-channel artifact detection algorithms such as FastICA+GPF (as shown below). The system presents all available feature value types, allowing users to select the energy values ​​of the α, θ, and γ bands in the frequency domain as output metrics. The time window is set to 2 seconds, and the feature value transmission method is TCP. A time window refers to a data set within a certain time range. In stream processing scenarios, data exists as a continuous stream, constantly being generated without a beginning or end. Using a time window, the data stream can be divided into multiple data blocks, which can then be processed based on the time window. This is a common setting for frequency domain feature analysis of EEG signals. FIG4 is a flow chart of a method for removing artifacts using the FastICA+GFP combination technique in an embodiment of the present application.

[0079] ②Step 2: Feature extraction. Temporarily store the real-time EEG data and use a sliding window to remove noise and artifacts from the current time point (different artifact removal algorithms vary) to obtain a clean signal. Then, extract frequency domain features from the 2 seconds before the current time point, with a step size of 1 second.

[0080] ③Step 3: Design the biofeedback system. The system includes a built-in breathing meditation training course, guiding individuals through the process of mastering breathing relaxation techniques. Five minutes of data collected from individuals in a quiet, stress-free state serves as a baseline. Energy values ​​in the alpha, theta, and gamma frequency bands are used for visualization. The visualization depicts a flower bud, which opens in low-anxiety situations and closes in high-anxiety situations.

[0081] Table 3 Classification of anxiety levels based on different frequency bands of EEG signals

[0082] ④ Step 4: Real-time feedback adjustment. The ErgoLAB real-time artifact processing and feature extraction system extracts the alpha, theta, and gamma frequency bands every 2 seconds and transmits them to the biofeedback system in real time via the TCP protocol. The flower buds on the biofeedback system interface begin to move and change. The individual needs to use the breathing relaxation methods learned to make the flower buds bloom as much as possible.

[0083] One specific embodiment of this application is applied to the context of collecting EEG signals to control a prosthesis. By collecting real-time EEG signals from a person with right-sided limb disability while performing forward, backward, left, and right motor imagery, a time-frequency analysis method is used to extract the corresponding event-related synchronization and desynchronization (ERS and ERD) of the μ and β rhythms. Machine learning methods are then used to classify these signals, and the prosthesis is controlled based on the classification results.

[0084] Specifically, the detailed process of the technical implementation in this embodiment includes:

[0085] ①Step 1: User configuration.

[0086] On the configuration interface, the user selects wireless Bluetooth transmission as the real-time data transmission method, enters device parameter information, sets the number of channels to 32, and the sampling rate to 512 Hz. Based on the device information, the ErgoLAB real-time artifact removal and feature extraction system recommends 0.1-70 Hz high-pass and low-pass filtering and 50 Hz notch filtering. The SOBI+SVM algorithm is recommended for artifact removal, and time-frequency analysis is selected, with the frequency range of interest set to 8-13 Hz for μ rhythm and 13-30 Hz for β rhythm.

[0087] ②Step 2: Feature value extraction.

[0088] The real-time EEG data is temporarily stored, and a sliding window method is used to remove noise and artifacts in a time window of several seconds from the current moment forward (different artifact removal algorithms are different) to obtain a clean signal. Then, according to the event coding of the start of motor imagery, the time after 1000ms and before 200ms is taken as the event-related time window and baseline, respectively, for short-time Fourier transform to extract the ERS and ERD of μ rhythm and β rhythm.

[0089] ③Step three: Classification.

[0090] Label setting: Label the data according to the requirements of imagination, divide the labeled data into training set and test set, create a 4-category model, create the model on the training set, adjust the model parameters on the validation set, use the test set to evaluate the training results, and select the model with the best accuracy for real-time arm control.

[0091] ④Step 4: Real-time control.

[0092] The ERS and ERD of μ rhythm and β rhythm extracted in real time are sent to the model using TCP protocol, and the classification results of the model are output for limb control.

[0093] A specific embodiment of this application is applied in the context of collecting EEG signals to assess the overall performance of operators. In this context, EEG signals are collected while the operator performs tasks of varying difficulty. Feature extraction algorithms are used to extract diverse eigenvalues ​​from dimensions such as time domain, frequency domain, time-frequency, and nonlinear analysis. These eigenvalues ​​are then labeled using questionnaire scale information. Machine learning methods are then used to classify the EEG data from tasks of varying difficulty, categorizing the operator's mental fatigue, workload, and stress state. Based on the classification results, the three dimensions are weighted to determine the operator's overall performance and reduce the occurrence of accidents.

[0094] Specifically, the detailed process of the technical implementation in this embodiment includes:

[0095] ①Step 1: User Configuration

[0096] On the configuration interface, the user selects wireless Bluetooth as the real-time data transmission method, enters device parameters, sets the number of channels to 64, and the sampling rate to 1024Hz. Based on the device information, the ErgoLAB real-time artifact removal and feature extraction system recommends 0.1-70Hz high-pass and low-pass filtering and 50Hz notch filtering. For artifact removal, the system recommends using a wavelet denoising algorithm, with a variety of eigenvalue types selected, including mean, variance, kurtosis, and skewness in the time domain; power spectral density and band energy in the frequency domain; short-time Fourier transforms in time and frequency; and sample entropy and Lyapunov exponents for nonlinear analysis. The time window is set to 2s, and the eigenvalue transmission method is TCP. The Lyapunov exponent represents the numerical characteristic of the average exponential divergence rate of adjacent trajectories in phase space. Also known as the Lyapunov characteristic index, it is one of the numerical features used to identify chaotic motion.

[0097] Time-frequency analysis is a joint analysis of EEG signals in the time domain and frequency domain. It aims to study the dynamic changes of signals in time and frequency, such as the event-related synchronization (ERS) of the μ (sensorimotor) rhythm that appears at the electrodes in the prefrontal and parietal lobes (sensorimotor areas) for a period of time after a motor imagery task.

[0098] Nonlinear analysis examines the nonlinear dynamic characteristics of EEG signals. EEG signals possess complex nonlinear dynamical properties, and nonlinear analysis aims to reveal nonlinear interactions, nonlinear dynamical behavior, and chaotic characteristics within the signals. Commonly used nonlinear analysis methods include phase space reconstruction, Lyapunov exponents, and nonlinear prediction.

[0099] ②Step 2: Feature value extraction

[0100] The real-time EEG data is temporarily stored, and a sliding window method is used to remove noise and artifacts from the current time point in a time window of several seconds (different artifact removal algorithms have different ones) to obtain a clean signal, and then the first 2 seconds of the current time point are taken for diversified feature extraction.

[0101] ③Step 3: Classification and Evaluation

[0102] The EEG data was labeled based on the questionnaire data and divided into training and test sets. Binary classification models were created for load, fatigue, and stress, respectively. The training set was used to train the models. Model optimization and parameter adjustment were performed using methods such as cross-validation. The test set was used to evaluate model performance. This process was repeated multiple times, and the model with the highest evaluation metrics, such as precision and recall, was selected.

[0103] The expert evaluation method was used to evaluate the weights of each model, which were W1, W2, and W3 respectively. The total score of comprehensive effectiveness was F = (W1*load (0, 100) + W2*fatigue (0, 100) + W3*stress (0, 100)) / 3.

[0104] ④Step 4: Real-time feature transmission and status warning

[0105] The diverse feature values ​​extracted in real time are sent to the model through the TCP protocol, and the classification results of the model are output respectively. The comprehensive efficiency score is calculated based on the model results and weights. The higher the comprehensive efficiency score, the worse the operator's working status. When the comprehensive efficiency score is higher than 80 points, an early warning is issued.

[0106] The method and system for real-time artifact processing and feature extraction of EEG signals proposed in this application can perform adaptive filtering and artifact removal on the EEG signal data stream based on parameter information such as the number of channels and sampling rate contained in the EEG device, and extract feature values ​​that meet the output indicators in real time based on the pre-selected output indicator matching feature extraction strategy. On the one hand, it can perform real-time artifact removal and feature extraction on EEG signals, and on the other hand, it can extract corresponding feature values ​​for different types of EEG devices and different output indicators in an integrated and automated manner, facilitating subsequent analysis and processing of EEG data. The method proposed in this application can support the export of feature values ​​channel by channel, making data application more convenient. Through multiple artifact removal methods, a high degree of automated processing can be achieved, eliminating the need for researchers to waste time and effort in identifying crisis components, and does not require the experience of researchers. Different artifact removal methods are integrated, and artifact removal can be performed on EEG data with single or fewer channels, as well as on EEG data with multiple channels.

[0107] Corresponding to the above method, the present application also provides a device for real-time artifact processing and feature extraction of EEG signals, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the method described above.

[0108] Example 2

[0109] The technical solution provided in the embodiment of the present application is mainly aimed at the defect of the existing technology in the weak ability to extract time feature information when extracting information from EEG signals using a neural network model. Specifically, for EEG signals, when they are applied in technical fields such as brain-computer interfaces, they can be mainly classified and processed based on the time feature information and spatial feature information in the EEG signals. The time feature information mainly refers to the information including amplitude, slope, peak and trough obtained by analyzing the waveform of the EEG signal. The spatial feature information mainly refers to the spatial distribution information of the acquired multi-channel EEG signal. The description of brain activity can be obtained by studying the correlation between the channels. The full extraction of the above-mentioned time feature information and spatial feature information in the neural network model can fully and more comprehensively and accurately understand the situation of brain activity, so as to facilitate subsequent classification and processing operations.

[0110] In the prior art, there is a convolutional neural network model capable of processing EEG signals, called the EEGNet model. Although it can also extract temporal feature information and spatial feature information, it extracts repeated temporal feature information in the convolutional layers of the second feature extraction module (block2) and the third feature extraction module (block3) of the model, resulting in feature redundancy between the convolutional layers and limiting the final classification effect of the model. Its ability to extract temporal feature information is relatively weak.

[0111] In response to the above technical problems, the embodiment of the present application provides a new convolutional neural network model architecture, and uses the model architecture to provide a method for processing EEG signals based on human intelligence, wherein the new convolutional neural network model architecture includes a first feature extraction module, a second feature extraction module, a third feature extraction module and a classification module connected in sequence, and the first feature extraction module, the second feature extraction module and the third feature extraction module all include a time convolution kernel for extracting time feature information, and the second feature extraction module also includes a spatial convolution kernel for extracting spatial feature information. By setting the time convolution kernel responsible for extracting time feature information in the above three feature extraction modules, it is possible to extract time feature information more comprehensively, so that the ability to extract time feature information is enhanced, and further different types of convolution layers can be set in different feature extraction modules, so that the characteristics of different types of convolution layers can be brought into play, so that the extracted time feature information can be continuously optimized, and the feature redundancy problem existing in the existing EEGNet model can also be solved. In the subsequent embodiments of this application, the specific architecture of the convolutional neural network model and the method for processing EEG signals will be introduced in detail.

[0112] The technical solution of the embodiment of the present application is described in detail below through the accompanying drawings. Figure 5 is a structural diagram of a convolutional neural network model in the embodiment of the present application. As shown in Figure 5, the convolutional neural network model includes a first feature extraction module 11, a second feature extraction module 12, a third feature extraction module 13 and a classification module 14 connected in sequence, wherein the EEG signal collected by the EEG acquisition device can be directly input, or input into the first feature extraction module 11 after a pre-processing process such as denoising, and then processed in sequence by subsequent modules, wherein the first feature extraction module 11, the second feature extraction module 12 and the third feature extraction module 13 all include a temporal convolution kernel for extracting temporal feature information, and the second feature extraction module 12 also includes a spatial convolution kernel for extracting spatial feature information.

[0113] FIG6 is a flow chart of a method for processing EEG signals based on human intelligence according to an embodiment of the present application. The method can be executed on an electronic device, including but not limited to a server, a local computer, etc. As shown in FIG6 , the method includes the following steps:

[0114] Step 101: Input the EEG signal to be processed into a first feature extraction module, a second feature extraction module, and a third feature extraction module connected in sequence to obtain temporal feature information and spatial feature information of the EEG signal;

[0115] The EEG signals to be processed in this step can be collected by an EEG signal acquisition device. Depending on the application scenarios of the embodiments of the present application, it can be an online processing scenario, in which case the EEG signals collected by the EEG signal acquisition device can be directly input into the convolutional neural network model provided by the embodiments of the present application for processing. Alternatively, it can be an offline processing scenario, in which case the EEG signals collected by the EEG signal acquisition device can be pre-stored in a storage module, and then when the EEG signals are processed according to the technical solution of the present application, they can be input into the convolutional neural network model for processing. The EEG signals to be processed can be directly collected by the EEG signal acquisition device, or they can be pre-processed after direct collection. The pre-processing process includes but is not limited to denoising, etc. In addition, the EEG signal acquisition device can be composed of multiple signal detection heads to collect multi-channel EEG signals.

[0116] In the embodiment of the present application, the first feature extraction module, the second feature extraction module and the third feature extraction module are all provided with convolution kernels for extracting time feature information or spatial feature information. Therefore, when the above-mentioned EEG signal to be processed is input into the first feature extraction module, the time convolution kernel therein will extract the time feature information in the EEG signal to obtain a first feature map, which includes time feature information; and the first feature map is further input into the second feature extraction module, the spatial convolution kernel in the second feature extraction module will extract spatial feature information from the first feature map, and the time convolution kernel in the second feature extraction module will further extract time feature information. The time feature information is a preliminary optimization of the time feature information in the first feature map, and the second feature extraction module will output a second feature map including spatial feature information and preliminary optimized time feature information; after the second feature map is further input into the third feature extraction module, the time convolution kernel in the third feature extraction module will further perform more fine optimization on the above-mentioned preliminary optimized time feature information to obtain the final optimized time feature information, and finally output the third feature map from the third feature module, which will include the above-mentioned spatial feature information and the final optimized time feature information.

[0117] It can be understood that the EEG signal to be processed in Example 2 of the present application can be equivalent to the EEG signal data stream that has undergone the filtering processing step and the artifact removal step in the above-mentioned Example 1. Example 2 can be used to extract the spatial feature information and temporal feature information of the EEG signal data stream that has undergone the filtering processing step and the artifact removal step.

[0118] The specific structures and functions of the first characteristic module, the second characteristic module and the third characteristic module involved in the embodiment of the present application will be described in detail in the embodiment shown in the subsequent Figure 7.

[0119] Step 102: input the temporal feature information and spatial feature information of the EEG signal to be processed into a classification module for classification processing to obtain a classification processing result of the EEG signal;

[0120] In this step, after obtaining the temporal feature information and spatial feature information of the above-mentioned EEG signal, it can be input into the classification module for classification processing. The specific classification results included in the above-mentioned classification processing results can be set according to actual needs. For example, if the EEG signals that control different parts of the body are understood, the classification results include the control of the corresponding different parts of the body. In some embodiments, the classification module in this step may include a fully connected layer and a normalization function, wherein the fully connected layer can play the role of mapping the feature information to each classification result, and the normalization function can be a softmax function, which can obtain the probabilities corresponding to different categories in different classification results, and the sum of the probabilities corresponding to all classification results is 1.

[0121] In the above embodiments of the present application, the first feature extraction module, the second feature extraction module and the third feature extraction module in the convolutional neural network model all include a temporal convolution kernel for extracting temporal feature information, which strengthens the extraction of temporal feature information in the EEG signal. At the same time, the spatial convolution kernel for extracting spatial feature information in the second feature extraction module realizes the extraction of spatial feature information in the EEG signal, thereby realizing more comprehensive and accurate feature information extraction in the EEG signal, which is conducive to improving the accuracy of the classification results obtained by the classification module based on the above-mentioned temporal feature information and spatial feature information.

[0122] Figure 7 is a schematic diagram of the specific architecture of a convolutional neural network model in an embodiment of the present application. As shown in Figure 7, the convolutional neural network model includes an input module 30, a first feature extraction module 31, a second feature extraction module 32, a third feature extraction module 33 and a classification module 34. The embodiment of the present application will provide a detailed description and explanation of each of the above modules.

[0123] The input module 30 is mainly used to receive EEG signals to be processed, which may include multi-channel EEG collected by an EEG acquisition device, and can perform pre-processing such as denoising on the EEG when necessary. After receiving the input EEG signal, the EEG signal is first sent to the first feature extraction module 31 for processing. In the embodiment of the present application, the EEG signal is processed as an image signal.

[0124] In some embodiments, the first feature extraction module 31 includes at least one depthwise separable convolution layer 41, and each depthwise separable convolution layer may include a first temporal convolution kernel for extracting temporal feature information. In this case, in the first feature extraction module, the steps of processing the EEG signal may include:

[0125] After the EEG signal to be processed is input into the first feature extraction module, a first feature map outputted from the first feature extraction module is obtained. The first feature map includes time feature information extracted by the first time convolution kernel.

[0126] In an embodiment of the present application, a depthwise separable convolution layer 41 can be set, and the depth of the depthwise separable convolution layer 41 can be set according to actual needs, for example, it can be set to values ​​such as 4 or 8; in some embodiments, the size of the first time convolution kernel can be set to (1, a), where a is a positive integer greater than 1, that is, the height and width distribution of the first time convolution kernel are set to 1 and a, and the first time convolution kernel can be used to perform feature extraction in the width direction of the image of the EEG signal to obtain the time feature information in the EEG signal.

[0127] In some embodiments, a batch normalization (BN) layer and an activation function may be further set in the first feature extraction module 31. The activation function may use a RELU activation function. The above-mentioned BN layer and RELU activation function may be sequentially set at the output end of the depthwise separable convolutional layer.

[0128] In the embodiment of the present application, the depthwise separable convolutional layer 41 used in the first feature extraction module 31 has the advantages of using fewer parameters and being able to achieve channel and region separation. Experimental verification shows that using the depthwise separable convolutional layer 41 alone to extract the temporal feature information of the EEG signal is better than the extraction effect of various other types of convolutional layers.

[0129] In the embodiment of the present application, the second feature extraction module 32 may include at least one ordinary convolution layer 42 and at least two residual blocks connected in sequence, at least one ordinary convolution layer 42 includes a spatial convolution kernel for extracting spatial feature information, and at least two residual blocks each include a second temporal convolution kernel for extracting temporal feature information. In the embodiment of the present application, the first feature extraction module 31 has already performed feature extraction and obtained a first feature map. Therefore, the specific processing steps for the second feature extraction module 32 are: after inputting the above-mentioned first feature map into the second feature extraction module 32, a second feature map output from the second feature extraction module 32 is obtained, and the second feature map includes the spatial feature information extracted by the spatial convolution kernel and the first optimized temporal feature information extracted by the second temporal convolution kernel.

[0130] In some embodiments, it may include a common convolution layer 42, the depth of which can be set according to actual needs, such as parameters such as 8, 16, etc.; the size of the spatial convolution kernel in the common convolution layer 42 is (channel, 1), where channel is the number of channels of the EEG signal, and the height and width distribution of the spatial convolution kernel are set to channel and 1. Through the spatial convolution kernel, feature extraction can be performed in the height direction of the image of the EEG signal to obtain spatial feature information in the EEG signal.

[0131] In the embodiment of the present application, as shown in FIG7 , the at least one residual block of the second feature extraction module 32 may include a first residual block and a second residual block connected in sequence, and the first residual block includes a first convolutional layer 43, a second convolutional layer 44, and a first skip connection 51 connected in sequence. The first skip connection 51 is used to add the input information of the first convolutional layer 43 and the output information of the second convolutional layer 44 to serve as the output information of the first residual block;

[0132] The second residual block includes a third convolutional layer 45, a fourth convolutional layer 46, and a second skip connection 52 connected in sequence, and the second skip connection 52 is used to add the input information of the third convolutional layer 45 and the output information of the fourth convolutional layer 46 to serve as the output information of the second residual block;

[0133] Among them, the first convolution layer 43, the second convolution layer 44, the third convolution layer 45 and the fourth convolution layer 46 all include the second temporal convolution kernel for extracting temporal feature information.

[0134] In some embodiments, the first convolution layer 43, the second convolution layer 44, the third convolution layer 45 and the fourth convolution layer 46 can be ordinary convolution layers, and their depths can be set as needed, such as 4, 8, 16 or 32, and the first convolution layer 43 and the second convolution layer 44 can be set to the same, and the third convolution layer 45 and the fourth convolution layer 46 can be set to the same, or the depths of the four convolution layers are the same. In some embodiments, the size of the second temporal convolution kernel in the first convolution layer and the second convolution layer is (1, a), a is a positive integer greater than 1, the size of the second temporal convolution kernel in the third convolution layer and the fourth convolution layer is (1, b), b is a positive integer greater than 1, and the above four convolution kernels are distributed as a and b in the width direction, so that they can obtain the temporal feature information in the EEG signal.

[0135] In some embodiments, a batch normalization (BN) layer and an activation function may be further set in the second feature extraction module 32. The activation function may use a RELU activation function. The above-mentioned BN layer and RELU activation function may be sequentially set at the output end of each of the above-mentioned convolutional layers.

[0136] Furthermore, a first average pooling layer can be set at the output end of the second feature extraction module 32. The pooling kernel size that can be adopted by the first average pooling layer is (1, 4), so as to perform average pooling processing on the second feature maps obtained by processing the above-mentioned convolutional layers to obtain the second feature map after average pooling processing.

[0137] The second feature extraction module provided in the embodiment of the present application has a strong ability to learn temporal feature information and enhances the parameter space of the network model. The two residual blocks therein can extract more refined temporal feature information compared to the first feature extraction module.

[0138] In this embodiment of the present application, the third feature extraction module 33 may include at least one hybrid dilated convolution layer, each of which includes a third temporal convolution kernel for extracting temporal feature information. The third feature extraction module 33 in this embodiment may specifically perform the following steps:

[0139] After the second feature map is input into the third feature extraction module 33, a third feature map output from the third feature extraction module 33 is obtained, and the third feature map includes the spatial feature information extracted by the above-mentioned spatial convolution kernel and the second optimized time feature information extracted by the third time convolution kernel.

[0140] In some embodiments, the third feature extraction module 33 may include a first hybrid dilated convolution layer and a second hybrid dilated convolution layer connected in sequence, that is, two hybrid dilated convolution layers. The hybrid dilated convolution layer can combine dilated convolution and ordinary convolution, and is a hybrid dilated convolution (HDC), which can process multi-scale features, ensure the spatial resolution of features and improve the generalization ability of the model. The depth of the hybrid dilated convolution layer can be set as needed, and as the depth increases, the size of the receptive field of the third temporal convolution kernel in the hybrid dilated convolution will gradually increase. The use of the hybrid dilated convolution layer in this embodiment can increase the representation ability of the convolutional neural network model and the effectiveness of the receptive field.

[0141] Specifically, in the embodiment of the present application, the size of the third temporal convolution kernel can be set to (1, c), where c is a positive integer greater than 1. The convolution kernel is set to c in the width direction so that it can obtain the temporal feature information in the EEG signal. And because the present embodiment uses a mixed dilated convolution, the embodiment of the present application has been verified through multiple ablation experiments to show that the technical solution of the present application can extract more comprehensive temporal feature information, thereby improving the accuracy of the final model classification result.

[0142] In some embodiments, a batch normalization layer may be further provided at the output ends of the first hybrid dilated convolution layer 47 and the second hybrid dilated convolution layer 48 .

[0143] In the embodiment of the present application, the depthwise separable convolution layer 41, the first convolution layer 43, the second convolution layer 44, the third convolution layer 45, the fourth convolution layer 46, the first mixed dilated convolution layer 47 and the second mixed dilated convolution layer 48 are all provided with temporal convolution kernels capable of extracting temporal feature information, but the implementation forms of each convolution kernel are different, and they can extract temporal feature information from different dimensions. Moreover, since the above-mentioned convolution layers are arranged in sequence, the temporal feature information can be continuously refined and optimized, and finally more comprehensive and accurate temporal feature information is obtained, thereby strengthening the extraction of temporal feature information of the input EEG signal.

[0144] In some embodiments, for the embodiment shown in Figure 7 above, the depths of the depthwise separable convolutional layer 41, the normal convolutional layer 42, the first convolutional layer 43, the second convolutional layer 44, the third convolutional layer 45, the fourth convolutional layer 46, the first mixed dilated convolutional layer 47, and the second mixed dilated convolutional layer 48 are set to an inverse bottleneck structure, that is, the depth of the depthwise separable convolutional layer 41 and the second mixed dilated convolutional layer 48 is set to the minimum, for example, 4, and the depth of each convolutional layer in the middle is set to other values ​​greater than 4, such as 8 or 16; or in some embodiments, for each convolutional layer in the middle, the depth of the normal convolutional layer 42 and the first mixed dilated convolutional layer 47 at both ends is still smaller, for example, 16, and the depth of the first convolutional layer 43, the second convolutional layer 44, the third convolutional layer 45, and the fourth convolutional layer 46 in the middle is 32. The depth of each convolutional layer is set to an inverse bottleneck structure, thereby reducing the amount of parameters used.

[0145] In this embodiment of the present application, an ELU activation function and a second average pooling layer connected in sequence may be further provided at the output end of the third feature extraction module 33. The second average pooling layer may use a pooling kernel size of (1, 8). Experimental verification shows that using the ELU activation function in this embodiment of the present application can achieve better accuracy than using ReLU activation.

[0146] In addition, the use of activation layers is reduced between multiple convolutional layers in the three feature extraction modules provided in the embodiment of the present application. For example, a single ELU activation function is set in the third feature extraction module 33, which is conducive to obtaining more accurate feature information and improving the accuracy of the final classification results.

[0147] In the embodiment of the present application, time feature information is first extracted in the first feature extraction module, and then spatial feature information is extracted in the second feature extraction module. Therefore, the technical solution provided in the embodiment of the present application is essentially a technical solution that first extracts time feature information and then extracts temporal feature information. Experiments have verified that its effect is better than the technical solution that first extracts spatial feature information and then extracts time feature information in the EEGNet model.

[0148] In an embodiment of the present application, the classification module 34 may further include a fully connected layer 49 and a loss function, wherein the fully connected layer 49 can map the temporal feature information and spatial feature information extracted by the above-mentioned feature extraction modules to each classification result, so as to achieve the purpose of obtaining the classification result based on the above-mentioned temporal feature information and spatial feature information; the normalization function can adopt the Softmax function to limit the sum of the probabilities of each classification result to 1.

[0149] The convolutional neural network model provided in the above embodiments of the present application is a model specifically used to decode and classify EEG signals. The model can be pre-trained, and then the pre-trained convolutional neural network model can be used to perform classification prediction of EEG signals.

[0150] The convolutional neural network model provided in the embodiment of the present application can be applied to the classification prediction of left-hand and right-hand imagined movements. A test embodiment of classification prediction using the existing collected left and right-hand imagined movement data sets is given below, and compared with the classification results of the EEGNet model in the prior art, it is found that it has better advantages in terms of the accuracy of the classification results, and the differences between different subjects are statistically significant (p < 0.05). In the above-mentioned imagined data set, the input image can be in 4-dimensional form. For example, the input image is represented as (288, 1, 22, 1000). After the various feature extraction modules of the convolutional neural network model provided in the above embodiment, the feature map of the final output can be represented as (288, A, 1, 30). In addition, the filling method of each convolution layer in this embodiment can be SAME, so the number of data points in the fourth dimension after processing by each convolution layer is unchanged. The following are the execution steps of the specific embodiment of the present application:

[0151] Step 1: Input the 4D EEG signal into the first feature extraction module. The convolution kernel size of the depthwise separation convolution layer in the first feature extraction module is (1, a), the depth of the convolution filter formed can be set to A, and the padding method is SAME. At this point, after processing by the first feature extraction module, the first feature map outputted by it can be represented as (288, A, 22, 1000);

[0152] Step 2: Input the first feature map into the second feature extraction module, where the size of the spatial convolution kernel of the ordinary convolution layer in the second feature extraction module is (channel, 1), where the value of channel is 22, and the depth of the convolution filter formed can be B. In order to form an inverse bottleneck structure, the value of B is greater than A, and the filling method is SAME. At this time, after passing through the ordinary convolution layer, the feature map obtained can be expressed as (288, B, 1, 1000), where the channel is 22, which extracts the spatial feature information of 22 channels and then compresses the 22 channels to 1; for the two residual blocks, the convolution kernel sizes of the first convolution layer and the second convolution layer of the first residual block are both (1, a), the depth of the convolution filter formed can be B, and the padding method is SAME, the convolution kernel sizes of the third convolution layer and the fourth convolution layer of the second residual block are both (1, b), the depth of the convolution filter formed can be B, and the padding method is SAME. After the above two residual blocks, the representation of the feature map remains unchanged and can still be expressed as (288, B, 1, 1000); the pooling kernel size of the first average pooling layer is (1, 4), so the feature map after processing by the first average pooling layer can be expressed as (288, B, 1, 250), which can be used as the second feature map;

[0153] Step 3. Input the second feature map into the third feature extraction module. The third feature extraction module includes two mixed void convolution layers, where the size of the convolution kernel is (1, c), the convolution filter depth formed by the first mixed void convolution layer is B, and the convolution filter depth formed by the second mixed void convolution layer is A. The filling method is SAME. The feature map representation method of the above-mentioned second feature map after processing by the first mixed void convolution layer remains unchanged. The feature map after processing by the second mixed void convolution layer can be expressed as (288, A, 1, 250), and then passes through the second average pooling layer. The size of the pooling kernel of the second average pooling layer is (1, 8), and the third feature map can be expressed as (288, A, 1, 30);

[0154] Step 4: After the above processing, the 30 data points in the fourth dimension of the output are tiled through the fully connected layer and then classified based on the 30 data points in each batch to obtain the classification results of the data set, for example, the left hand, right hand, foot, or tongue. This enables the use of the convolutional neural network model provided in the embodiments of the present application to predict and classify EEG signals. For the classification results, the prediction ability of the model can be statistically evaluated by loss value, accuracy, F1 value, or confusion matrix.

[0155] Corresponding to the above-mentioned method embodiment, the embodiment of the present application also provides an EEG signal processing device based on human intelligence, which is executed based on a new convolutional neural network model. The convolutional neural network model includes a first feature extraction module, a second feature extraction module, a third feature extraction module and a classification module connected in sequence. The first feature extraction module, the second feature extraction module and the third feature extraction module all include a temporal convolution kernel for extracting temporal feature information, and the second feature extraction module also includes a spatial convolution kernel for extracting spatial feature information. It may include an input module and a result acquisition module, wherein the input module is used to input the EEG signal to be processed into the first feature extraction module, the second feature extraction module and the third feature extraction module connected in sequence to obtain the temporal feature information and spatial feature information of the EEG signal; the result acquisition module is used to input the temporal feature information and spatial feature information of the EEG signal to be processed into the classification module for classification processing to obtain the classification processing result of the EEG signal.

[0156] In some embodiments, the first feature extraction module includes at least one depthwise separable convolution layer, each depthwise separable convolution layer includes a first temporal convolution kernel for extracting temporal feature information;

[0157] The above-mentioned step of inputting the EEG signal to be processed into the first feature extraction module, the second feature extraction module and the third feature extraction module connected in sequence includes:

[0158] After the EEG signal to be processed is input into the first feature extraction module, a first feature map outputted from the first feature extraction module is obtained, where the first feature map includes the temporal feature information extracted by the first temporal convolution kernel.

[0159] In some embodiments, the first feature extraction module includes a depthwise separable convolution layer, and the size of the first temporal convolution kernel is (1, a), where a is a positive integer greater than 1.

[0160] In some embodiments, the second feature extraction module includes at least one common convolution layer and at least one residual block connected in sequence, and the at least one common convolution layer includes a spatial convolution kernel for extracting spatial feature information, and the at least one residual block includes a second temporal convolution kernel for extracting temporal feature information;

[0161] The above-mentioned step of inputting the EEG signal to be processed into the first feature extraction module, the second feature extraction module and the third feature extraction module connected in sequence includes:

[0162] After the first feature map is input into the second feature extraction module, a second feature map output from the second feature extraction module is obtained, where the second feature map includes spatial feature information extracted by the spatial convolution kernel and the first optimized temporal feature information extracted by the second temporal convolution kernel.

[0163] In some embodiments, the second feature extraction module includes a first residual block and a second residual block connected in sequence:

[0164] The first residual block includes a first convolutional layer, a second convolutional layer, and a first skip connection connected in sequence, wherein the first skip connection is used to add input information of the first convolutional layer and output information of the second convolutional layer to serve as output information of the first residual block;

[0165] The second residual block includes a third convolutional layer, a fourth convolutional layer, and a second skip connection connected in sequence, wherein the second skip connection is used to add input information of the third convolutional layer and output information of the fourth convolutional layer to serve as output information of the second residual block;

[0166] The first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer all include the second temporal convolution kernel.

[0167] In some embodiments, the size of the spatial convolution kernel is (channel, 1), where channel is the number of output channels of the EEG signal, the size of the second temporal convolution kernel in the first convolution layer and the second convolution layer is (1, a), a is a positive integer greater than 1, and the size of the second temporal convolution kernel in the third convolution layer and the fourth convolution layer is (1, b), b is a positive integer greater than 1.

[0168] In some embodiments, the third feature extraction module includes at least one mixed dilated convolution layer, each of which includes a third temporal convolution kernel for extracting temporal feature information;

[0169] The above-mentioned step of inputting the EEG signal to be processed into the first feature extraction module, the second feature extraction module and the third feature extraction module connected in sequence includes:

[0170] After the second feature map is input into the third feature extraction module, a third feature map output from the third feature extraction module is obtained, where the third feature map includes the spatial feature information extracted by the spatial convolution kernel and the second optimized temporal feature information extracted by the third temporal convolution kernel.

[0171] In some embodiments, the third feature extraction module includes a first mixed dilated convolution layer and a second mixed dilated convolution layer connected in sequence, and the size of the third temporal convolution kernel is (1, c), where c is a positive integer greater than 1.

[0172] In some instances, the depth of the depthwise separable convolutional layer, the ordinary convolutional layer, the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the first mixed dilated convolutional layer, and the second mixed dilated convolutional layer constitute an inverse bottleneck structure.

[0173] In some embodiments, the output end of the second feature extraction module is provided with a first average pooling layer, and the output end of the third feature extraction module is provided with an ELU activation function and a second average pooling layer connected in sequence.

[0174] An embodiment of the present application provides an EEG signal processing device based on human intelligence, which corresponds to the method embodiment shown in any one of Figures 5-7 above, and can execute the above method and achieve corresponding technical effects. The embodiment of the present application will not be described in detail, and the specific methods and technical effects achieved can refer to the above embodiments.

[0175] Example 3

[0176] Brainwaves are bioelectric signals generated when the brain is active and information is transmitted between a large number of neurons. By collecting and analyzing the EEG signals of a target object, the target's health status can be identified. The health status can be healthy or unhealthy. The EEG signals include: a portion of the EEG signals of the target object before it is stimulated by a preset stimulus, and a portion of the EEG signals of the target object after it is stimulated by a preset stimulus. The portion of the EEG signals before it is stimulated by a preset stimulus is usually used as the baseline of the EEG signals. The health status of the target object is generally analyzed by comparing the portion of the EEG signals after it is stimulated by a preset stimulus with the baseline.

[0177] When collecting an EEG baseline, the subject is typically required to remain calm and avoid unexpected stimulation. However, during the actual collection process, the subject may be exposed to unexpected stimulation, which can cause interference signals to be superimposed on some of the collected EEG signals, leading to fluctuations in the EEG baseline. If the subject's health status is determined directly based on EEG signals with fluctuating baselines, the accuracy of the determined health status will be low.

[0178] To ensure accurate determination of the subject's health status, the related art typically resamples the subject's EEG signals until the desired EEG signal, free of unexpected stimulation, is obtained. This demonstrates the low efficiency of obtaining the desired EEG signal in the related art. This desired EEG signal can be used to accurately determine the subject's health status.

[0179] In view of this, an embodiment of the present application provides an EEG signal correction method based on human intelligence, which can collect multimodal physiological signals, and the multimodal physiological signals include: EEG signals and target physiological signals other than EEG signals. After determining that the target object is subjected to non-preset stimulation based on the target physiological signal, the method can correct the EEG signal to correct the baseline of the EEG signal. The similarity between the corrected EEG signal and the EEG signal collected without the non-preset stimulation is greater than the similarity threshold. It can be seen that the method provided in the embodiment of the present application can obtain the EEG signal for accurately determining the health status of the target object without re-collecting the EEG signal, thereby improving the efficiency of obtaining the EEG signal.

[0180] The embodiment of the present application provides a method for correcting EEG signals based on human intelligence, which is applied to a multimodal physiological signal acquisition device (hereinafter referred to as the acquisition device). The method includes:

[0181] Step A: Collect multimodal physiological signals of the target object.

[0182] The multimodal physiological signal includes an EEG signal and a target physiological signal other than the EEG signal. The target physiological signal may be at least one of an electrodermal activity (EDA) signal (abbreviated as an EDA signal) and a skin temperature (SKT) signal (abbreviated as a SKT signal). For example, the target physiological signal may be an EDA signal.

[0183] It is understood that the acquisition device may include: an EEG signal acquisition unit and a target physiological signal acquisition unit, and the target physiological signal acquisition unit may include: at least one of a skin electrodermal signal acquisition unit and a skin temperature signal acquisition unit. The acquisition device may synchronously acquire EEG signals and target physiological signals of a target object (e.g., a human body) through the EEG signal acquisition unit and the target physiological signal acquisition unit.

[0184] Step B: If it is determined based on the target physiological signal that the target object is subjected to non-preset stimulation, then the EEG signal in the multimodal physiological signal is corrected.

[0185] In an embodiment of the present application, the acquisition device can detect whether the target object is subjected to non-preset stimulation outside the preset stimulation application period based on the target physiological signal in the multimodal signal. If the acquisition device determines that the target object is subjected to non-preset stimulation outside the preset stimulation application period based on the target physiological signal, the EEG signal can be corrected to correct the EEG signal's baseline.

[0186] Among them, the similarity between the corrected EEG signal and the reference EEG signal is greater than the similarity threshold, that is, the corrected EEG signal is relatively similar to the reference EEG signal. The similarity between the corrected EEG signal and the reference EEG signal can refer to: the similarity between the waveform of the corrected EEG signal and the waveform of the reference EEG signal. The reference EEG signal is an EEG signal collected when the target object is not subjected to non-preset stimulation. For example, except for the non-preset stimulation, the collection environment of the reference EEG signal is the same as the collection environment of the EEG signal.

[0187] As can be seen from the above description, compared to the pre-corrected EEG signal, the corrected EEG signal eliminates interference signals superimposed on the EEG signal due to the target object being subjected to non-preset stimulation. This ensures that the health status of the target object determined based on the corrected EEG signal is highly accurate.

[0188] It can be understood that Example 3 of the present application is an example of another method for filtering and removing artifacts from EEG signal data streams. Compared with Example 1, the method provided in Example 3 not only requires the collection of EEG signal data streams, but also requires the collection of target physiological signals, and uses the above-mentioned target physiological signals to correct abnormal signals in the above-mentioned EEG signal data streams, that is, this process can also achieve the purpose of filtering and removing artifacts from EEG signal data streams.

[0189] In the embodiment of the present application, the acquisition device may perform step B after completing the acquisition of the multimodal physiological signal. Alternatively, the acquisition device may perform step B during the acquisition of the multimodal physiological signal. In this way, the acquisition efficiency of the desired EEG signal can be further improved.

[0190] In summary, the embodiment of the present application provides an EEG signal correction method based on human intelligence, and the acquisition device can acquire multimodal physiological signals, which include: EEG signals and target physiological signals other than EEG signals. And after the acquisition device determines that the target object is subjected to non-preset stimulation based on the target physiological signal, it can correct the EEG signal, and the similarity between the corrected EEG signal and the EEG signal collected when it is not subjected to non-preset stimulation is greater than the similarity threshold. It can be seen that the method provided in the embodiment of the present application can obtain the EEG signal used to accurately determine the health status of the target object without re-acquiring the EEG signal of the target object, thereby improving the acquisition efficiency of the EEG signal.

[0191] The present application also provides another method for correcting EEG signals based on human intelligence, which may include:

[0192] Step 1: Collect multimodal physiological signals of the target object.

[0193] The target object may be a human body. The multimodal physiological signal includes an EEG signal and a target physiological signal other than the EEG signal. The target physiological signal may be at least one of an EDA signal and an SKT signal. For example, the target physiological signal may be an EDA signal.

[0194] It is understood that the acquisition device may include: an EEG signal acquisition unit and a target physiological signal acquisition unit, and the target physiological signal acquisition unit may include: at least one of a skin electrodermal signal acquisition unit and a skin temperature signal acquisition unit. The acquisition device may synchronously acquire EEG signals and target physiological signals of a target object (e.g., a human body) through the EEG signal acquisition unit and the target physiological signal acquisition unit.

[0195] In the embodiments of the present application, the EDA signal is closely related to the target's emotions, arousal, and attention. Furthermore, the EDA signal is highly stable, easy to measure, and highly sensitive, making it the most effective and sensitive physiological parameter for reflecting changes in individual sympathetic nerve excitability. Furthermore, both the EDA signal and the EEG signal are high-dimensional time series signals.

[0196] Step 2: Detect whether the target physiological signal in the multimodal physiological signal has a mutation.

[0197] In the embodiments of the present application, after the target is stimulated, all multimodal physiological signals will change. Because the target physiological signals of the target change more significantly than the EEG signals, the acquisition device can determine whether the target has received non-preset stimulation outside the preset stimulation application period by detecting whether the target physiological signals have changed suddenly, thereby ensuring a high degree of accuracy in the determination.

[0198] If the acquisition device determines that the target physiological signal does not suddenly change outside the application period, step 3 may be performed. If the acquisition device determines that the target physiological signal suddenly changes outside the application period, step 4 may be performed.

[0199] It can be understood that if the acquisition device determines that the difference between the signal value of the target physiological signal collected at the current sampling moment and the signal value of the target physiological signal collected at the previous sampling moment is greater than the second threshold, and the current sampling moment is outside the application period of the preset stimulus, it can be determined that the target physiological signal has undergone a mutation.

[0200] The second threshold value may be pre-stored by the acquisition device. The preset stimulation is used to apply a desired stimulation to the target object during the process of acquiring the EEG signal of the target object. The application period of the preset stimulation may be pre-stored by the acquisition device.

[0201] Step 3: Ensure that the target object is not exposed to any unintended stimulation.

[0202] If the acquisition device determines that the target physiological signal has not undergone a sudden change, it can be determined that the target object has not been subjected to non-preset stimulation outside the preset stimulation application period, and accordingly there is no need to correct the collected EEG signal.

[0203] It is understandable that when the acquisition device executes step 2 during the acquisition of multimodal physiological signals, if the acquisition device determines that the target physiological signal currently being acquired has not undergone a sudden change, step 2 may continue to be executed until the acquisition of multimodal physiological signals is completed.

[0204] In the case where the acquisition device performs step 2 after completing the multimodal physiological signal acquisition, if the acquisition device determines that the acquired target physiological signal has not undergone a sudden change, it can be determined that the target object has not been subjected to non-preset stimulation.

[0205] Step 4: Determine whether the target object is subjected to non-preset stimulation.

[0206] If the acquisition device determines that the target physiological signal has a sudden change, it can be determined that the target object has been subjected to a non-preset stimulus outside the preset stimulus application period, and then step five can be executed.

[0207] Step 205: Correct the EEG signal in the multimodal physiological signal.

[0208] The acquisition device can correct abnormal signals in the EEG signal to eliminate interference signals superimposed on the EEG signal due to the target being subjected to unintended stimulation, thereby achieving the effect of correcting the EEG signal baseline. The abnormal signal refers to the portion of the EEG signal collected from the target being subjected to the unintended stimulation until recovery.

[0209] In the embodiments of the present application, the EEG signal and the target physiological signal are collected synchronously. Therefore, when the target physiological signal undergoes a sudden change, the EEG signal will also change. Based on this, the acquisition device can determine that the portion of the EEG signal collected from the time the target physiological signal undergoes a sudden change to the time the target physiological signal recovers is an abnormal signal in the EEG signal.

[0210] That is, the start sampling time of the abnormal signal is the moment when the target physiological signal undergoes a sudden change, and the end sampling time is the moment when the target physiological signal recovers. Both the moment of sudden change and the moment of recovery occur outside the preset stimulus application period. The moment when the target physiological signal undergoes a sudden change is the sampling time at which the difference between the signal value and the signal value at the previous sampling time first exceeds a second threshold. The recovery time is the sampling time at which, starting from the moment of sudden change, the difference between the signal value and the signal value at the previous sampling time first becomes less than or equal to the second threshold.

[0211] In the embodiment of the present application, the EEG signal may include multiple signal values. The acquisition device may correct each abnormal signal value included in the abnormal signal in sequence according to the order of sampling moments.

[0212] It is understood that the acquisition device can correct abnormal signals in the EEG signals during the acquisition of multimodal physiological signals, that is, the acquisition device can correct abnormal signals in real time. In this case, the acquisition device can correct each abnormal signal value in the abnormal signal in the order of sampling time from earliest to latest.

[0213] Alternatively, the acquisition device can correct abnormal signals in the EEG signal after completing the acquisition of the multimodal physiological signal. In this case, the acquisition device can correct each abnormal signal value in the abnormal signal in the order of sampling time from early to late, or from late to early.

[0214] In an embodiment of the present application, for each abnormal signal value in the abnormal signal, the acquisition device can correct the abnormal signal value based on the first signal value set corresponding to the abnormal signal value. The first signal value set includes N signal values ​​with consecutive sampling moments. N is an integer greater than 1. The sampling moment of the target signal value among the N signal values ​​is adjacent to the sampling moment of the abnormal signal value. The target signal value is the signal value with the earliest sampling moment or the signal value with the latest sampling moment among the N signal values. That is, the sampling moments of the N signal values ​​are all earlier than the sampling moment of the abnormal signal value, or are all later than the sampling moment of the abnormal signal value.

[0215] The difference between the statistical value of the second signal value set corresponding to the corrected abnormal signal value and the statistical value of the first signal value set is less than a first threshold. The second signal value set includes: the corrected abnormal value and N-1 consecutive signal values ​​at sampling times in the first signal value set, where the N-1 signal values ​​include the target signal value. The first threshold may be pre-stored by the acquisition device.

[0216] It is understood that if the acquisition device corrects each abnormal signal value in order from earliest to latest sampling time, the target signal value may be the signal value with the latest sampling time in the first signal value set. If the acquisition device corrects each abnormal signal value in order from latest to earliest sampling time, the target signal value may be the signal value with the earliest sampling time in the first signal value set.

[0217] For example, assuming that the sampling moments of the signal values ​​included in the first signal value set are all earlier than a certain abnormal signal value, that is, the signal values ​​are located before the certain abnormal signal value, and the first signal value set includes: signal value 1, signal value 2, signal value 3, and signal value 4. Then, the second signal value set may include: signal value 2, signal value 3, signal value 4, and the abnormal signal value after correction.

[0218] Assume that the sampling time of each signal value included in the first signal value set is later than a certain abnormal signal value, that is, each signal value is located after the certain abnormal signal value, and the first signal value set includes: signal value 1, signal value 2, signal value 3, and signal value 4. Then, the second signal value set can include: the corrected abnormal signal value, signal value 1, signal value 2, and signal value 3.

[0219] It is understood that the statistical value may include at least one of a standard deviation and a mean. For example, the statistical value may be the standard deviation. If the statistical value includes the standard deviation and the mean, the difference between the standard deviation of the second signal value set and the standard deviation of the first signal value set may be less than the first threshold, and the difference between the mean of the second signal value set and the mean of the first signal value set may also be less than the first threshold.

[0220] Optionally, the mean may be an arithmetic mean, a geometric mean, or a root mean square.

[0221] Optionally, the N signal values ​​in the first signal value set may be a plurality of consecutive signal values ​​collected by the collection device within a preset time period. The preset time period may be pre-stored by the collection device, for example, 5 seconds (s).

[0222] It is understandable that the second signal value set corresponding to each corrected abnormal signal value can be used as the first signal value set corresponding to the next abnormal signal value to be corrected. That is, the corrected abnormal signal value can be used to correct the next abnormal signal value.

[0223] For example, assume that the abnormal signal includes: abnormal signal value 1 and abnormal signal value 2, arranged in order of sampling time from earliest to latest. The first signal value set corresponding to abnormal signal value 1 is: signal value 1, signal value 2, and signal value 3. Assuming that the sampling times of signal values ​​1 to 3 are all earlier than the sampling time of abnormal signal value 1, the second signal value set corresponding to abnormal signal value 1 can be signal value 2, signal value 3, and the corrected abnormal signal value 1. This second signal value set can be the first signal value set corresponding to abnormal signal value 2.

[0224] In the embodiment of the present application, the acquisition device can not only acquire the target physiological signal of the target object while acquiring the EEG signal of the target object, but also acquire the auxiliary physiological signal of the target object. The auxiliary physiological signal can include one of the following signals: electromyogram (EMG) signal, electrocardiogram (ECG) signal, electro-oculogram (EOG) signal and photoplethysmography (PPG) signal.

[0225] EMG signals are bioelectrical currents generated by the contraction of surface muscles. ECG signals are typically displayed as waveforms, typically consisting of the P wave, QRS wave, and T wave. The P wave represents atrial contraction, the QRS wave represents ventricular contraction, and the T wave represents ventricular relaxation.

[0226] Photoplethysmography (PPG) signals are detected using photoelectric technology and reflect changes in blood volume in peripheral blood vessels caused by cardiac activity. When light of a specific wavelength strikes the skin surface, the contraction and dilation of blood vessels with each heartbeat affect the transmission or reflection of the light. As the light passes through skin tissue and reflects back to the photoelectric receiver, it experiences a certain degree of attenuation. While other tissues (such as muscle, bone, and veins) absorb relatively constant light, arteries experience varying absorption due to the pulsation of blood. Therefore, when the optical signal is converted into an electrical signal, the resulting signal can be divided into a DC signal and an AC signal. The AC signal can reflect the characteristics of blood flow. It is understood that transmission and reflection vary depending on the test location; generally, transmission is used on the fingers, while reflection is more commonly used on the wrist.

[0227] As can be seen from the above description, the acquisition device can simultaneously monitor changes in multiple physiological signals, meaning it is a multimodal device capable of simultaneously acquiring multiple physiological signals. This allows the capture of physiological data from multiple angles, enabling a more comprehensive monitoring of the target's health status, ensuring the accuracy and reliability of the analyzed health status. This health status can include both physiological and psychological health. This psychological health can be reflected by emotions, stress, and other factors.

[0228] In the embodiments of the present application, the acquisition device not only simultaneously collects multiple physiological signals but also performs feature fusion processing on the collected physiological signals to obtain a fused signal. The acquisition device then processes this signal using a classification model to determine the health status of the target object. The classification model is pre-stored by the acquisition device.

[0229] It is understandable that the acquisition device can align and perform feature fusion processing on multiple physiological signals through the hardware circuit in the acquisition device and the embedded algorithm burned into the hardware circuit to generate a fused signal.

[0230] Before performing feature fusion processing on multiple physiological signals, the acquisition device can perform feature extraction on the collected physiological signals. For example, the acquisition device can extract at least one of the time domain features (such as mean and variance) and frequency domain features of the physiological signals. For example, the acquisition device can extract both the time domain features and the frequency domain features of the physiological signals. Feature extraction is the process of converting the original physiological signals into feature vectors.

[0231] It is understood that the acquisition device can process the physiological signal using a time domain feature extraction method (or a frequency domain feature extraction method) to extract the time domain features (or frequency domain features) of the physiological signal. Optionally, the time domain feature extraction method can include: wavelet transform and short-time Fourier transform. The frequency feature extraction method can include: discrete Fourier transform and power spectral density.

[0232] Optionally, the classification model may be obtained by training a plurality of sample data using a machine learning algorithm. The machine learning algorithm may be one of a support vector machine, an artificial neural network, and a decision tree. Each sample data may include: a sample physiological signal and a health status corresponding to the sample physiological signal.

[0233] It is understood that the acquisition device can also determine pulse oximeter oxygen saturation (SpO2), heart rate (HR), and blood pressure based on the PPG signal. The acquisition device can also determine blood pressure based on the ECG signal.

[0234] It is understandable that the order of the steps of the EEG signal correction method based on human intelligence provided in the embodiment of the present application can be appropriately adjusted, and the steps can be increased or decreased accordingly according to the situation. Any person skilled in the art who can easily think of a modified method within the technical scope disclosed in this application should be included in the scope of protection of this application, so it will not be repeated here.

[0235] In summary, the embodiment of the present application provides an EEG signal correction method based on human intelligence, and the acquisition device can acquire multimodal physiological signals, which include: EEG signals and target physiological signals other than EEG signals. And after the acquisition device determines that the target object is subjected to non-preset stimulation based on the target physiological signal, it can correct the EEG signal, and the similarity between the corrected EEG signal and the EEG signal collected when it is not subjected to non-preset stimulation is greater than the similarity threshold. It can be seen that the method provided in the embodiment of the present application can obtain the EEG signal used to accurately determine the health status of the target object without re-acquiring the EEG signal of the target object, thereby improving the acquisition efficiency of the EEG signal.

[0236] The present application provides an apparatus for correcting EEG signals based on human intelligence, which can be used to perform the method for correcting EEG signals based on human intelligence provided in the above method embodiment. The apparatus includes:

[0237] An acquisition module is used to acquire multimodal physiological signals of a target object, where the multimodal physiological signals include: EEG signals and target physiological signals other than EEG signals;

[0238] a correction module, configured to correct the EEG signal if it is determined based on the target physiological signal that the target object is subjected to non-preset stimulation;

[0239] The similarity between the corrected EEG signal and the reference EEG signal is greater than a similarity threshold, and the reference EEG signal is an EEG signal collected when the target object is not subjected to non-preset stimulation.

[0240] Optionally, the correction module can be used to:

[0241] Correct abnormal signals in EEG signals;

[0242] Among them, the abnormal signal is a part of the EEG signal collected during the period from the target object being subjected to non-preset stimulation to recovery.

[0243] Optionally, the EEG signal includes: multiple signal values. The correction module can be used to:

[0244] The abnormal signal values ​​included in the abnormal signal are corrected in sequence according to the order of sampling moments.

[0245] Optionally, the correction module can be used to:

[0246] For each abnormal signal value in the abnormal signal, correct the abnormal signal value based on the first signal value set corresponding to the abnormal signal value;

[0247] The first signal value set includes N signal values ​​with consecutive sampling times, the sampling time of the target signal value among the N signal values ​​is adjacent to the sampling time of the abnormal signal value, and the target signal value is the signal value with the earliest sampling time or the signal value with the latest sampling time among the N signal values;

[0248] The difference between the statistical value of the second signal value set corresponding to the corrected abnormal signal value and the statistical value of the first signal value set is less than the first threshold value. The second signal value set includes: the corrected abnormal value, and N-1 consecutive signal values ​​at the sampling moment in the first signal value set, and the N-1 signal values ​​include the target signal value.

[0249] Optionally, the statistical value includes at least one of a standard deviation and a mean.

[0250] Optionally, the device may further include:

[0251] The first determining module is configured to determine that the target object is subjected to a non-preset stimulus if it is determined that a sudden change occurs in the target physiological signal.

[0252] Optionally, the device may further include:

[0253] The first determination module is configured to determine that a sudden change has occurred in the target physiological signal if a difference between a signal value of the target physiological signal collected at a current sampling moment and a signal value of the target physiological signal collected at a previous sampling moment is greater than a second threshold, and the current sampling moment is outside a preset stimulation application period.

[0254] Optionally, the device may further include:

[0255] The third determination module is used to determine part of the EEG signals collected from the time when the target physiological signal suddenly occurs to the time when the target physiological signal recovers as abnormal signals.

[0256] Optionally, the target EEG signal is at least one of a skin temperature signal and a skin electrical activity signal.

[0257] In summary, the embodiment of the present application provides an EEG signal correction device based on human intelligence, which can collect multimodal physiological signals, including: EEG signals and target physiological signals other than EEG signals. And after the acquisition device determines that the target object is subjected to non-preset stimulation based on the target physiological signal, it can correct the EEG signal, and the similarity between the corrected EEG signal and the EEG signal collected when it is not subjected to the non-preset stimulation is greater than the similarity threshold. It can be seen that the device can obtain the EEG signal used to accurately determine the health status of the target object without re-collecting the EEG signal of the target object, thereby improving the efficiency of acquiring the EEG signal.

[0258] FIG8 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device can be the electronic device in Example 2 or the acquisition device in Example 3. Referring to FIG8 , the electronic device 400 includes: a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, such as via a bus 402. Optionally, the electronic device 400 may further include a transceiver 404. It should be noted that in actual applications, the number of transceivers 404 is not limited to one, and the structure of the acquisition device 400 does not constitute a limitation on the embodiments of the present application.

[0259] The processor 401 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 401 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0260] Bus 402 may include a path for transmitting information between the aforementioned components. Bus 402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, for example. Bus 402 may be divided into an address bus, a data bus, a control bus, and so on. For ease of illustration, FIG8 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0261] The memory 403 is used to store a computer program corresponding to the EEG signal correction method based on human intelligence according to the above-mentioned embodiment of the present application, and the computer program is controlled and executed by the processor 401. The processor 401 is used to execute the computer program stored in the memory 403 to implement the contents of the above-mentioned method embodiment 1, embodiment 2, or embodiment 3.

[0262] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it can implement the real-time artifact processing and feature extraction method provided in the above-mentioned embodiment 1, the EEG signal processing method based on human intelligence provided in the embodiment 2, or the EEG signal correction method based on human intelligence provided in the embodiment 3.

Claims

1. A method for real-time artifact processing and feature extraction of EEG signals, wherein: The method comprises: Receive the EEG signal data stream collected by EEG equipment in real time; The sliding window method is used to segment the EEG signal data stream in real time; Obtain parameter information of the EEG device including the number of channels and sampling rate, and perform corresponding filtering processing steps and artifact removal steps on the EEG signal data stream segments based on the parameter information; For one or more pre-selected output indicator matching feature extraction strategies, characteristic values ​​that meet the output indicators are extracted in real time from the EEG signal data stream segments that have undergone filtering processing steps and artifact removal steps according to the matching feature extraction strategies from angles including time domain, frequency domain, time-frequency domain and / or nonlinear analysis.

2. The method according to claim 1, wherein: Before receiving the EEG signal data stream and performing artifact processing and feature extraction, the method further includes: obtaining device information of the EEG device of each input EEG signal data stream, wherein the device information includes the number of channels and the sampling rate.

3. The method according to claim 2, wherein: The method also includes the step of obtaining user configuration for real-time artifact processing and feature extraction, wherein the user configuration includes a time window size of a sliding window, a filtering processing strategy, an artifact removal strategy, an eigenvalue output index, and an eigenvalue transmission support protocol selection.

4. The method according to claim 1, wherein: The supported protocols for transmitting the EEG signal data stream collected by the EEG device and the extracted feature values ​​include one or more of the TCP protocol, the UDP protocol, the wireless Bluetooth protocol, the MQTT protocol, the RS-232 / RS-485 protocol and the LSL protocol.

5. The method according to claim 1, wherein: The filtering process step includes one or more of low-pass filtering, high-pass filtering, band-pass filtering and notch filtering; The artifact removal step includes one or more of independent component analysis, grid-based natural language structure analysis, blind source signal separation and SVM artifact removal.

6. The method according to claim 5, wherein: In the EEG signal recognition anxiety state scenario, the energy values ​​of the alpha band, the theta band and the gamma band are used as output indicators, and the artifact removal steps used include independent component analysis and grid-based natural language structure analysis processing; In the scenario of collecting EEG signals to control prostheses, the characteristic values ​​of the event-related synchronization and desynchronization output indicators of the corresponding μ rhythm and β rhythm are extracted in real time from the EEG signal data stream segments that have undergone filtering processing and artifact removal steps in the time-frequency domain analysis perspective according to the matching feature extraction strategy.

7. The method according to claim 1, wherein: The step of extracting in real time from the EEG signal data stream segments that have undergone the filtering process and the artifact removal process to obtain feature values ​​that meet the output index in accordance with the matching feature extraction strategy from the perspectives including time domain, frequency domain, time-frequency domain and / or nonlinear analysis includes: From the perspective of time domain analysis, based on statistical algorithms or Hjorth algorithms, one or more of the mean, variance, standard deviation, kurtosis, skewness and autocorrelation coefficient are extracted in real time from the EEG signal data stream segments that have undergone filtering and artifact removal steps; From the perspective of frequency domain analysis, based on any one of the fast Fourier transform algorithm, the periodogram method, the Welch method, the multi-window method and the autoregressive model, the energy value and / or the power value are extracted in real time from the EEG signal data stream segments after the filtering processing step and the artifact removal step; From the perspective of time-frequency domain analysis, feature values ​​are extracted in real time from the EEG signal data stream segments that have undergone filtering and artifact removal steps based on short-time Fourier transform or continuous wavelet transform. From the perspective of nonlinear analysis, one or more of Shannon entropy, approximate entropy, sample entropy and sorting entropy are extracted in real time from the EEG signal data stream segments after filtering and artifact removal steps based on recursive variable analysis and complexity.

8. The method according to claim 1, wherein: The steps of filtering and removing artifacts for segmenting the EEG signal data stream based on parameter information include: When the number of EEG signal channels collected by the EEG device is less than a preset threshold, a pre-trained neural network model is used to segment the EEG signal data stream and perform appropriate filtering processing steps and artifact removal, wherein the neural network model is obtained by supervised learning and training using a large data scale of EEG signals containing different types of artifacts and / or different signal-to-noise ratios.

9. The method according to claim 1, wherein: The step of extracting in real time from the EEG signal data stream segments that have undergone the filtering process and the artifact removal process to obtain feature values ​​that meet the output index according to the matching feature extraction strategy from the perspectives including time domain, frequency domain, time-frequency domain and / or nonlinear analysis is performed based on a convolutional neural network model, and the convolutional neural network model includes a first feature extraction module, a second feature extraction module, and a third feature extraction module that are connected in sequence, and the first feature extraction module, the second feature extraction module, and the third feature extraction module all include a time convolution kernel for extracting time feature information, and the second feature extraction module also includes a space convolution kernel for extracting space feature information. The method includes: The EEG signal to be processed is input into the first feature extraction module, the second feature extraction module and the third feature extraction module which are connected in sequence, so as to obtain the temporal feature information and the spatial feature information of the EEG signal.

10. The method according to claim 9, wherein: The convolutional neural network model also includes a classification module, and the method also includes: The time characteristic information and the space characteristic information of the electroencephalogram signal to be processed are input into the classification module for classification processing to obtain the classification processing result of the electroencephalogram signal.

11. The method according to claim 10, wherein: The first feature extraction module includes at least one depth-separable convolutional layer, each of the depth-separable convolutional layers includes a first temporal convolution kernel for extracting temporal feature information; The step of inputting the to-be-processed EEG signal into the first feature extraction module, the second feature extraction module and the third feature extraction module connected in sequence comprises: After the EEG signal to be processed is input into the first feature extraction module, a first feature map outputted from the first feature extraction module is obtained, wherein the first feature map includes the time feature information extracted by the first time convolution kernel.

12. The method according to claim 11, wherein: The second feature extraction module includes at least one common convolution layer and at least one residual block connected in sequence, the at least one common convolution layer includes a spatial convolution kernel for extracting spatial feature information, and the at least one residual block includes a second temporal convolution kernel for extracting temporal feature information; The step of inputting the to-be-processed EEG signal into the first feature extraction module, the second feature extraction module and the third feature extraction module connected in sequence comprises: After the first feature map is input into the second feature extraction module, a second feature map output from the second feature extraction module is obtained, wherein the second feature map includes spatial feature information extracted by the spatial convolution kernel and first optimized temporal feature information extracted by the second temporal convolution kernel.

13. The method according to claim 12, wherein: The second feature extraction module includes a first residual block and a second residual block connected in sequence: The first residual block includes a first convolutional layer, a second convolutional layer, and a first skip connection connected in sequence, wherein the first skip connection is used to add input information of the first convolutional layer and output information of the second convolutional layer to serve as output information of the first residual block; The second residual block includes a third convolutional layer, a fourth convolutional layer, and a second skip connection connected in sequence, and the second skip connection is used to add input information of the third convolutional layer and output information of the fourth convolutional layer to serve as output information of the second residual block; The first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer all include the second temporal convolution kernel.

14. The method according to claim 13, wherein: The third feature extraction module includes at least one mixed hole convolution layer, each mixed hole convolution layer includes a third time convolution kernel for extracting time feature information; The step of inputting the to-be-processed EEG signal into the first feature extraction module, the second feature extraction module and the third feature extraction module connected in sequence comprises: After the second feature map is input into the third feature extraction module, a third feature map output from the third feature extraction module is obtained, and the third feature map includes the spatial feature information extracted by the spatial convolution kernel and the second optimized time feature information extracted by the third time convolution kernel.

15. The method according to claim 14, wherein: The third feature extraction module includes a first mixed dilated convolution layer and a second mixed dilated convolution layer connected in sequence, and the size of the third temporal convolution kernel is (1, c), where c is a positive integer greater than 1.

16. The method according to claim 1, wherein: Also includes: Collecting target physiological signals other than the EEG signal data stream; If it is determined based on the target physiological signal that the target object is subjected to non-preset stimulation, correcting the EEG signal data stream; Among them, the similarity between the corrected EEG signal data stream and the reference EEG signal is greater than a similarity threshold, and the reference EEG signal is an EEG signal collected when the target object is not subjected to the non-preset stimulation.

17. The method according to claim 16, wherein: Correcting the EEG signal data stream includes: Correcting abnormal signals in the EEG signal data stream; The abnormal signal is a portion of the EEG signal collected during the period from when the target object is subjected to non-preset stimulation to when it recovers.

18. The method according to claim 17, wherein: The EEG signal data stream includes: a plurality of signal values; and correcting the abnormal signal includes: According to the order of sampling time, each abnormal signal value included in the abnormal signal is corrected in sequence.

19. The method according to claim 18, wherein: Correcting each abnormal signal value included in the abnormal signal in sequence, including: For each abnormal signal value in the abnormal signal, correcting the abnormal signal value based on a first signal value set corresponding to the abnormal signal value; The first signal value set includes N signal values ​​with consecutive sampling times, the sampling time of the target signal value among the N signal values ​​is adjacent to the sampling time of the abnormal signal value, and the target signal value is the signal value with the earliest sampling time or the signal value with the latest sampling time among the N signal values; The difference between the statistical value of the second signal value set corresponding to the corrected abnormal signal value and the statistical value of the first signal value set is less than a first threshold, and the second signal value set includes: the corrected abnormal value, and N-1 signal values ​​consecutively at the sampling moment in the first signal value set, and the N-1 signal values ​​include the target signal value.

20. The method according to any one of claims 16 to 19, wherein: The method further comprises: If it is determined that the target physiological signal has a sudden change, it is determined that the target object has been subjected to non-preset stimulation.

21. The method according to claim 20, wherein: The method further comprises: If the difference between the signal value of the target physiological signal collected at the current sampling moment and the signal value of the target physiological signal collected at the previous sampling moment is greater than a second threshold, and the current sampling moment is outside the application period of the preset stimulus, it is determined that the target physiological signal has undergone a mutation.

22. A system for real-time artifact processing and feature extraction of EEG signals, wherein: The system includes: A user configuration module is used to receive one or more pre-selected output indicator matching feature extraction strategies and set output indicators, and obtain parameter information including the number of channels and sampling rate of the EEG device; A built-in processing module is used to perform a corresponding filtering processing step and an artifact removal step on the EEG signal data stream segment based on the parameter information, match the feature extraction strategy for one or more pre-selected output indicators, and extract the feature values ​​that meet the output indicators from the EEG signal data stream segment that has undergone the filtering processing step and the artifact removal step in real time according to the matched feature extraction strategy in the angles including time domain, frequency domain, time-frequency domain and / or nonlinear analysis; The data transmission module is used to receive the EEG signal data stream collected by the EEG device in real time, and transmit the characteristic values ​​that meet the output indicators extracted in real time.

23. A device for real-time artifact processing and feature extraction of EEG signals, comprising a processor and a memory, wherein: The memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the method as claimed in any one of claims 1 to 21.

24. A terminal device, wherein: The terminal device applies the steps of the method according to any one of claims 1 to 21.

25. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 21 are implemented.

Images