Head brain wave feature recognition method and computer program product

Abstract

The invention provides a head brain wave feature recognition method and a computer program product, relates to the field of biological feature recognition processing, and solves the problem that brain wave feature extraction and classification accuracy is affected due to the fact that the quality of brain wave signals is poor. Obtaining a processed signal to be processed; performing feature extraction to obtain feature information; based on a deep learning classifier, performing automatic classification according to the feature information, and identifying the brain waves to obtain a brain wave classification result; the step of obtaining the brain wave signals of the head comprises the substeps that at least two electrodes are used for collecting the brain wave signals in the head area, and the electrodes are distributed in the head area in the mode of combining partial symmetry and partial asymmetry. According to the scheme, the electrodes are designed in a symmetrical and asymmetrical combined mode, tiny differences of head areas of left and right brain hemispheres can be captured, functional activities of left and right frontal lobes are effectively distinguished, and the accuracy of brain wave feature extraction and classification is improved.

Description

Technical Field

[0001] The present invention relates to the field of biological feature recognition processing, and in particular to a head brain wave feature recognition method and a computer program product. Background Art

[0002] This section is intended to provide background or context for embodiments of the invention set forth in the claims. The descriptions herein may include concepts that may be explored, but not necessarily concepts that have been previously thought of or explored. Therefore, unless noted herein, the content described in this section is not prior art with respect to the specification and claims of this application, and is not admitted to be prior art by inclusion in this section.

[0003] With the rapid development of brain science and neurotechnology, brain (EEG, Electroencephalography) based signal recognition and processing technology has been widely used in medical treatment, neural regulation, brain-computer interface (BCI) and other fields. Different areas of the brain do not emit the same brain wave frequency at the same time. The EEG signal between the electrodes placed on the scalp consists of many waves with different characteristics. Even the large amount of data received from a single EEG recording makes interpretation difficult. Each person's brain wave pattern is unique.

[0004] Existing brain wave signal acquisition equipment usually requires electrodes to be placed in multiple areas of the head. However, traditional EEG acquisition methods face problems such as inconvenient electrode placement and large signal noise interference. Especially in portable and non-invasive devices, how to effectively collect and process EEG signals from the head area and realize feature extraction and classification is a technical problem that needs to be solved urgently.

[0005] Traditional EEG acquisition technology relies on a large number of electrodes covering the entire scalp to ensure that signals from different brain regions can be collected. This type of method is not only complicated, but also has a great impact on the user's comfort and convenience. Especially in non-laboratory environments, such as portable devices or mobile environments, traditional multi-electrode helmet-type devices are bulky and susceptible to external interference.

[0006] The head is a relatively smooth surface, suitable for the stable placement of electrodes and easy to use. However, the EEG signals collected from the head are usually weak and easily interfered by other bioelectric signals (such as electrooculography, electromyography, etc.), resulting in poor signal quality and affecting the accuracy of EEG feature extraction and classification. For this problem, the existing technology lacks an effective solution. Summary of the invention

[0007] To solve the problem that the quality of electroencephalogram (EEG) signals in the prior art is poor, which affects the accuracy of brain wave feature extraction and classification, the purpose of the present invention is to provide a method and application for feature recognition of head brain waves and a computer program product.

[0008] To solve the above technical problems, in a first aspect, according to some embodiments, the present invention provides a method for feature recognition of head brain waves, including:

[0009] Obtain the brain wave signal of the head, preprocess the brain wave signal to obtain a processed signal to be processed;

[0010] Extract features from the signal to be processed to obtain feature information;

[0011] Based on a deep learning classifier, automatically classify according to the feature information, identify the brain waves, and obtain a brain wave classification result;

[0012] Interact with the user based on the brain wave classification result;

[0013] Among them, the obtaining of the brain wave signal of the head specifically includes:

[0014] Collect brain wave signals in the head area using at least two electrodes, where the electrodes are distributed in the head area in a combination of partially symmetric and partially asymmetric manners.

[0015] In some embodiments, the preprocessing of the brain wave signal to obtain a processed signal to be processed specifically includes:

[0016] Process the brain wave signal using a band-pass filter, and the filtering range is 1 Hz - 40 Hz;

[0017] Utilize the adaptive blind source separation (BSS) technology, combined with the independent component analysis (ICA) algorithm, to effectively separate and eliminate the artifact signals in the brain wave signal;

[0018] Normalize the brain wave signal to adjust the amplitude of the brain wave signal to a unified range.

[0019] In some embodiments, the extracting of features from the signal to be processed to obtain feature information specifically includes:

[0020] Perform time-domain feature extraction and frequency-domain feature extraction on the signal to be processed respectively;

[0021] Analyze the extracted time-domain feature signal and frequency-domain feature signal to obtain the analyzed feature information.

[0022] In some embodiments, the time-domain feature extraction specifically includes:

[0023] Extract the mean and standard deviation features by calculating the overall trend and fluctuation amplitude of the brain wave signal;

[0024] Extract the autocorrelation feature of the brain wave signal to measure the degree of self-similarity of the signal in time;

[0025] Extract the peak and energy of the brain wave signal according to the peak and energy of the brain wave signal within a specific time window.

[0026] In some embodiments, the frequency domain feature extraction specifically includes:

[0027] Convert the brain wave signal to the frequency domain through fast Fourier transform (FFT);

[0028] Analyze the energy of different frequency components and extract the features of the α-wave band, β-wave band, θ-wave band, and δ-wave band.

[0029] In some embodiments, the analysis of the extracted time domain feature signal and frequency domain feature signal to obtain the analyzed feature information specifically includes:

[0030] Perform wavelet transform and empirical mode decomposition (EMD) on the brain wave signal for time-frequency domain analysis of the signal;

[0031] The wavelet transform specifically includes: decomposing the brain wave signal on multiple scales and extracting the local time and frequency features of the brain wave signal;

[0032] The empirical mode decomposition (EMD) specifically includes: decomposing the original signal of the brain wave into multiple intrinsic mode functions (IMFs) and extracting the essential frequency and fluctuation mode of the brain wave signal.

[0033] In some embodiments, the deep learning-based classifier automatically classifies according to the feature information, specifically including:

[0034] The deep learning model structure includes: a convolutional neural network and a recurrent neural network. The convolutional neural network is used to extract the spatial features in the brain wave signal, and the recurrent neural network is used to capture the correlation in the brain wave time series;

[0035] The convolutional neural network is used to extract the spatial features in the signal, specifically including:

[0036] Spatial feature extraction: Perform spatial transformation through common spatial patterns to obtain a feature matrix;

[0037] Construct a CNN network: It includes multiple convolutional layers and fully connected layers. The input of the CNN network is a feature matrix transformed by CSP, or the spatio-temporal dimension of the EEG signal is added to the 2D EEG topographic map;

[0038] Feature learning: The CNN learns the feature matrix through the network and automatically extracts spatial features through the convolutional layers;

[0039] Weight analysis and feature screening: By analyzing the weight matrix of the fully connected layer of the CNN network, define the screening criteria for the CSP feature matrix to obtain a dimension-reduced and efficient EEG feature set F;

[0040] Construct a classifier: Construct a new CNN classifier according to the extracted feature set F. The CNN classifier is used for the final EEG signal classification task.

[0041] Training and optimization: Use the stochastic gradient descent optimization method to train the CNN network. By calculating the negative gradient and updating the weights, optimize the network performance;

[0042] Classification: Use the trained CNN classifier to classify new EEG signals and identify different brain wave patterns;

[0043] Performance evaluation: Evaluate the performance and accuracy of the CNN classifier in the EEG signal classification by comparing with the baseline method or other state-of-the-art methods.

[0044] In some embodiments, the time-domain feature extraction specifically includes:

[0045] Feature extraction: Extract features from the EEG signal to obtain valuable information for classification from the redundant information. The valuable information includes time-domain features, and the time-domain features include kurtosis coefficient;

[0046] Construct an RNN model: Create an RNN network that includes at least 100 recurrent neurons; each input contains a feature value;

[0047] Serialize the input: Serialize the time series data of the EEG signal. Each training sample contains multiple consecutive EEG signal values, and the target is a sequence that is shifted by one time step relative to the training sample;

[0048] Train the RNN: Use dynamic RNN units to process the input sequence and perform training. During the training process, the RNN model learns how to predict the output of the next time step based on the current and previous inputs;

[0049] Classification: Classify the extracted features to determine the degree of association between the obtained brain wave signal state and the classification match. Among them, the RNN model classifies or predicts the EEG signals at each time step to identify different electroencephalogram activity states or detect abnormal events;

[0050] Attention mechanism: Helps the encoder calculate by generating weight sequences of different dimensions to obtain intermediate encodings;

[0051] Post - processing and evaluation: Input the features extracted by the RNN model into a classifier to identify the category of the EEG data, where each category corresponds to a specific brain activity;

[0052] Evaluate the performance of the model: Evaluate the model so that the recognition accuracy can reach the preset requirements and the robustness of the algorithm framework under the premise of real - time data processing.

[0053] In some embodiments, interacting with the user based on the brain wave classification results specifically includes:

[0054] When it is determined to be a preset state according to the classification result, trigger the corresponding feedback mechanism according to the preset rules corresponding to the preset state; and / or,

[0055] Optimize the feedback rules and provide personalized suggestions according to the matching situation between the user's brain wave information and the user's behavior.

[0056] In a second aspect, an embodiment of the present invention further provides a computer program product, including a computer program, where the computer program is stored in a computer - readable storage medium; when a processor of an electronic device reads the computer program from the computer - readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of the method according to any one of the above - mentioned first aspects.

[0057] The above technical solution of the present invention has at least the following beneficial technical effects: In the solution of the embodiment of the present invention, feature recognition is performed through head brain waves, and a brain wave classification result is obtained and interacted with the user. Although the brain wave signals in the head area are relatively weak, they contain rich prefrontal information, especially activities related to emotions, cognition, and attention. The present invention uses a specially designed electrode array to optimize signal acquisition through a special structural layout, that is, at least two electrodes are used to collect brain wave signals in the head area, and the electrodes are distributed in the head area in a combination of partial symmetry and partial asymmetry, which can maximize the signal strength in the head area. Moreover, the electrode layout adopts a combined design of symmetry and asymmetry, enabling it to capture the subtle differences in the head areas of the left and right cerebral hemispheres, thereby providing a more accurate spatial resolution in emotional cognitive tasks. Compared with the traditional single-symmetric electrode layout, the present application adopts a combination of partial symmetry and partial asymmetry, which can effectively distinguish the functional activities of the left and right prefrontal lobes. Moreover, the solution of the present application can be applied to portable devices or mobile environments, and is more portable and has better signal quality and better effects compared with the traditional multi-electrode helmet devices that are bulky and vulnerable to external interference. Moreover, the present application preprocesses the brain wave signals, eliminates the problem of poor signal quality caused by the interference of other bioelectric signals (such as electrooculogram, electromyogram, etc.), improves the accuracy of brain wave feature extraction and classification, effectively collects and processes electroencephalogram signals from the head area, and realizes feature extraction and classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0059] Figure 1 It is a schematic diagram of the electrode layout of a head brain wave signal acquisition device provided by an embodiment of the present invention.

[0060] Figure 2 It is a flowchart of a method for feature recognition of head brain waves provided by an embodiment of the present invention;

[0061] Figure 3 It is a schematic block diagram of a hardware device provided by an embodiment of the present invention;

[0062] Figure 4 It is a schematic diagram of the overall structure of a wearable brain wave acquisition device provided by an embodiment of the present invention;

[0063] Figure 5 It is a schematic block diagram of a signal processing module provided by an embodiment of the present invention;

[0064] Among them, Figures 1 to 5 the corresponding relationship between the reference numerals in the drawings and the component names is as follows:

[0065] 1 - First electrode region, 2 - Second electrode region, 3 - Non - zero electrode;

[0066] 11 - First reference electrode, 21 - Second reference electrode. Detailed implementation manners

[0067] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0068] In addition, in the following description, the descriptions of well - known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present invention.

[0069] It should be noted that the sequence numbers mentioned in this application do not necessarily represent that they must be strictly executed in the order of the sequence numbers in the actual specific implementation process. The sequence numbers are used to distinguish each step for the convenience of description to prevent confusion.

[0070] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0071] Figure 1 Shows the symmetric and asymmetric layout designs of the electrodes in the head in the present invention.

[0072] The first electrode region 1 (within the dashed box) and the second electrode region 2 (within the dashed box) are respectively located in the prefrontal regions on the left and right sides of the head, and are used to capture the brain wave signals of the left and right hemispheres. Specifically, the first electrode region 1 is used to capture the brain wave signals of the region of the left hemisphere of the brain, and the second electrode region 2 is used to capture the brain wave signals of the region of the right hemisphere of the brain, or the first electrode region 1 is used to capture the brain wave signals of the region of the right hemisphere of the brain, and the second electrode region 2 is used to capture the brain wave signals of the region of the left hemisphere of the brain, which can be adjusted according to actual needs. Among them, the electrode arrangements in the first electrode region 1 and the second electrode region 2 are partially symmetric and partially asymmetric.

[0073] The non - zero electrode 3 is located at the center of the head and is used to correct the electroencephalogram signals.

[0074] Furthermore, the first electrode region 1 and the second electrode region 2 may also have reference electrodes, specifically as Figure 1As shown, the first reference electrode 11 and the second reference electrode 21 are respectively located on the earlobe or other body parts to provide a stable reference signal.

[0075] The present invention adopts an integrated wearable device, which includes multiple electrodes for collecting brain wave signals in the head area. These electrodes can have the following characteristics:

[0076] Flexible material electrodes: To improve wearing comfort, the electrodes are made of a flexible material with strong conductivity, which can closely adhere to the head surface and adapt to the bending and changes of the skin, reducing the contact resistance. And / or,

[0077] Asymmetric and symmetric combined electrode layout: The electrode array is distributed on the left and right sides of the head, slightly biased towards the frontal lobe. This asymmetric and symmetric layout design enables the electrodes to capture the differential activities of the left and right brains, and is particularly suitable for the detection of functions such as cognition, emotion, and attention. And / or,

[0078] Number of electrodes: At least two to five electrodes are used for signal acquisition, and if necessary, it can also be increased to four or six to improve the signal spatial resolution. Each electrode corresponds to a different signal channel to ensure the diversity and comprehensiveness of data acquisition.

[0079] For the signal acquisition circuit, the following method can be adopted:

[0080] Low-noise amplifier: The electroencephalogram (EEG) signal voltage is very weak, usually at the microvolt level. To ensure the signal strength, the present invention uses a low-noise amplifier to preliminarily amplify the signal. The gain range of the amplifier is set to more than 10,000 times, which can ensure that the signal strength can be effectively recognized by the subsequent processing system.

[0081] High sampling rate: The acquisition circuit uses an ADC (analog-to-digital converter) with a high sampling rate (at least 500 Hz) to ensure that the details of the rapidly changing EEG signals can be captured and meet the requirements of subsequent frequency analysis.

[0082] As Figure 3 shown, the hardware device 300 of the present invention consists of the following parts:

[0083] Wearable electrode device 301: It includes signal acquisition electrodes and flexible circuits.

[0084] Embedded processing unit 302: Real-time processes and analyzes the collected brain wave signals.

[0085] Wireless transmission module 303: Transmits the brain wave signals and processing results to the intelligent terminal, supporting mobile applications and remote monitoring.

[0086] This application can be integrated into a wearable brain wave acquisition device, such as Figure 4As shown, the overall structure 400 of the wearable device in the present invention can be as follows:

[0087] Head electrode array 401: The electrodes are embedded in a flexible headband and can firmly adhere to the head skin.

[0088] Processing unit 402: It has a built-in signal processor for real-time processing of the collected brain wave signals.

[0089] Wireless module 403: Supports Bluetooth or Wi-Fi connections to transmit the processed signals to external intelligent devices.

[0090] Power supply unit 404: Provides the required power for the device, usually using a lightweight rechargeable battery.

[0091] As Figure 5 shown, the signal processing module 500 of the present invention mainly includes the following parts:

[0092] Signal acquisition module 501: Obtains the original brain wave signals from the electrode sensors.

[0093] Signal preprocessing module 502: Includes filters, BSS and ICA artifact removal, and signal normalization.

[0094] Filter: Used to remove 50Hz power frequency noise and other high-frequency and low-frequency interferences.

[0095] BSS and ICA artifact removal: Applies adaptive blind source separation (BSS) and independent component analysis (ICA) to remove electrooculogram and electromyogram artifacts.

[0096] Signal normalization: Performs normalization processing on the artifact-removed signals.

[0097] Feature extraction module 503: Time domain features: Extracts statistical quantities such as mean and variance. Frequency domain features: Uses fast Fourier transform (FFT) to extract frequency band features such as α, β, and θ. Time-frequency domain features: Combines wavelet transform and EMD for time-frequency domain analysis.

[0098] Classification module 504: Inputs the extracted features into deep learning classifiers (CNN and RNN) for classification.

[0099] Feedback module 505: Provides real-time feedback according to the classification results.

[0100] The specific implementation manner of the present invention is carried out around five key steps: collection, preprocessing, feature extraction, classification, and feedback of head brain wave signals, and combines improved hardware design and advanced algorithm models to achieve efficient head brain wave feature recognition. The following is a detailed description.

[0101] A method for feature recognition of head brain waves, as Figure 2As shown in the figure, it includes:

[0102] S201: Obtain the brain wave signal of the head, preprocess the brain wave signal, and obtain the processed signal to be processed;

[0103] Among them, the obtaining of the brain wave signal of the head specifically includes:

[0104] Collect the brain wave signal in the head area using at least two electrodes, where the electrodes are distributed in the head area in a manner combining partial symmetry and partial asymmetry.

[0105] The hardware support required for obtaining the signal can refer to the above solution, and can be obtained through electrodes or other means.

[0106] The method of preprocessing is not limited here. It mainly denoises the collected signal and eliminates interference signals.

[0107] S202: Extract features from the signal to be processed to obtain feature information;

[0108] Extract features in the time domain and frequency domain, or specific feature information can be extracted according to actual needs.

[0109] S203: Based on a deep learning classifier, automatically classify according to the feature information, identify the brain wave, and obtain a brain wave classification result;

[0110] Use a deep learning classifier for classification. The deep learning classifier is a trained model, and the feature information is classified according to the model to obtain the corresponding result

[0111] S204: Interact with the user based on the brain wave classification result.

[0112] Specifically, the interaction process can be: If the user is in a certain specific state, such as fatigue, concentration, anxiety, etc., the device generates feedback mechanisms with different effects triggered according to preset rules, such as sound prompts, music push, vibration alarms, visual signals, etc., or provides personalized suggestions based on the matching situation between the user's previous brain wave data and their actual behavior, such as training to improve concentration.

[0113] The solution of the embodiment of the present invention performs feature recognition through head brain waves, obtains the brain wave classification result and interacts with the user. Although the brain wave signal in the head area is relatively weak, it contains rich prefrontal information, especially activities related to emotion, cognition, and attention. The present invention uses a specially designed electrode array to optimize signal acquisition through a special structural layout, that is, at least two electrodes are used to collect brain wave signals in the head area, and the electrodes are distributed in the head area in a combination of partial symmetry and partial asymmetry, which can maximize the signal strength in the head area. Moreover, the electrode layout adopts a combined design of symmetry and asymmetry, enabling it to capture the minute differences in the head areas of the left and right cerebral hemispheres, thereby providing a more accurate spatial resolution in emotional cognitive tasks. Compared with the traditional single symmetric electrode layout, the present application adopts a combination of partial symmetry and partial asymmetry, which can effectively distinguish the functional activities of the left and right prefrontal lobes. Moreover, the solution of the present application can be applied to portable devices or mobile environments, and is more lightweight and has better signal quality and better effects compared with the traditional multi-electrode helmet devices that are bulky and vulnerable to external interference. Moreover, the present application preprocesses the brain wave signal to eliminate the problem of poor signal quality caused by the interference of other bioelectric signals (such as electrooculogram, electromyogram, etc.), improves the accuracy of brain wave feature extraction and classification, effectively collects and processes electroencephalogram signals from the head area, and realizes feature extraction and classification.

[0114] For the specific arrangement of the electrodes, reference can be made to Figure 1 as shown, and the above embodiments, and no further examples will be given here.

[0115] In the solution of the present invention, although the brain wave signal in the head area is relatively weak, it contains rich prefrontal information, especially activities related to emotion, cognition, and attention. The present invention uses a specially designed electrode array to optimize signal acquisition through a special structural layout, a signal acquisition design with local feature enhancement, to maximize the signal strength in the head area. The electrode layout adopts a combined design of symmetry and asymmetry, enabling it to capture the minute differences in the head areas of the left and right hemispheres, thereby providing a more accurate spatial resolution in emotional cognitive tasks. Different from the traditional single symmetric electrode layout, this combined design can effectively distinguish the functional activities of the left and right prefrontal lobes.

[0116] Optionally, as one of the embodiments, the preprocessing of the brain wave signal to obtain the processed signal to be processed specifically includes:

[0117] Processing the brain wave signal using a band-pass filter, and the filtering range is 1 Hz - 40 Hz;

[0118] Using the adaptive blind source separation (BSS) technology, combined with the independent component analysis (ICA) algorithm, to effectively separate and eliminate the artifact signals in the brain wave signal;

[0119] Perform normalization processing on the brain wave signal to adjust the amplitude of the brain wave signal to a unified range.

[0120] The following is a detailed description.

[0121] In the collected original EEG signals, there are usually a large number of bioelectric interference signals mixed in (such as eye movement signals, muscle activity signals, etc.). Therefore, preprocessing is a key step to improve the accuracy of subsequent feature extraction and recognition.

[0122] First, use a band-pass filter to process the collected signals, and set the filtering range to 1 - 40 Hz. This frequency band covers most of the EEG frequencies related to cognition, attention, and emotion, and at the same time removes high-frequency noise (such as 50 Hz power frequency noise) and low-frequency interference signals (such as skin electrical activity).

[0123] The present invention also adopts a unique adaptive blind source separation (BSS) technology, combined with the independent component analysis (ICA) algorithm, to effectively separate the artifact signals mixed in the EEG signals. The specific steps are as follows:

[0124] BSS algorithm: By dynamically identifying the characteristics of each source in the original signal, the BSS method can separate signals from different biological sources (such as electrooculogram, electromyogram) from the real EEG signals, and no prior knowledge is required, which is applicable to different individuals and environments.

[0125] ICA algorithm: Independent component analysis (ICA) further extracts the independent components of the signal, making the contribution of a specific signal source more significant, thereby effectively removing the artifact components. Compared with the traditional fixed filtering method, ICA can adaptively adjust and eliminate the artifacts generated under different individuals or environments.

[0126] To ensure the comparability of signals under different users and different environments, the EEG signals after denoising processing need to be normalized. By means of normalization (such as Z-score), the amplitude of the signal is adjusted to a unified range for subsequent feature extraction and classification.

[0127] The present invention proposes an artifact cancellation algorithm based on adaptive blind source separation (BSS). This algorithm can effectively separate artifacts (such as eye movement and muscle activity signals) and effective brain wave signals in the head brain wave signals. Different from the existing artifact cancellation techniques based on fixed threshold filters, the BSS method realizes the adaptive removal of artifact signals under different individuals and different environments by dynamically adjusting signal components, improving the purity of the signals. Combining ICA (independent component analysis) with an adaptive filter can self-adjust parameters according to signal characteristics during real-time processing. Especially in a mobile environment, such as when the wearer is walking or moving, it can greatly reduce the influence of motion artifacts on the signals.

[0128] Optionally, as one of the embodiments, the extracting feature information from the signal to be processed specifically includes:

[0129] Performing time-domain feature extraction and frequency-domain feature extraction on the signal to be processed respectively;

[0130] Analyzing the extracted time-domain feature signal and frequency-domain feature signal to obtain the analyzed feature information.

[0131] Optionally, as one of the embodiments, the time-domain feature extraction specifically includes:

[0132] Extracting the mean and standard deviation features by calculating the overall trend and fluctuation amplitude of the brain wave signal;

[0133] Extracting the autocorrelation feature of the brain wave signal to measure the self-similarity degree of the signal in time;

[0134] Extracting the peak value and energy of the brain wave signal according to the peak value and energy of the brain wave signal within a specific time window.

[0135] Optionally, as one of the embodiments, the frequency-domain feature extraction specifically includes:

[0136] Converting the brain wave signal to the frequency domain through fast Fourier transform (FFT);

[0137] Analyzing the energy of different frequency components and extracting the features of the α-wave frequency band, β-wave frequency band, θ-wave frequency band, and δ-wave frequency band.

[0138] Optionally, as one of the embodiments, the analyzing the extracted time-domain feature signal and frequency-domain feature signal to obtain the analyzed feature information specifically includes:

[0139] Performing wavelet transform and empirical mode decomposition (EMD) on the brain wave signal for time-frequency domain analysis of the signal;

[0140] The wavelet transform specifically includes: decomposing the brain wave signal at multiple scales and extracting the local time and frequency characteristics of the brain wave signal;

[0141] The empirical mode decomposition (EMD) specifically includes: decomposing the original signal of the brain wave into multiple intrinsic mode functions (IMFs) and extracting the essential frequency and fluctuation mode of the brain wave signal.

[0142] The following is a detailed description.

[0143] In feature extraction, the preprocessed signal contains various useful frequency components and time variation information.

[0144] To extract features that can reflect the user's mental state or behavior from it, the present invention adopts the following method:

[0145] Time-domain feature extraction, the following common features can be extracted in the time domain:

[0146] Mean and standard deviation: Calculate the overall trend and fluctuation amplitude of the signal.

[0147] Autocorrelation: Measure the self-similarity degree of the signal in time and reflect the periodic characteristics of the signal.

[0148] Peak value and energy: The peak value and energy of the signal within a specific time window, used to identify the peak of short-term brain activity.

[0149] In frequency-domain feature extraction, the signal is transformed into the frequency domain through the fast Fourier transform (FFT) to analyze the energy of different frequency components.

[0150] Particularly extract the features of the following frequency bands:

[0151] Alpha wave (8 - 13 Hz): Related to the relaxation and meditation states.

[0152] Beta wave (13 - 30 Hz): Related to the focused and tense states.

[0153] Theta wave (4 - 8 Hz): Related to the memory and intuitive states.

[0154] Delta wave (1 - 4 Hz): Usually related to the deep sleep or unconscious states.

[0155] For time-frequency domain feature extraction, the present invention combines the wavelet transform and the empirical mode decomposition (EMD) to perform time-frequency domain analysis on the signal:

[0156] Wavelet transform: Decompose the signal at multiple scales and extract the local time and frequency characteristics of the signal, which is suitable for capturing short-term brain wave changes, such as mood fluctuations or sudden cognitive transitions.

[0157] EMD method: By decomposing the original signal into multiple Intrinsic Mode Functions (IMFs), the essential frequencies and fluctuation patterns of the signal are extracted, which is particularly suitable for the analysis of complex and multi-level brain wave signals.

[0158] In the embodiments of the present invention, in order to better capture the spatio-temporal characteristics of brain wave signals in the head region, the present invention adopts a multi-scale analysis method. By combining wavelet transform and EMD (Empirical Mode Decomposition) technology, the multi-level characteristics of the signal can be decomposed and extracted, and the weak fluctuations hidden in the EEG signal can be captured. This method not only extracts traditional frequency components such as alpha waves and beta waves, but also excavates potential characteristics in the low-frequency band through a fine-grained decomposition process, such as the performance of slow waves (delta waves) and theta waves in different mental states. This is very important for applications such as concentration monitoring and emotion recognition.

[0159] Optionally, as one of the embodiments, the deep learning-based classifier automatically classifies according to the feature information, specifically including:

[0160] The deep learning model structure includes: a convolutional neural network and a recurrent neural network. The convolutional neural network is used to extract the spatial features in the brain wave signal, and the recurrent neural network is used to capture the correlations in the brain wave time series;

[0161] The convolutional neural network is used to extract the spatial features in the signal, specifically including:

[0162] Spatial feature extraction: Through common spatial patterns for spatial transformation, a feature matrix is obtained;

[0163] Constructing a CNN network: It includes multiple convolutional layers and fully connected layers. The input of the CNN network is the feature matrix after CSP transformation, and the spatio-temporal dimensions of the EEG signal are added to the 2D EEG topographic map;

[0164] Feature learning: The CNN learns the feature matrix through the network and automatically extracts spatial features through the convolutional layers;

[0165] Weight analysis and feature screening: By analyzing the weight matrix of the fully connected layer of the CNN network, a screening criterion for the CSP feature matrix is defined to obtain a dimension-reduced and efficient EEG feature set F;

[0166] Constructing a classifier: A new CNN classifier is constructed according to the extracted feature set F, and the CNN classifier is used for the final EEG signal classification task.

[0167] Training and optimization: Use the stochastic gradient descent optimization method to train the CNN network, and optimize the network performance by calculating the negative gradient and updating the weights;

[0168] Classification: Classify new EEG signals using the trained CNN classifier to identify different brain wave patterns;

[0169] Performance evaluation: Evaluate the performance and accuracy of the CNN classifier in the EEG signal classification by comparing it with baseline methods or other state-of-the-art methods.

[0170] Optionally, as one of the embodiments, the time-domain feature extraction specifically includes:

[0171] Feature extraction: Extract features from the EEG signals to obtain valuable information for classification from redundant information. The valuable information includes time-domain features, and the time-domain features include kurtosis coefficients;

[0172] Construct an RNN model: Create an RNN network with at least 100 recurrent neurons; each input contains a feature value;

[0173] Serialize the input: Serialize the time series data of the EEG signals. Each training sample contains a continuous plurality of the EEG signal values, and the target is a sequence that is shifted by one time step relative to the training sample;

[0174] Train the RNN: Use dynamic RNN cells to process the input sequence and perform training. During the training process, the RNN model learns how to predict the output of the next time step based on the current and previous inputs;

[0175] Classification: Classify the extracted features to judge the degree of association between the obtained brain wave signal state and the classification match. The RNN model classifies or predicts the EEG signals at each time step to identify different electroencephalogram activity states or detect abnormal events;

[0176] Attention mechanism: Help the encoder calculate by generating weight sequences of different dimensions to obtain intermediate encodings;

[0177] Post-processing and evaluation: Input the features extracted by the RNN model into a classifier to identify the category of the EEG data, where each category corresponds to a specific brain activity;

[0178] Evaluate the performance of the model: Evaluate the model so that the recognition accuracy can reach the preset requirements and the robustness of the algorithm framework under the premise of real-time data processing.

[0179] The following is a detailed description.

[0180] The present invention uses a deep learning-based classifier to automatically classify the extracted features. The model structure includes:

[0181] Convolutional Neural Network (CNN): Used to extract spatial features in signals. By moving the convolutional kernel, it can automatically identify specific patterns in the brain wave signals of the head, such as brain wave patterns related to concentration and emotional changes.

[0182] Recurrent Neural Network (RNN): Used to capture correlations in time series, especially suitable for processing EEG signals with temporal correlations, such as sustained concentration and emotional changes.

[0183] During the brain wave recognition process, the process of identifying brain wave signals from spatial features can be as follows:

[0184] Spatial feature extraction: Through methods such as Common Spatial Pattern (CSP: an abbreviation for Common Spatial Pattern, which is one of the widely used feature extraction algorithms in the field of Brain-Computer Interface (BCI)), perform spatial transformation to obtain a feature matrix. CSP is an effective spatial filter for extracting sensorimotor rhythms in EEG signals and can be used for real-time brain-computer interfaces.

[0185] Construct a CNN network: Construct a CNN network that may contain multiple convolutional layers and fully connected layers. The input to the network is the feature matrix transformed by CSP, or the spatio-temporal dimensions of the EEG signal are added to the 2D EEG topographic map.

[0186] Feature learning: The CNN learns the feature matrix through the network and automatically extracts spatial features through convolutional layers. The convolutional layer can capture local spatial patterns in the EEG signal, while the fully connected layer is used to integrate these features for classification.

[0187] Weight analysis and feature screening: By analyzing the weight matrix of the fully connected layer of the CNN network, define the screening criteria for the CSP feature matrix to obtain a reduced-dimensional and efficient EEG feature set F.

[0188] Construct a classifier: Construct a new CNN classifier based on the extracted feature set F. This classifier will be used for the final EEG signal classification task.

[0189] Training and optimization: Use optimization methods such as Stochastic Gradient Descent (SGD) to train the CNN network, and optimize the network performance by calculating the negative gradient and updating the weights.

[0190] Classification: Use the trained CNN classifier to classify new EEG signals and identify different brain wave patterns, such as signals under different thinking tasks.

[0191] Performance evaluation: Evaluate the performance and accuracy of CNN in EEG signal classification by comparing it with baseline methods or other state-of-the-art methods.

[0192] During the brain wave recognition process, the process of recognizing brain wave signals in temporal features can be referred to as follows:

[0193] Feature extraction: Extract features from EEG signals to obtain valuable information for classification from redundant information. Here, it mainly includes temporal features (such as kurtosis coefficient).

[0194] Construct an RNN model: Create an RNN network, which includes a certain number of recurrent neurons. For example, an RNN network with 100 recurrent neurons can unfold the training samples into multiple time segments, and each input contains a feature value (the value at that moment).

[0195] Serialize the input: Serialize the time series data of EEG signals. Each training sample may contain multiple consecutive EEG signal values, and the target is a sequence that is shifted by one time step relative to the training sample.

[0196] Train the RNN: Use dynamic RNN cells (such as tf.nn.dynamic_rnn) to process the input sequence and perform training. During the training process, the RNN learns how to predict the output of the next time step based on the current and previous inputs.

[0197] Classification: Classify the extracted features to judge the degree of association between the obtained brain wave state and the classification match. The RNN can classify or predict the EEG signals at each time step to identify different brain electrical activity states or detect abnormal events.

[0198] Attention mechanism: In some advanced applications, an attention mechanism may be introduced to help the RNN model better identify and focus on the key features in EEG signals. The attention mechanism helps the encoder calculate better intermediate encodings by generating weight sequences of different dimensions.

[0199] Post-processing and evaluation: Input the features extracted by the RNN model into a classifier (such as XGBoost) to identify the categories of EEG data. Each category corresponds to a specific brain activity. Evaluate the performance of the model to ensure that the recognition accuracy can reach a high level under the premise of real-time data processing, and ensure the robustness of the algorithm framework.

[0200] Through continuous training, the deep learning model can adaptively adjust its weights, optimize the classification accuracy, especially showing high generalization ability in the signal differences of different individuals for adaptive optimization.

[0201] In an embodiment of the present invention, in terms of feature classification, a self-learning neural network classifier based on deep learning is proposed for the first time. This classifier can automatically adjust weights during the training process and optimize the processes of feature extraction and classification according to the physiological differences of different users (such as skin conductivity, scalp thickness, etc.), so as to achieve personalized brain wave feature recognition. By using a hybrid architecture that combines a convolutional neural network (CNN) with a recurrent neural network (RNN), it is possible to capture the spatial features of the signal while taking into account the changes in the time series. Different from traditional static classifiers, this deep learning model can be continuously updated according to the real-time state of the user, further improving the recognition accuracy.

[0202] The existing technologies use relatively fixed mathematical calculation methods such as the frequency domain and time domain. For example, the calculation process for time domain analysis is relatively traditional. Compared with the existing technologies, the solution of the present application proposes a deep learning network that integrates time domain and frequency domain features to calculate and classify features. In the index verification, based on the combined application of convolutional and recurrent methods, more high-precision signal classification types can be obtained, such as attention, emotional changes, concentration, fatigue, anxiety characteristics, etc., and better results can be achieved.

[0203] Optionally, as one of the embodiments, interacting with the user based on the brain wave classification result specifically includes:

[0204] When it is determined that it is a preset state according to the classification result, trigger the corresponding feedback mechanism according to the preset rules corresponding to the preset state; and / or,

[0205] Optimize the feedback rules according to the matching situation between the user's brain wave information and the user's behavior, and provide personalized suggestions.

[0206] The present invention designs a real-time feedback mechanism that can interact with the user based on the brain wave classification result. The specific implementation methods include:

[0207] Real-time feedback: When the classification result shows that the user is in a certain specific state (such as fatigue, concentration, anxiety), the device will trigger different feedback mechanisms according to the preset rules, such as sound prompts, music push, vibration alarms, visual signals, etc. That is, when it is determined that it is a preset state according to the classification result, trigger the corresponding feedback mechanism according to the preset rules corresponding to the preset state.

[0208] Adaptive learning: During the long-term use of the device, it will continuously optimize the feedback rules by recording the matching situation between the user's brain wave data and their actual behavior (such as operating the device, emotional state), and provide personalized suggestions, such as training to improve concentration, relaxation exercises, etc. That is, optimize the feedback rules according to the matching situation between the user's brain wave information and the user's behavior, and provide personalized suggestions.

[0209] The feedback mechanism can be as follows: When a specific state (such as inattentiveness, mood swings, etc.) is detected, the system gives corresponding prompts or conducts training through a preset feedback mechanism:

[0210] Vibration feedback: For example, when it is detected that the user's attention has declined, the device vibrates to remind the user.

[0211] Sound prompt: When the user has been in a tense state for a long time, a sound is emitted to remind them to relax.

[0212] App recommendation: The system can give personalized suggestions through a mobile app, such as taking a short break or doing relaxation exercises.

[0213] Specifically, the real-time feedback can be: When the classification result shows that the user is in a certain specific state (such as fatigue, concentration, anxiety), the device will trigger different feedback mechanisms according to preset rules, such as sound prompts, music pushes, vibration alarms, visual signals, etc.

[0214] Specifically, the adaptive learning can be: During the long-term use of the device, it will continuously optimize the feedback rules by recording the matching situation between the user's brain wave data and their actual behaviors (such as operating the device, emotional state), and provide personalized suggestions, such as concentration improvement training, relaxation exercises, etc.

[0215] The user adaptive feedback system designed by the present invention can dynamically adjust the working mode of the device according to real-time brain wave characteristics and classification results. For example, in a brain-computer interface (BCI) application, when the system detects that the user is concentrated, it can increase the interaction frequency of the system, and when it detects that the user is fatigued, the system provides an alarm, automatic adjustment, and actively sets to reduce the interaction frequency. In addition, the system can also perform self-learning through long-term monitoring data of brain wave characteristics, predict the user's behavior pattern, and provide personalized suggestions for the user in daily applications, such as relaxation training or concentration training.

[0216] An embodiment of the present invention also provides a computer program product, including a computer program, where the computer program is stored in a computer-readable storage medium; when the processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of the method according to any one of the above embodiments.

[0217] The present invention can be used in different application scenarios, such as:

[0218] Emotion monitoring scenario: The user wears the device for daily activities, and the system detects their mood swings in real time, such as providing feedback in stress management or emotion regulation.

[0219] Brain-computer interface applications: Users interact with external devices through the device, such as controlling virtual objects or executing specific commands through brain waves.

[0220] Attention monitoring applications: When used for driving or operating machinery, the device can detect the user's attention level in real time and issue alerts to improve safety.

[0221] Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0222] It should be understood that the above specific embodiments of the present invention are only for illustrative or explanatory purposes of the principles of the present invention and do not constitute a limitation to the present invention. Therefore, any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and scope of the present invention shall be included within the scope of protection of the present invention. In addition, the appended claims of the present invention are intended to cover all variations and modifications that fall within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A method for identifying features of head brain waves, characterized in that: include: Acquire the brain wave signal of the head, pre-process the brain wave signal, and obtain the processed signal to be processed; Extracting features of the signal to be processed to obtain feature information; A deep learning-based classifier automatically classifies the brain waves according to the feature information, identifies the brain waves, and obtains brain wave classification results; Interact with users based on brainwave classification results; Wherein, the obtaining of the brain wave signal of the head specifically includes: At least two electrodes are used to collect brain wave signals in the head area, wherein the electrodes are distributed in the head area in a combination of partially symmetrical and partially asymmetrical manner.

2. The method according to claim 1, characterized in that The preprocessing of the brain wave signal to obtain the processed signal to be processed specifically includes: Processing the brain wave signal using a bandpass filter, wherein the filtering range is 1 Hz-40 Hz; Using adaptive blind source separation technology combined with independent component analysis algorithm, the artifact signal in the brain wave signal is effectively separated and eliminated; The brain wave signal is standardized to adjust the amplitude of the brain wave signal to a uniform range.

3. The method according to claim 1, characterized in that The feature extraction of the signal to be processed to obtain feature information specifically includes: Performing time domain feature extraction and frequency domain feature extraction on the signal to be processed respectively; The extracted time domain characteristic signal and the frequency domain characteristic signal are analyzed to obtain analyzed characteristic information.

4. The method according to claim 3, characterized in that The time domain feature extraction specifically includes: By calculating the overall trend and fluctuation amplitude of the brain wave signal, the mean and standard deviation features are extracted; Extracting the autocorrelation feature of the brain wave signal to measure the degree of self-similarity of the signal in time; The peak value and energy of the brain wave signal are extracted according to the peak value and energy of the brain wave signal within a specific time window.

5. The method according to claim 3, characterized in that: The frequency domain feature extraction specifically includes: Converting the brain wave signal to the frequency domain by fast Fourier transform (FFT); The energy of the different frequency components is analyzed to extract the characteristics of the α wave band, β wave band, θ wave band, and δ wave band.

6. The method according to claim 3, characterized in that The step of analyzing the extracted time domain characteristic signal and the frequency domain characteristic signal to obtain the analyzed characteristic information specifically includes: Performing wavelet transform and empirical mode decomposition (EMD) on the brain wave signal, and performing time-frequency domain analysis on the signal; The wavelet transform specifically includes: decomposing the brain wave signal at multiple scales to extract local time and frequency characteristics of the brain wave signal; The empirical mode decomposition specifically includes: extracting the essential frequency and fluctuation mode of the brain wave signal by decomposing the original signal of the brain wave into multiple intrinsic mode functions.

7. The method according to claim 1, characterized in that The deep learning-based classifier automatically classifies according to the feature information, specifically including: The deep learning model structure includes: a convolutional neural network and a recurrent neural network, wherein the convolutional neural network is used to extract spatial features in the brain wave signal, and the recurrent neural network is used to capture the correlation in the brain wave time series; The convolutional neural network is used to extract spatial features in the signal, specifically including: Spatial feature extraction: spatial transformation is performed through common spatial patterns to obtain feature matrices; Constructing a CNN network: including multiple convolutional layers and fully connected layers, the input of the CNN network is a feature matrix after CSP transformation, adding the spatiotemporal dimension of the EEG signal to the 2D EEG topography; Feature learning: The CNN learns the feature matrix through the network and automatically extracts spatial features through the convolution layer; Weight analysis and feature screening: By analyzing the weight matrix of the fully connected layer of the CNN network, the screening criteria of the CSP feature matrix are defined to obtain the EEG feature set F with efficient dimensionality reduction; Constructing a classifier: Constructing a new CNN classifier based on the extracted feature set F, and the CNN classifier is used for the final EEG signal classification task. Training and optimization: Use the stochastic gradient descent optimization method to train the CNN network and optimize the network performance by calculating negative gradients and updating weights; Classification: Use the trained CNN classifier to classify new EEG signals and identify different brain wave patterns; Performance Evaluation: The performance and accuracy of the CNN classifier in classifying the EEG signals were evaluated by comparison with baseline methods or other state-of-the-art methods.

8. The method according to claim 3, characterized in that The time domain feature extraction specifically includes: Feature extraction: extract features from EEG signals to obtain valuable information for classification from redundant information, wherein the valuable information includes time domain features, and the time domain features include kurtosis coefficients; Build an RNN model: Create an RNN network with at least 100 recurrent neurons; each input contains a feature value; Serialization input: Serialize the time series data of the EEG signal, each training sample contains a plurality of continuous EEG signal values, and the target is a sequence after being shifted by one time step relative to the training sample; Training RNN: Using dynamic RNN units to process the input sequence and perform training, where during the training process, the RNN model learns how to predict the output of the next time step based on the current and previous inputs; Classification: classifying the extracted features to determine the degree of association between the acquired brain wave signal state and the classification match, wherein the RNN model classifies or predicts the EEG signal at each time step to identify different brain electrical activity states or detect abnormal events; Attention mechanism: helps the encoder calculate by generating weight sequences of different dimensions to obtain intermediate encoding; Post-processing and evaluation: inputting the features extracted by the RNN model into a classifier to identify the categories of the EEG data, wherein each category corresponds to a specific brain activity; Evaluate model performance: Evaluate the model so that the recognition accuracy can meet the preset requirements and the robustness of the algorithm framework under the premise of real-time data processing.

9. The method according to claim 1, characterized in that: The interacting with the user based on the brain wave classification result specifically includes: When it is determined according to the classification result that it is a preset state, triggering a corresponding feedback mechanism according to a preset rule corresponding to the preset state; and / or, Based on the matching of the user's brain wave information and the user's behavior, the feedback rules are optimized and personalized suggestions are provided.

10. A computer program product, characterized in that The method comprises a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of the method according to any one of claims 1 to 9.

Images