Multi-Level DNN for EEG Mental State Recognition

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional devices for recognizing mental behavioral attributes based on bio-signals, such as EEG, face limitations in identifying complex features and dependencies among brain regions due to lateralization effects and are biased by gender or handedness, making them unsuitable for accurate mental state recognition.

Innovation Solution

An electronic device employing a multi-level deep neural network (DNN) model that processes EEG and other bio-signals to identify invariant mental behavioral attributes, accounting for lateralization, gender, and handedness, and tracks changes in mental states, providing diagnostic and predictive reports.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional supervised machine learning models are used for mental state recognition, then the device can identify simple features from limited inputs, but it fails to identify complex features and relationships among different brain regions due to lateralization effects

Engineering Contradiction:
Improverecognition accuracyVSAvoidability to identify complex features
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent segments the brain signal processing into multiple independent feature extraction channels corresponding to different brain regions, allowing each channel to process signals independently before combining results. This segmentation approach enables the system to handle complex features and relationships among brain regions while maintaining simplicity in individual processing streams.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent transitions from traditional 2D signal processing to a multi-dimensional approach by incorporating spatial information from multiple EEG electrodes arranged in specific patterns. This dimensional expansion allows the system to capture complex spatial relationships and dependencies among different brain regions that conventional 1D or 2D models cannot detect.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Ease of manufacture

If conventional machine learning models are trained with hand-crafted features, then the model can learn from bio-signals, but it introduces bias due to lateralization of brain functions related to gender or handedness

Engineering Contradiction:
Improvemodel training capabilityVSAvoidbias in identification
Core Design Contradiction:
Ease of manufactureVSReliability

Solution Approach 1:

The patent extracts and removes bias-related components from the feature representation by identifying and eliminating features that correlate with gender or handedness. This extraction approach allows the model to focus on task-relevant features while discarding confounding variables that introduce bias in mental state recognition.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies different processing strategies to different brain regions based on their functional characteristics. By tailoring the feature extraction and processing to local regional properties rather than applying uniform processing across all electrodes, the system reduces the impact of lateralization effects while maintaining sensitivity to genuine neural patterns.

Inventive Principle:
Principle #3Local quality

3Device complexity

If conventional devices use limited input sets for training, then the model training is simpler, but the device cannot accurately identify complex relationships and dependencies among brain regions

Engineering Contradiction:
Improvemodel training complexityVSAvoididentification of complex relationships
Core Design Contradiction:
Device complexityVSMeasurement precision

Solution Approach 1:

The patent performs preliminary feature extraction and dimensionality reduction on the raw EEG signals before feeding them to the main classification model. This preliminary processing step simplifies the input data structure while preserving critical information about complex relationships among brain regions, making the subsequent model training more efficient and accurate.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent introduces intermediate processing layers that act as mediators between raw signal inputs and final classification outputs. These intermediate layers extract and represent complex relationships in a simplified form, enabling the model to learn from limited training data while still capturing intricate dependencies among different brain regions.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS11141088B2Electronic device for recognition of mental behavioral attributes based on deep neural networks
Publication Date: 2021.10.12 SONY GROUP CORP
  • US11141088B2 patent drawing
  • US11141088B2 patent drawing
  • US11141088B2 patent drawing

AI summary

An electronic device that handles recognition of mental behavioral, affect, emotional, mental states, mental health, or mood-based attributes based on deep neural networks (DNNs), stores a set of EEG signals and a set of bio-signals associated with a subject. The electronic device trains a plurality of first recognition models on a training set of EEG signals and a training set of bio-signals associated with different training subjects. The electronic device trains a second recognition model on a feature vector from output layers of the plurality of first recognition models. The electronic device estimates a plurality of dependency or relationship data by application of the trained plurality of first recognition models on the set of EEG signals and bio-signals. The electronic device identifies a mental behavioral attribute of the subject by application of the trained second recognition model on the plurality of signals and their relationship data.