EEG Visual Reconstruction via Confusability Matrix
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for reconstructing images from EEG signals face challenges such as insufficient information, artifacts, and localization issues, and rely on expensive and non-portable technologies like fMRI, which lack temporal resolution and are constrained by assumptions about stimulus-response mappings.
Innovation Solution
A computer-implemented method and system that use scalp EEG signals to generate visual category reconstructions by receiving and processing EEG signals through a trained pattern classifier, constructing a confusability matrix, and generating a multidimensional visual representational space to determine relevant subspaces and reconstruct visual appearances, avoiding the need for explicit encoding models and leveraging the temporal resolution of EEG.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If fMRI is used to reconstruct images from brain activity, then fine details of brain activity can be generated, but the method is expensive, non-portable, and lacks temporal resolution
Solution Approach 1:
The patent uses EEG signals as a simplified copy or alternative representation of brain activity instead of relying on complex fMRI measurements. By training classification models on EEG data, the system reconstructs visual stimuli from this less complex but still informative signal source, achieving comparable reconstruction capability without the drawbacks of fMRI
Solution Approach 2:
The patent replaces the mechanical fMRI system with an electrical/chemical EEG-based system. Instead of using magnetic fields and blood flow detection, the invention uses electrical signals from EEG electrodes combined with machine learning classification to achieve image reconstruction, thereby eliminating the need for expensive and non-portable fMRI equipment
2Measurement precision
If fMRI is used to measure brain activity, then detailed measurements can be obtained, but temporal resolution is insufficient for tracking percept development over time
Solution Approach 1:
The patent changes the measurement parameter from blood flow (fMRI) to electrical activity (EEG). This parameter change enables millisecond-level temporal resolution while maintaining sufficient spatial information through the use of multiple EEG channels and sophisticated classification algorithms that can extract meaningful patterns from the electrical signals
3Device complexity
If EEG signals are used for image reconstruction, then portability and cost-effectiveness are improved, but the signals contain insufficient information and artifacts
Solution Approach 1:
The patent introduces classification models as intermediary processing layers between the raw EEG signals and the final image reconstruction. These models act as mediators that filter out artifacts, amplify relevant neural patterns, and translate the noisy EEG data into meaningful visual representations, thereby compensating for the inherent limitations of EEG signal quality
Solution Approach 2:
The patent transforms the low-dimensional, noisy EEG signal space into a higher-dimensional representational space through classification and decoding processes. By projecting EEG data into multiple feature dimensions and combining information across channels and time points, the system recovers sufficient information content despite the limitations of individual EEG measurements
Data Source
AI summary
There is provided a system and method for generating visual category reconstruction from electroencephalography (EEG) signals. The method includes: receiving scalp EEG signals; using a trained pattern classifier, determining pairwise discrimination of the EEG signals, the pattern classifier trained using a training set comprising EEG signals associated with a subject experiencing different known visual identities; determining discriminability estimates of the pairwise discrimination of the EEG signals by constructing a confusability matrix; generating a multidimensional visual representational space; determining visual features for each dimension of the visual representational space by determining weighted sums of image stimulus properties; identifying subspaces determined to be relevant for reconstruction; and reconstructing the visual appearance of a reconstruction target using estimated coordinates of the target and a summed linear combination of the visual features proportional with the coordinates of the target in the visual representational space.