EEG-fNIRS Fusion Neural Network for Brain Activity Determination

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Brain-computer interface (BCI) systems using EEG or fNIRS standalone technologies face limitations in performance due to vulnerability to human motion noise and low temporal resolution, respectively.

Innovation Solution

A method and electronic device that combine EEG and fNIRS signals using a neural network model with convolutional layers and an fNIRS-guided attention layer to enhance brain activity determination, improving performance by extracting and fusing features from both signals.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If EEG signal is used for brain activity measurement, then temporal resolution is improved, but vulnerability to human motion noise increases

Engineering Contradiction:

Improvetemporal resolutionVSAvoidhuman motion noise vulnerability

Core Design Contradiction:

SpeedVSObject-affected harmful factors

Solution Approach 1:

The patent combines EEG and fNIRS signals into a hybrid BCI system that integrates the high temporal resolution of EEG with the motion robustness of fNIRS. The neural network model processes both signal types simultaneously, allowing the system to leverage EEG's speed advantages while compensating for its noise vulnerability through fNIRS guidance.

Inventive Principle:

Principle #5Merging (Combining)

2Object-affected harmful factors

If fNIRS signal is used for brain activity measurement, then resistance to human motion noise is improved, but temporal resolution deteriorates

Engineering Contradiction:

Improvehuman motion noise resistanceVSAvoidtemporal resolution

Core Design Contradiction:

Object-affected harmful factorsVSSpeed

Solution Approach 1:

The fNIRS signal acts as a guiding intermediary that provides motion-robust hemodynamic information to complement EEG's fast neural responses. The attention mechanism uses fNIRS features to modulate EEG processing, allowing the system to maintain temporal precision while filtering out motion-related artifacts through the slower but more stable fNIRS channel.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Device complexity

If standalone EEG or fNIRS system is used, then device complexity is reduced, but measurement precision deteriorates

Engineering Contradiction:

Improvesystem structure simplicityVSAvoidbrain activity determination accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent creates a composite signal processing system that treats EEG and fNIRS as complementary components. The neural network architecture integrates both signal streams with dedicated processing branches and fusion layers, creating a composite measurement approach that achieves superior precision compared to either standalone system while managing complexity through modular design.

Inventive Principle:

Principle #40Composite materials

Data Source

PatentUS12178616B2Method of determining brain activity and electronic device performing the method

Publication Date: 2024.12.31 FOUND FOR RES & BUSINESS SEOUL NAT UNIV OF SCI & TECH

AI summary

An electronic device according to an example embodiment includes a processor, and a memory operatively connected to the processor and including instructions executable by the processor, wherein when the instructions are executed, the processor is configured to collect an EEG signal measuring brain activity and an fNIRS signal measuring the brain activity, and output a result of determining a type of the brain activity from a trained neural network model using the EEG signal and the fNIRS signal, and the neural network model may be trained to, extract an EEG feature from the EEG signal, extract an fNIRS feature from the fNIRS signal, extract a fusion feature based on the EEG signal and the fNIRS signal, and output the result of determining the type of the brain activity based on the EEG feature and the fusion feature.