Cross-Modality Bio-Signal Learning with Masked Frequency Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models for bio-signal analysis face challenges due to data scarcity, particularly the lack of large, labeled high-quality datasets, especially in modalities like EMG, ECG, and PPG, hindering effective training and generalization across different physiological domains.
Innovation Solution
Pretraining a machine learning model using EEG data to extract frequency-domain features, leveraging a convolutional neural network (CNN) for local temporal features and a transformer network for global temporal features, followed by a masking operation to reconstruct frequency-domain representations, enabling generalization across different bio-signal domains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models are trained on data from data-rich domains (EEG), then model performance and generalization are improved, but the ability to directly apply to data-scarce domains (EMG, ECG, PPG) deteriorates due to domain mismatch
Solution Approach 1:
The patent applies preliminary action by pretraining the encoder on EEG data before fine-tuning on target domain data. The encoder is first trained to extract frequency-domain features from EEG signals, then transferred to other modalities. This preliminary training on data-rich EEG data provides a strong foundation that improves performance on data-scarce target domains without requiring extensive training data in each target domain.
Solution Approach 2:
The patent uses an intermediary approach by introducing a shared encoder that processes multiple signal modalities through a common representation space. The encoder acts as an intermediary that learns modality-agnostic frequency-domain features, enabling transfer between different bio-signal domains (EEG to EMG/ECG/PPG) while maintaining domain adaptability through modality-specific decoders.
2Reliability
If large, labeled, high-quality datasets are collected for training, then model training quality and generalization improve, but data collection cost, complexity, and time requirements increase
Solution Approach 1:
The patent applies copying by using EEG data as a proxy or copy to train the encoder, which is then transferred to other modalities. Instead of collecting large labeled datasets for each target domain (EMG, ECG, PPG), the system copies the training approach from EEG and adapts it to other domains through the shared encoder, significantly reducing data collection requirements while maintaining training quality.
Solution Approach 2:
The patent performs preliminary action by concentrating data collection and labeling efforts on EEG data, which is then used to train the encoder before applying it to other domains. This preliminary data preparation on one modality (EEG) eliminates the need for extensive data collection in each target domain, reducing overall data collection complexity and time requirements.
3Reliability
If modality-specific models are trained separately for each bio-signal type, then domain-specific performance is optimized, but computational resources and training time requirements increase
Solution Approach 1:
The patent applies universality by creating a shared encoder that serves multiple functions across different bio-signal modalities. Instead of training separate models for each modality (EEG, EMG, ECG, PPG), the encoder is trained once on EEG data and then used for all target domains through modality-specific decoders. This universal encoder significantly reduces computational resources and training time while maintaining domain-specific performance through the decoders.
Solution Approach 2:
The patent merges the training process by combining multiple domain-specific models into a single shared encoder architecture. The encoder processes all modalities through a unified representation space, merging the computational burden into one training process rather than separate training processes for each modality, thereby reducing overall computational resources and training time requirements.
Data Source
AI summary
A computer-implemented method includes processing a time-series input signal using an encoder to produce an encoded representation, segmenting the encoded representation into a plurality of patches, applying a masking operation to a subset of the patches to produce a masked encoded representation, processing the masked encoded representation using a transformer to generate contextual features, processing the contextual features using a decoder to generate a predicted frequency-domain representation of the time-series input signal, and adjusting parameters of the encoder and parameters of the transformer to minimize a loss between the predicted frequency-domain representation and a reference frequency-domain representation derived from the time-series input signal.