Masked Image Autoencoder Training Without Manual Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The need for human subject-matter expert intervention in labeling training data for machine learning models is tedious, expensive, and prone to errors, making the training process costly and inefficient.
Innovation Solution
A self-supervised learning method using autoencoders to encode and decode partially masked images, comparing unaugmented and augmented model outputs to determine total loss, and improving the autoencoder based on this loss to create an improved model without human intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If human subject-matter experts label training data for machine learning models, then model training accuracy is improved, but training cost and time consumption increase significantly
Solution Approach 1:
The system performs self-supervised learning by automatically generating training data and labels without human intervention. The autoencoder model learns to reconstruct images by encoding and decoding them, creating its own training signals from unstructured image data, thereby eliminating the need for manual labeling while maintaining training effectiveness
Solution Approach 2:
The patent transforms the training approach by changing from supervised learning (requiring labeled data) to self-supervised learning (using unlabeled data). This parameter change in the learning paradigm allows the model to learn from raw image data through reconstruction tasks, reducing dependency on human-labeled datasets
2Measurement precision
If human subject-matter experts label training data for machine learning models, then model training accuracy is improved, but training cost increases
Solution Approach 1:
The system performs self-supervised learning by automatically generating training data and labels without human intervention. The autoencoder model learns to reconstruct images by encoding and decoding them, creating its own training signals from unstructured image data, thereby eliminating the need for manual labeling while maintaining training effectiveness
Solution Approach 2:
The patent uses readily available unstructured image data from various sources (cameras, satellites, drones) as training material. These images serve as disposable training samples that don't require expensive human labeling, allowing the model to learn from large volumes of low-cost or free image data
3Measurement precision
If traditional supervised learning is used with labeled data, then model accuracy is improved, but automation extent decreases
Solution Approach 1:
The system performs self-supervised learning by automatically generating training data and labels without human intervention. The autoencoder model learns to reconstruct images by encoding and decoding them, creating its own training signals from unstructured image data, thereby eliminating the need for manual labeling while maintaining training effectiveness
Solution Approach 2:
The patent transforms the training approach by changing from supervised learning (requiring labeled data) to self-supervised learning (using unlabeled data). This parameter change in the learning paradigm allows the model to learn from raw image data through reconstruction tasks, reducing dependency on human-labeled datasets
Data Source
AI summary
Self-supervised learning of a machine learning model using images. The computing device masks a section of each image in an image database to generate a partially masked image. An autoencoder encodes each partially masked image to generate one or more encodings representing each partially masked image. The autoencoder decodes each of the one or more encodings into one or more decoded encodings representing each partially masked image previously input into the autoencoder. The computing device compares each of the one or more decoded encodings of each partially masked image with the corresponding image from the image database to generate an unaugmented model output. The computing device augments each image according to a data augmentation policy to generate an augmented model output. The computing device determines a total loss by comparing the unaugmented model output to the augmented model output. The autoencoder is improved based upon the total loss.