Self-Supervised Feature Separation for Reduced Calculation Cost
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing contrastive learning methods in machine learning require large computational resources due to the need to input pairs of positive and negative examples to the encoder, leading to high calculation costs.
Innovation Solution
The proposed information processing apparatus uses a machine learning model where features are made farther apart by updating the model's parameters based on a loss function that calculates the similarity between features, eliminating the need for positive examples and reducing computational costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If contrastive learning is used to reduce human cost for teaching data generation, then self-supervised learning capability is improved, but calculation amount increases due to needing to input pairs of positive and negative examples to the encoder
Solution Approach 1:
The patent extracts and removes the negative example processing component from the contrastive learning framework. By eliminating the need to input negative examples to the encoder, the calculation amount is reduced while maintaining the self-supervised learning capability through positive example pairing only.
Solution Approach 2:
The patent segments the contrastive learning process into distinct components: positive example processing and negative example processing. By separating these functions and removing the negative example segment, the system achieves reduced computational load while preserving the essential self-supervised learning functionality.
2Reliability
If pairs of positive and negative examples are input to the encoder for contrastive learning, then training effectiveness is improved, but device complexity increases
Solution Approach 1:
The patent extracts the negative example processing requirement from the training system. By removing this component, the system complexity is reduced while maintaining training effectiveness through alternative mechanisms that rely only on positive example pairs.
3Power
If the decoder's model size is made smaller than the encoder's, then calculation cost is reduced, but manufacturing precision requirements increase
Solution Approach 1:
The patent applies asymmetry by deliberately designing the decoder with a smaller model size than the encoder. This asymmetric configuration reduces calculation cost while the system maintains performance through the encoder's larger capacity compensating for the decoder's reduced complexity.
Data Source
AI summary
According to one embodiment, the information processing apparatus includes a processor. The processor extracts a plurality of features from a plurality of training data by using a machine learning model. The processor generates a prediction result relating to a task, from the training data and teaching data corresponding to the training data. The processor calculates a similarity between features with respect to the plurality of features. The processor updates a parameter of the machine learning model, based on the prediction result and the similarity, in such a manner that the features become farther from each other.