Data Classification With Adaptive Label Embeddings for Lower-Memory Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning methods, such as backpropagation, local learning, and forward learning, face limitations in computational efficiency, memory usage, and performance due to symmetry of weights, increased parameters, and lack of auxiliary networks, limiting their effectiveness in data classification tasks.
Innovation Solution
A data classification method and apparatus that utilize dictionary contrastive learning to train a network model by directly comparing local features from individual layers with adaptive label embeddings, minimizing memory consumption and parameter count while improving classification performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If backpropagation method is used, then classification performance is improved, but memory consumption increases due to storing local activation of each layer
Solution Approach 1:
The patent extracts only the necessary features (local activations) from each layer that are needed for gradient calculation, rather than storing all intermediate computations. This selective extraction reduces memory usage while maintaining the ability to compute gradients for backpropagation.
Solution Approach 2:
The patent employs asymmetric weight updates where weights are updated differently during forward and backward passes, breaking the symmetry constraint of traditional backpropagation. This allows for more flexible memory management and reduced storage requirements while maintaining training effectiveness.
2Productivity
If local learning method with auxiliary network is used, then computational efficiency is improved, but number of parameters and memory consumption increase significantly
Solution Approach 1:
The patent merges the auxiliary network functionality directly into the main network architecture, eliminating the need for separate auxiliary networks at each layer. This integration reduces the total number of parameters while maintaining the computational efficiency benefits of local learning.
Solution Approach 2:
The patent creates a universal auxiliary structure that serves multiple layers simultaneously, rather than having dedicated auxiliary networks for each layer. This multi-functional approach reduces parameter count while preserving computational efficiency.
3Quantity of substance
If forward learning method without auxiliary network is used, then memory consumption is reduced, but classification performance decreases due to inability to transform local features effectively
Solution Approach 1:
The patent introduces label embedding vectors as intermediary representations that bridge local features and classification labels. These embeddings serve as a mediator that transforms local features into a form suitable for classification without requiring complex auxiliary networks, thus maintaining low memory usage while improving performance.
Solution Approach 2:
The patent transforms local features by changing their parameter representation through label embeddings, converting them into a suitable format for loss calculation and classification. This parameter transformation enables effective feature utilization without increasing memory consumption.
Data Source
AI summary
Proposed are a data classification method and apparatus. The data classification method that is performed by the data classification apparatus includes extracting features from input data through a learning network model and outputting prediction results based on the features, and the learning network model compares local features derived through an individual layer other than the final layer of the learning network model with label embedding vectors corresponding to a classification label.