Object Detection Model Training via Distillation Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object detection models face challenges in performing effectively with smaller batch sizes, as contrastive loss methods used in feature representation learning are difficult to apply.
Innovation Solution
An object detection apparatus that trains a model using a feature representation learning technique involving a distillation loss, where input data is augmented and fed into both a first and second network model to calculate and apply distillation losses for objects and classes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If contrastive loss is used for feature representation learning, then performance is improved with large batch sizes, but it becomes difficult to apply with small batch sizes
Solution Approach 1:
The patent changes the loss function parameter from contrastive loss to distillation loss, which fundamentally alters how the model learns features. The distillation loss uses knowledge distillation between teacher and student networks, allowing effective training with small batch sizes by leveraging soft labels and gradient information from the teacher model, thus resolving the batch size dependency issue of contrastive loss
Solution Approach 2:
The patent introduces a teacher network as an intermediary that generates soft labels and guidance signals for the student network. This teacher-student framework acts as a mediator that enables effective feature representation learning even when batch sizes are small, by providing additional gradient information and reducing the reliance on large batches for contrastive learning
Data Source
AI summary
An object detection apparatus for providing a training technique for improving the performance of an object detection model includes a memory storing an object detection model and a processor that trains the object detection model. The processor augments input data to generate first augmented data and second augmented data, respectively inputs the first augmented data and the second augmented data to a first network model and a second network model, and trains the object detection model based on an output of the first network model and an output of the second network model.