Knowledge Distillation for Lightweight Scene Flow Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Bi-point flow net models for scene flow prediction are inefficient due to high computation requirements and deteriorated performance from hierarchical single flow embedding, and existing knowledge distillation methods fail to achieve lightweight models and are limited to 2D computer vision.
Innovation Solution
A method for distilling knowledge from a teacher model to a student model using a loss function generated through bi-directional flow embedding and a flow predictor layer, enabling lightweight student models by calculating differentiation and applying it to an inversed Softmax function, and combining with a multi-scale loss function for training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If bi-point flow net uses hierarchical single flow embedding, then flow prediction performance deteriorates, but computation time increases
Solution Approach 1:
The patent segments the flow embedding process into multiple independent bidirectional flow embedding layers instead of using a single hierarchical layer. Each embedding layer processes flow information independently, allowing parallel computation and improving both prediction performance and computational efficiency.
Solution Approach 2:
The patent transitions from a single-dimensional hierarchical embedding approach to a multi-dimensional bidirectional embedding structure. By embedding flow information in multiple directions (bidirectional) across different layers, the model captures more comprehensive flow patterns without increasing computation time.
2Reliability
If knowledge distilling method is applied to 2D computer vision model, then flow prediction performance enhances, but model lightweighting is not achieved
Solution Approach 1:
The patent changes the parameters of the knowledge distillation process by using bidirectional flow embedding and multiple hierarchical prediction results as teacher model outputs. The loss function incorporates weights derived from inversed Softmax of prediction differences, enabling effective knowledge transfer that reduces student model size while maintaining performance.
Solution Approach 2:
The patent creates a student model that copies the essential features of the teacher model through knowledge distillation. The student model learns from the teacher's hierarchical prediction results and bidirectional flow embedding patterns, achieving a lightweight version that maintains predictive accuracy.
3Reliability
If knowledge distilling method is applied without student model generation, then flow prediction performance enhances, but model lightweighting is not achieved
Solution Approach 1:
The patent explicitly generates a student model that copies the teacher model's architecture but with reduced complexity. The student model learns from the teacher's bidirectional flow embedding and hierarchical predictions, creating a lightweight version that maintains performance without requiring the full teacher model structure.
Data Source
AI summary
An apparatus for distilling knowledge for a scene flow prediction model includes: a student model former forming a student model to have single bidirectional flow embedding and a flow predictor structure of a teacher model; a weight generator generating a weight based on a plurality of hierarchical prediction results of the teacher model and predetermined ground truth data; a function generator generating a loss function by using the weight and the plurality of prediction results; and a knowledge distiller distilling the knowledge of the teacher model to the student model by using the loss function.