Object Detection Query Distillation for Quantized Transformer Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Transformer-based object detectors require significant memory and computing resources due to large parameter counts and FLOPs, limiting their deployment on resource-constrained platforms, and quantization for compression leads to performance degradation.
Innovation Solution
Minimize the information difference between a student model and a teacher model by maximizing the entropy of queries in a compressed neural network, using 2-bit, 3-bit, or 4-bit quantization, and perform foreground-aware query matching to improve model precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If quantization is performed to compress the neural network, then the quantity of network parameters and activation values is reduced, but model precision is degraded
Solution Approach 1:
The patent changes the parameter representation from fixed-point to floating-point format in the quantized model, allowing the model to maintain precision while using fewer bits. This involves modifying how parameters are stored and processed in the compressed model to preserve their numerical accuracy despite reduced bit-width representation
Solution Approach 2:
The patent performs preliminary quantization-aware training before final model compression. During this phase, the model is trained with quantization operations simulated in the training pipeline, allowing it to adapt to the reduced precision environment beforehand. This preliminary action prepares the model to maintain performance after actual quantization compression is applied
2Productivity
If the neural network is compressed for efficient online inference, then deployment on resource-constrained platforms becomes possible, but model precision sharply degrades
Solution Approach 1:
The patent employs mixed-precision quantization where different parts of the neural network are quantized to different precision levels. Critical layers that contribute most to model accuracy are maintained at higher precision (e.g., 8-bit or full precision), while less critical layers are compressed to lower precision (e.g., 4-bit or 2-bit). This selective parameter precision approach maintains overall model precision while achieving significant compression for efficient deployment on resource-constrained platforms
Solution Approach 2:
The patent implements quantization-aware training as a preliminary step before final model compression for deployment. During this training phase, the model learns with simulated quantization effects, allowing it to adapt its parameters and representations to work effectively in the compressed format. This preliminary adaptation ensures that when the model is deployed with actual quantization for efficient inference, it maintains higher precision compared to models that are quantized without such preparation
Data Source
Figure 1A
Figure 1B~1C
Figure 1D
AI summary
This application relates to the artificial intelligence field, and discloses a data processing method, including: obtaining a first query (query) corresponding to a first neural network and a second query corresponding to a second neural network, where the first query is a query obtained by maximizing an information entropy of a query corresponding to the first neural network, the first neural network is a model obtained by compressing the second neural network, and the first neural network and the first neural network are used for object detection; determining a first loss based on the first query and the second query, where the first loss indicates to minimize an information difference between the second query and the first query; and updating the first neural network and the first query based on the first loss. In this application, an information difference between a query (query) of a student model and a query of a teacher model is minimized, to minimize an information difference between the student model and the teacher model, and improve model precision of a compressed model.