Query Recollection Training for Object Detection Error Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Query-based object detection methods face challenges due to unbalanced responsibility among decoding stages, lack of training emphasis on later stages, and cascading errors from the sequential structure, which impede the convergence and accuracy of final predictions.
Innovation Solution
The Query Recollection (QR) training strategy collects intermediate queries and feeds them to downstream stages, providing geometric progression of supervision signals and allowing later stages to view prior queries, while Selective Query Recollection (SQR) selectively forwards queries from the prior two stages, reducing computing costs and enhancing supervision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sequential decoding stages are used for query-based object detection, then the model can progressively refine predictions, but cascading errors propagate through stages and later stages degrade true-positives while upgrading false-positives
Solution Approach 1:
The patent implements feedback by allowing later decoding stages to access and learn from intermediate queries produced by earlier stages. Specifically, intermediate queries from stage s-1 are fed back as additional inputs to stage s, enabling the model to correct its own mistakes by comparing refined queries against earlier intermediate representations. This feedback mechanism prevents cascading errors by giving later stages the opportunity to reconsider predictions that may have deteriorated during sequential processing.
Solution Approach 2:
The patent applies preliminary action by pre-collecting and preserving intermediate queries from all previous stages before they are potentially degraded by subsequent refinements. These intermediate queries are stored and made available as supervision signals for later stages, ensuring that valuable intermediate predictions are not lost before later stages can learn from them and potentially recover or improve upon them.
2Ease of manufacture
If all decoding stages are supervised in an equivalent manner, then training is simplified, but later stages do not receive sufficient training emphasis despite bearing greater responsibility for final predictions
Solution Approach 1:
The patent applies local quality by differentiating the supervision mechanism across different decoding stages. Instead of uniform supervision, later stages receive enhanced supervision through access to multiple intermediate queries from previous stages, while earlier stages receive standard supervision. This localized differentiation ensures that stages with greater responsibility for final predictions (later stages) receive proportionally more training emphasis and supervision signals, improving overall prediction accuracy without overly complicating the training of earlier stages.
3Device complexity
If intermediate queries are cascaded sequentially through all stages, then the model maintains a simple processing structure, but errors accumulate and the sequential structure impedes later stages from seeing prior queries during training
Solution Approach 1:
The patent applies dimensionality change by transforming the sequential one-dimensional processing flow into a two-dimensional structure where intermediate queries from multiple previous stages are made available to later stages. Instead of querying only the immediately preceding stage, each stage s can access intermediate queries from stages s-1, s-2, and potentially earlier, creating a broader information dimension that reduces error accumulation while maintaining reasonable structural complexity.
Data Source
AI summary
Disclosed herein are training strategies for query-based object detectors, referred to herein as Query Recollection (QR). In one variation or QR, dense query recollection, every intermediate query is collected and independently forwarded to every downstream stage. In a second variation or QR, selective query recollection, intermediate queries are collected from the two nearest previous stages and forwarded to the next downstream stage. This eliminates the phenomena wherein intermediate stages of the decoder produce more accurate results than later stages of the decoder.