Image Object Detection With Decoupled Position and Appearance Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Attention network-based detection models in computer vision suffer from slow convergence speed and reduced detection accuracy due to coupled appearance and positional features, leading to challenges in precise object positioning.
Innovation Solution
Perform spatial positional encoding on an initial object position to obtain first spatial positional codes, process these codes with query vectors and image features using a second detection model to decouple appearance and positional features, determining a positional offset and size information for precise object positioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If anchor points or anchor boxes are assigned to different truth values according to preset rules with balanced distribution of positive and negative samples, then detection accuracy is improved, but convergence speed deteriorates due to massive post-processing such as non-maximum suppression
Solution Approach 1:
The patent segments the detection task into two independent parts: appearance feature extraction and positional feature extraction. By using separate neural network branches for appearance and position, the model avoids the need for complex post-processing while maintaining accurate detection. The segmentation allows each branch to specialize, improving convergence speed without sacrificing detection accuracy.
Solution Approach 2:
The patent extracts positional information as a separate feature from appearance features. By taking out the positional encoding and processing it independently through a dedicated positional encoding layer, the model eliminates the need for non-maximum suppression and other post-processing steps, thereby improving convergence speed while preserving detection accuracy.
2Adaptability or versatility
If learnable encoded vectors are used as templates in attention network for querying on feature map, then balance between positive and negative samples is achieved, but detection accuracy deteriorates due to limited template coverage
Solution Approach 1:
The patent applies local quality by using spatial positional encoding that adapts to different positions in the feature map. Instead of using uniform templates, the model generates position-specific encodings that capture local characteristics. This allows the model to maintain sample balance while improving detection accuracy by adapting to local variations in the image data.
Solution Approach 2:
The patent adds a spatial dimension to the template matching process by incorporating positional encodings. By transforming the problem from simple template matching to multi-dimensional feature comparison (appearance + position), the model achieves better sample balance and improved detection accuracy through enhanced feature discrimination.
3Device complexity
If appearance feature and positional feature are coupled in attention network-based detection model, then model complexity is reduced, but positioning precision deteriorates leading to slow convergence speed
Solution Approach 1:
The patent segments the feature extraction process into independent appearance and position branches. By separating the processing of appearance features and positional features, the model achieves high positioning precision while maintaining manageable complexity through modular architecture. Each branch can be optimized independently.
Solution Approach 2:
The patent introduces spatial positional encoding as an intermediary mechanism that bridges appearance and position features. This intermediary layer processes positional information separately before combining it with appearance features, enabling precise positioning while keeping the overall model structure organized and computationally efficient.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Embodiments of the present disclosure disclose a method, an apparatus, a device and a medium for identifying a target object in an image. The method comprises: performing spatial positional encoding on an initial position of the target object in the image, to obtain a plurality of first spatial positional codes; processing the plurality of first spatial positional codes, a plurality of query vectors corresponding to the plurality of first spatial positional codes, an image feature of the image, and a second positional code that matches the image feature, to obtain an appearance feature and a positional feature of the target object on the image; determining, based on the positional feature and the plurality of first spatial positional codes, a positional offset of the target object relative to the initial position; determining, based on the appearance feature, size information of the target object in the image; and determining, based on the initial position and the positional offset of the target object, a final position of the target object in the image. The embodiments of the present disclosure can realize decoupling of an appearance feature and a positional feature of a target object in an image, thereby realizing high-precision positioning for the target object in the image.