Label Tracking for High-Frequency Offset Propagation in Patch Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing patch-matching algorithms face inefficiencies due to occluding objects disrupting the propagation of offsets, leading to re-creation of offsets rather than propagation, especially in images with complex structures like brick walls with occluding objects.
Innovation Solution
The use of labels to track high-frequency offsets allows for efficient propagation by identifying frequently used offsets across the image, bypassing the need for neighbor-to-neighbor propagation, and using these offsets to map source image patches to target image patches even in the presence of occluding objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If nearest-neighbor propagation is used to propagate offsets through the source image, then processing efficiency is improved for simple images, but processing efficiency deteriorates when occluding objects are present because offsets must be re-created rather than propagated
Solution Approach 1:
The patent segments the offset propagation process by introducing label-based tracking that separates the propagation mechanism from occlusion interference. Labels are assigned to offset types (e.g., brick wall offsets vs. lamp post offsets) and propagated independently, allowing the system to maintain high productivity for regular patterns while reliably handling occlusions through label switching.
Solution Approach 2:
The patent changes the parameter representation from raw offset vectors to labeled offset categories. This parameter transformation allows the system to generalize offset patterns (e.g., all brick wall offsets labeled as 'wall_type_1') and automatically adapt when occlusions are detected, resolving the contradiction between efficiency and reliability.
2Use of energy by moving object
If offset vectors are propagated from neighboring patches, then the number of computations is reduced, but computational resources are wasted when offsets must be re-created due to occluding objects
Solution Approach 1:
The patent performs preliminary action by pre-classifying and labeling offset patterns before propagation. Labels are established upfront based on local image characteristics, allowing the system to quickly identify when propagation is appropriate versus when re-creation is needed, thereby optimizing computational resource usage without sacrificing productivity.
Solution Approach 2:
The patent implements feedback through label-based verification during propagation. When an offset is propagated, the system checks whether the label matches the target patch characteristics. If mismatched (indicating occlusion), the system triggers re-creation with updated labels, creating a feedback loop that optimizes computational resource allocation dynamically.
3Measurement precision
If traditional patch-matching algorithms search for similar patches without using offset labels, then accuracy is maintained through exhaustive search, but processing time increases significantly
Solution Approach 1:
The patent applies partial action by using labels to guide the search process rather than performing exhaustive searches. Labels provide a partial but sufficient indication of offset patterns, allowing the system to achieve high accuracy without the time cost of complete enumeration. The label-based approach performs just enough search to establish initial labels, then propagates them efficiently.
Solution Approach 2:
The patent transforms the search parameter from exhaustive coordinate-based exploration to label-based pattern recognition. This parameter change reduces the search space dramatically while maintaining accuracy, as labels encode the essential characteristics of offset patterns without requiring detailed patch-by-patch verification.
Data Source
AI summary
Certain embodiments involve using labels to track high-frequency offsets for patch-matching. For example, a processor identifies an offset between a first source image patch and a first target image patch. If the first source image patch and the first target image patch are sufficiently similar, the processor updates a data structure to include a label specifying the offset. The processor associates, via the data structure, the first source image patch with the label. The processor subsequently selects certain high-frequency offsets, including the identified offset, from frequently occurring offsets in the data structure. The processor uses these offsets to identify a second target image patch, which is located at the identified offset from a second source image patch. The processor associates, via the data structure, the second source image patch with the identified offset based on a sufficient similarity between the second source image patch and the second target image patch.