Visual Object Removal Using Composite Pixel Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for removing visual objects from images or video streams are inefficient in selecting and replacing image elements, particularly in real-time applications, due to the need for extensive visual adjustments at transitions and lack of consideration for geometric similarity.
Innovation Solution
A method that uses a combination of appearance and geometry similarity measures to select and replace pixels, incorporating a resolution pyramid for iterative improvement and a cost function that minimizes visual disruption, allowing for efficient removal and reinsertion of visual objects with scaling, rotation, and perspective transformation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If image content areas are selected based on visual similarity to replace removed objects, then the replacement content closely matches the background, but extensive manual visual adjustment is required at transitions
Solution Approach 1:
The patent copies pixel values from source areas to target areas using a cost function that automatically identifies the best matching pixels. Instead of manual copying and adjustment, the system automatically copies pixel data from locations with lowest cost values, eliminating manual visual adjustment while maintaining high visual similarity.
Solution Approach 2:
The patent changes the parameter selection from simple visual similarity to a composite cost function that includes multiple parameters: visual similarity, geometric similarity, and distance metrics. This parameter transformation enables automatic optimization of replacement quality without manual intervention.
2Manufacturing precision
If areas for replacement are selected based on visual similarity and distance, then appropriate replacement content is identified, but the selection process is computationally intensive and slow
Solution Approach 1:
The patent segments the image processing into discrete pixel-level operations with a unified cost function. By breaking down the complex selection process into individual pixel cost calculations based on visual similarity, geometric similarity, and distance, the system achieves both high quality selection and improved processing efficiency through systematic decomposition.
Solution Approach 2:
The patent calculates cost functions for all pixels in the image, which is more comprehensive than traditional methods that only examine candidate regions. This excessive action ensures that the best possible replacement pixels are identified throughout the entire image, not just in pre-selected areas, improving both quality and efficiency.
3Ease of manufacture
If traditional image inpainting methods are used to remove visual objects, then object removal is achieved, but geometric relationships and spatial continuity are not preserved
Solution Approach 1:
The patent combines multiple similarity measures (visual similarity, geometric similarity) and distance metrics into a composite cost function. This composite approach integrates both appearance-based and geometry-based criteria, ensuring that replacement pixels maintain both visual consistency and spatial relationships with surrounding structures.
Solution Approach 2:
The patent replaces traditional mechanical inpainting algorithms with a cost function-based selection mechanism. Instead of using fixed geometric transformations or manual warping, the system uses a mathematical cost function to automatically select and place pixels, preserving natural geometric relationships and spatial continuity.
Data Source
Figure 1~3
AI summary
The present invention relates to a method and an image processing system for removing a visual object from an image, for example from a moving image. According to the method, an image is provided in which a visual object to be removed is selected, for which a mask is determined. Pixels outside the mask that are intended to be used for replacing pixels inside the mask are selected. This is done on the basis of a similarity measure, which comprises an appearance similarity measure that represents the visual similarity between a previously selected pixel for replacing the pixel to be replaced and the pixel to be selected. According to the invention, the similarity measure further comprises a geometry similarity measure, which represents the closeness of the pixel to be selected to the pixel among the pixels outside the mask that was previously used for replacing the pixel adjacent to the pixel to be replaced. Then the pixels inside the mask are replaced by copying the properties of the selected pixels to the respective pixels to be replaced.