Object Boundary Stabilization Using Depth and Color Image Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth sensors in image processing systems suffer from inaccuracies and artifacts such as shadowy areas and boundary fluctuations, leading to unstable object boundaries in image sequences, which are visually unpleasant and affect segmentation accuracy.
Innovation Solution
An image-processing apparatus that utilizes both depth and color images to refine object boundaries by removing zero-depth artifacts, dangling-pixels, and self-shadows, and applies a moving-template filter for smoothing, thereby stabilizing object boundaries and improving segmentation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth-based object segmentation is used, then object boundary separation is improved, but boundary stability and accuracy deteriorate due to shadowy areas and fluctuations in depth maps
Solution Approach 1:
The patent combines depth map data with color image data to perform object segmentation. By merging the depth information (which provides good object-background separation) with color information (which provides stable boundary definition), the system achieves both accurate separation and stable boundaries, resolving the contradiction between measurement precision and reliability
Solution Approach 2:
The patent uses color image data as an intermediary to stabilize the object boundaries obtained from depth maps. The color information acts as a mediator that fills in the unstable regions caused by shadowy areas and depth fluctuations, providing a reliable boundary definition while maintaining the segmentation effectiveness of depth-based methods
2Measurement precision
If depth map data is used for segmentation, then object-background separation is improved, but visual quality deteriorates due to artifacts and boundary fluctuations
Solution Approach 1:
The patent converts the harmful effect of depth map artifacts into a benefit by using the color image to identify and correct unstable boundary regions. The artifacts in depth maps highlight areas where additional color-based processing is needed, and this combination eliminates the visual artifacts while preserving the accurate object delineation provided by depth data
3Reliability
If color image alone is used for segmentation, then boundary stability is improved, but object-background separation accuracy deteriorates
Solution Approach 1:
The patent performs a two-stage segmentation process: first using depth map data to achieve initial object-background separation, then using color image data to refine and stabilize the boundaries. This segmented approach allows each data type to contribute its strength - depth for separation accuracy and color for boundary stability - resolving the contradiction between these two objectives
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Image-processing apparatus and method for object boundary stabilization in an image of a sequence of image frames are disclosed. The image-processing apparatus includes an image processor that receives a depth image (112a) of a scene from a first-type of sensor and a color image (110a) of the scene from the second-type of sensor. The scene may comprise at least an object-of-interest (118a, 120a, 118b, 120b). A first object mask of the object-of-interest is generated by a depth thresholding operation on the received depth image. Dangling-pixels artifact (124) present on a first object boundary of the first object mask, are removed. The first object boundary is smoothened using a moving-template filter on the color image. A second object mask having a second object boundary is generated based on the smoothening of the first object boundary. The object-of-interest from the color image is extracted based on the generated second object mask.