Human Object Segmentation with Ground Plane Feet Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional object segmentation techniques often inaccurately segment human objects from images due to missing feet regions, especially when the feet are close to the ground plane or obscured by shadows, and require undesirable pre-capture of static background images or noisy depth data.
Innovation Solution
An object segmentation apparatus that uses a combination of color and depth sensors to generate point clouds, detect the ground plane by clustering vectors, and recover missing feet regions by identifying foreground pixels in a defined area based on distinct image parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional object segmentation techniques are used to segment human objects from images, then the segmentation process is simple and fast, but the feet region is missing or inaccurate due to proximity to ground plane and shadow interference
Solution Approach 1:
The patent combines multiple segmentation approaches (color-based segmentation, depth-based segmentation, and shadow detection) into a unified system. By merging these different techniques, the system achieves more accurate feet region segmentation than any single method could provide alone, resolving the contradiction between segmentation accuracy and system complexity.
Solution Approach 2:
The patent introduces shadow detection as an intermediary component that identifies shadow regions and provides correction information to the segmentation process. This intermediary helps distinguish between actual foot boundaries and shadow artifacts, improving segmentation accuracy without requiring complete system redesign.
2Ease of operation
If pre-determined static background image subtraction is used for segmentation, then the segmentation process is straightforward, but it requires capturing a sequence of images when the object is not present, which is time-consuming and impractical
Solution Approach 1:
The patent performs preliminary shadow detection and ground plane identification on the current image frame before final segmentation. By preparing shadow masks and ground plane models in advance during the segmentation process itself, the system eliminates the need for separate background capture sessions, reducing time loss while maintaining ease of operation.
3Loss of information
If depth sensor data is used for segmentation, then depth information is available for 3D reconstruction, but noisy depth values with invalid measurements lead to erroneous segmentation results
Solution Approach 1:
The patent applies different processing qualities to different regions of the depth image. Valid depth regions are processed with full depth information, while regions with noisy or invalid depth values are handled differently using color-based segmentation or shadow detection. This local quality approach maintains depth information completeness while ensuring segmentation reliability in problematic areas.
Solution Approach 2:
The patent converts the harmful effect of shadow regions (which cause segmentation errors) into a beneficial feature by using shadow detection to identify these regions and adjust the segmentation process accordingly. The shadow detection mechanism, which could be seen as an additional complexity, actually improves reliability by preventing erroneous segmentation in shadowed areas.
4Productivity
If simple color-based segmentation is used, then the processing is fast and computationally efficient, but the feet region near the ground plane is not accurately segmented due to similar colors and shadow interference
Solution Approach 1:
The patent divides the segmentation process into multiple stages: initial color-based segmentation for fast processing, followed by shadow detection to identify problematic regions, and then targeted refinement in those regions. This multi-level segmentation approach maintains high processing speed for most of the image while applying more precise methods only where needed, preserving both productivity and feet region accuracy.
Data Source
AI summary
An object segmentation system that includes a first type of sensor, a second type of sensor and a control circuitry. The first-type of sensor captures a sequence of color image frames of a scene. The second-type of sensor captures a depth image for each corresponding color image frame of the sequence of color image frames. The control circuitry generates a point cloud for an input color image frame. The control circuitry segments a foreground human object from a background of the input color image frame. The control circuitry detects a ground plane for the scene captured in the input color image frame. The control circuitry recovers a feet region in a defined region from a level of the detected ground plane. The control circuitry extracts the foreground human object with the recovered feet region from the background of the input color image frame, based on the detection of ground plane.