Occluded Object Detection Using Map and Point Cloud Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems inaccurately label images with occluded objects, leading to incorrect ground truth data and increased manual effort in training machine learning models, due to dynamic and static objects obstructing the view in sensor data.
Innovation Solution
Systems and methods for detecting occluded objects in images and sensor data representations by combining image processing techniques with point cloud processing, using classifications and 3D-2D projections to determine occlusions, generating accurate ground truth data with reduced human intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional systems use map labels to generate ground truth labels for images, then the labeling process is automated and efficient, but the accuracy of ground truth data deteriorates when objects are occluded in the images
Solution Approach 1:
The patent introduces an occlusion detection module as an intermediary between the map-based labeling system and the final ground truth generation. This module analyzes sensor data (images, point clouds, depth maps) to detect occluded objects and generates occlusion masks that prevent mislabeling. The intermediary layer resolves the contradiction by maintaining automated efficiency while improving accuracy through selective correction of occluded regions
Solution Approach 2:
The system performs preliminary occlusion detection and mask generation before final label creation. By analyzing sensor data and detecting occlusions in advance, the system prepares correction information that is then applied during ground truth generation. This preliminary action prevents inaccurate labels from being created in the first place, resolving the accuracy-efficiency tradeoff
2Measurement precision
If conventional systems manually quality check and update labels for occluded objects, then the accuracy of ground truth data improves, but the time and manual effort required increases significantly
Solution Approach 1:
The system implements self-service through automated occlusion detection and correction. The occlusion detection module autonomously analyzes sensor data, identifies occluded objects, generates occlusion masks, and applies corrections to ground truth labels without human intervention. This self-service capability maintains high accuracy while eliminating the time-consuming manual quality check process
Solution Approach 2:
The patent replaces the mechanical manual quality check process with an automated computational system. Instead of human labelers visually inspecting and correcting labels, the system uses machine learning models and sensor data processing to automatically detect occlusions and generate corrections, substituting human effort with automated technology
3Measurement precision
If systems process multiple sensor data types (images, point clouds, depth maps) to detect occlusions, then the detection accuracy improves, but the computational complexity and processing time increases
Solution Approach 1:
The patent segments the occlusion detection process into distinct functional modules: image processing module, point cloud processing module, depth map processing module, and occlusion detection module. Each module processes specific sensor data types independently and contributes to the final occlusion detection result. This segmentation manages complexity by dividing the overall system into manageable, specialized components
Solution Approach 2:
The system leverages multiple dimensions of sensor data (2D images, 3D point clouds, depth maps) to detect occlusions. By processing information from different dimensional representations of the same scene, the system achieves more accurate occlusion detection than any single modality could provide alone, using the complementary information from each data type
Data Source
AI summary
In various examples, detecting occluded objects within images or other sensor data representations for autonomous or semi-autonomous systems and applications is described herein. Systems and methods described herein may determine when objects are occluded at portions of images using various techniques. For example, an image may be processed in order to determine classifications associated with objects depicted by the image and, the classifications, along with labels that are projected on the image using a map, may then be used to determine whether one or more of the objects are occluded in the image. For another example, a map may be used to determine first distances to points within an environment and a point cloud may be used to determine second distances to the points within the environment. The distances may then be used to determine whether one or more objects are occluded within the image.