2D Occupancy and Occlusion Views for Depth Map Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for determining the type and state of an object of interest, such as fall detection, face inefficiencies and inaccuracies due to high information complexity in 3D depth maps and issues with occlusion by other objects, leading to suboptimal performance.
Innovation Solution
Generating three 2D occupancy views and three 2D occlusion views from depth maps, feeding these views into separate inputs of a convolutional neural network, with a three-staged architecture including frontend, middle, and averaging sections to process and combine information effectively, utilizing more layers for occupancy views to handle occlusion and reduce dimensionality through max-pooling layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If 3D depth maps are used for object detection, then the amount of information available increases, but the efficiency of neural networks decreases due to difficulty in extracting information
Solution Approach 1:
The patent segments the 3D depth map information into multiple 2D views (front, side, top views) and processes them separately through parallel neural network branches. This segmentation reduces the information complexity in each view while preserving the total information content, enabling more efficient processing by neural networks.
Solution Approach 2:
The patent transforms 3D depth map data into multiple 2D projections by introducing different viewing dimensions. Instead of processing the full 3D data directly, the system creates 2D representations from different angles (front, side, top views), making the information more accessible to 2D convolutional neural networks while maintaining the spatial relationships needed for accurate detection.
2Reliability
If standard 2D view methods are used in neural networks, then detection success rate increases, but accuracy decreases when the object of interest is occluded by other objects
Solution Approach 1:
The patent applies different processing strategies to different parts of the multi-view data. Visible regions are processed through standard convolutional layers, while occluded regions are handled by occlusion-aware modules that use contextual information from other views to infer the hidden object parts. This local differentiation improves accuracy under occlusion conditions.
Solution Approach 2:
The patent merges information from multiple 2D views (front, side, top) and combines visible and occluded region processing results. By integrating data from different viewing angles and processing streams, the system reconstructs a complete understanding of the object even when parts are occluded, improving detection accuracy.
3Measurement precision
If twelve different projective greyscale images are computed by rotating a virtual camera, then 3D shape recognition improves, but aliasing artifacts are produced and computational efficiency decreases
Solution Approach 1:
Instead of computing all twelve projective views, the patent uses a selective approach by computing only three essential 2D views (front, side, top) that provide sufficient geometric information for accurate detection. This partial action reduces computational complexity and avoids aliasing artifacts while maintaining recognition accuracy.
Solution Approach 2:
The patent creates simplified 2D copies of the 3D depth map from different viewing angles rather than rotating the virtual camera to generate all twelve projective images. These copied views are sufficient for the detection task and avoid the computational burden and artifacts of full 360-degree projection.
Data Source
AI summary
The disclosed subject matter relates to a method for determining a type and a state of an object of interest, comprising: generating a depth map of a scene by means of a depth sensor, the scene containing the object of interest and an occlusion object lying between the depth sensor and the object of interest; computing three 2D occupancy views, each in a different viewing direction, and three 2D occlusion views; feeding each of said occupancy and occlusion views into a different input of a trained convolutional neural network; receiving both a class and a bounding box of the object of interest from the convolutional neural network; and determining the type of the object of interest and the state of the object of interest.The disclosed subject matter further relates to a system for carrying out said method.