Object Counting Using Image and Depth Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object counting systems using only image information face errors due to 'object grouping' and 'over segmentation', where multiple objects are misidentified as a single entity or vice versa, and struggle to determine proper camera installation angles and heights, leading to inaccurate counting and installation verification.
Innovation Solution
A method and apparatus utilizing both image and depth sensors to acquire and process image and depth maps, generating corrected depth maps with edge information, identifying depth patterns, and comparing them with reference patterns to accurately count objects and verify camera installation, reducing errors and improving installation validation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If only image information is used for object counting, then the system complexity is low, but the counting accuracy deteriorates due to object grouping and over segmentation errors
Solution Approach 1:
The patent combines image information from an image sensor with depth information from a depth sensor to create a multi-modal sensing system. This merging of different information types enables more accurate object counting by using depth data to distinguish between objects that appear grouped together in 2D images and to prevent over-segmentation of single objects, thereby resolving the technical contradiction between counting accuracy and system complexity.
Solution Approach 2:
The patent transitions from 2D image information to 3D depth information by incorporating depth sensors. This dimensional change allows the system to perceive spatial relationships and object boundaries more accurately, enabling differentiation between closely positioned objects and proper segmentation of objects based on their actual spatial extent, thus improving counting accuracy without excessive complexity increase.
2Measurement precision
If camera installation angle and height are not verified, then the installation process is simple, but the object detection accuracy deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where the system captures images and depth maps, processes them to detect objects and calculate their positions, then uses this information to determine whether the camera installation angle and height are appropriate. This feedback loop allows the system to automatically verify installation quality and provide guidance for adjustment, ensuring detection accuracy while maintaining installation simplicity through automated verification.
Solution Approach 2:
The system performs self-verification of installation parameters by automatically processing captured images and depth maps to assess whether the camera is properly positioned. This self-service capability eliminates the need for manual installation verification by technicians, maintaining installation simplicity while ensuring detection accuracy through automated quality control.
Data Source
Figure 1
Figure 2(a)~3
Figure 4~4(d)
AI summary
According to one embodiment of the present invention, a method for counting objects involves using an image sensor and a depth sensor, and comprises the steps of: acquiring an image from the image sensor and acquiring a depth map from the depth sensor, the depth map indicating depth information on the subject in the image; acquiring boundary information on objects in the image; applying the boundary information to the depth map to generate a corrected depth map; identifying the depth pattern of the objects from the corrected depth map; and counting the identified objects.