Real-Time Human Tracking with Color-Depth Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image recognition systems, both fixed and movable, struggle to accurately recognize and track individuals when they are in close proximity, leading to potential failures in response mechanisms such as path adjustment and collision avoidance.
Innovation Solution
An electronic apparatus equipped with a first sensor for color imaging and a second sensor for depth imaging, utilizing a neural network model to identify regions of interest and track objects based on intersection information between color and depth images, even when objects are close by, by merging regions and tracking positions using multiple sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a fixed image recognition system is used, then the system can recognize persons at a distance, but the system fails to recognize persons when they are in close proximity
Solution Approach 1:
The patent combines color image data from a first sensor with depth image data from a second sensor to create a more robust representation of the target object. By merging information from both sensors, the system can recognize persons whether they are far away (using color features) or close by (using depth features), thus resolving the contradiction between distance-specific recognition and adaptability across distances
Solution Approach 2:
The patent introduces depth information as an additional dimension beyond traditional color image data. By utilizing the depth dimension from the second sensor, the system gains the ability to distinguish objects at close distances where color features alone are insufficient, thereby expanding recognition capability across different distance ranges
2Adaptability or versatility
If a movable image recognition system is used, then the system can operate in close proximity to persons, but the system fails to track persons when the distance becomes very short
Solution Approach 1:
The patent merges color image information with depth image information to maintain reliable tracking in close proximity. When the movable system approaches a person, the depth sensor provides accurate distance measurements that complement the color image data, ensuring continuous reliable tracking even when feature points become scarce at very short distances
Solution Approach 2:
The depth image from the second sensor acts as an intermediary that bridges the gap when color image-based tracking fails at close distances. The depth information provides an alternative basis for tracking that becomes particularly valuable when the movable system is in very close proximity to the target, thus maintaining tracking reliability
3Measurement precision
If only color imaging is used, then the system can identify objects at a distance, but the system cannot accurately identify objects in close proximity
Solution Approach 1:
The patent merges the output of a neural network model processing color images with depth image data from a second sensor. This combination allows the system to accurately identify objects whether they are far away (where color features dominate) or close by (where depth features become critical), thereby improving overall identification capability across all distance ranges
Solution Approach 2:
The patent creates a composite data representation by combining color image data and depth image data, analogous to composite materials in engineering. This composite information structure leverages the strengths of both sensors - color information for distant objects and depth information for close objects - achieving accurate identification across the full range of distances
Data Source
AI summary
An electronic apparatus is provided. The electronic apparatus includes a first sensor configured to obtain a color image, a second sensor configured to obtain a depth image, a memory storing a neural network model, and a processor configured to, based on a first color image being received from the first sensor, obtain a first region of interest by inputting the first color image to the neural network model, and identify whether a distance between an object included in the first region of interest and the electronic apparatus is less than a threshold distance.