Mobile Robot Person Tracking Using Fused Face and Depth Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mobile robotic systems face challenges in autonomously tracking and following individuals within environments, particularly when the person's face or head is not visible, and in navigating around obstacles while maintaining a consistent line of sight.
Innovation Solution
The system employs a combination of face detection using a color video camera and motion detection with a depth video camera to determine the location of individuals, transitioning to a following mode where the robotic device uses torso blob-based tracking and repositioning of cameras to maintain tracking, even when the face is not visible, and incorporates obstacle avoidance to ensure continuous following.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If face detection method is used to track persons, then tracking accuracy is improved when face is visible, but tracking fails when face or head is not visible
Solution Approach 1:
The patent segments the detection task into two independent modules: face detection (using color video camera) and motion detection (using depth video camera). Each module operates independently to detect different aspects of the target, allowing the system to function even when one module fails (e.g., when face is not visible). The results from both modules are then fused to achieve robust tracking.
Solution Approach 2:
The patent merges the detection results from the face detection method and motion detection method through result fusion. This combination allows the system to leverage the strengths of both methods: face detection provides high accuracy when face is visible, while motion detection ensures continuous tracking capability even when face is not visible. The fused results enable reliable person location determination under varying conditions.
2Measurement precision
If mobile robotic device follows person closely, then tracking accuracy is improved, but obstacle avoidance capability deteriorates
Solution Approach 1:
The patent introduces an intermediary obstacle avoidance system that operates independently from the tracking system. This mediator monitors the environment for obstacles and intervenes when potential collisions are detected, allowing the robotic device to maintain close following distance for accurate tracking while preventing harmful collisions. The obstacle avoidance system acts as a protective layer between the tracking objective and potential hazards.
3Duration of action of stationary object
If robotic device repositions cameras to maintain line of sight, then continuous tracking is improved, but system complexity increases
Solution Approach 1:
The patent implements dynamic camera repositioning that adapts to the tracking situation. The cameras automatically adjust their positions and orientations to maintain optimal viewing angles for both face detection and motion detection. This dynamic adjustment enables continuous tracking throughout the environment while the system intelligently manages the complexity through automated control algorithms that respond to real-time detection needs.
Data Source
AI summary
Tracking and following technique embodiments are presented that are generally employed to track and follow a person using a mobile robotic device having a color video camera and a depth video camera. A computer associated with the mobile robotic device is used to perform various actions. Namely, in a tracking mode, a face detection method and the output from the color video camera is used to detect potential persons in an environment. In addition, a motion detection method and the output from the depth video camera is also used to detect potential persons in the environment. Detection results obtained using the face and motion detection methods are then fused and used to determine the location of one or more persons in the environment. Then, in a following mode, a mobile robotic device following method is used to follow a person whose location was determined in the tracking mode.