Object-Tracking Robot Navigation Using DRL for Obstacle Avoidance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robot navigation systems face challenges in tracking moving objects while navigating through dynamic environments, as they often lose sight of the target object when avoiding obstacles, leading to failures in consistent tracking.
Innovation Solution
A deep reinforcement learning (DRL) network is trained to combine object tracking and navigation rewards, using tracking quality, navigation path quality, and traversability rewards to guide the robot in maintaining object visibility and avoiding obstacles, enabling simultaneous localization and mapping (SLAM) in both static and dynamic environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the robot prioritizes obstacle avoidance during navigation, then collision risk is reduced, but the robot loses sight of the target object leading to tracking failure
Solution Approach 1:
The patent combines object tracking and navigation into a unified reinforcement learning framework. The DRL agent simultaneously learns tracking policies and navigation policies, merging two previously separate functions into one integrated system that optimizes both tracking reliability and collision avoidance together through combined reward signals
Solution Approach 2:
The system implements feedback mechanisms where the tracking quality reward and navigation reward continuously inform the DRL agent about its performance. The agent uses this feedback to adjust its actions in real-time, balancing the trade-off between maintaining target visibility and avoiding obstacles based on the current environmental context
2Ease of operation
If the robot uses traditional navigation methods to avoid obstacles, then collision avoidance is improved, but the robot cannot maintain continuous tracking of the moving object
Solution Approach 1:
The patent merges traditional navigation capabilities with object tracking by integrating them into a single DRL framework. The agent learns to navigate through environments while continuously tracking objects, combining what were previously separate operational modes into a unified intelligent system
Solution Approach 2:
The system transitions from static, pre-programmed navigation paths to dynamic, adaptive navigation behavior. The DRL agent continuously adjusts its navigation strategy based on real-time tracking quality and environmental conditions, enabling flexible adaptation to moving targets and dynamic obstacles
3Measurement precision
If the robot focuses on maintaining object visibility, then tracking quality is improved, but the robot may collide with obstacles in the environment
Solution Approach 1:
The dual reward system provides continuous feedback to the DRL agent about both tracking precision and collision risk. The tracking quality reward reinforces actions that maintain object visibility, while the navigation reward penalizes actions that lead to collisions, enabling the agent to learn the optimal balance between these competing objectives
Solution Approach 2:
The system dynamically adjusts the weighting and prioritization of different reward components based on the situation. When tracking quality is poor, the system emphasizes actions that improve visibility, while when obstacles are detected, it shifts priority toward safe navigation, effectively changing the optimization parameters in real-time
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A system and method of tracking an object and navigating an object tracking robot includes receiving tracking sensor input representing the object and an environment at multiple times, responsive to the tracking sensor input, calculating positions of the robot and the object at the multiple times, and using a computer implemented deep reinforcement learning (DRL) network trained as a function of tracking quality rewards and robot navigation path quality rewards, the DRL network being responsive to the calculated positions of the robot and the object at the multiple times to determine possible actions specifying movement of the object tracking robot from a current position of the robot and target, determine quality values (Q-values) for the possible actions, and select an action as a function of the Q-values. A method of training the DRL network is also included.