Robot Navigation with Simultaneous Local Planning and Q-Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional robot navigation techniques lack a simultaneous approach for local path planning and learning, leading to inefficiencies in handling environmental uncertainties and requiring offline learning or a priori training data, which limits adaptability.
Innovation Solution
A method and system for robotic navigation that combines Dynamic Window Approach (DWA) for local path planning with Next best Q-learning (NBQ) for real-time learning, balancing exploration and exploitation, using a Q-tree that dynamically adapts to environmental changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If online planning is used for robot navigation, then the robot can adapt to environmental changes in real-time, but it suffers from repeated planning for minor positional modifications which reduces efficiency
Solution Approach 1:
The navigation problem is segmented into global path planning (offline) and local path planning (online). The global planner provides a rough path divided into way-points, while the local planner handles real-time adjustments. This segmentation allows the system to maintain adaptability through local planning while improving efficiency by avoiding repeated global planning.
Solution Approach 2:
Global path planning is performed in advance to obtain a preliminary path with way-points. This preliminary action provides a framework that guides subsequent local planning, reducing the computational burden of real-time decision-making while maintaining adaptability to environmental changes.
2Productivity
If learning algorithms are used to circumvent repeated planning, then navigation efficiency improves, but they require offline learning or a priori training data which limits adaptability
Solution Approach 1:
The patent merges online planning and learning algorithms into a unified framework where they guide each other simultaneously. The Q-learning component learns from real-time interactions with the environment, while the planner provides structured guidance. This combination achieves both efficiency (by learning from experience) and adaptability (by continuously learning in the current environment without requiring pre-training).
Solution Approach 2:
The system performs self-learning through Q-learning, where the robot learns navigation strategies through its own interactions with the environment. The Q-tree is automatically updated with new experiences, enabling the system to improve its performance over time without external training data while maintaining adaptability to new situations.
3Ease of manufacture
If conventional learning and planning algorithms act independently, then each can be optimized separately, but they fail to guide each other simultaneously which reduces overall navigational performance
Solution Approach 1:
The patent integrates planning and learning into a unified architecture where the planner generates candidate actions and the learning component (Q-learning) evaluates and improves upon them. The Q-tree stores learned knowledge that guides future planning decisions. This merging allows both components to benefit from each other, improving navigational performance while maintaining implementation feasibility through modular integration.
Data Source
AI summary
In conventional robot navigation techniques learning and planning algorithms act independently without guiding each other simultaneously. A method and system for robotic navigation with simultaneous local path planning and learning is disclosed. The method discloses an approach to learn and plan simultaneously by assisting each other and improve the overall system performance. The planner acts as an actuator and helps to balance exploration and exploitation in the learning algorithm. The synergy between dynamic window approach (DWA) as a planning algorithm and a disclosed Next best Q-learning (NBQ) as a learning algorithm offers an efficient local planning algorithm. Unlike the traditional Q-learning, dimension of Q-tree in the NBQ is dynamic and does not require to define a priori.