Robot Local Path Planning with Simultaneous Q-Learning Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional robot navigation techniques struggle with simultaneous local path planning and learning, as existing learning algorithms either work offline or require a priori training data, and are not capable of adapting to changing environments in real-time.
Innovation Solution
The proposed method and system for robotic navigation employ a Simultaneous Learning and Planning Algorithm (SLPA) within the sensing range (SLPA-SR), which synergistically combines dynamic window approach (DWA) for planning and Next Best Q-learning (NBQ) for learning, allowing for dynamic Q-tree dimensioning and real-time adaptation to environmental uncertainties.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If learning algorithms are used to circumvent repeated planning, then planning efficiency is improved, but the algorithms require offline operation or a priori training data
Solution Approach 1:
The patent merges online planning and offline learning into a unified framework where the robot performs local path planning using Dynamic Window Approach (DWA) while simultaneously updating Q-values through reinforcement learning. This integration allows the system to benefit from both the real-time responsiveness of planning algorithms and the efficiency gains from learned policies, eliminating the need to choose between offline learning and online planning.
Solution Approach 2:
The patent implements dynamic Q-tree dimensioning where the size and structure of the Q-tree adapt based on the robot's exploration progress and environmental complexity. The Q-tree expands automatically as new states are discovered during navigation, allowing the learning component to grow dynamically rather than requiring pre-defined state spaces, thus enabling real-time adaptation.
2Measurement precision
If conventional learning algorithms require a priori training data, then learning accuracy is improved, but the system cannot adapt to changing environments in real-time
Solution Approach 1:
The patent implements continuous feedback loops where the robot's navigation experiences are immediately processed to update Q-values. The reinforcement learning component receives real-time feedback from the environment through rewards and penalties based on navigation success, obstacle avoidance, and goal proximity, allowing the system to learn and adapt continuously during operation rather than relying on pre-collected training data.
Solution Approach 2:
The system performs self-learning by generating its own training data through actual navigation tasks. Rather than requiring external training datasets, the robot autonomously explores the environment, collects experience tuples (state, action, reward, next state), and uses this self-generated data to continuously improve its navigation policy through reinforcement learning.
3Manufacturing precision
If online planning is performed for every positional modification, then navigation accuracy is improved, but computational resources are wasted due to repeated planning
Solution Approach 1:
The patent pre-computes and stores Q-values for potential states and actions in a Q-tree structure during exploration phases. When the robot encounters similar states during navigation, it can directly retrieve pre-computed Q-values instead of performing full path planning, significantly reducing computational overhead while maintaining navigation accuracy through the learned policy guidance.
4Device complexity
If the Q-tree dimension is fixed in advance, then system complexity is reduced, but the system cannot adapt to different environmental scales
Solution Approach 1:
The patent implements dynamic Q-tree dimensioning where the Q-tree automatically adjusts its size, granularity, and structure based on the environmental scale and complexity encountered during navigation. The system monitors exploration progress and environmental features to dynamically expand or contract the Q-tree dimensions, allowing the same system to effectively handle environments of varying scales without manual reconfiguration.
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
In conventional robot navigation techniques learning and planning algorithms act independently without guiding each other simultaneously. A method and system for robotic navigation with simultaneous local path planning and learning is disclosed. The method discloses an approach to learn and plan simultaneously by assisting each other and improve the overall system performance. The planner acts as an actuator and helps to balance exploration and exploitation in the learning algorithm. The synergy between dynamic window approach (DWA) as a planning algorithm and a disclosed Next best Q-learning (NBQ) as a learning algorithm offers an efficient local planning algorithm. Unlike the traditional Q-learning, dimension of Q-tree in the NBQ is dynamic and does not require to define a priori.