Deep Reinforcement Learning Robot Control for Collision-Free Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mobile robot autonomous driving methods face challenges such as getting stuck in local minima, excessive dependence on accurate maps, and lack of generalization in various environments, particularly in complex and crowded settings, which affect their navigation and collision avoidance capabilities.
Innovation Solution
A robot control method based on deep reinforcement learning (DRL) using an actor-critic algorithm, where an agent learns a shared policy through state data from a robot cluster to navigate and avoid obstacles, allowing for fast, reliable navigation without requiring sequential environment structure generation or path planning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If rule-based algorithms (artificial potential field, dynamic window access) are used for local planning, then the control logic is simple and easy to implement, but the robot becomes caught in local minima and lacks generalization in various environments
Solution Approach 1:
The patent replaces traditional mechanical rule-based control algorithms with a deep reinforcement learning-based neural network controller. The DRL agent learns optimal navigation policies through simulation training, substituting the rigid rule-based system with an adaptive intelligent system that generalizes across different environments without getting trapped in local minima.
Solution Approach 2:
The patent changes the control parameters from fixed rule-based thresholds to dynamically learned policies from deep reinforcement learning. The neural network processes sensor inputs and generates control commands based on learned patterns, allowing the robot to adapt to various environments while maintaining reliable navigation performance.
2Reliability
If deep reinforcement learning is used for robot control, then the robot achieves better generalization and avoids local minima, but the training complexity and computational resources increase
Solution Approach 1:
The patent performs preliminary training of the deep reinforcement learning agent in a simulated environment before deploying to real robots. The simulation phase pre-trains the neural network policies, allowing the system to achieve reliable navigation without requiring complex real-time training infrastructure during actual operation.
Solution Approach 2:
The patent uses simulation environments as copies of real-world scenarios to train the DRL agent. By creating virtual replicas of navigation tasks and environments, the system can perform extensive training iterations without requiring physical robots, thereby reducing training system complexity and computational resource requirements.
3Reliability
If sequential environment structure generation and path planning are used, then the navigation is safe and reliable, but the response time is slow and cannot adapt to dynamic conditions
Solution Approach 1:
The patent implements continuous control through the deep reinforcement learning agent, which generates smooth, continuous control commands rather than discrete sequential planning steps. The neural network processes sensor inputs and outputs control actions in a continuous manner, enabling fast response to dynamic conditions while maintaining navigation safety through learned policies.
Data Source
AI summary
Provided is a system and method for controlling a robot. The method includes: training an agent through an actor-critic algorithm for deep reinforcement learning (DRL), wherein the training of the agent includes: identifying a robot cluster including a plurality of robots each configured to move from a starting point to a destination; obtaining initial state data from the robot cluster, wherein the initial state data includes information about a first location; sending, by an actor, an action to the robot cluster based on the initial state data; obtaining late state data from the robot cluster after the robot cluster has moved based on the action, wherein the late state data includes information about a second location reached by the robot cluster; and inputting a reward to a critic, wherein the reward is based on the initial state data and the late state data.