Mobile Cleaning Robot Escape Behavior Using Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous cleaning robots often get stuck due to obstacles or entanglement with textiles, requiring user intervention to resolve, which is undesirable for autonomous operation.
Innovation Solution
A mobile cleaning robot equipped with a sensor circuit and controller circuit that uses a trained robot escape behavior model, developed through reinforcement learning, to detect stuck states and generate control signals to actuators to escape such situations autonomously.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If the cleaning robot uses autonomous navigation to move about the environment, then the robot can perform cleaning tasks independently, but the robot may get stuck due to obstacles or narrow spaces
Solution Approach 1:
The system performs preliminary detection of stuck states by monitoring sensor data (wheel rotation, motor current, vibration) before the robot becomes completely immobilized. Escape policies are pre-trained through reinforcement learning to handle various stuck scenarios, enabling the robot to proactively resolve issues before they compromise the cleaning mission.
Solution Approach 2:
The system continuously monitors sensor data from motors, wheels, and environmental sensors to detect stuck states. Based on this feedback, the controller applies trained escape policies that adjust motor commands in real-time to resolve the stuck condition, creating a closed-loop control system that improves reliability while maintaining autonomy.
2Productivity
If the robot encounters stuck states frequently, then the robot may require user intervention, but this reduces cleaning productivity and increases loss of time
Solution Approach 1:
The robot autonomously detects stuck states through sensor monitoring and self-resolves them by executing escape policies that adjust motor commands and wheel rotations. This self-service capability eliminates the need for user intervention, preventing productivity loss and ensuring continuous cleaning operation without time interruptions.
Solution Approach 2:
The reinforcement learning model pre-trains multiple escape policies for various stuck scenarios encountered during cleaning missions. When a stuck state is detected, the system quickly applies the appropriate pre-trained policy, minimizing the time lost and maintaining high cleaning productivity throughout the mission.
3Extent of automation
If the robot uses complex sensor circuits and trained models to detect and resolve stuck states, then the robot can autonomously escape from stuck states, but the device complexity increases
Solution Approach 1:
The system uses a unified reinforcement learning framework that processes multiple sensor inputs (motor current, wheel rotation, vibration) through a single trained model to handle various stuck scenarios. This universal approach consolidates what could be multiple specialized detection and resolution systems into one integrated control architecture, managing complexity while achieving high-level autonomy.
Solution Approach 2:
The escape policies dynamically adjust motor control parameters (torque, speed, rotation direction) based on the detected stuck state. Rather than adding complex mechanical components, the system achieves autonomous escape by intelligently modifying operational parameters through the trained model, maintaining relatively simple hardware while enabling sophisticated autonomous behavior.
Data Source
AI summary
Described herein are systems, devices, and methods for controlling a mobile cleaning robot to escape from a stuck state using a learned robot escape behavior model. The model is trained using reinforcement learning at a cloud-computing device or networked devices. A mobile cleaning robot comprises a drive system, a sensor circuit to collect sensor data associated with a detected stuck state, and a controller circuit that can receive the trained robot escape behavior model, and apply the sensor data associated with the detected stuck state to the trained robot escape behavior model to determine an escape policy. The drive system or one or more actuators of the mobile robot can remove the mobile robot from the stuck state according to the determined escape policy.