ASRS Robot Controller With Motion-Constrained Route Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated storage and retrieval systems face challenges in efficiently controlling robotic container-handling vehicles due to the need for precise route planning and collision avoidance, particularly when physical constraints such as momentum and deceleration times are not adequately considered in current reinforcement learning methods.
Innovation Solution
Implementing a controller that applies constraints on robot movements to mimic physical limitations, such as requiring robots to maintain direction or stop before changing, and using reinforcement learning to optimize control policies, incorporating observations of the environment and other robots to enhance prediction and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If reinforcement learning methods are used to learn control policies for robotic vehicles, then route planning and collision avoidance capabilities are improved, but the physical constraints such as momentum and deceleration times are not adequately considered leading to unrealistic control actions
Solution Approach 1:
The patent transforms the continuous control problem into a discrete state-space problem by defining specific states (position, velocity, acceleration) and actions (accelerate, maintain, decelerate) with discrete transitions. This discretization allows reinforcement learning to operate effectively while naturally incorporating physical constraints through the state transition model, resolving the contradiction between learning efficiency and physical realism.
Solution Approach 2:
The patent implements dynamic constraints by defining state transitions that reflect physical realities: vehicles can only change velocity within certain bounds, position changes are constrained by velocity, and acceleration is limited. These dynamic constraints are embedded in the environment model, ensuring that learned policies automatically comply with physical limitations while maintaining route planning effectiveness.
2Device complexity
If discrete time steps and simple state models are used in reinforcement learning, then computational complexity is reduced, but the ability to accurately represent complex robot states and control actions is limited
Solution Approach 1:
The patent segments the complex control problem into distinct discrete states (position, velocity, acceleration levels) and actions (accelerate, maintain, decelerate). This segmentation simplifies the state space for reinforcement learning while maintaining sufficient precision to represent the essential dynamics of the robotic vehicles, balancing computational tractability with accurate state representation.
Solution Approach 2:
The patent adds temporal dimension to the state representation by including velocity and acceleration states alongside position, creating a multi-dimensional state space that captures the dynamic nature of the system. This dimensional expansion allows the discrete model to represent complex temporal patterns in vehicle motion without requiring continuous variables, maintaining precision while managing complexity.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
The disclosure relates to a controller for controlling a robot in an automated storage and retrieval system. The robot is configured to travel along the x and y dimensions of a storage grid of columns and to lift storage containers from or lower storage containers into the columns. A processor is configured to receive an observation at a current time step, input the observation to a control policy to generate an action for the controlled robot and output the action to control the controlled robot. The action is one of staying at the current column or moving to an adjacent column along the x or y dimension. If the robot has moved to the current column at the time step preceding the current time step, the adjacent column is constrained to be in a direction the robot has moved from the different column to the current column.