Robot Motion Control Using Occupancy-Coupled Multi-Agent Q-Functions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for trajectory planning in multi-agent systems, such as autonomous vehicles in mixed traffic, face exponential runtime and memory requirements, making them impractical for larger state and action spaces.
Innovation Solution
A method that models interactions among agents using a coupling term based on occupancy measures, reducing computational complexity and memory requirements by decoupling Q-functions from the product space of all agents' states, allowing for efficient trajectory planning and control policy derivation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Q-functions are established for each agent in a multi-agent system to determine optimal movement policies, then the accuracy of trajectory planning is improved, but the computational complexity and memory requirements increase exponentially
Solution Approach 1:
The patent segments the joint Q-function into individual Q-functions for each agent, where each agent's Q-function depends only on its own state and the occupancy measures of other agents. This segmentation transforms the exponential complexity of the joint Q-function into polynomial complexity of individual Q-functions, while still capturing multi-agent interactions through occupancy measures.
Solution Approach 2:
The patent introduces occupancy measures as a new dimensional representation that aggregates spatial-temporal information about other agents' positions. Instead of considering the full product space of all agents' states, the system projects interactions onto the occupancy measure dimension, reducing the state space from exponential to polynomial size while preserving essential interaction information.
2Measurement precision
If Q-functions are established for each agent in a multi-agent system to determine optimal movement policies, then the accuracy of trajectory planning is improved, but the memory requirements increase exponentially
Solution Approach 1:
The patent segments the joint Q-function into individual Q-functions for each agent, where each agent's Q-function depends only on its own state and the occupancy measures of other agents. This segmentation transforms the exponential memory requirements of storing the full joint Q-function into polynomial memory requirements for storing individual Q-functions and occupancy measures.
Solution Approach 2:
The patent extracts the interaction information from the full joint state space and represents it separately through occupancy measures. This extraction allows the system to store only the essential interaction information (agent positions and occupancy probabilities) rather than the complete joint state space, dramatically reducing memory requirements while preserving interaction effects.
3Measurement precision
If the method models each agent's behaviour as a Q-function over the product space of the states of all agents, then the accuracy of movement policy determination is improved, but the computation time increases to exceed three days for small scenarios
Solution Approach 1:
The patent segments the computation of the joint Q-function into separate computations of individual Q-functions. Each agent's Q-function is computed independently based on its own state and occupancy measures of other agents, eliminating the need to compute the full joint Q-function over the product space. This segmentation reduces computation time from exponential to polynomial complexity.
Solution Approach 2:
The patent changes the computational dimension from the full product space of all agents' states to the occupancy measure space. By computing Q-functions over individual agent states combined with occupancy measures rather than the joint state space, the system reduces the computational dimension from exponential to polynomial, making real-time computation feasible.
Data Source
AI summary
A method for controlling a robot is described. The method includes acquiring sensor data representing an environment of the robot, identifying one or more objects in the environment of the robot from the sensor data, associating the robot and each of the one or more objects with a respective agent of a multiagent system, determining, for each agent of the multiagent system, a quality measure which includes a reward term for a movement action at a position and a coupling term which depends on the probabilities of the other agents occupying the same position as the agent at a time, determining a movement policy of the robot that selects movement actions with a higher value of the quality measure determined for the robot with higher probability than movement actions with a lower value of the quality measure, and controlling the robot according to the movement policy.