Robotic Navigation With Dual-Channel Action Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional reinforcement learning (RL) methods struggle with exploring large action spaces in robotic navigation due to exponential growth in the number of unique action combinations, especially with multiple actuators, and lack specific requirements on embedding quality, making it challenging to learn efficient and robust policies.
Innovation Solution
A method and system using a Dual Channel (DC) action embedding approach with a General Value Function Network (GVFN) for directed exploration and a Dual Channel Training (DCT) based autoencoder decoder model to compute embeddings that predict future states and reconstruct actions, enabling efficient navigation in low-dimensional spaces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional reinforcement learning methods are used to handle large action spaces, then the system can process complex robotic navigation tasks, but the exploration of action spaces becomes insufficient and policy learning becomes intractable due to exponential growth in unique action combinations
Solution Approach 1:
The patent transforms the high-dimensional action space into a low-dimensional embedding space where actions are represented as continuous vectors. This dimensional transformation allows the RL agent to explore a compressed representation space rather than the original exponential action space, making policy learning tractable while preserving the ability to handle complex robotic navigation tasks with multiple actuators
2Ease of operation
If hand-crafted heuristics are used to map RL outputs to actions, then the system can handle discrete action choices, but the approach becomes intractable as the number of actions increases
Solution Approach 1:
The patent replaces hand-crafted heuristic mapping mechanisms with a learned neural network embedding model. Instead of using complex rule-based heuristics to map RL outputs to discrete actions, the system learns continuous action embeddings that naturally capture the mapping relationships, simplifying the operation while handling large numbers of actions efficiently
3Ease of manufacture
If conventional embedding approaches are used that assume prior embedding knowledge or hierarchical structure, then the system can learn action representations, but the approach becomes inapplicable when such prior knowledge is unavailable
Solution Approach 1:
The patent implements a self-supervised learning approach where the embedding model learns action representations directly from interaction data without requiring pre-defined hierarchical structures or prior embedding knowledge. The model automatically discovers the embedding space structure through reinforcement learning, making it applicable to unknown action spaces while maintaining ease of implementation
Data Source
AI summary
The present disclosure provides a Reinforcement Learning (RL) based architecture to efficiently learn action embeddings in low dimensional space. In conventional methods, the embeddings are learnt with the sole objective of improving policy learning, and there are no specific requirements on the quality of the embeddings. Initially, the system receives a goal to be reached by a mobile robot and a current location of the mobile robot is obtained. Simultaneously current transition dynamics associated with the plurality of directional actuators are obtained using a Reinforcement Learning (RL) technique. Further, a plurality of embeddings is computed based on the current location of the mobile robot and the current transition dynamics using a trained Dual Channel Training (DCT) based autoencoder decoder model. Finally, a displacement vector for current navigation of the mobile robot is computed based on the computed plurality of embeddings using the RL technique.