Machine Learning Planner for Continuous State Space Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current planning methods for physical systems often rely on discrete state spaces, which can be restrictive and difficult to establish, especially in real-world applications where state spaces are continuous and high-dimensional, limiting their applicability in modeling and solving Markov decision processes.
Innovation Solution
A machine learning device and method that learns an encoder function and transition function from training data to define a Markov decision problem in a continuous latent space, allowing for dynamic programming-based planning without prior knowledge of the state space, enabling planning for physical systems with continuous and high-dimensional observable states.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If discrete state spaces are used for planning, then planning can be performed using standard Markov decision processes, but the approach becomes restrictive and difficult to establish for real-world physical systems with continuous and high-dimensional states
Solution Approach 1:
The patent replaces the traditional mechanical/discrete state space modeling approach with a neural network-based continuous state space representation. The neural network learns to map continuous physical states to discrete planning states, substituting the need for manual discrete model construction with a learned continuous representation that can handle high-dimensional physical systems.
Solution Approach 2:
The patent transforms the state space representation from discrete to continuous by introducing a neural network that maps continuous physical states to latent planning states. This parameter change allows the system to handle continuous and high-dimensional states while maintaining the structured approach needed for planning.
2Extent of automation
If CNN-based architectures are applied on 2D lattices as state spaces, then value iteration networks can be trained end-to-end, but this imposes restrictions on the models that can be learnt and requires prior knowledge of state space
Solution Approach 1:
The patent moves from 2D lattice-based state spaces to a continuous state space represented in a different dimensional framework. By using a neural network to map continuous states to a latent space, the system eliminates the restriction of 2D lattice structures while maintaining the ability to perform end-to-end training through backpropagation.
Solution Approach 2:
The patent introduces a dynamic, learnable state space representation that adapts to the specific problem domain. Instead of fixing the state space structure a-priori as in CNN-based 2D lattice approaches, the neural network learns the appropriate representation dynamically during training, increasing the range of learnable models.
3Adaptability or versatility
If state space is not known a-priori, then the approach can be more flexible, but standard Markov decision process solving methods cannot be directly applied
Solution Approach 1:
The patent performs preliminary action by training the neural network to learn the state space representation and transition dynamics before actual planning occurs. The pre-training phase learns encoder functions and transition models from data, preparing the system for subsequent planning tasks without requiring a-priori knowledge of the specific state space structure.
Solution Approach 2:
The patent introduces an intermediary neural network that bridges the gap between unknown continuous state spaces and structured planning problems. The neural network acts as a mediator that learns to represent continuous states in a form suitable for planning, enabling standard planning methods to be applied after the intermediary learning stage.
Data Source
Figure 1a~1b
Figure 1c~1d
Figure 2a~2b
AI summary
Some embodiments are directed to a machine learning device for learning a planner for a physical system. A machine-learnable encoder function may be configured to map an observable state to a latent state in a latent space, and a machine-learnable latent transition function may be configured to map an action and a first latent state to a second latent state.