Self-Learning Neural Control With Mediated Environment Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural networks require precise formulation of problems and target variables for training, limiting their ability to solve new, unknown problems without modeling the entire environment.
Innovation Solution
A method involving a first agent with a first neural network that incorporates emotional inputs and a second neural network to model the environment, allowing autonomous learning and adaptation without predefined references.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional artificial neural networks are used for training, then the target variable must be precisely specified and the problem must be precisely formulated, but this limits the ability to solve new, unknown problems without modeling the entire environment
Solution Approach 1:
The patent introduces a second neural network as an intermediary that generates expected reactions from the first neural network's output vectors. This mediator network allows the system to learn without requiring precise problem formulation or target variables, as the second network provides expected outcomes that guide the learning process without needing to model the entire environment.
Solution Approach 2:
The system implements feedback by comparing the second neural network's expected reactions with actual outcomes and using this comparison to train the first neural network. This feedback mechanism enables autonomous learning and adaptation to new problems without requiring predefined target variables or comprehensive environment modeling.
2Reliability
If monitored training with reference vectors is used, then training can be performed with optimization methods, but reference vectors are necessary which limits autonomous learning
Solution Approach 1:
The system achieves self-service by having the second neural network generate expected reactions that serve as training targets for the first neural network. This eliminates the need for external reference vectors or human-provided training data, allowing the system to autonomously generate its own training signals and learn without external supervision.
Solution Approach 2:
The feedback loop compares expected reactions from the second network with actual outputs, automatically generating training signals that enable reliable training while maintaining full automation. The system uses its own internal predictions as references, eliminating dependence on external reference vectors.
3Adaptability or versatility
If reinforcement learning is used to train self-acting systems, then maximum future reward can be achieved, but the entire world including all rules and axioms must be modeled for training
Solution Approach 1:
The second neural network acts as an intermediary that replaces the need for comprehensive world modeling. Instead of requiring all environmental rules and axioms to be explicitly modeled, the second network learns to predict outcomes based on the first network's actions, providing a simplified path to reinforcement learning without full environment specification.
Solution Approach 2:
The system achieves universality by using the second neural network to handle multiple functions: generating expected reactions, providing training targets, and enabling learning across different problems without requiring problem-specific world models. This multi-functional approach allows the same architecture to solve diverse problems without redundant environment modeling.
Data Source
AI summary
A method is provided for controlling a technical system using a first neural network of an agent. A first input vector and a current state (ht) of the first network are converted together into a new state (ht+1) of the first network, from which state a first output vector of the first network is generated. The first output vector of the first network is fed to a second neural network. A first output vector of the second network representing an expected reaction of the second network to the first output vector of the first network, is generated from the new state (wt+1) of the second network. The first output vector of the second network is compared to the first input vector of the first network, in order to train the first network.