Deep Reinforcement Module Networks for Autonomous Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial intelligence systems for managing sensor networks face challenges in efficiently tasking and optimizing resources due to high computational costs, memory requirements, and the complexity of training large networks, especially in dynamic environments like space situational awareness, where automation is crucial but costly and complex to develop and maintain.
Innovation Solution
The introduction of deep reinforcement modules (DReMs) that can be trained individually and connected to form a network, allowing for modular, self-optimizing systems that can adapt to changing mission environments by decomposing complex functions into simpler modules, enabling efficient resource allocation and data fusion across interconnected devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If conventional large deep learning networks are used for autonomous decision-making, then decision-making capability is improved, but computational cost and memory resources increase significantly
Solution Approach 1:
The patent divides a large deep learning network into multiple smaller Deep Reinforcement Modules (DReMs), each capable of performing specific decision-making functions independently. These modules can be trained separately and then composed together, reducing the computational burden and memory requirements while maintaining autonomous decision-making capability.
Solution Approach 2:
The patent combines multiple small DReM networks into a unified system that functions as a complete autonomous decision-making architecture. By merging specialized modules rather than using a single large network, the system achieves both computational efficiency and comprehensive decision-making capability.
2Adaptability or versatility
If deep reinforcement architectures are trained with substantial stochastic training and memorized experiences, then complex functions are learned, but the risk for catastrophic forgetting increases
Solution Approach 1:
By segmenting the learning process into multiple specialized DReM modules, each module can focus on learning specific functions with targeted training data. This reduces the need for extensive stochastic training and memorization across a single large network, thereby reducing catastrophic forgetting while maintaining complex function learning capability.
3Ease of operation
If traditional hard-coded tasking algorithms are developed and maintained for individual systems, then resource management is achieved, but development cost and optimization complexity increase
Solution Approach 1:
The DReM networks are trained to autonomously perform tasking and resource management functions without requiring manual hard-coding. The systems learn optimal resource allocation and tasking strategies through reinforcement learning, eliminating the need for expensive development and continuous optimization of traditional algorithms.
Solution Approach 2:
The patent replaces traditional mechanical hard-coded algorithms with learned neural network models. Instead of manually programming tasking rules, the system uses DReM networks that automatically learn and adapt resource management strategies, reducing development costs and optimization complexity.
Data Source
AI summary
A novel architecture for a network of deep reinforcement modules that enables cross-functional and multi-system coordination of autonomous systems for self-optimization with a reduced computational footprint is disclosed. Each deep reinforcement module in the network is comprised of either a single artificial neural network or a deep reinforcement module sub-network. DReMs are designed independently, decoupling each requisite function. Each module of a deep reinforcement module network is trained independently through deep reinforcement learning. By separating the functions into deep reinforcement modules, reward functions can be designed for each individual function, further simplifying the development of a full suite of algorithms while also minimizing training time. Following training, the DReMs are integrated into the full deep reinforcement module network, which is then refined through additional reinforcement training or genetic multi-objective optimization to maximize the overall performance of the network.