Pre-trained Neural Network Pool for Efficient AI Model Assembly
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The computational limitations and time required to train large neural networks and ensembles of machine learning systems pose a significant barrier to progress in artificial intelligence, especially for difficult classification problems.
Innovation Solution
A machine-learning computer system uses a pool of predefined functions and pre-trained networks to accelerate the building and training of large neural networks by iteratively selecting and evaluating connections between predefined function nodes or networks, allowing for efficient computational processing and parallelization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large neural networks are built to solve difficult classification problems, then classification accuracy is improved, but computational time and resources increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-training multiple smaller neural networks on different subsets of training data before combining them into an ensemble. These pre-trained networks are then integrated through a combining network, which learns to optimally aggregate their predictions. This approach avoids training one large network from scratch, significantly reducing computational time while maintaining or improving classification accuracy on difficult problems.
2Adaptability or versatility
If the size of neural network is increased to improve performance, then model capacity is improved, but training complexity increases
Solution Approach 1:
The patent segments the training process by dividing the training data into multiple subsets and training separate smaller neural networks on each subset. These segmented networks are then combined through a integrating network. This segmentation reduces the complexity of training individual networks while achieving the model capacity of a larger network through ensemble combination, making the overall training process more manageable and scalable.
3Productivity
If parallel processing on GPUs is used to speed up training, then training speed is improved, but computational resources increase
Solution Approach 1:
The patent segments both the training data and the neural network architecture into multiple smaller components that can be trained in parallel on GPU resources. By training multiple smaller networks simultaneously on different data subsets and then combining their outputs, the system achieves faster training speeds through parallel processing while distributing computational resources more efficiently across multiple smaller tasks rather than concentrating them in one large network.
Data Source
AI summary
A computer system uses a pool of predefined functions and pre-trained networks to accelerate the process of building a large neural network or building a combination of (i) an ensemble of other machine learning systems with (ii) a deep neural network. Copies of a predefined function node or network may be placed in multiple locations in a network being built. In building a neural network using a pool of predefined networks, the computer system only needs to decide the relative location of each copy of a predefined network or function. The location may be determined by (i) the connections to a predefined network from source nodes and (ii) the connections from a predefined network to nodes in an upper network. The computer system may perform an iterative process of selecting trial locations for connecting arcs and evaluating the connections to choose the best ones.