Variable Epoch Genetic Algorithm for Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine-learning techniques, such as genetic algorithms and backpropagation, are inefficient for generating and training neural networks, especially when dealing with large data sets or complex neural network structures, and often result in overfitting, making them costly and time-consuming for organizations to implement effectively.
Innovation Solution
An automated model building system that combines a genetic algorithm with variable epoch sizes and selective backpropagation, allowing for the generation and training of neural networks applicable to multiple machine-learning problems like regression, classification, and reinforcement learning, by using optimization techniques like derivative-free optimizers or extreme learning machines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If genetic algorithms are used to generate neural networks, then model accuracy can be achieved comparable to data scientist-created models, but the number of iterations required to converge increases significantly
Solution Approach 1:
The patent applies dynamics by making the epoch size variable rather than fixed. The system dynamically adjusts the number of epochs between genetic algorithm iterations based on convergence metrics. When the model reaches a satisfactory accuracy threshold, the system reduces the epoch size to minimize unnecessary iterations, thereby resolving the contradiction between achieving high accuracy and reducing convergence time.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring model performance metrics after each genetic algorithm iteration. This feedback is used to determine whether to continue with full epoch cycles or to terminate early when convergence criteria are met, allowing the system to balance accuracy requirements with time efficiency.
2Quantity of substance
If backpropagation is used to train large neural networks with large data sets, then model training can be performed, but the training speed becomes slow
Solution Approach 1:
The patent segments the training process into distinct phases: genetic algorithm-based model generation phases followed by selective backpropagation refinement phases. This segmentation allows the system to handle large data sets efficiently by using the genetic algorithm for broad model exploration and only applying computationally intensive backpropagation when necessary, thereby maintaining training speed while processing large quantities of data.
Solution Approach 2:
The system changes parameters dynamically by adjusting the frequency and intensity of backpropagation based on the complexity of the data set and the current model performance. For large data sets, the system reduces backpropagation frequency and uses smaller batch sizes, allowing it to train on extensive data while maintaining acceptable training speeds.
3Reliability
If backpropagation is used extensively, then neural networks can be trained effectively, but overfitting occurs reducing generalization to other data sets
Solution Approach 1:
The patent maintains continuity of useful action by combining genetic algorithm operations with selective backpropagation throughout the training process. The genetic algorithm continuously generates diverse model architectures that inherently resist overfitting, while periodic backpropagation refinements improve training effectiveness. This continuous, balanced approach ensures both effective training and good generalization to unseen data.
Solution Approach 2:
The system applies partial backpropagation action rather than continuous full-strength backpropagation. By applying backpropagation selectively and partially, the system achieves sufficient training effectiveness while avoiding the excessive action that leads to overfitting, thereby maintaining both training quality and model generalization capability.
4Measurement precision
If specialized machine learning platforms are used for specific problem types, then problem-specific performance is optimized, but the system cannot efficiently handle multiple types of machine learning problems
Solution Approach 1:
The patent implements universality by creating a hybrid training framework that can handle multiple types of machine learning problems (classification, regression, reinforcement learning) through a single unified system. The genetic algorithm component provides problem-agnostic model generation, while the selective backpropagation component adapts to specific problem types, allowing the system to maintain high accuracy across diverse problem domains without requiring separate specialized platforms.
Data Source
AI summary
A method includes generating, by a processor of a computing device, a first plurality of models (including a first number of models) based on a genetic algorithm and corresponding to a first epoch of the genetic algorithm. The method includes determining whether to modify an epoch size for the genetic algorithm during a second epoch of the genetic algorithm based on a convergence metric associated with at least one epoch that is prior to the second epoch. The second epoch is subsequent to the first epoch. The method further includes, based on determining to modify the epoch size, generating a second plurality of models (including a second number of models that is different than the first number) based on the genetic algorithm and corresponding to the second epoch. Each model of the first plurality of models and the second plurality of models includes data representative of neural networks.