Chromosome Representation Learning in Evolutionary Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hyperparameter optimization techniques for machine learning algorithms face challenges in efficiently exploring high-dimensional configuration spaces, particularly due to the presence of compensatory and antagonistic parameters, which often result in suboptimal performance trade-offs between objectives like accuracy and training time.
Innovation Solution
The integration of Principal Component Analysis (PCA) to detect and model parameter correlations, allowing for the generation of virtual chromosomes that improve the genetic algorithm's efficiency by decoupling optimization variables and biasing mutation operators towards better solutions, thereby accelerating the optimization process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional genetic algorithms are used to explore the parameter configuration space, then the entire space can be adequately explored through strong diversification mechanisms, but the optimization process is time-consuming and cannot leverage correlations between parameters
Solution Approach 1:
The patent transforms the parameter representation from original configuration space to a rotated coordinate system using principal component analysis. This parameter transformation allows the genetic algorithm to operate in a space where parameter correlations are captured, enabling more efficient optimization that leverages compensatory relationships between parameters while reducing the time required to explore the configuration space effectively
Solution Approach 2:
The patent introduces principal component analysis as an intermediary transformation layer between the original parameter space and the genetic algorithm optimization process. This intermediary rotates the coordinate system to align with directions of maximum variance, creating a new representation that preserves exploration capability while enabling the algorithm to efficiently exploit parameter correlations and reduce optimization time
2Reliability
If the genetic algorithm optimizes for one objective such as inference accuracy, then accuracy can be improved, but training time increases and other objectives like training efficiency are degraded
Solution Approach 1:
By transforming parameters into the principal component space, the patent enables the genetic algorithm to simultaneously optimize multiple objectives including both accuracy and training time. The rotated parameter representation captures compensatory relationships, allowing the algorithm to find configurations that balance competing objectives rather than optimizing for a single metric at the expense of others
Data Source
AI summary
Principal component analysis (PCA) accelerates and increases accuracy of genetic algorithms. In an embodiment, a computer generates many original chromosomes. Each original chromosome contains a sequence of original values. Each position in the sequences in the original chromosomes corresponds to only one respective distinct parameter in a set of parameters to be optimized. Based on the original chromosomes, many virtual chromosomes are generated. Each virtual chromosome contains a sequence of numeric values. Positions in the sequences in the virtual chromosomes do not correspond to only one respective distinct parameter in the set of parameters to be optimized. Based on the virtual chromosomes, many new chromosomes are generated. Each new chromosome contains a sequence of values. Each position in the sequences in the new chromosomes corresponds to only one respective distinct parameter in the set of parameters to be optimized. The computer may be configured based on a best new chromosome.