Guided ANN Architecture Evolution Beyond Explored Search Domains
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for discovering novel artificial neural network (ANN) architectures require significant computational resources and are limited to exploring small subspaces of the entire ANN architecture space, making it challenging to identify architectures that could significantly outperform existing ones.
Innovation Solution
A guided evolutionary growth process is employed to discover novel ANN architectures by starting from a minimum possible configuration, incorporating pairwise similarities with reference architectures to guide the evolution away from explored domains, using incremental layer additions and retraining, and calculating fitness scores based on performance and cumulative similarities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If current methods for discovering novel ANN architectures are used, then some architectures can be identified, but the exploration is limited to small subspaces of the entire ANN architecture space and requires significant computational resources
Solution Approach 1:
The ANN architecture space is segmented into multiple subspaces, each representing a different architecture type (e.g., fully connected, convolutional, recurrent). The system independently discovers novel architectures within each subspace using evolutionary growth processes, then aggregates results to achieve comprehensive coverage of the entire architecture space without requiring exhaustive search of all possible architectures simultaneously
Solution Approach 2:
The system performs preliminary actions by pre-defining multiple architecture types and their corresponding evolutionary growth rules before the discovery process begins. This allows the system to efficiently explore different subspaces by applying appropriate growth strategies tailored to each architecture type, reducing the computational overhead of on-the-fly architecture definition and validation
2Measurement precision
If exhaustive search of ANN architecture space is performed, then comprehensive coverage is achieved, but computational resources and time requirements increase significantly
Solution Approach 1:
The system employs dynamic evolutionary growth processes that adaptively explore the ANN architecture space. Instead of static exhaustive search, the evolutionary algorithms dynamically adjust the search trajectory based on fitness evaluations, allowing the system to efficiently navigate toward promising regions of the architecture space while avoiding exhaustive enumeration of all possible architectures
Solution Approach 2:
The system changes key parameters of the search process, including the architecture type being explored, the evolutionary growth rules applied, and the fitness evaluation criteria. By varying these parameters across multiple independent discovery processes, the system achieves comprehensive coverage of the architecture space with reduced time requirements compared to a single exhaustive search
3Adaptability or versatility
If evolutionary growth process is used to discover novel architectures, then uncharted domains are populated, but the process requires calculating pairwise similarities and fitness scores for multiple candidate architectures
Solution Approach 1:
The complexity of the discovery process is segmented into distinct modular components: architecture generation modules that create candidate architectures, similarity calculation modules that compute pairwise similarities using fingerprints, fitness evaluation modules that assess performance, and selection modules that choose architectures for further evolution. This modular segmentation makes the complex process more manageable and efficient
4Adaptability or versatility
If multiple existing ANN architectures are used as reference for guided evolutionary growth, then the discovery process is guided away from explored domains, but calculating cumulative similarities requires comparing each candidate with all reference architectures
Solution Approach 1:
The system performs preliminary action by pre-calculating the fingerprints of all reference architectures before the evolutionary growth process begins. These pre-computed fingerprints serve as a reference library that enables efficient similarity calculations during the discovery process, avoiding the need to re-compute fundamental architecture characteristics repeatedly
Data Source
AI summary
Methods, systems and apparatuses for discovering novel artificial neural network architectures (ANN) architecture are disclosed. One method includes calculating ANN architecture fingerprints including an ANN architecture fingerprint of each of a plurality of existing ANN architectures, creating a plurality of next-generation candidate ANN architectures, calculating a plurality of next-generation candidate ANN architecture fingerprints including an ANN architecture fingerprint of each of the plurality of next-generation candidate ANN architectures, calculating ANN architecture pairwise similarities between each of the plurality of existing ANN architectures and each of the plurality of next-generation candidate ANN architectures using the plurality of existing ANN architecture fingerprints and the plurality of next-generation candidate ANN architecture fingerprints, retraining each of the plurality of next-generation candidate ANN architectures on the training dataset, obtaining a performance score of each of the next-generation candidate ANN architectures, and calculating a fitness score for each of the next-generation candidate ANN architectures.