Deep CNN Evolution with Filter-Wise Recombination and Error Propagation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing genetic algorithms for training neural networks, particularly convolutional neural networks (CNNs), suffer from inefficiencies in convergence speed and accuracy due to excessive variability and disruption in offspring populations, leading to reduced accuracy and prolonged training times, especially in deep networks.
Innovation Solution
A novel genetic algorithm (EvoCNN) that employs filter-wise recombination and propagating error corrections to evolve CNNs, ensuring entire filters are copied during recombination and using recursive error corrections to mutate weights, thereby constraining the search space and enhancing convergence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional genetic algorithms are used to train deep CNNs, then the search space exploration is extensive, but the convergence speed is slow and accuracy is reduced due to excessive variability and disruption
Solution Approach 1:
The patent segments the CNN into hierarchical layers and groups of neurons, allowing genetic operations to be performed at multiple levels (individual neuron weights, groups of weights, entire filters, and layers) rather than treating the entire network as a single chromosome. This segmentation reduces variability by maintaining structural integrity at each level while still enabling comprehensive search space exploration.
Solution Approach 2:
The patent applies different genetic operations to different parts of the CNN based on their specific characteristics. For example, filters are recombined as complete units to preserve local feature detection capabilities, while individual weights within filters can undergo mutation. This localized approach maintains useful patterns in high-impact regions while allowing exploration in less critical areas.
2Adaptability or versatility
If traditional genetic algorithms with random mutation are used, then diversity in the population is maintained, but convergence accuracy is reduced and training time increases
Solution Approach 1:
The patent incorporates feedback mechanisms where the performance of parent networks is evaluated, and this information guides the selection and combination of features in offspring. Fitter chromosomes are preferentially selected for breeding, and their successful patterns are preserved and propagated. This feedback loop maintains population diversity while directing evolution toward higher accuracy solutions.
Solution Approach 2:
The patent performs preliminary evaluation of parent networks before genetic operations, identifying high-performing filters and weight patterns. These pre-identified useful patterns are then preserved and combined in offspring through selective recombination, rather than relying solely on random mutation. This preliminary action ensures that diversity is maintained through selection rather than random disruption.
3Reliability
If entire filters are recombined as complete units, then useful filter patterns are preserved, but the recombination operation becomes more complex
Solution Approach 1:
The patent segments the CNN into hierarchical layers and groups of neurons, allowing genetic operations to be performed at multiple levels (individual neuron weights, groups of weights, entire filters, and layers) rather than treating the entire network as a single chromosome. This segmentation reduces variability by maintaining structural integrity at each level while still enabling comprehensive search space exploration.
Solution Approach 2:
The patent merges multiple parent networks by combining their filters and weight patterns at different hierarchical levels. During recombination, corresponding filters from parent networks are combined to create offspring, preserving useful patterns from both parents. This merging operation maintains reliability by systematically combining complete filter units rather than randomly mixing individual weights.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An efficient technique of machine learning is provided for training a plurality of convolutional neural networks (CNNs) with increased speed and accuracy using a genetic evolutionary model. A plurality of artificial chromosomes may be stored representing weights of artificial neuron connections of the plurality of respective CNNs. A plurality of pairs of the chromosomes may be recombined to generate, for each pair, a new chromosome (with a different set of weights than in either chromosome of the pair) by selecting entire filters as inseparable groups of a plurality of weights from each of the pair of chromosomes (e.g., "filter-by-filter" recombination). A plurality of weights of each of the new or original plurality of chromosomes may be mutated by propagating recursive error corrections incrementally throughout the CNN. A small random sampling of weights may optionally be further mutated to zero, random values, or a sum of current and random values.