Automated Evolutionary Synthesis of Custom Convolution Filters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for automatically synthesizing neural networks, such as template-based methods and filter shaping, face challenges with overfitting and computational complexity, particularly when dealing with large datasets, leading to reduced prediction accuracy and inefficiency in generating custom convolution filters for deep neural networks.
Innovation Solution
An automated evolutionary process is employed to build custom convolution filters by incrementally adding new connections and weight variables, pruning, and merging them, while reinitializing the neural network to avoid overfitting and leverage specialized hardware like GPUs, thereby improving prediction accuracy and reducing computational costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If template-based methods and filter shaping are used to automatically synthesize neural networks, then the synthesis process can be automated, but overfitting occurs and prediction accuracy decreases
Solution Approach 1:
The patent changes the parameters of the neural network by adding new connections and weight variables incrementally, then pruning and merging them. This dynamic parameter adjustment allows the network to adapt to the training data without overfitting, improving prediction accuracy while maintaining automation.
Solution Approach 2:
The patent employs a dynamic evolutionary process where the neural network structure is continuously modified through adding, pruning, and merging connections. This dynamic approach allows the network to evolve optimally for the given task, resolving the contradiction between automation and prediction accuracy.
2Extent of automation
If template-based methods and filter shaping are used to generate custom convolution filters, then the synthesis process can be automated, but computational complexity increases
Solution Approach 1:
The patent extracts only the necessary connections and weight variables from the neural network through a pruning process. By removing redundant connections and keeping only the essential ones, the computational complexity is reduced while maintaining the automation of the synthesis process.
Solution Approach 2:
The patent merges connections with similar weight patterns into consolidated filters. This merging process reduces the total number of parameters and computations required, thereby decreasing computational complexity while preserving the automated generation capability.
3Reliability
If the neural network size is increased to improve generalization ability, then prediction accuracy on new data may improve, but computational costs increase
Solution Approach 1:
The patent optimizes the parameters of the neural network by incrementally adding and pruning connections. This results in a network with the minimal necessary size to achieve good generalization, avoiding the computational costs associated with unnecessarily large networks.
Solution Approach 2:
The patent discards redundant connections through pruning and recovers computational efficiency by maintaining only the essential network structure. This allows the network to achieve good generalization ability with reduced computational costs.
Data Source
AI summary
Mechanisms are provided for synthesizing a computer implemented neural network. An initially trained neural network is received and modified by introducing a new hidden layer of neurons and new connections that connect the new hidden layer of neurons to an output layer and a previous layer of neurons previously directly connected to the output layer of neurons to generate a modified neural network. The modified neural network is trained through one or more epochs of machine learning to generate modified weight values for the new connections and the new connections are pruned based on the modified weight values to remove a subset of the new connections and leaving remaining connections in the modified neural network. A merge operation is performed on the remaining connections in the modified neural network to generate a custom convolution filter and modified neural network. The modified neural network is then retrained for deployment.