PolyCNN Seed Filter Polynomial Transformations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training deep convolutional neural networks (CNNs) is computationally expensive and prone to over-fitting due to the large number of parameters that need to be learned for each convolutional filter at each layer.
Innovation Solution
The polynomial convolutional neural network (PolyCNN) reduces computational complexity by learning only one convolutional filter at each layer, with other filters being polynomial transformations of this seed filter, known as the 'early fan-out' or 'late fan-out' embodiments, significantly reducing the number of learnable parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If standard CNN architecture with multiple convolutional filters is used, then representational power and performance are improved, but computational complexity and number of learnable parameters increase significantly
Solution Approach 1:
The seed filter serves multiple functions by generating multiple response maps through polynomial transformations. Instead of requiring separate learnable filters for each response map, a single seed filter performs the work of multiple filters by applying different polynomial transformations (e.g., squaring, cubing) to generate diverse feature representations, thereby reducing computational complexity while maintaining representational power
Solution Approach 2:
The invention changes the parameters of the seed filter through polynomial transformations to generate multiple response maps. By applying different polynomial functions (e.g., x^2, x^3, x^4) to the same seed filter weights, the system generates diverse feature representations without requiring separate learnable parameters for each transformation, thus reducing the number of learnable parameters while maintaining adaptability
2Reliability
If standard CNN with fully learnable convolutional filters is used, then performance on perception tasks is improved, but training computational cost increases
Solution Approach 1:
The invention creates copies of the seed filter's response map through polynomial transformations rather than learning separate filters. The seed filter is applied to input data to generate a response map, and then polynomial transformations (e.g., squaring, cubing) are applied to this response map to generate multiple transformed response maps that serve as inputs to subsequent layers, eliminating the need to relearn filter weights for each transformation
3Adaptability or versatility
If standard CNN with large number of parameters is used, then model capacity is improved, but over-fitting risk increases
Solution Approach 1:
A single seed filter with fewer parameters performs the work of multiple filters through polynomial transformations, reducing the total number of learnable parameters in the model. This parameter reduction decreases model capacity slightly but significantly reduces over-fitting risk, especially on small datasets, while the polynomial transformations maintain sufficient representational power through non-linear feature generation
Data Source
AI summary
The invention proposes a method of training a convolutional neural network in which, at each convolutional layer, weights for one seed convolutional filter per layer are updated during each training iteration. All other convolutional filters are polynomial transformations of the seed filter, or, alternatively, all response maps are polynomial transformations of the response map generated by the seed filter.