One-Dimensional Convolution Segmentation for Neural Network Speed
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network models require substantial memory and computing power, and multiple trainings are often necessary to achieve performance requirements, leading to inefficiencies in processing time and resource utilization.
Innovation Solution
A novel technique is introduced that modifies lower convolution layers of neural networks from two-dimensional to separate one-dimensional convolutions, reducing complexity and increasing speed without compromising accuracy, and uses class-specific weight fillers to enhance convergence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional neural network models with two-dimensional convolution layers are used, then model accuracy can be maintained, but processing time and computational resource requirements increase substantially
Solution Approach 1:
The patent segments the two-dimensional convolution operation into separate one-dimensional convolution operations. Instead of applying a 2D filter directly to 2D image data, the method first applies 1D convolutions along horizontal dimensions, then applies 1D convolutions along vertical dimensions to the intermediate results. This segmentation reduces computational complexity and processing time while maintaining the same feature extraction capability and model accuracy as traditional 2D convolutions.
2Measurement precision
If conventional neural network models with two-dimensional convolution layers are used, then model accuracy can be maintained, but computational power requirements increase substantially
Solution Approach 1:
The patent segments the two-dimensional convolution operation into separate one-dimensional convolution operations. Instead of applying a 2D filter directly to 2D image data, the method first applies 1D convolutions along horizontal dimensions, then applies 1D convolutions along vertical dimensions to the intermediate results. This segmentation reduces computational complexity and processing time while maintaining the same feature extraction capability and model accuracy as traditional 2D convolutions.
3Reliability
If multiple trainings are performed to achieve performance requirements, then model performance can be optimized, but resource utilization efficiency decreases
Solution Approach 1:
The patent changes the fundamental parameter of convolution operation from two-dimensional to separate one-dimensional operations. This parameter change reduces the computational burden of each training iteration, allowing models to converge faster with fewer training cycles. The reduced computational complexity per iteration improves resource utilization efficiency while still achieving the required model performance through optimized training processes.
Data Source
AI summary
A mechanism is described for facilitating smart convolution in machine learning environments. An apparatus of embodiments, as described herein, includes one or more processors including one or more graphics processors, and detection and selection logic to detect and select input images having a plurality of geometric shapes associated with an object for which a neural network is to be trained. The apparatus further includes filter generation and storage logic (“filter logic”) to generate weights providing filters based on the plurality of geometric shapes, where the filter logic is further to sort the filters in filter groups based on common geometric shapes of the plurality of geographic shapes, and where the filter logic is further to store the filter groups in bins based on the common geometric shapes, wherein each bin corresponds to a geometric shape.