Depth-wise Partitioning of Deep Convolution Networks for Edge Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for executing deep convolution networks on computationally constrained devices at the network edge face inefficiencies due to overlapping pixel partitions and inadequate adaptability to pre-trained models, leading to reduced inference speed and high memory requirements.
Innovation Solution
A system and method for depth-wise partitioning of deep convolution networks, utilizing a master-slave network architecture that estimates runtime, generates depth-wise indices, and partitions input data and operations based on filter depth, allowing for efficient distribution and execution on slave devices, thereby reducing pixel overlaps and improving inference speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing partitioning methods are used to execute deep convolution networks on edge devices, then the network can be distributed for execution, but pixel overlaps occur and inference speed decreases
Solution Approach 1:
The patent applies segmentation by dividing the convolution operation into depth-wise partitions. Instead of traditional spatial partitioning that causes pixel overlaps, the input feature maps are segmented along the depth dimension (channel dimension), creating non-overlapping partitions that can be processed independently in parallel on different edge devices.
Solution Approach 2:
The patent transitions from spatial partitioning (2D division of image pixels) to depth-wise partitioning (3D division along the channel dimension). This dimensional change eliminates pixel overlaps while maintaining parallel processing capability, as each partition operates on distinct channel data without spatial interference.
2Ease of operation
If deep convolution networks are executed on computationally constrained edge devices, then data processing can be performed closer to the source, but memory requirements become excessively high
Solution Approach 1:
The patent segments the deep convolution network into multiple depth-wise partitions, where each partition contains a subset of filters operating on corresponding input channels. This segmentation reduces the memory footprint on each individual edge device, as only the necessary filters and parameters for that specific partition need to be stored locally.
Solution Approach 2:
By partitioning along the depth dimension rather than storing entire network layers, the patent reduces memory requirements on edge devices. The depth-wise segmentation allows selective loading of only required filter sets, dramatically reducing the quantity of stored parameters compared to executing full network layers locally.
3Adaptability or versatility
If existing partitioning methods are used, then distribution across multiple devices is achieved, but adaptability to pre-trained models is inadequate
Solution Approach 1:
The patent applies local quality by preserving the original filter depth and connectivity patterns of pre-trained models within each depth-wise partition. Each partition maintains the local characteristics of the original network architecture, allowing direct utilization of pre-trained filters and weights without requiring complex reconfiguration or adaptation of the overall network structure.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A growing need for inferencing to be run on fog devices exists, in order to reduce the upstream network traffic. However, being computationally constrained in nature, executing complex deep inferencing models on such devices has been proved difficult. A system and method for partitioning of deep convolution neural network for execution of computationally constraint devices at a network edge has been provided. The system is configured to use depth wise input partitioning of convolutional operations in deep convolutional neural network (DCNN). The convolution operation is performed based on an input filter depth and number of filters for determining the appropriate parameters for partitioning based on an inference speedup method. The system uses a master-slave network for partitioning the input. The system is configured to address these problems by depth wise partitioning of input which ensures speedup inference of convolution operations by reducing pixel overlaps.