Fully Convolutional Network Tiling for Variable-Size Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Fully convolutional networks (FCNs) deployed on hardware accelerators are typically statically compiled for fixed input sizes, making it difficult to process inputs with varying or unknown sizes efficiently, and existing techniques for dynamic processing incur significant overhead and inaccuracies due to neighbor pixel handling.
Innovation Solution
The described techniques tile input data into fixed-size portions, determine valid and dummy regions, and stitch outputs to generate an accurate final output, optimizing computation efficiency and reducing memory usage by leveraging hardware constraints and data sharing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fully convolutional network is statically compiled for fixed input sizes on hardware accelerators, then computation efficiency is improved, but adaptability to process inputs with varying or unknown sizes deteriorates
Solution Approach 1:
The input data is divided into multiple fixed-size tiles that can be processed independently by the statically compiled FCN. Each tile is processed separately through the hardware accelerator, and the results are then stitched together to form the final output. This segmentation allows the system to maintain the efficiency of fixed-size processing while achieving versatility in handling variable-sized inputs.
2Adaptability or versatility
If a fully convolutional network is dynamically compiled to process inputs with varying sizes, then adaptability is improved, but computation efficiency and overhead deteriorate
Solution Approach 1:
The system performs preliminary actions by dividing the input into fixed-size tiles before processing. This pre-processing step enables the use of a statically compiled FCN that is optimized for fixed-size inputs, thereby maintaining high computation efficiency. The tiling approach eliminates the need for dynamic compilation while still allowing the network to handle variable-sized inputs effectively.
3Productivity
If fixed-size inputs are used for statically compiled FCN, then computation efficiency is improved, but measurement precision of pixel-wise results deteriorates due to inaccurate boundary pixels
Solution Approach 1:
The system applies local quality by differentiating between accurate and inaccurate pixel regions in the output. Boundary pixels that are affected by the fixed-size tiling are identified as having lower quality or accuracy. The stitching process carefully handles these regions, combining results from multiple tiles while accounting for the reduced accuracy at boundaries. This approach maintains overall computation efficiency while being transparent about the localized precision limitations.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing inference computations of a fully convolutional neural network receiving inputs with different sizes. One of the methods include receiving a new input to be processed by a fully convolutional neural network, the new input having a first size different from a fixed size that the fully convolutional neural network is configured to process; determining, one or more fixed-size inputs from the new input, each fixed-size input having the fixed size; obtaining a respective fixed-size output generated by the fully convolutional neural network performing inference computations for each of the one or more fixed-size inputs; and generating, from the respective fixed-size outputs comprising one or more invalid pixel values, a final output that is equivalent to an output that would be generated by processing the new input using the fully convolutional neural network.