Deep CNN Query Optimization with Incremental View Maintenance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural network inferences are computationally expensive, especially for tasks like occlusion-based explanation and object recognition in videos, which require repeated inferences on slightly modified inputs, leading to significant computational redundancy and latency.
Innovation Solution
The system employs query-optimized occlusion-based explanations by generating views that materialize tensors from a convolutional neural network, performing partial re-computations based on occlusion changes, and generating heatmaps to indicate the contribution of image portions to the network's output, thereby minimizing redundant computations through incremental view maintenance and multi-query optimization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If repeated inferences are performed on slightly modified inputs for occlusion-based explanation and object recognition in videos, then accurate results are achieved, but computational overhead and latency increase significantly
Solution Approach 1:
The system performs preliminary full inference on the original image to generate initial feature maps and predictions. These preliminary results are stored and reused as the basis for subsequent incremental inferences, avoiding redundant computations while maintaining accuracy for occlusion-based explanation and video object recognition.
Solution Approach 2:
The inference process is segmented into a full initial inference and multiple incremental updates. Each incremental inference only processes the modified portions (occluded regions or changed video frames) rather than reprocessing the entire image or frame sequence, significantly reducing computational overhead while preserving measurement precision.
2Reliability
If full re-computation is performed for each modified input, then accurate results are ensured, but computational redundancy increases
Solution Approach 1:
Instead of performing complete re-computation for each modified input, the system applies partial action by only reprocessing the affected regions (occluded patches or changed frame differences). This partial re-computation approach maintains reliability by focusing computational resources on the specific modifications while avoiding redundant processing of unchanged portions.
Solution Approach 2:
The system discards the notion of complete re-computation and recovers computational efficiency by reusing previously computed feature maps and intermediate results. The incremental inference mechanism recovers only the necessary updates based on input changes, significantly reducing computational redundancy while ensuring accurate results through selective re-computation.
Data Source
AI summary
A method may include generating views materializing tensors generated by a convolutional neural network operating on an image. Determining the outputs of the convolutional neural network operating on the image with a patch occluding various portions of the image. The outputs being determined by generating queries on the views that performs, based at least on the changes associated with occluding different portions of the image, partial re-computations of the views. A heatmap may be generated based on the outputs of the convolutional neural network. The heatmap may indicate the quantities to which the different portions of the image contribute to the output of the convolutional neural network operating on the image. Related systems and articles of manufacture, including computer program products, are also provided.