Adaptive Resolution Inference for Neural Network Speed
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for predicting the minimum image resolution required for accurate inference in neural network models are computationally intensive, leading to prolonged inference times.
Innovation Solution
An inference apparatus comprising a resolution prediction unit, a resolution conversion unit, and an inference unit, which uses a first model to predict the minimum resolution and a second model to infer labels from the converted resolution, while reducing computational load by utilizing part of the activation output from a given layer in the first model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the resolution of the image is reduced to shorten inference time, then the inference time is shortened, but the inference accuracy is reduced
Solution Approach 1:
The patent applies dynamics by making the image resolution adaptive rather than fixed. The resolution prediction unit dynamically determines the appropriate resolution for each input image based on its characteristics, allowing the system to adjust between high resolution (for accuracy) and low resolution (for speed) depending on the specific inference task and image content.
Solution Approach 2:
The patent changes the resolution parameter dynamically. Instead of using a fixed resolution for all inferences, the system predicts and adjusts the resolution parameter based on the input image characteristics, enabling optimal balance between inference speed and accuracy for each specific case.
2Measurement precision
If a large amount of computation is used to predict the resolution, then the resolution prediction accuracy is improved, but the inference time is prolonged
Solution Approach 1:
The patent segments the neural network model into multiple layers, where earlier layers process lower-resolution feature maps and later layers process higher-resolution feature maps. This segmentation allows the resolution prediction to be performed efficiently on downsampled feature maps rather than full-resolution images, reducing computational overhead while maintaining prediction accuracy.
Solution Approach 2:
The patent performs preliminary downsampling of the input image to a lower resolution before feeding it to the resolution prediction unit. This preliminary action reduces the computational burden of resolution prediction while still enabling accurate prediction, as the prediction is made on a simplified representation of the input.
3Measurement precision
If the full-resolution image is used for inference, then the inference accuracy is maintained, but the computational load increases
Solution Approach 1:
The patent makes the inference resolution dynamic by using the predicted resolution to guide the inference process. The system infers labels at the predicted resolution rather than always using full resolution, dynamically adapting the computational resources used based on the specific inference requirements.
Solution Approach 2:
The patent introduces an intermediate resolution level between the original full-resolution image and the final inference. The resolution prediction unit acts as an intermediary that determines the appropriate intermediate resolution, balancing the need for accuracy (closer to full resolution) with the need to reduce computational load (lower resolution).
Data Source
AI summary
The resolution prediction unit takes data of a certain resolution as an input and predicts a minimum resolution, among multiple resolution candidates, by which a label of the data can be inferred with a predetermined accuracy, using a first model including multiple layers. The resolution conversion unit converts a resolution of the data to the predicted resolution. The inference unit takes a resolution-converted data as an input and infers a label of the data using a second model including multiple layers and a part of activation output from a given layer in the first model.