On-Device AI Frame Processing for Variable Resolution Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI models face challenges with handling varying input resolutions, leading to high latency, increased memory requirements, and power consumption due to zero padding and redundant computations, especially in video inference and dynamic reshaping of input sizes.
Innovation Solution
An in-place transformation method using smart compute unit kernels and an in-place transformation layer to handle multiple resolutions efficiently, reducing the need for reinitialization and re-caching by detecting and transforming regions of interest (RoI) within images or video frames, and operating only on selective areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If zero padding is applied to support different input resolutions, then hardware compatibility is ensured, but latency increases and memory requirements increase
Solution Approach 1:
The patent implements dynamic resolution handling by detecting the actual resolution of input images and adjusting the computational graph accordingly. Instead of using fixed zero-padding, the system dynamically reshapes input tensors to match the actual image dimensions, allowing the same model to handle multiple resolutions without padding. This dynamic adaptation resolves the contradiction by maintaining hardware compatibility while eliminating the latency and memory overhead of zero-padding.
Solution Approach 2:
The patent changes the parameter of input tensor shape from fixed padded dimensions to variable actual image dimensions. By using preprocessor directives with pre-calculated values based on detected resolution, the system adapts the computational graph parameters to match the actual input size. This parameter change allows the model to maintain compatibility across different hardware while reducing the need for zero-padding, thereby lowering latency and memory requirements.
2Adaptability or versatility
If zero padding is applied to support different input resolutions, then hardware compatibility is ensured, but memory requirement increases
Solution Approach 1:
The system dynamically adjusts the computational graph based on the actual input image resolution, eliminating the need for fixed zero-padding memory allocation. By detecting the real resolution and using preprocessor directives to configure the graph accordingly, the memory required is exactly matched to the actual input size rather than allocating memory for padded dimensions. This dynamic approach maintains hardware compatibility while reducing memory requirements.
Solution Approach 2:
The patent changes the memory allocation parameter from fixed padded size to variable actual size. Through preprocessor directives that receive pre-calculated values based on detected resolution, the system allocates memory dynamically matching the actual image dimensions. This parameter change ensures hardware compatibility while minimizing memory usage by eliminating unnecessary padding memory.
3Adaptability or versatility
If conventional AI models process varying input sizes, then multiple resolutions are supported, but reinitialization and re-caching are required
Solution Approach 1:
The patent performs preliminary detection of the input image resolution before processing, using this information to pre-configure the computational graph through preprocessor directives. By detecting the resolution in advance and using it to pre-calculate graph configuration parameters, the system avoids the need for reinitialization and re-caching when handling different resolutions. This preliminary action maintains multiple resolution support while eliminating the time loss associated with reinitialization.
Solution Approach 2:
The patent introduces an intermediary layer between the input image and the computational graph - a resolution detection and preprocessor directive system. This intermediary detects the actual resolution and translates it into preprocessor directives that configure the computational graph accordingly. This intermediary mechanism allows the system to support multiple resolutions without requiring reinitialization and re-caching, as the graph is pre-configured based on the detected resolution.
4Reliability
If video inference processes all frames, then complete information is processed, but redundant computations occur
Solution Approach 1:
The patent applies local quality by detecting changes in RoI scale between consecutive frames and selectively processing only the regions that have changed. Instead of processing the entire frame uniformly, the system identifies the specific local regions (RoIs) that contain new information and applies computational operations only to those regions. This maintains information completeness for changed areas while significantly reducing power consumption by avoiding redundant computations in unchanged regions.
Solution Approach 2:
The patent implements partial action by processing only the necessary portions of video frames - specifically the RoI regions that have changed in scale or contain new information. Rather than processing all frames completely (excessive action), the system selectively processes only the changed regions (partial action), thereby reducing redundant computations and power consumption while maintaining the reliability of processing complete information where needed.
Data Source
AI summary
A method for processing an input frame for an on-device AI model is provided. The method may include obtaining an input frame. The method may include building at least one kernel independent of the scale of the input frame by passing input variables to the at least one kernel using preprocessor directives independent of the scale of the input frame. The method may include inputting the input frame to the on-device AI model including the at least one kernel independent of the scale of the input frame. The method may include processing the input frame in the on-device AI model.