Convolutional Layer Input Scaling for Lower Power and Delay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for scaling inputs to be processed by machine learning models, such as image processing, involve inefficient use of dedicated hardware and software components like camera ISP pipelines and CPU/DSP scaling, leading to high computing resource and power costs, as well as inter-processor communication delays.
Innovation Solution
Utilizing one or more convolutional layers of a machine learning model to perform input scaling, allowing for non-integer stride lengths and avoiding separate hardware and software components, thus reducing inefficiencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If dedicated scaling hardware (camera ISP pipeline) or software components (CPU/DSP) are used for input scaling, then input scaling can be performed, but computing resource cost and power consumption increase significantly
Solution Approach 1:
The patent merges the input scaling function with the machine learning model processing pipeline by implementing scaling operations within the neural network processor itself. The scaling is performed as part of the standard data flow through convolutional layers, eliminating the need for separate dedicated scaling hardware or software components. This integration allows the same processing units to perform both scaling and ML inference, significantly reducing power consumption and resource usage.
Solution Approach 2:
The neural network processor is designed to perform multiple functions: it can execute standard ML inference operations and simultaneously perform input scaling operations. The processing units that normally execute convolutional layers are also capable of performing scaling calculations, making the system multi-functional and eliminating the need for specialized scaling hardware.
2Reliability
If dedicated scaling hardware (camera ISP pipeline) or software components (CPU/DSP) are used for input scaling, then input scaling can be performed, but device area increases
Solution Approach 1:
The patent combines the scaling functionality into the existing neural network processor architecture. By implementing scaling operations within the same processing units that perform ML inference (using convolutional layers with appropriate stride configurations), the system eliminates the need for additional dedicated scaling hardware blocks, thereby conserving device area while maintaining full scaling capability.
3Reliability
If dedicated scaling hardware (camera ISP pipeline) or software components (CPU/DSP) are used for input scaling, then input scaling can be performed, but processing delay increases
Solution Approach 1:
The patent performs scaling operations as part of the preliminary data preparation stage within the ML pipeline itself. By configuring convolutional layers with specific stride parameters, the scaling is performed automatically as data flows into the network, eliminating the need for separate pre-processing scaling steps that would add inter-processor communication delays and processing time.
Solution Approach 2:
The patent merges the scaling operation with the ML inference pipeline execution. Since scaling is performed within the same processing units and data flow path as the ML model operations, there are no inter-processor communication delays or additional processing stages, significantly reducing overall processing time compared to using separate CPU/DSP or ISP pipeline scaling.
4Reliability
If conventional scaling methods are used, then input scaling can be performed, but computing resource cost increases
Solution Approach 1:
The patent merges scaling operations with ML inference operations within the same processing units. The neural network processor executes both scaling and inference using the same computational resources, eliminating the need for separate dedicated scaling engines and reducing overall computing resource consumption.
Solution Approach 2:
The ML model itself performs the scaling operation as part of its native processing capabilities. By configuring convolutional layers with appropriate stride parameters, the model scales inputs automatically during its normal operation without requiring external scaling services or additional computational resources.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques and apparatus for efficient scaling of inputs to be processed by a machine learning model. An example method generally includes receiving, by a machine learning model, an input having a starting size in a plurality of dimensions. The method further includes scaling, by the machine learning model, the input in one or more dimensions of the plurality of dimensions to generate a scaled input, wherein the input is scaled in each respective dimension of the one or more dimensions based on a respective stride length determined based on a starting size in the respective dimension and a target size in the respective dimension, and the respective stride length associated with at least one dimension in the one or more dimensions comprises a non-integer value. The method further includes generating, by the machine learning model, an inference based on the scaled input.