Non-Integer Scaling in Neural Network Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning systems lack an efficient method for performing non-integer scaling of spatial data within neural networks, which is necessary for emerging techniques like scale selective machine learning and dimensionally-aware neural networks. Existing approaches force non-integer scaling to be done outside the neural network structure, resulting in performance, cost, and power consumption penalties.
Innovation Solution
The proposed solution involves repurposing conventional deep learning layers to perform efficient non-integer scaling within existing hardware. This includes enhancing hardware mappers in deep learning hardware accelerators to support computationally efficient non-integer scaling and extending the capabilities of existing hardware accelerators to perform this scaling operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If non-integer scaling is performed outside the neural network structure using conventional methods, then scaling functionality is achieved, but performance, cost, and power consumption are penalized
Solution Approach 1:
The patent merges non-integer scaling operations with standard convolutional neural network operations by representing scaling factors as rational numbers N/D and implementing them through N convolutions with D-stride. This integration allows scaling to be performed within the existing neural network computational framework rather than as a separate preprocessing or postprocessing step, thereby improving efficiency and reducing power consumption.
Solution Approach 2:
The patent enhances existing hardware accelerators to perform multiple functions: standard convolution operations, pooling operations, and non-integer scaling operations. By making the hardware universal and capable of handling different operation types through configuration rather than requiring dedicated hardware for each function, the system achieves better resource utilization and reduced overhead.
2Adaptability or versatility
If non-integer scaling is performed outside the neural network structure, then scaling is achieved, but device complexity increases
Solution Approach 1:
The patent introduces dynamic configurability to existing hardware accelerators, allowing them to adapt their operation mode based on the task requirements. The hardware can be configured to perform standard convolutions, pooling, or non-integer scaling operations by adjusting control parameters such as stride values and kernel configurations, rather than requiring fixed dedicated hardware for each function.
Solution Approach 2:
The patent changes the operational parameters of existing hardware components to achieve non-integer scaling. By modifying parameters such as convolution stride, kernel size, and padding configurations, the same hardware architecture can perform scaling operations with different factors (e.g., 0.5x, 1.5x, 2.0x) without requiring structural modifications.
3Ease of manufacture
If conventional deep learning layers are repurposed for non-integer scaling, then hardware reuse is improved, but operational complexity increases
Solution Approach 1:
The patent segments the non-integer scaling operation into discrete computational steps that mirror standard convolution operations. By representing the scaling factor N/D as N convolutions with D-stride, each step can be independently implemented using existing hardware modules, making the overall process manageable and compatible with current hardware architectures.
Data Source
AI summary
Processing image data using deep neural networks is critical to many systems that desire to understand objects and their environment using camera sensors. Image scaling is a fundamental processing task required when managing image data. Although it is possible to scale image data using standard computer or graphics processors it would be highly advantageous in terms of throughput, latency and power consumption to manage image scaling using dedicated neural network hardware. The inventions contained herein provides methods to use existing neural network hardware to preform image scaling functions. Further, the inventions contained herein describe additional circuitry that can be added to neural network hardware to further enhance image scaling capabilities and efficiencies.