Neural Network Convolution Using Sparsity Maps and Storage Pointers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face high computation costs and significant memory demands due to the need for extensive MAC operations and data processing, particularly in operations like transposed and dilated convolutions, which increase latency and memory footprint without leveraging sparse compute acceleration.
Innovation Solution
The use of storage pointers and sparsity maps to perform deep learning operations, such as convolutions, by generating sparsity maps based on hyperparameters and using storage pointers to read data elements from memory, thereby reducing memory footprint and accelerating computations by avoiding unnecessary data storage and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If transposed and dilated convolutions are performed using conventional methods, then the DNN achieves high accuracy, but memory footprint and computation time increase significantly
Solution Approach 1:
The patent extracts and stores only the non-zero data elements from the input tensor in a compressed format, rather than storing the entire tensor including all zero elements. This extraction of essential information reduces memory footprint while preserving the computational integrity needed for high accuracy DNN operations.
Solution Approach 2:
The patent changes the representation parameters of the tensor by using sparsity maps to indicate the presence and location of non-zero elements, and storage pointers to access them efficiently. This parameter transformation allows the system to work with a compressed representation that requires less memory while maintaining computational accuracy.
2Reliability
If transposed and dilated convolutions are performed using conventional methods, then the DNN achieves high accuracy, but computation latency increases
Solution Approach 1:
The patent extracts only the non-zero elements for processing, eliminating unnecessary computations on zero elements. This selective extraction reduces the number of MAC operations required, thereby reducing computation latency while maintaining the accuracy of the convolution results.
Solution Approach 2:
The patent performs preliminary actions by pre-computing and storing sparsity maps and storage pointers that identify the locations of non-zero elements. This preliminary organization of data allows the convolution operation to directly access only the necessary elements, avoiding the time cost of scanning or processing zero elements during the actual computation.
3Reliability
If conventional convolution methods are used, then all data elements are processed ensuring accuracy, but unnecessary data storage and processing of zeros increases memory usage
Solution Approach 1:
The patent extracts and stores only the non-zero data elements in a compressed format, using sparsity maps to track their positions. This extraction eliminates the need to store zero elements while preserving all information necessary for accurate computation, directly reducing memory usage.
Solution Approach 2:
The patent applies different storage qualities to different parts of the data: non-zero elements are stored in a compressed, optimized format with metadata (sparsity maps and storage pointers), while zero elements are not stored at all. This local differentiation of storage quality reduces overall memory usage while maintaining computation accuracy.
4Reliability
If conventional convolution methods are used, then complete data is processed ensuring correctness, but processing zeros reduces computational efficiency
Solution Approach 1:
The patent extracts and processes only the non-zero elements that contribute to the computation, using sparsity maps and storage pointers to identify and access them efficiently. This extraction eliminates wasteful processing of zero elements, directly improving computational efficiency while maintaining correctness through the use of sparsity information.
Solution Approach 2:
The patent performs preliminary computation by pre-identifying and organizing non-zero elements using sparsity maps and storage pointers before the actual convolution operation. This preliminary action ensures that during the computation phase, only relevant data elements are accessed and processed, improving efficiency without sacrificing correctness.
Data Source
AI summary
Deep learning operations (e.g., transposed convolution, resized convolution, dilated convolution, etc.) may be performed with sparsity maps and storage pointers. A deep learning operation has a tensor, which can be used to generate an upsampled tensor by adding new data elements (e.g., zeros) into the tensor. One or more sparsity maps may be generated based on one or more parameters of the first deep learning operation. The sparsity map may include elements indicating whether a data element in the upsampled tensor is a data element in the tensor or is a new data element. One or more storage pointers may be generated. A storage pointer may indicate a location (e.g., a memory address) where one or more data elements of the tensor are stored in a memory. An output of the deep learning operation may be performed using data elements in the tensor, the sparsity maps, and the storage pointers.