Data Processing Circuit for Structured Sparsity in Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence technologies based on deep learning face challenges in applying deep neural networks to devices with limited hardware resources due to high computation and storage demands, which existing hardware and instruction sets fail to efficiently support, particularly in mobile phones and embedded devices.
Innovation Solution
A data processing circuit and method that implement structured sparsity on tensor data, utilizing a control circuit, storage circuit, and operation circuit to perform sparsity on one dimension of tensor data, specifically supporting sparsity in neural networks during inference and training processes, thereby reducing computational and storage requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks are designed deeper to improve algorithm performance, then computation and storage space requirements increase, but hardware resources become insufficient
Solution Approach 1:
The patent extracts and removes redundant components from the neural network through structured sparsity. By identifying and eliminating zero or near-zero weight connections in a systematic manner, the network achieves compression without significant loss of computational capability, thus reducing storage and computation requirements while maintaining algorithm performance.
Solution Approach 2:
The patent changes the sparsity parameter of the neural network weights from dense to structured sparse representation. Through parameter transformation and reorganization, the network achieves compact storage and reduced computation while preserving essential computational functions, resolving the contradiction between network depth and hardware resource requirements.
2Device complexity
If existing hardware and instruction sets are used, then device complexity remains low, but they cannot efficiently support sparsity operations
Solution Approach 1:
The patent segments the sparsity operation process into distinct functional modules: sparsity detection unit, sparsity mask generation unit, and data processing unit. This segmentation allows each module to be optimized independently, improving overall processing efficiency while keeping individual component complexity manageable.
Solution Approach 2:
The patent introduces a sparsity mask as an intermediary representation that bridges the original dense data and the compressed sparse data. The sparsity mask contains information about which elements should be retained and which can be discarded, enabling efficient sparsity operations without requiring fundamental changes to existing hardware architecture.
3Productivity
If structured sparsity is implemented to reduce computation and storage, then processing efficiency improves, but existing hardware cannot efficiently support it
Solution Approach 1:
The patent designs a hardware architecture where the sparsity processing units can work with existing standard data formats and memory structures. The system maintains compatibility with conventional hardware while adding specialized sparsity handling capabilities, allowing multi-functional operation without requiring complete hardware redesign.
Data Source
AI summary
The present disclosure discloses a data processing circuit, a data processing method, and related products. The data processing circuit is implemented as a computing apparatus included in a combined processing apparatus. The combined processing apparatus further includes an interface apparatus and other processing apparatus. The computing apparatus interacts with other processing apparatus to jointly complete a user specified computation operation. The combined processing apparatus further includes a storage apparatus. The storage apparatus is connected to the computing apparatus and other processing apparatus, respectively. The storage apparatus is used to store data of the computing apparatus and other processing apparatus. The solution disclosed in the present disclosure provides hardware implementation for operations related to structured sparsity, which can simplify processing and improve processing efficiency of a machine.