Hybrid Digital-Analog PIM Block for Depth-wise Convolution Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Non-volatile Processing-In-Memory (NVPIM) architectures face inefficiencies when processing Depth-wise Convolutional Neural Networks (DCNNs) due to low memory array utilization, particularly in resource-constrained systems, as they are primarily designed for standard convolutions, leading to suboptimal execution of Depth-wise convolution layers.
Innovation Solution
A hybrid digital/analog mode NVPIM architecture is introduced, allowing the PIM block to switch between modes for efficient processing of Depth-wise and Point-Wise convolution layers, where the PIM block operates in digital mode for Depth-wise convolution and analog mode for Point-Wise convolution, optimizing memory utilization and computation efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If NVPIM architecture is used for Depth-wise convolution, then computation can be performed in-memory, but memory array utilization is low
Solution Approach 1:
The PIM block dynamically switches between digital mode and analog mode based on the convolution type being performed. For Depth-wise convolution, it operates in digital mode with enhanced memory utilization techniques, while for Point-Wise convolution, it switches to analog mode. This dynamic adaptability resolves the contradiction by optimizing memory array utilization for each specific computation type while maintaining computation efficiency.
Solution Approach 2:
The system changes operational parameters by switching between digital and analog modes, and by adjusting memory mapping strategies based on the convolution layer type. This parameter change enables the same hardware architecture to efficiently handle different convolution operations with appropriately optimized memory utilization patterns.
2Productivity
If NVPIM architecture is designed for standard convolution, then computation is efficient, but it is suboptimal for Depth-wise convolution layers
Solution Approach 1:
The PIM block is designed with multi-functionality to handle both standard convolution and Depth-wise convolution efficiently. By incorporating digital mode with optimized memory access patterns and analog mode, the architecture becomes universal and adaptable to different convolution layer types, resolving the contradiction between computation efficiency and convolution layer compatibility.
Solution Approach 2:
The system dynamically adapts its operational mode based on the input convolution type, making it versatile for different computational tasks while maintaining high efficiency for each specific operation type.
3Adaptability or versatility
If resource-constrained systems use DCNN, then cognitive tasks can be performed, but power budget is limited
Solution Approach 1:
The PIM architecture performs computation directly within the memory array, eliminating the need to move data between separate memory and processing units. This self-service approach reduces energy consumption by eliminating data transfer operations, enabling resource-constrained systems to perform cognitive tasks within limited power budgets.
Solution Approach 2:
The patent replaces the traditional von Neumann architecture's mechanical data movement between memory and processor with in-situ computation within the memory array itself, significantly reducing energy consumption while maintaining cognitive task capability.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This hybrid approach enhances inference efficiency for both Depth-wise and Point-Wise convolution layers, improving overall performance in resource-constrained systems by maximizing memory array utilization and reducing computational overhead.
Implementation Method 1
Resistive random-access memory (ReRAM) based non-volatile PIM (NVPIM) emerges as a promising solution of accelerating DCNN executions. The high cell density of ReRAM allows large on-chip ReRAM arrays to be implemented on the chip to store the parameters of the DCNN while proper functions, e.g., vector-matrix multiplications (VMM), can be directly performed in the ReRAM arrays and their peripheral circuits.
Data Source
AI summary
Embodiments of apparatus and method for matrix multiplication using processing-in-memory (PIM) are disclosed. In an example, an apparatus for matrix multiplication includes an array of tiles that each include one or more PIM blocks. A PIM block may include a hybrid-mode PIM block that may be configured into a digital mode or an analog mode. The PIM block configured into digital mode may perform operations associated with depth-wise (DW) convolution. On the other hand, a PIM block configured into analog mode may perform operations associated with point-wise (PW) convolution. A controller may be used to configure the PIM block into either digital mode or analog mode, depending on the computations.