Deep Learning Processing Apparatus Parallel Matrix Accumulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning processors, such as GPUs, face inefficiencies in convolution operations and inverse operations during CNN training due to high power consumption and complex thread scheduling, leading to low processing efficiency.
Innovation Solution
A deep learning processing apparatus with matrix multiply-add modules and matrix accumulation modules that perform parallel operations on intermediate matrices, optimizing convolution operations and reducing power consumption by storing and accumulating results efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional deep learning processors (such as GPUs) are used to perform convolution operations and inverse operations during CNN training, then the processing can be performed using general-purpose hardware, but the power consumption is high and the processing efficiency is low due to complex thread scheduling
Solution Approach 1:
The patent divides the deep learning processing system into two distinct segments: a general-purpose processing apparatus for providing training instructions and a dedicated deep learning processing apparatus for executing convolution and inverse operations. This segmentation allows the dedicated apparatus to optimize for specific operations, improving processing efficiency while reducing power consumption compared to using general-purpose hardware for all tasks.
Solution Approach 2:
The patent introduces a dedicated deep learning processing apparatus as an intermediary between the general-purpose processing apparatus and the training data. This intermediary is specifically designed to handle convolution and inverse operations, bridging the gap between general-purpose control and specialized computation, thereby achieving both high efficiency and energy optimization.
2Speed
If conventional deep learning processors are used, then hardware flexibility is maintained, but the thread scheduling complexity increases and processing speed decreases
Solution Approach 1:
The patent extracts the complex thread scheduling functionality from the deep learning processing apparatus and relocates it to the general-purpose processing apparatus. This extraction allows the dedicated deep learning processor to focus solely on high-speed convolution and inverse operations without the overhead of managing complex thread schedules, thereby achieving both high processing speed and reduced scheduling complexity.
Solution Approach 2:
Instead of having the deep learning processor manage its own complex thread scheduling, the patent inverts the approach by having the general-purpose processor handle scheduling and instruction generation, while the dedicated processor executes operations in a simplified, streamlined manner. This inversion eliminates the need for complex thread management within the deep learning processor itself.
Data Source
AI summary
Embodiments of the present disclosure propose a deep learning processing apparatus and method, device and storage medium, relating to the field of artificial intelligence. A deep learning processing apparatus includes: at least one matrix multiply-add module, configured to perform a matrix multiply-add operation of a convolution kernel parameter value matrix of a convolutional layer in a convolutional neural network and a first error gradient value matrix to obtain a plurality of intermediate matrices; a storage apparatus, configured to store the plurality of intermediate matrices without reshaping elements in the plurality of intermediate matrices; and a plurality of matrix accumulation modules, configured to read the plurality of intermediate matrices from the storage apparatus and perform a matrix accumulation operation based on the plurality of intermediate matrices according to a convolution scheme of the convolutional layer in parallel, to obtain a second error gradient value matrix for the convolutional layer.