Neural Network Convolution Processing Units Parallel Data Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network operations, such as convolution and matrix multiplication, require complex data rearrangement, leading to slower processing speeds and cumbersome operation steps, particularly in tasks like image and speech processing.
Innovation Solution
A method and apparatus that utilize multiple operation units to acquire and process image data in parallel, performing multiplication operations with pre-stored weight data to generate intermediate results, which are then summed to produce operation results, thereby simplifying data reading logic and increasing processing efficiency without additional hardware resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a reading module is used to read data from external storage and send it to a matrix multiplication unit, then data can be obtained for operations, but the data needs to be rearranged which leads to complex reading logic and cumbersome operation steps
Solution Approach 1:
The patent divides the data processing task into multiple operation units that can work in parallel. Each operation unit handles a specific portion of the matrix multiplication independently, eliminating the need for complex centralized data rearrangement logic. The data is divided into multiple rows, with each row processed by a dedicated operation unit simultaneously.
Solution Approach 2:
The patent transitions from sequential processing to parallel processing by introducing multiple operation units working simultaneously on different data rows. This dimensional change from single-unit sequential operation to multi-unit parallel operation reduces the complexity of data reading logic while improving processing speed.
2Productivity
If data is read and rearranged to meet matrix multiplication requirements, then operations can be performed, but the operation steps become cumbersome and processing speed decreases
Solution Approach 1:
The patent pre-arranges data in a specific format before it enters the operation units. The data is organized in advance so that when operation units read it, they can directly perform matrix multiplication without additional rearrangement steps. This preliminary data preparation eliminates time-consuming rearrangement operations during the actual computation.
Solution Approach 2:
The patent ensures continuous operation by having multiple operation units process data simultaneously without interruption. The parallel architecture allows continuous matrix multiplication operations to proceed without the stopping and starting that would occur with sequential rearrangement and processing steps, thereby reducing total operation time.
3Productivity
If multiple operation units process data in parallel, then processing speed increases and reading logic simplifies, but additional hardware resources would be needed
Solution Approach 1:
The patent designs operation units that can perform multiple functions: data reading, matrix multiplication, and result accumulation. Each operation unit is a multi-functional component that handles various tasks within the neural network operation, reducing the need for separate dedicated hardware for each function and thereby minimizing additional hardware resource requirements.
Solution Approach 2:
The patent combines multiple operation units into a unified processing architecture where they work together to complete the matrix multiplication task. The operation units are integrated into a single system that shares common resources such as memory access and control logic, effectively merging what would otherwise be separate hardware components into a cohesive unit that achieves parallel processing without proportionally increasing hardware complexity.
Data Source
AI summary
In one aspect, an operation method based on a neural network: calculating, according to the sizes of a convolution kernel and an original image, a total number of operation cycles and an image matrix corresponding to each operation cycle; for the image matrix, a plurality of operation units concurrently acquiring the image data and performing a product operation on the image data and pre-stored weight data, to obtain intermediate data; summing the intermediate data to obtain an operation result; and compiling statistics on all the operation results to obtain a target operation result. The overall operation speed is increased in a unit time; the data read logic is simplified; and the bandwidth requirement of a single operation unit for data is reduced. A convolution operation of any size can be performed, and the convolution operation efficiency is improved, thereby increasing the image processing speed.