Matrix Processor Architecture for AI Computing Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processors, including central and graphics processing units, face inefficiencies in computing and memory access for artificial intelligence applications due to their fine granularity instruction streams and general-purpose architectures, leading to low performance and redundant circuit logic, resulting in high resource consumption and costs.
Innovation Solution
A processor architecture that includes a host interaction device, instruction control device, off-chip memory, on-chip cache, and array processing device, which exchanges data and instructions with a granularity of matrices, converting external instructions into memory access and computing instructions to perform matrix operations efficiently, allowing parallel processing of matrix sub-blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If fine granularity instruction streams and arithmetic logic units are used for basic computing operations, then basic computing operations (addition, multiplication) can be performed, but computing efficiency is low due to frequent data loading and storing
Solution Approach 1:
The patent segments the instruction stream into coarse-grained matrix operation instructions that operate on entire matrix blocks rather than individual elements. The array processing device is segmented into multiple processing elements that can simultaneously operate on different portions of matrices, reducing the overhead of fine-grained instruction execution and improving computing efficiency.
Solution Approach 2:
The patent merges multiple fine-grained computing operations into a single coarse-grained matrix operation instruction. By combining numerous individual addition and multiplication operations into one unified matrix operation, the system reduces instruction overhead and enables more efficient utilization of the array processing device.
2Adaptability or versatility
If general-purpose processor architecture is used, then various computing tasks can be supported, but computing efficiency is low and redundant circuit logics are generated
Solution Approach 1:
The patent applies local quality by designing a processor architecture specifically optimized for matrix operations used in artificial intelligence applications. The array processing device and associated memory structures are tailored to the specific computational patterns of AI workloads, providing high efficiency for these tasks without the redundancy of general-purpose circuit logic.
Solution Approach 2:
The patent achieves universality through the array processing device that can handle various matrix operations (multiplication, addition, activation functions) through a single unified architecture. The processing elements can be configured to perform different operations on different data sets, providing versatile AI computing capability without requiring separate dedicated hardware for each function.
3Ease of operation
If fine granularity memory access modes and data paths are used, then fine granularity memory access is ensured, but computing efficiency is low due to frequent data loading and storing
Solution Approach 1:
The patent implements a nested memory hierarchy with on-chip cache memory and off-chip memory working together. The on-chip cache provides fast access for active matrix data, while the off-chip memory stores larger datasets. This nested structure allows the system to maintain fine-grained memory access capability while reducing the frequency of slow off-chip memory accesses.
Solution Approach 2:
The patent changes the dimension of memory access by operating on entire matrix blocks rather than individual elements. This block-level memory access pattern reduces the total number of memory access operations required, as each access retrieves or stores a substantial portion of the data needed for computation, thereby reducing data loading and storing time.
Data Source
AI summary
The present application discloses a processor and a method for executing an instruction on a processor. A specific implementation of the processor includes: a host interaction device, an instruction control device, an off-chip memory, an on-chip cache and an array processing device, wherein the host interaction device is configured to exchange data and instructions with a host connected with the processor, wherein the exchanged data has a granularity of a matrix; the off-chip memory is configured to store a matrix received from the host, on which a matrix operation is to be performed; and the instruction control device is configured to convert an external instruction received from the host to a series of memory access instructions and a series of computing instructions and execute the converted instructions. The implementation can improve the execution efficiency of a deep learning algorithm.