Tensor Operator Compilation for Compute-in-Memory Parallel Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The current compute-in-memory architecture is limited by low processing parallelism due to the use of scalar instruction sets, which results in inefficient data transfer and high power consumption, and the storage-computing separated architecture restricts performance and energy efficiency improvement.
Innovation Solution
An operator compilation method is employed to generate tensor instructions for the compute-in-memory system, utilizing a compilation operator to improve processing parallelism by compiling source code into tensor instructions, including operation codes, tensor addresses, data types, and lengths, and storing them in an instruction storage list.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If scalar instruction sets are used in compute-in-memory architecture, then data transfer frequency is reduced, but processing parallelism is low
Solution Approach 1:
The patent changes the instruction set parameter from scalar to tensor instructions, enabling batch processing of multiple data elements simultaneously. This parameter change allows the system to maintain reduced data transfer frequency while achieving high processing parallelism through tensor operations that process multiple values in parallel.
Solution Approach 2:
The patent segments the data processing task into tensor operations that can be executed in parallel across multiple execution units. By dividing the computational workload into independent tensor instructions, the system achieves both low data transfer frequency and high processing parallelism simultaneously.
2Quantity of substance
If storage-computing separated architecture is used, then storage capacity is increased, but computing performance is restricted
Solution Approach 1:
The patent merges storage and computing functions by implementing compute-in-memory architecture where computing operations are performed directly within the memory device. This merging enables the system to achieve both high storage capacity and improved computing performance, as data processing occurs at the storage location rather than requiring data transfer to a separate processor.
Solution Approach 2:
The patent introduces tensor instructions as an intermediary between the high-level programming interface and the physical memory hardware. These tensor instructions enable efficient computation directly in memory, serving as a mediator that allows the storage-computing separated architecture to achieve both high storage capacity and computing performance.
3Device complexity
If scalar instructions process data one by one, then instruction simplicity is maintained, but processing parallelism is low
Solution Approach 1:
The patent transitions from scalar (1D) processing to tensor (multi-dimensional) processing, enabling simultaneous operation on multiple data elements across different dimensions. This dimensional change allows the instruction set to maintain relative simplicity while achieving high processing parallelism through multi-dimensional data structures and operations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An operator compilation method and apparatus are disclosed, to implement tensor data processing in a compute-in-memory architecture, and improve processing parallelism of computing tasks in the compute-in-memory architecture. The method includes: obtaining a compilation operator, where the compilation operator indicates a compilation rule for source code; compiling the source code based on the compilation operator, to obtain one or more tensor instructions, where the tensor instruction is used to execute a computing task in a compute-in-memory system, and the tensor instruction includes one or more of the following instruction segments: an operation code, a tensor address, a data type, a data bit width, and a tensor length; and storing the tensor instruction into an instruction storage list, where the instruction storage list is used to store a compiled tensor instruction.