Tensor Instruction Compilation for Compute-in-Memory Parallelism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The current compute-in-memory architecture is limited by low processing parallelism due to the use of scalar instruction sets, which results in inefficient data transfer and high power consumption, and the storage-computing separated architecture restricts performance and energy efficiency improvement.
Innovation Solution
An operator compilation method is employed to generate tensor instructions for a compute-in-memory system, utilizing a compilation operator to improve processing parallelism by compiling source code into tensor instructions, including operation codes, tensor addresses, data types, and lengths, and storing these instructions in an instruction storage list.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If scalar instruction sets are used in compute-in-memory architecture, then the architecture can be implemented, but processing parallelism remains low
Solution Approach 1:
The patent transforms scalar instructions into tensor instructions by changing the data dimensionality parameter. Instead of processing single scalar values, the system processes multi-dimensional tensor data structures, enabling parallel computation across multiple data elements simultaneously while maintaining the same compute-in-memory hardware architecture.
Solution Approach 2:
The patent introduces dimensional expansion by transitioning from scalar (0-dimensional) to tensor (multi-dimensional) data structures. This dimensional change allows the system to process batches of data in parallel within the same instruction cycle, significantly improving processing parallelism without requiring additional hardware complexity.
2Loss of energy
If storage-computing separated architecture is used, then data transfer can be performed, but power consumption increases due to frequent data transfer
Solution Approach 1:
The patent merges storage and computing operations by implementing compute-in-memory architecture where tensor instructions are executed directly within the memory array. This integration eliminates the need for frequent data transfers between separate storage and computing units, reducing power consumption while maintaining high computing performance through in-situ processing.
3Productivity
If data is transferred frequently between memory and processor, then computing tasks can be executed, but transmission power consumption increases
Solution Approach 1:
The patent extracts the computing function from the central processor and relocates it directly into the memory array through compute-in-memory architecture. Tensor instructions are executed in-place within the memory, eliminating the need for repeated data transfers between memory and processor, thereby reducing transmission power consumption while maintaining computing task execution capability.
Data Source
AI summary
An operator compilation method includes: obtaining a compilation operator, where the compilation operator indicates a compilation rule for a source code; compiling the source code based on the compilation operator to obtain one or more tensor instructions, where the tensor instruction is used to execute a computing task in a compute-in-memory system, and the tensor instruction comprises one or more of the following instruction segments: an operation code, a tensor address, a data type, a data bit width, or a tensor length; and storing the tensor instruction into an instruction storage list, where the instruction storage list is used to store a compiled tensor instruction.