Layered architecture system

Abstract

The application relates to the computer technical field, in particular to a layered architecture system, the system comprising a logic chip, a memory chip and a control processor, the logic chip comprising a plurality of in-memory computing units and a static random access memory as a first memory level; the memory chip comprising a dynamic random access memory as a second memory level; the memory chip is vertically integrated with the logic chip through a three-dimensional stacking technology; and the control processor is used for realizing concurrent and overlapped execution of data movement operations and computing tasks by executing first type instructions and second type instructions.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a layered architecture system. Background Technology

[0002] With the development of Artificial Intelligence (AI), various related technologies have emerged and developed rapidly. For example, combining Digital In-Memory Computing (DIMC) technology and 3D chip stacking technology has created an AI accelerator architecture.

[0003] However, in some cases, the general instruction set is not fully applicable to this AI accelerator architecture, affecting the computational efficiency of the AI ​​accelerator. Utility Model Content

[0004] To address the above issues, embodiments of this application provide an instruction execution method and system thereof, a server, a computer program product, and a storage medium.

[0005] In a first aspect, embodiments of this application provide an instruction execution method applied to a control processor, the controlled processor comprising a hierarchical architecture system, the hierarchical architecture system comprising at least a first memory level and a second memory level, the method comprising: decoding and executing a first type of instruction, the first type of instruction being used to control the transfer of data blocks between the first memory level and the second memory level; decoding and executing a second type of instruction, the second type of instruction being used to assign a first computing task to an in-memory computing unit; and concurrently executing subsequent first type of instructions associated with the second computing task while the in-memory computing unit is executing the first computing task.

[0006] In one possible implementation of the first aspect described above, the first type of instruction is an explicit cross-memory level data move instruction; the format of the first type of instruction includes an opcode, a destination address, a source address, and a data block size.

[0007] In one possible implementation of the first aspect described above, the second type of instruction includes instructions for dispatching atomic computational tasks; the second type of instruction may have a format including operands, the operands pointing to operand addresses in a first memory level; the operand addresses include at least the address of the input vector and the address of the weight matrix.

[0008] In one possible implementation of the first aspect described above, the execution of the second type of instruction triggers the in-memory computing unit to perform a complete and multi-cycle vector-matrix multiplication operation.

[0009] In one possible implementation of the first aspect above, the first memory level is a static random access memory (SRAM) integrated into a logic chip containing in-memory computing units; the second memory level is a dynamic random access memory (DRAM) integrated into a separate memory chip; the logic chip and the memory chip are vertically integrated through three-dimensional stacking technology, and high-speed data interaction between the two memory levels is achieved through high-density through-silicon vias (TSVs).

[0010] In one possible implementation of the first aspect described above, the first type of instructions and the second type of instructions are generated by the compiler; the first type of instructions include a load data instruction and a store data instruction; the load data instruction is used to load a data block from the second memory level to the first memory level, and the store data instruction is used to store a data block from the first memory level to the second memory level.

[0011] Secondly, embodiments of this application provide a layered architecture system, which includes a logic chip, a memory chip, and a control processor. The logic chip includes multiple in-memory computing units and a static random access memory (SRAM) as a first memory layer. The memory chip includes a dynamic random access memory (DRAM) as a second memory layer. The memory chip is vertically integrated with the logic chip through three-dimensional stacking technology. The control processor is used to implement concurrent overlapping execution of data movement operations and computing tasks by executing a first type of instruction and a second type of instruction.

[0012] In one possible implementation of the second aspect described above, the logic chip and the memory chip are interconnected via high-density through-silicon vias (TSVs) to enable high-speed data interaction between the first memory level and the second memory level.

[0013] In one possible implementation of the second aspect described above, the system further includes a compiler module for generating the first type of instructions and the second type of instructions based on the AI ​​computation graph, so that the control processor can coordinate the parallel execution of data transmission and computation tasks at the instruction level.

[0014] In one possible implementation of the second aspect above, the layered architecture system is encapsulated in the AI ​​accelerator slot of the server motherboard and communicates with the main processor via a high-speed interconnect interface to perform artificial intelligence computing tasks as a heterogeneous coprocessing unit of the host system.

[0015] Thirdly, embodiments of this application provide a server, the server comprising: a memory for storing instructions executed by one or more processors of the server; and a processor, one of the processors of the server, for executing the instructions stored in the memory to implement the first aspect above, and any method mentioned in the first aspect.

[0016] Fourthly, embodiments of this application provide a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the first aspect described above, and computer program code of the instruction execution method mentioned in the first aspect.

[0017] Fifthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the first aspect and the instruction execution method mentioned in the first aspect.

[0018] The beneficial effects of the second, third, fourth and fifth aspects mentioned above can be referred to the first aspect mentioned above, as well as the relevant descriptions in the various possible implementations of the first aspect, which will not be repeated here. Attached Figure Description

[0019] Figure 1 According to some embodiments of this application, a schematic diagram of a layered architecture system is shown; Figure 2 According to some embodiments of this application, a schematic flowchart of an instruction execution method is shown; Figure 3 According to some embodiments of this application, a format of LD3D instructions is shown; Figure 4 According to some embodiments of this application, a format for ST3D instructions is shown; Figure 5 According to some embodiments of this application, a format for an atomized computational task dispatch instruction is shown; Figure 6 According to some embodiments of this application, a schematic diagram of instruction execution interaction is shown. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Digital in-memory computing technology can refer to the technology of physically integrating in-memory computing units with storage units, thereby reducing data transfer by performing computations within the memory array.

[0022] 3D chip stacking technology can refer to the vertical integration of logic chips and memory chips to improve memory bandwidth and capacity.

[0023] In some embodiments, digital in-memory computing technology and 3D chip stacking technology can be combined to form a new AI accelerator architecture. This AI accelerator architecture can be, for example, the layered architecture system mentioned below, which may include a first memory layer (located at the bottom layer) and a second memory layer (located at the top layer).

[0024] The first memory tier can be composed of high-performance DIMC logic chips, which may include Static Random-Access Memory (SRAM) with high-speed read / write but limited capacity. The second memory tier can be composed of Dynamic Random-Access Memory (DRAM) chips.

[0025] In response, some related technologies, such as the standard RISC-V (an open-source instruction set architecture) and other general-purpose instruction sets, typically apply memory operation instructions (such as Load / Store) to traditional, hardware-managed cache levels. This makes it difficult to directly and efficiently manage two-level heterogeneous memory, such as that of AI accelerator architectures, which have significantly different characteristics and are visible to software.

[0026] Therefore, in some embodiments, high-bandwidth memory-processing-in-memory (HMBCM) technology can be used for dual-level heterogeneous memory. However, the control model in this technology typically requires the central processing unit (CPU) to preload a microprogram consisting of low-level, primitive Reduced Instruction Set Computer (RISC) instructions (e.g., a single multiply-accumulate operation) into the storage device, and then trigger execution via memory commands. This approach leaves the complex control burden to the central CPU, but it is difficult to command dedicated hardware to perform complex tasks in a concise manner at the instruction set architecture (ISA) level, thus affecting the computational efficiency of AI accelerators.

[0027] To address this, this application provides an instruction execution method. In this method, while the processor executes one instruction from a second type of instruction to initiate a computational task in the in-memory computing unit, it can concurrently execute subsequent instructions from a first type of instruction. This allows the data transfer operation between the first and second memory levels, prepared for future computational tasks, to overlap with the computational task currently being executed by the in-memory computing unit. This enables instruction set expansion with collaborative optimization at the main control ISA level, bridging the gap between high-level software and underlying hardware, achieving fine-grained and high-efficiency control over data flow and computational tasks, and effectively improving the computational efficiency of AI accelerators.

[0028] The instruction execution method mentioned in the embodiments of this application will be described below with reference to the accompanying drawings. Figure 1 The diagram shows the structure of a layered architecture system, which may include logic chips, memory chips, and a control processor (main CPU).

[0029] The logic chip (or DIMC logic chip) may include multiple in-memory computing units and static random access memory (SRAM) as the first memory tier (Tier-1). The memory chip may include dynamic random access memory (DRAM) as the second memory tier (Tier-2). In some embodiments, the memory chip can be vertically integrated with the logic chip using three-dimensional (3D) stacking technology, and high-speed data interaction between the two memory tiers can be achieved through high-density through-silicon vias (TSVs).

[0030] The control processor can be used to enable concurrent overlapping execution of data transfer operations and computational tasks through first-type instructions and second-type instructions.

[0031] In some embodiments, such as Figure 2 The schematic diagram of the instruction execution method involved in the embodiment of this application shown can be executed by the layered architecture system mentioned above, and the method can include: S101: Decode and execute the first type of instruction.

[0032] In some alternative embodiments, the first type of instruction can be used to control the transfer of data blocks between a first memory level and a second memory level. For example, the first type of instruction can be an explicit cross-memory level data move instruction. In some embodiments, the explicit cross-memory level data move instruction may also include LD3D (Load from 3D DRAM) instructions and ST3D (Store to 3D DRAM) instructions.

[0033] Specifically, when the processor executes the LD3D instruction (load data instruction), a data block can be loaded from the second memory level to the first memory level. For example... Figure 3 The format of the LD3D instruction shown can include the opcode, the destination SRAM address (dest_sram), the source DRAM address (src_dram), and the data block size (size).

[0034] When the processor is controlled to execute the ST3D instruction (store data instruction), a block of data can be stored from the first memory level to the second memory level. For example... Figure 4 The ST3D instruction format shown can include an opcode, source SRAM address (src_sram), destination DRAM address (dest_dram), and data block size (size).

[0035] Upon receiving a first type of instruction, the control processor can decode and execute the first type of instruction, and send request data to the second memory level to request that data blocks required for future computational tasks to be executed by the in-memory computing unit be loaded into the first memory level.

[0036] S102: Decode and execute the second type of instruction.

[0037] In some optional embodiments, the second type of instructions can be used to assign computational tasks to in-memory computing units. For example, the second type of instructions may include atomic computational task assignment instructions. Wherein, such as Figure 5 The format of the atomic computation task dispatch instruction shown includes operands that point to operand addresses located in a first memory level. The operand addresses include at least the input vector address (vec_addr) and the weight matrix address (mat_addr).

[0038] In some alternative embodiments, a second type of instruction can characterize a complex, multi-cycle hardware operation as a single CPU instruction. Execution of a second type of instruction can trigger an in-memory computing unit to perform a complete, multi-cycle vector-matrix multiplication operation.

[0039] In response, the control processor can decode and execute the second type of instruction upon receiving it, thereby assigning the first computational task to the in-memory computing unit.

[0040] S103: When the in-memory computing unit is executing the first computing task, the first type of instruction subsequently associated with the second computing task is executed concurrently.

[0041] In some optional embodiments, while the in-memory computing unit executes the first computing task, the control processor can receive a first type of instruction, which may come from a compiler, etc., so that the control processor can execute the first type of instruction to prefetch data blocks associated with a subsequent second computing task.

[0042] Specifically, the control processor sends request data to the second memory level, thereby loading data blocks associated with a subsequent second computation task from the second memory level to the first memory level.

[0043] It is understandable that during this process, the execution of the first computational task by the in-memory computing unit overlaps with the execution of the first type of instruction by the control processor. Thus, this application, in its hierarchical architecture system structure, provides both a first type of instruction and a second type of instruction, enabling the control processor to "unload" the computational task to the in-memory computing unit after executing the second type of instruction, and concurrently execute the next instruction.

[0044] In this way, instruction set extensions can be collaboratively optimized at the host ISA level to bridge the gap between high-level software and low-level hardware, enabling fine-grained and high-efficiency control over data flow and computing tasks.

[0045] In some other alternative embodiments, such as Figure 6 The diagram shows the instruction execution interaction.

[0046] In some embodiments, the layered architecture system may further include a compiler module. The compiler module can generate a first type of instruction and a second type of instruction based on the AI ​​computation graph. The second type of instruction controls the processor to assign computation tasks to in-memory computing units, while the first type of instruction controls the movement of data between different memory levels, enabling the processor to coordinate the parallel execution of data transfer and computation tasks at the instruction level.

[0047] Specifically, the compiler can generate the corresponding code for the Nth-level computation, and then send a second type of instruction to the control processor (main CPU) to initiate the Nth-level computation. The control processor can then assign the corresponding first computation task to the in-memory computing unit based on the received second type of instruction, and the in-memory computing unit will execute the Nth-level computation.

[0048] Simultaneously, the compiler can send a first type of instruction to the control processor to prefetch data from layer N+1. Based on the received first type of instruction, the control processor can send a request to the second memory level to request loading the data block from layer N+1 from the second memory level to the first memory level. Then, the second memory level can respond to the request and transfer the data block from layer N+1 to the first memory level.

[0049] Therefore, after the first computation task is completed, the control processor can assign a second computation task to the in-memory computing unit based on the second type of instructions sent by the compiler related to the second computation task, to execute the N+1th level computation. It can be understood that the in-memory computing unit performs the N+1th level computation based on prefetching data blocks in the first memory level.

[0050] In response, while the in-memory computing unit is performing the N+1 level of computation, the control processor can concurrently execute operations related to the N+2 level of computation.

[0051] In this way, the time for data loading from the second memory level to the first memory level overlaps with the time for the in-memory computing unit to execute the current computing task. This enables concurrent execution of the control plane and data plane, as supported by the ISA layer.

[0052] In other words, the atomicity of the second type of instructions allows the control processor to be released immediately after the computing task is assigned, thereby concurrently executing subsequent first type of instructions. This overlaps the data prefetch delay with the execution time of the current computing task, significantly improving the computing efficiency and energy efficiency of the AI ​​accelerator.

[0053] In some embodiments, the layered architecture system can be deployed as an artificial intelligence acceleration unit in a server. Specifically, the layered architecture system can be encapsulated in an AI accelerator slot on a server motherboard and communicate with the main processor via a high-speed interconnect interface to perform artificial intelligence computing tasks as a heterogeneous coprocessing unit of the host system.

[0054] This application also provides a server, including: a memory and one or more processors, wherein the memory is used to store instructions executed by the one or more processors, and the processors are used to execute the instruction execution method mentioned in any of the above embodiments of this application.

[0055] This application also provides a computer-readable medium storing instructions that, when executed on a server, cause the server to perform the instruction execution method mentioned in any of the above embodiments of this application.

[0056] This application also provides a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the instruction execution method mentioned in any of the above embodiments of this application.

[0057] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0058] It should be noted that, in the examples and description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0059] Although this application has been illustrated and described with reference to certain embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

[0060] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A layered architecture system, characterized in that, The system includes a logic chip, a memory chip, and a control processor. The logic chip includes multiple in-memory computing units and a static random access memory as the first memory level. The memory chip includes dynamic random access memory as a second memory level; The memory chip is vertically integrated with the logic chip using three-dimensional stacking technology; The control processor is used to achieve concurrent overlapping execution of data movement operations and computation tasks by executing a first type of instruction and a second type of instruction.

2. The layered architecture system according to claim 1, characterized in that, The logic chip and the memory chip are interconnected through high-density through-silicon vias to enable high-speed data interaction between the first memory level and the second memory level.

3. The layered architecture system according to claim 1, characterized in that, The first memory level is static random access memory, and the second memory level is dynamic random access memory.

4. The layered architecture system according to claim 1, characterized in that, The control processor is configured to execute two different types of instructions: the first type of instruction is an explicit cross-memory level data movement instruction, used to control the transfer of data blocks between the first memory level and the second memory level; the second type of instruction is a computation task dispatch instruction, used to dispatch computation tasks to the in-memory computing unit.

5. The layered architecture system according to claim 4, characterized in that, The first type of instruction includes a load data instruction and a store data instruction; the load data instruction is used to load a data block from the second memory level to the first memory level, and the store data instruction is used to store a data block from the first memory level to the second memory level.

6. The layered architecture system according to claim 4, characterized in that, The second type of instruction is an atomic computation task dispatch instruction; the second type of instruction includes operands, the operands pointing to operand addresses in the first memory level, the operand addresses including at least the address of the input vector and the address of the weight matrix.

7. The layered architecture system according to claim 6, characterized in that, The in-memory computing unit is configured to perform atomic, multi-cycle vector-matrix multiplication operations, each of which is triggered by a second type of instruction executed by the control processor.

8. The layered architecture system according to claim 1, characterized in that, The control processor is configured to concurrently execute a first type of instruction associated with a second computing task while the in-memory computing unit is performing a first computing task, thereby achieving time overlap between data transfer operations and computing tasks.

9. The layered architecture system according to claim 1, characterized in that, The system also includes a compiler module, which generates the first type of instructions and the second type of instructions based on the AI ​​computation graph, so that the control processor can coordinate the parallel execution of data transmission and computation tasks at the instruction level.

10. The layered architecture system according to claim 1, characterized in that, The layered architecture system is encapsulated in the AI ​​accelerator slot of the server motherboard and communicates with the main processor through a high-speed interconnect interface to perform artificial intelligence computing tasks as a heterogeneous coprocessing unit of the host system.

Images