Digital memory computing accelerator, instruction set architecture and data processing method thereof

Abstract

The invention discloses a digital in-memory computing accelerator, an instruction set architecture and a data processing method thereof, and belongs to the technical field of in-memory computing. The digital memory computing accelerator comprises a chip-level module, a core-level module and a unit-level module, the chip-level module is used for acquiring a target calculation task, splitting the target calculation task into a plurality of core-level calculation tasks according to a load distribution strategy and distributing the core-level calculation tasks to the core-level module; the core-level module comprises a control unit, a calculation unit and a cache unit, and the control unit comprises an instruction fetching module, a decoding module and an instruction scheduler; an instruction fetching module reads a to-be-executed instruction corresponding to the core-level calculation task, a decoding module decodes the to-be-executed instruction to generate a control signal and sends the control signal to a calculation unit, and the to-be-executed instruction is an instruction of a calculation instruction set architecture in a digital memory; the calculation unit executes corresponding calculation operation according to the control signal; the cache unit caches intermediate data generated by executing the corresponding calculation operation.

Description

Technical Field

[0001] This application relates to the field of in-memory computing technology, and in particular to a digital in-memory computing accelerator, instruction set architecture, and data processing method thereof. Background Technology

[0002] With the rapid development of computationally intensive applications such as artificial intelligence and big data analytics, the "memory wall" problem under the traditional von Neumann architecture is becoming increasingly prominent. Frequent data movement between the processor and memory has become a major bottleneck for system performance and energy efficiency. Compute-In-Memory (CIM), as an emerging computing paradigm, can effectively reduce data movement and significantly improve system performance and energy efficiency by performing computational operations directly within the storage unit.

[0003] In-memory computing architectures can be broadly categorized into analog in-memory computing and digital in-memory computing. Analog in-memory computing accelerators utilize the physical characteristics of memory devices to directly perform multiplication and addition operations in the current and voltage domains. While offering high energy efficiency, their accuracy is limited and they are susceptible to variations in manufacturing processes and the environment. Digital in-memory computing accelerators, on the other hand, embed digital logic circuits within memory cells, implementing computational functions through bit-level operations. This results in better computational accuracy and stability, making them more suitable for applications requiring high precision.

[0004] However, current digital in-memory computing accelerators have significant limitations. When faced with increasingly complex and diverse deep neural network (DNN) structures, existing digital in-memory computing accelerators lack a systematic multi-level hardware abstraction mechanism, making it difficult to cope with tasks of different scales and adapt to different computing modes and data flow characteristics, resulting in low resource utilization efficiency and inability to process tasks efficiently. Summary of the Invention

[0005] This application provides a digital in-memory computing accelerator, instruction set architecture, and data processing method thereof, which can solve the problem that existing digital in-memory computing accelerators are unable to cope with tasks of different scales, resulting in low resource utilization efficiency and inability to efficiently process tasks.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, a digital in-memory computing accelerator is provided, comprising: a chip-level module, a core-level module, and a unit-level module; wherein: The chip-level module includes: multiple core-level modules connected via an on-chip network, used to acquire a target computing task and, according to a load distribution strategy, split the target computing task into multiple core-level computing tasks and distribute them to the multiple core-level modules; The core-level module includes: a control unit, a computing unit, and a cache unit. The computing unit is composed of the unit-level modules and includes at least an in-memory computing unit. The control unit includes: an instruction fetch module, a decoding module, and an instruction scheduler, wherein: The instruction fetching module is used to read the instruction to be executed corresponding to the core-level computing task, wherein the instruction to be executed is an instruction of the digital in-memory computing instruction set architecture; The decoding module is used to decode the instruction to be executed and generate control signals; The instruction scheduler is used to send the control signal to the computing unit to drive the computing unit to perform corresponding computing operations; The computing unit is configured to perform corresponding computing operations according to the control signal; wherein, when the instruction type of the instruction to be executed is an in-memory computing instruction, the in-memory computing unit is configured to perform in-memory computing operations according to the control signal. The cache unit is used to cache intermediate data generated during the execution of the corresponding calculation operation.

[0007] Secondly, a data processing method based on the aforementioned digital in-memory computing accelerator is provided, including: The chip-level module acquires the target computing task and, according to the load distribution strategy, splits the target computing task into multiple core-level computing tasks and distributes them to multiple core-level modules. The control unit of the core-level module reads the instruction to be executed corresponding to the core-level computing task, decodes the instruction to be executed to generate a control signal, sends the control signal to the computing unit to drive the computing unit to execute the corresponding computing operation, and sends the intermediate data generated by the execution of the corresponding computing operation to the cache unit for caching. The instruction to be executed is an instruction of the digital in-memory computing instruction set architecture. The computing unit executes the corresponding computing operation according to the drive of the control signal. When the instruction type of the instruction to be executed is an in-memory computing instruction, the in-memory computing operation is executed according to the drive of the control signal.

[0008] Thirdly, an instruction set architecture is provided, including: Instructions that rely on the functions provided by the chip-level module, the core-level module, and the unit-level module, the instructions are used to instruct the execution of the functions provided by the corresponding modules. The execution operations include at least one of the following: calculation operations, communication operations, control operations, and memory access operations. The instructions include a preset number of format types and instruction fields. The instructions adopt a fixed-length instruction encoding format. The instruction fields include at least one of the following: opcode, register, function field, and immediate value. The function field supports new operations and custom functions.

[0009] The digital in-memory computing accelerator, instruction set architecture, and data processing method provided in this application can clearly define the physical composition and logical functions of the digital in-memory computing accelerator through three-level hardware modules from the unit level, core level, to the chip level. The chip-level modules can support workload allocation strategies for large-scale tasks, splitting the target computing task into multiple core-level computing tasks and allocating them to multiple core-level modules. Each core-level module can process different instruction sequences in parallel or cooperate to complete large-scale computing tasks, thereby realizing flexible inter-core data flow and pipelined parallel operation. It can efficiently execute instruction sequences and achieve multi-core collaboration, greatly improving the efficiency of task processing and effectively enhancing the overall system performance and throughput.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0012] Figure 1 This invention provides a schematic diagram of the structure of a digital in-memory computing accelerator according to an exemplary embodiment of the present application. Figure 2 This invention provides a schematic diagram of the structure of another digital in-memory computing accelerator according to an exemplary embodiment of the present application. Figure 3 This paper shows a schematic diagram of the structure of a chip-level module provided in an application example of this application; Figure 4 This paper shows a schematic diagram of the core-level module provided in an application example of this application; Figure 5 This paper shows a schematic diagram of the structure of a unit-level module provided in an application example of this application; Figure 6 A flowchart illustrating a data processing method based on a digital in-memory computing accelerator provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0013] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0014] To address the problem that existing digital in-memory computing accelerators are unable to handle tasks of different scales, resulting in low resource utilization efficiency and inefficient task processing, this application provides a digital in-memory computing accelerator, as well as a data processing method and instruction set architecture based on the digital in-memory computing accelerator.

[0015] First, the terms used in this application will be explained.

[0016] Deep Neural Networks (DNNs) are a type of artificial neural network with a multi-layered structure. They use deep, non-linear transformations to progressively abstract and represent input data, enabling the learning and recognition of complex patterns. DNNs have achieved significant success in fields such as computer vision and natural language processing. However, their complex structure and massive number of parameters also bring huge computational and storage requirements, making them one of the main application scenarios for in-memory computing technology.

[0017] Compute-In-Memory (CIM): CIM is a novel computing paradigm that reduces or eliminates latency and energy consumption caused by frequent data movement between the processor and memory in traditional von Neumann architectures by performing computational operations directly within memory cells. This technology is particularly well-suited for data-intensive applications such as deep neural networks, effectively overcoming the "memory wall" and "power wall" problems.

[0018] Digital in-memory computing architecture: Digital in-memory computing architecture is an implementation of in-memory computing. It achieves digital computing functions within the storage cells of Static Random Access Memory (SRAM). Compared to analog in-memory computing, digital in-memory computing offers higher computational accuracy and stability, making it more suitable for applications with high precision requirements.

[0019] Analog in-memory computing architecture: Analog in-memory computing architecture is an in-memory computing scheme implemented in the analog domain (such as the current, voltage, or time domain). It directly performs analog multiplication and addition operations by utilizing the physical characteristics of the storage medium (such as impedance, conductance, etc.) to replace traditional digital operations, thereby improving energy efficiency. However, its accuracy is usually affected by material properties and environmental factors.

[0020] Static Random Access Memory (SRAM): SRAM is a common semiconductor memory that uses bistable circuitry (typically six transistors) to store each bit. As long as power is maintained, the data remains unchanged without the need for refresh operations. SRAM is relatively mature in integrated circuit technology, has high integration density, and fast read / write speeds, but its bit density is relatively low, resulting in higher cost per unit area compared to other storage media such as DRAM.

[0021] Instruction Set Architecture (ISA): An instruction set architecture is a specification that defines how a processor executes instructions, including instruction format, opcode, and addressing mode. For in-memory computing architectures, dedicated instruction set designs can fully leverage their unique hardware capabilities to improve computational efficiency. General-purpose and scalable instruction set architecture designs can support different types of in-memory computing hardware platforms and diverse computing tasks.

[0022] In-memory compute macrocell (CIM Macro): An in-memory compute macrocell is a fundamental component of an in-memory compute architecture, typically consisting of a memory array (such as SRAM) and dedicated compute logic. Multiple macrocells can be combined to construct larger-scale in-memory compute units or architectures, effectively solving the "memory wall" problem in traditional compute architectures.

[0023] Matrix-Vector Multiplication (MVM): Matrix-vector multiplication is one of the core computational operations in deep neural networks, especially prevalent in convolutional neural networks and fully connected layers. In in-memory computing architectures, MVM operations can be efficiently executed directly within memory units without moving data out to separate computation units, thus significantly improving computational efficiency and energy efficiency.

[0024] Single Instruction Multiple Data (SIMD): SIMD is a parallel computing paradigm that allows a single instruction to perform the same operation on multiple data elements simultaneously. In in-memory computing architectures, SIMD enables multiple in-memory arrays to execute the same instruction in parallel based on the data stored in their respective memory locations, improving computational parallelism and processing efficiency.

[0025] Example 1 Figure 1 A schematic diagram of the structure of a digital in-memory computing accelerator illustrated in an exemplary embodiment of this application is shown. Figure 1 As shown, the digital in-memory computing accelerator mainly includes: a chip-level module 10, a core-level module 20, and a unit-level module 30; wherein: The chip-level module 10 includes: multiple core-level modules 20 connected via a network-on-chip (NoC) for acquiring target computing tasks and distributing the target computing tasks into multiple core-level computing tasks to the multiple core-level modules 20 according to a load distribution strategy. The core-level module 20 includes: a control unit 201, a computing unit 202, and a cache unit 203. The computing unit 202 is composed of unit-level modules 30, and the computing unit 202 includes at least an in-memory computing unit. The control unit 201 includes: an instruction fetch module 2011, a decoding module 2012, and an instruction scheduler 2013, wherein: The instruction fetch module 2011 is used to read the instruction to be executed corresponding to the core-level computing task, wherein the instruction to be executed is the instruction of the digital in-memory computing instruction set architecture provided in this application embodiment (see the description below for details); the decoding module 2012 is used to decode the instruction to be executed to generate control signals; the instruction scheduler 2013 is used to send the control signals to the computing unit 202 to drive the computing unit 202 to perform corresponding computing operations; The calculation unit 202 is used to perform corresponding calculation operations according to the drive of the control signal. When the instruction type of the instruction to be executed is an in-memory calculation instruction, the in-memory calculation unit is used to perform in-memory calculation operations according to the drive of the control signal. Cache unit 203 is used to cache intermediate data generated during the execution of corresponding calculation operations.

[0026] The digital in-memory computing accelerator provided in this application adopts a hierarchical abstract design for its digital in-memory computing hardware architecture, consisting of unit-level, core-level, and chip-level architectures from bottom to top. The chip-level module 10 is composed of multiple core-level modules 20, and the computing units within the core-level modules 20 are composed of unit-level modules 30. This three-tiered hardware architecture clearly defines the organization, functional characteristics, and interrelationships of computing resources at each level, providing a unified and scalable hardware view for subsequent instruction set design, compilation optimization, and execution implementation. It also supports data interaction and collaborative work between different levels within the overall architecture.

[0027] In this embodiment, the chip-level module 10 serves as the highest-level organizational structure of the digital in-memory computing accelerator. Multiple core-level modules are connected via an efficient on-chip network (NoC) (such as a two-dimensional mesh structure). The NoC is responsible for data exchange and synchronous communication between the core-level modules. This multi-core on-chip interconnection organization enables the digital in-memory computing accelerator provided in this embodiment to support workload allocation strategies for large-scale tasks, achieving flexible inter-core data flow and pipelined parallel operations, thereby efficiently executing large-scale deep learning models.

[0028] As an optional implementation method in this embodiment, such as Figure 2As shown, the chip-level module 10 also includes an on-chip global memory 40, which is connected to multiple core-level modules 20 via an on-chip network. The on-chip global memory 40 is used to store the weights of the entire neural network model, large-scale feature map data, or intermediate results shared by multiple cores. The chip-level module 10 typically integrates a relatively large global memory 40 to support data access for each core-level module 20.

[0029] In one application example Figure 3 A schematic diagram of the structure of a chip-level module 10 illustrated in an exemplary embodiment of this application is shown. Figure 3 As shown, the chip-level module 10 includes 16 core-level modules, namely core 0, core 1... core 15, and the 16 core-level modules and the global storage unit are interconnected through NoC.

[0030] In this embodiment, the digital in-memory computing accelerator provided in this application adopts a multi-core collaborative execution mechanism. At the chip level, collaborative computing and data synchronization among multiple core-level modules 20 are achieved through an on-chip network. Each core-level module 20 can process different instruction sequences in parallel or cooperate to complete large-scale computing tasks. Data transmission between core-level modules 20 is achieved through SEND / RECV communication instructions, and execution synchronization among core-level modules 20 is achieved through WAIT / BARRIER synchronization instructions. This collaborative mechanism supports flexible workload allocation strategies, enabling both task-level parallelism (different core-level modules 20 process different computing tasks) and data-level parallelism (multiple core-level modules 20 cooperate to process a single large-scale task), thereby fully leveraging the computing potential of the multi-core architecture and effectively improving the overall system performance and throughput.

[0031] Furthermore, in this embodiment, the digital in-memory computing accelerator provided in this application adopts an instruction execution pipeline design. Each core-level module uses a three-stage pipelined processing architecture to complete instruction processing, including: instruction fetch (IF), instruction decode (ID), and instruction execution (IE) stages. In the instruction fetch stage, the instruction fetch module 2011 reads the instruction to be executed corresponding to the core-level computing task from the instruction memory. In the instruction decode stage, the decode module 2012 is responsible for parsing the instruction opcode and operands, reading the corresponding operands from the register file, processing the data dependencies between instructions, and generating control signals. In the instruction execution stage, the instruction scheduler 2013 distributes the control signals to the corresponding computing units 202 to execute the corresponding computing operations. In this embodiment, the instruction processing is completed through a three-stage pipelined processing architecture, which can efficiently execute instruction sequences and achieve multi-core collaboration, greatly improving the efficiency of task processing.

[0032] As an optional implementation in this embodiment, the instruction scheduler 2013 is used to check the readiness status of the computing unit 202. If the computing unit 202 is busy, it notifies the instruction fetch module 2011 to stop reading instructions. If the computing unit 202 is idle, it performs data conflict detection. If a data conflict occurs, it notifies the instruction fetch module 2011 to stop reading instructions. If no data conflict occurs, the instruction fetch module 2011 continues to execute the instruction reading operation.

[0033] In this embodiment, as an optional implementation, the instruction scheduler 2013 performs data conflict detection in the following way: it compares the data address of the instruction to be executed with the address recorded in the scoreboard of the computing unit 202. If there is address overlap, it is determined that a data conflict has occurred; otherwise, it is determined that no data conflict has occurred.

[0034] In this embodiment, a scoreboard is maintained for each computing unit to record the ready state (e.g., busy or idle) and the data read / write address range of the currently executing instructions. By comparing the data address of the instruction to be executed with the address recorded in the scoreboard of the computing unit, it is determined whether a data conflict has occurred. When a data conflict is detected, the instruction fetch module 2011 is notified to stop the instruction fetching operation, and necessary pipeline pauses are automatically inserted to ensure the correctness of the execution result. In this embodiment, the conflict control mechanism based on the scoreboard can ensure the correct and efficient execution of the instruction sequence.

[0035] As an optional implementation method in this embodiment, such as Figure 2 As shown, the core-level module 20 also includes a data transmission unit 204, wherein the data transmission unit 204 is used to transmit intermediate data between the computing unit and the cache unit, data between each core-level module, and data between the core-level module and the on-chip global memory.

[0036] In this embodiment, the data transmission unit 204 realizes data transmission between core modules 20 through SEND / RECV communication instructions, and realizes execution synchronization between cores through WAIT / BARRIER synchronization instructions, thereby achieving collaborative computing and data synchronization between multiple cores, giving full play to the computing potential of the multi-core architecture, and effectively improving the overall system performance and throughput.

[0037] As an optional implementation in this embodiment, the cache unit 203 specifically includes: local memory and a register file (RF). The local memory is used to cache weight data, input / output feature map data, and other intermediate data during the computation process. Optionally, the local memory is typically divided into multiple segments and addressed using a unified address space shared with the computation units within the core module, thereby improving data access efficiency and flexibility. The register file (RF) contains a set of general-purpose registers (GRF) and special-purpose registers (SRF). General-purpose registers are used to temporarily store instruction operands and temporary computation results; special-purpose registers may be used to store core state, configuration information, or control parameters for specific operations. Data binding or transfer can occur between general-purpose registers and special-purpose registers under certain circumstances.

[0038] In this embodiment, the computing unit 202 of the core-level module 20 is composed of unit-level modules 30. As an optional implementation in this embodiment, the unit-level module 30 further includes a vector computing unit and a scalar computing unit. Specifically, the instruction scheduler 2013 sends control signals to the computing units in the following manner: according to the instruction type of the instruction to be executed, the control signals are sent to the computing units corresponding to the instruction type; the vector computing unit is used to execute vector computing operations according to the control signals when the instruction type is a vector computing instruction; the scalar computing unit is used to execute scalar computing operations according to the control signals when the instruction type is a scalar computing instruction. Through the collaborative work of these different types of computing units, each completing its corresponding computing operation, complex computational tasks are accomplished in a coordinated manner.

[0039] In one application example Figure 4 A schematic diagram of the core-level module 20 shown in an exemplary embodiment of this application is illustrated. Figure 4As shown, the core-level module 20 includes: an instruction memory that stores instructions to be executed corresponding to core-level computing tasks; an instruction fetch module responsible for fetching instructions; a decoding module responsible for decoding instructions; and an instruction scheduling module responsible for scheduling and execution phase control, sending the decoded control signals to each computing unit. In this embodiment, the instruction scheduler adopts a scoreboard-based conflict control mechanism to handle computational resource conflicts and data conflicts between instructions, ensuring the correct and efficient execution of the instruction sequence; the computing units include scalar computing units, vector computing units, and in-memory computing units. The local memory is divided into segments 0-k, storing intermediate data of the vector computing units and in-memory computing units in segments; registers are divided into dedicated registers and general-purpose registers. General-purpose registers are used to temporarily store instruction operands and temporary calculation results; dedicated registers may be used to store the status, configuration information, or control parameters of specific operations of the core module. Data binding or transfer can be performed between general-purpose registers and dedicated registers under certain circumstances.

[0040] In this embodiment, the core-level module 20 serves as the smallest independently controllable computation execution unit in the digital in-memory computing accelerator. Each module 20 possesses its own independent instruction fetching and decoding logic, instruction memory, and a complete set of computing and storage resources, enabling it to autonomously complete complex computational tasks.

[0041] In one application example Figure 5 A schematic diagram of the structure of a unit-level module 30 according to an exemplary embodiment of this application is shown. In this embodiment, the unit-level module 30 is a basic functional unit that performs specific computational operations. These computational units constitute the basic modules of the core-level module 20, such as... Figure 5 As shown, the unit-level module 30 mainly includes an in-memory computation unit, a vector computation unit, and a scalar computation unit.

[0042] The following is a detailed description of each computing unit.

[0043] (1) The in-memory computing unit (CIM Compute Unit) is a module that performs in-memory computing tasks such as core matrix-vector multiplication (MVM).

[0044] like Figure 5 As shown, the in-memory computing unit comprises multiple macrogroups. Each macrogroup consists of multiple macrounits. For example... Figure 5 As shown, a macrounit group can contain K macrounits. These macrounits typically store different weight information from the neural network and share the same input activation data. This organization allows multiple macrounits to process the same input data in parallel, thus significantly improving computational throughput. The entire in-memory computing unit typically consists of T such macrounit groups to support larger-scale parallel computing.

[0045] A macrocell is a storage and computing unit composed of multiple storage elements organized in a two-dimensional array. For example... Figure 5 As shown, a macrocell typically consists of H rows and W columns of elements, each element usually consisting of an array of m rows and n columns of storage cells, where n corresponds to the basic data bit width. Internally, it integrates corresponding 1-bit computational logic circuits, which are shared by the m rows of storage cells. It activates specific rows through word lines and interacts with data through bit lines, realizing bit-level operations between stored data and input data. The total storage capacity of a macrocell can be represented as W × H × m × n bits. Inside the macrocell, all elements in the same row share the same input feature data, while weight data is mapped and stored in different elements in a specific way. The macrocell has the ability to effectively aggregate and accumulate the computation results of elements through internal adder trees and accumulators, thereby completing partial sums and even the final matrix-vector multiplication calculations. To optimize power consumption, if some columns of elements in the macrocell are not participating in the current effective computation, their corresponding computation and subsequent processing logic can be selectively turned off.

[0046] (2) The Vector Compute Unit is responsible for performing common vector operations in deep neural networks. These operations are usually performed after the in-memory computation unit completes MVM. Its functions include, but are not limited to: activation function calculation (such as ReLU, Sigmoid, etc.), pooling operations, quantization, and element-wise addition (such as residual connections).

[0047] (3) The Scalar Compute Unit mainly performs scalar arithmetic operations, address calculation, loop control, condition judgment, instruction jump and other control flow related operations, and participates in instruction blocking and scheduling decisions.

[0048] Through the aforementioned three-tiered hardware abstraction from the unit level, core level to the chip level, the digital in-memory computing accelerator provided in this application not only clearly defines the physical composition and logical functions of the digital in-memory computing architecture, but also provides a stable and easy-to-understand hardware model for upper-layer software (such as compilers). This enables compilers to effectively map and schedule complex computing tasks to specific hardware resources, fully leveraging the advantages of in-memory computing, while also laying a solid foundation for the versatility, flexibility, and future scalability of instruction set architectures.

[0049] The digital in-memory computing accelerator provided in this application clearly defines its physical composition and logical functions through three-level hardware modules from the unit level, core level to the chip level. The chip-level module can support workload allocation strategies for large-scale tasks, splitting the target computing task into multiple core-level computing tasks and allocating them to multiple core-level modules. Each core-level module can process different instruction sequences in parallel or cooperate to complete large-scale computing tasks, thereby realizing flexible inter-core data flow and pipelined parallel operation. It can efficiently execute instruction sequences and achieve multi-core collaboration, greatly improving task processing efficiency and effectively enhancing the overall system performance and throughput.

[0050] Example 2 This application also provides a data processing method based on a digital in-memory computing accelerator. Figure 6 A flowchart illustrating a data processing method for a digital in-memory computing accelerator provided in an exemplary embodiment of this application is shown. This data processing method employs the methods described above... Figures 1 to 5 The digital in-memory computing accelerator is implemented in any of the embodiments shown. The following is only a brief description of this data processing method; for other matters not covered herein, please refer to the relevant descriptions in the section on digital in-memory computing accelerators.

[0051] like Figure 6 As shown, the data processing method of this digital in-memory computing accelerator includes the following steps: S101, The chip-level module obtains the target computing task and, according to the load distribution strategy, splits the target computing task into multiple core-level computing tasks and distributes them to multiple core-level modules. S102, The control unit of the core module reads the instruction to be executed corresponding to the core computing task, decodes the instruction to be executed to generate a control signal, sends the control signal to the computing unit to drive the computing unit to execute the corresponding computing operation, and sends the intermediate data generated by the execution of the corresponding computing operation to the cache unit for caching; S103. The calculation unit performs corresponding calculation operations according to the control signal; wherein, when the instruction type of the instruction to be executed is an in-memory calculation instruction, the in-memory calculation operation is performed according to the control signal.

[0052] In this embodiment, the chip-level module supports a workload allocation strategy for large-scale tasks, splitting the target computing task into multiple core-level computing tasks and distributing them to multiple core-level modules. Each core-level module can process different instruction sequences in parallel or collaborate to complete large-scale computing tasks. This enables flexible inter-core data flow and pipelined parallel operations, thereby efficiently executing large-scale deep learning models.

[0053] Furthermore, in this embodiment, the digital in-memory computing accelerator provided in this application adopts an instruction execution pipeline design. Each core-level module uses a three-stage pipelined processing architecture to complete instruction processing, including: instruction fetch (IF), instruction decode (ID), and instruction execution (IE) stages. In the instruction fetch stage, the control unit reads the instruction to be executed corresponding to the core-level computing task from the instruction memory. In the instruction decode stage, the control unit is responsible for parsing the instruction opcode and operands, reading the corresponding operands from the register file, processing the data dependencies between instructions, and generating control signals. In the instruction execution stage, the control unit distributes the control signals to the corresponding computing units to execute the corresponding computational operations. In this embodiment, the instruction processing is completed through a three-stage pipelined processing architecture, which can efficiently execute instruction sequences and achieve multi-core collaboration, greatly improving the efficiency of task processing.

[0054] As an optional implementation method in this embodiment, before the core-level module reads the instructions to be executed corresponding to the core-level computing task, the data processing method provided in this application embodiment further includes: The control unit of the core module checks the readiness status of the computing unit. If the computing unit is busy, the operation of reading instructions is stopped. If the computing unit is idle, data conflict detection is performed. If a data conflict occurs, the operation of reading instructions is stopped. If no data conflict occurs, the control unit continues to execute the operation of reading instructions.

[0055] As an optional implementation in this embodiment, the instruction scheduler of the core module performs data conflict detection, including: comparing the data address of the instruction to be executed with the address recorded in the scoreboard of the computing unit; if there is address overlap, it is determined that a data conflict has occurred; otherwise, it is determined that no data conflict has occurred.

[0056] In this embodiment, a scoreboard is maintained for each computing unit to record its readiness status (e.g., busy or idle) and the data read / write address range of the currently executing instructions. By comparing the data address of the instruction to be executed with the address recorded in the scoreboard of the computing unit, a data conflict is determined. When a data conflict is detected, the control unit stops reading the instruction and automatically inserts necessary pipeline pauses to ensure the correctness of the execution result. In this embodiment, the conflict control mechanism based on the scoreboard can ensure the correct and efficient execution of the instruction sequence.

[0057] The data processing method provided in this application embodiment is based on a digital in-memory computing accelerator with three-level hardware modules from unit level, core level to chip level. The chip-level module can support a workload allocation strategy for large-scale tasks, splitting the target computing task into multiple core-level computing tasks and allocating them to multiple core-level modules. Each core-level module can process different instruction sequences in parallel or cooperate to complete large-scale computing tasks, thereby realizing flexible inter-core data flow and pipeline parallel operation. It can efficiently execute instruction sequences and realize multi-core collaboration, greatly improving the efficiency of task processing and effectively enhancing the overall system performance and throughput.

[0058] Example 3 This application also provides an instruction set architecture. This instruction set architecture is based on the above-described... Figures 1 to 5 The digital in-memory computing accelerator implementation in any of the illustrated embodiments follows the design principles of Reduced Instruction Set Computer (RISC) and aims to provide a complete, efficient, and scalable programming interface for the digital in-memory computing accelerator. This instruction set architecture includes instructions dependent on chip-level modules, chip-level modules themselves, and the functions provided by those modules. These instructions instruct the execution of operations related to the functions provided by the corresponding modules. Specifically, the execution operations include at least one of the following: computation operations, communication operations, control operations, and memory access operations. Through the flexible combination of these instructions, diverse data flow and computational requirements in deep learning models can be efficiently supported, thereby fully leveraging the advantages of digital in-memory computing in terms of data processing bandwidth and energy efficiency.

[0059] As an optional implementation in this application, the instructions in the instruction set architecture access the storage resources in the digital in-memory computing accelerator through virtual addresses; wherein, all addressable storage resources in the digital in-memory computing accelerator are mapped to the same continuous virtual address space, and the virtual address in the instruction is converted into the physical address of the storage resource in the digital in-memory computing accelerator through a preset address mapping mechanism.

[0060] To achieve efficient and unified management of diverse storage resources (including the storage space inherent in the in-memory computing array itself, input / output caches, local memory, and global memory) in digital in-memory computing accelerators, the instruction set adopts a virtual address space-based design. This application maps all addressable storage resources within the digital in-memory computing accelerator to the same contiguous virtual address space. When instructions perform memory access operations, they use virtual addresses, eliminating the need for additional segment identifiers or storage type information within the instructions. At the hardware level, a dedicated address mapping mechanism converts the virtual address (VA) in the instructions into the physical address (PA) of the corresponding physical storage unit. Alternatively, this conversion process can be represented by an address mapping function F, which can be implemented based on segment tables, page tables, or a simpler method of adding an offset to a base address. The conversion process is: PA = VA - Offset, where "Offset" = F(VA), representing the starting address offset of a specific storage resource in the virtual address space.

[0061] In this embodiment, the virtual address space design simplifies instruction formatting, allowing instructions to access any storage location simply by including a virtual address, thus reducing instruction coding complexity. Furthermore, it enhances storage management flexibility, enabling the layout and mapping of physical storage resources to be flexibly adjusted according to application requirements and system configuration. This effectively avoids address space fragmentation issues that can occur with traditional segmented management, improving storage space utilization. Moreover, when the digital in-memory computing accelerator needs to adjust or expand storage resources (such as adding new memory types or changing the size of existing memory), only the address mapping configuration needs modification, minimizing the impact on the instruction set's compatibility and making it easier to adapt to diverse data flows and evolving application needs.

[0062] As an optional implementation in this application embodiment, the instructions in the instruction set include: a preset number of format types and instruction fields. In this embodiment, the instruction set mainly adopts a fixed-length instruction encoding format, such as 32-bit instruction encoding, to simplify the hardware's instruction fetching and decoding logic and improve execution efficiency. According to the organization of the opcode and operand segments, the instructions can be divided into several general format types. Specifically, the instruction format types can be divided into R-type, I-type, and J-type instructions. These format types provide a regular structure for the hardware decoding and software processing (such as compiler instruction generation) of the instructions. In some embodiments, the instruction field includes at least one of the following: opcode, register, function field, immediate value; wherein, the function field supports new operations and custom functions. In this embodiment, typical instruction fields include: (1) opcode, located in the high bits of the instruction (e.g., bits 31-26), used to indicate the basic operation type to be performed by the instruction. (2) source register (RS, RT), used to specify the index of the general or special register where the source operands involved in the operation are located. (3) destination register (RD), used to specify the index of the general or special register where the operation result is stored. (4) Additional registers (RE, RF), which serve as additional register operands in specific instructions. (5) Function field (Funct), which in R-type instructions usually serves as the function code (Funct) to further distinguish the specific operation. (6) Immediate value, used to store constant values, address offsets, or partial jump addresses. Its bit width and position vary depending on the specific subtype of I-type and J-type instructions.

[0063] Typical instruction format types and their field assignments are shown in Table 1, the format definitions of each instruction field are shown in Table 2, and the assembly code definitions, function descriptions, and examples of each instruction are shown in Table 3.

[0064] Table 1.

[0065] Table 2.

[0066] Table 3.

[0067] In this embodiment, the instruction set architecture supports new computations and custom functions, such as different multicast data streams and sparse neural network inference operations, by extending the Flags field of the CIM_MVM instruction; and it is compatible with new network layer algorithms by extending the Funct field of the SIMD and REDUCE instructions. When the network structure is adjusted or a new algorithm is introduced, the instruction set provided in this embodiment can significantly reduce design costs and improve system flexibility.

[0068] In summary, this embodiment establishes a unified interface connecting applications and hardware through the hardware structure and instruction set architecture specific to the operating modes of digital in-memory computing accelerators. The instruction set architecture supports diverse in-memory computing operations, providing dedicated instructions for various operations such as matrix-vector multiplication, bitwise operations, and data movement. The flexible combination of instructions within the instruction set architecture enables support for complete deep neural network workloads. Furthermore, the instruction set architecture possesses a flexible extension mechanism to adapt to different hardware implementations and application scenarios.

[0069] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0070] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A digital in-memory computing accelerator, characterized in that, include: Chip-level modules, core-level modules, and unit-level modules; among which: The chip-level module includes: multiple core-level modules connected via an on-chip network, used to acquire a target computing task and, according to a load distribution strategy, split the target computing task into multiple core-level computing tasks and distribute them to the multiple core-level modules; The core-level module includes: a control unit, a computing unit, and a cache unit. The computing unit is composed of the unit-level modules and includes at least an in-memory computing unit. The control unit includes: an instruction fetch module, a decoding module, and an instruction scheduler, wherein: The instruction fetching module is used to read the instruction to be executed corresponding to the core-level computing task, wherein the instruction to be executed is an instruction of the digital in-memory computing instruction set architecture; The decoding module is used to decode the instruction to be executed and generate control signals; The instruction scheduler is used to send the control signal to the computing unit to drive the computing unit to perform corresponding computing operations; The computing unit is configured to perform corresponding computing operations according to the control signal; wherein, when the instruction type of the instruction to be executed is an in-memory computing instruction, the in-memory computing unit is configured to perform in-memory computing operations according to the control signal. The cache unit is used to cache intermediate data generated during the execution of the corresponding calculation operation.

2. The digital in-memory computing accelerator according to claim 1, characterized in that, The instruction scheduler is also used to check the readiness status of the computing unit. If the computing unit is busy, it notifies the instruction fetch module to stop reading instructions. If the computing unit is idle, it performs data conflict detection. If a data conflict occurs, it notifies the instruction fetch module to stop reading instructions. If no data conflict occurs, the instruction fetch module continues to execute the instruction reading operation.

3. The digital in-memory computing accelerator according to claim 1, characterized in that, The instruction scheduler performs data conflict detection in the following ways: The data address of the instruction to be executed is compared with the address recorded in the scoreboard of the computing unit. If there is address overlap, a data conflict is determined to have occurred; otherwise, no data conflict is determined to have occurred.

4. The digital in-memory computing accelerator according to claim 1, characterized in that, The chip-level module further includes: an on-chip global memory, connected to multiple core-level modules via the on-chip network; the core-level module further includes: a data transmission unit, wherein: The data transmission unit is used to transmit intermediate data between the computing unit and the cache unit, data between each of the core-level modules, and data between the core-level modules and the on-chip global memory.

5. The digital in-memory computing accelerator according to claim 1, characterized in that, The computing unit further includes: a vector computing unit and a scalar computing unit; The vector calculation unit is used to perform vector calculation operations according to the control signal when the instruction type is a vector calculation instruction; The scalar calculation unit is used to perform scalar calculation operations according to the control signal when the instruction type is a scalar calculation instruction.

6. A data processing method based on the digital in-memory computing accelerator according to any one of claims 1 to 5, characterized in that, include: The chip-level module acquires the target computing task and, according to the load distribution strategy, splits the target computing task into multiple core-level computing tasks and distributes them to multiple core-level modules. The control unit of the core module reads the instruction to be executed corresponding to the core computing task, decodes the instruction to be executed to generate a control signal, sends the control signal to the computing unit to drive the computing unit to perform the corresponding computing operation, and sends the intermediate data generated by the execution of the corresponding computing operation to the cache unit for caching. The instruction to be executed is an instruction of the digital in-memory computing instruction set architecture. The computing unit performs corresponding computing operations according to the control signal; wherein, when the instruction type of the instruction to be executed is an in-memory computing instruction, the in-memory computing operation is performed according to the control signal.

7. The method according to claim 6, characterized in that, Before the core-level module reads the instruction to be executed corresponding to the core-level computing task, the method further includes: The control unit of the core module checks the readiness status of the computing unit. If the computing unit is busy, the operation of reading instructions is stopped. If the computing unit is idle, data conflict detection is performed. If a data conflict occurs, the operation of reading instructions is stopped.

8. The method according to claim 7, characterized in that, The control unit of the core module performs data conflict detection, including: The data address of the instruction to be executed is compared with the address recorded in the scoreboard of the computing unit. If there is address overlap, a data conflict is determined to have occurred; otherwise, no data conflict is determined to have occurred.

9. An instruction set architecture based on the digital in-memory computing accelerator according to any one of claims 1 to 5, characterized in that, include: Instructions that rely on the functions provided by the chip-level module, the core-level module, and the unit-level module, the instructions are used to instruct the execution of the functions provided by the corresponding modules. The execution operations include at least one of the following: calculation operations, communication operations, control operations, and memory access operations. The instructions include a preset number of format types and instruction fields. The instructions adopt a fixed-length instruction encoding format. The instruction fields include at least one of the following: opcode, register, function field, and immediate value. The function field supports new operations and custom functions.

10. The instruction set architecture according to claim 9, characterized in that, The instruction accesses storage resources in the digital in-memory computing accelerator via a virtual address; wherein all addressable storage resources in the digital in-memory computing accelerator are mapped to the same continuous virtual address space, and the virtual address in the instruction is converted into the physical address of the storage resource in the digital in-memory computing accelerator through a preset address mapping mechanism.

Images