An in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control

By using a shared ring buffer and multi-node collaborative control in-memory computing accelerator device, the control communication overhead and intermediate result transfer problems between the global control node and the computing node are solved, achieving efficient computing node utilization and pipeline continuity, and reducing memory access power consumption and transmission latency.

CN123117218APending Publication Date: 2026-11-17GUIZHOU POWER GRID CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611114977.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-27
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing in-memory computing accelerators have high overhead in control communication between global control nodes and computing nodes, high overhead in transporting intermediate results between multiple computing nodes, and are prone to arbitration conflicts and bandwidth bottlenecks.

Method used

The architecture adopts a shared ring buffer and multi-node collaborative control. Through hierarchical control between global control nodes and local controllers, fine-grained control commands are reduced. Intermediate results are directly transferred between adjacent computing nodes using a shared ring buffer. Storage partitions that operate in a ping-pong manner are used to reduce waiting time.

Benefits of technology

It reduces global control communication overhead and software scheduling complexity, reduces intermediate result handling overhead, improves computing node utilization and pipeline continuity, and reduces memory access power consumption and transmission latency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123117218A_ABST
    Figure CN123117218A_ABST
Patent Text Reader

Abstract

This invention discloses an in-memory computing accelerator device based on a shared ring buffer and multi-node collaborative control, belonging to the field of in-memory computing accelerator technology. It includes a shared ring buffer memory, multiple computing nodes, and a global control node. The multiple computing nodes are connected sequentially to form a ring structure, and each set of shared buffers is located between two adjacent computing nodes. This invention generates the control timing sequence of the in-memory computing unit based on the task description through a local controller. The global control node only issues the task description, input feature map, and synchronization information, without issuing cycle-by-cycle micro-control signals, reducing global control communication overhead and software scheduling complexity. Intermediate results are directly transmitted between adjacent computing nodes via the shared buffer, avoiding intermediate results being relayed through the global bus, reducing memory access power consumption and transmission latency, and improving computing node utilization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of in-memory computing accelerator technology, specifically to an in-memory computing accelerator device based on a shared ring buffer and multi-node collaborative control. Background Technology

[0002] As the computational load and number of parameters in neural network models continue to increase, traditional AI accelerators employ a structure that separates computation and storage units. Weight data, input feature maps, and intermediate results need to be frequently moved between memory and computation units, resulting in high memory access power consumption, large data transfer latency, and significant pressure on on-chip bus bandwidth. In-memory computing architectures, by performing multiplication and accumulation calculations within or near the memory array, reduce data movement and have become an important technological direction for AI acceleration chips. Existing in-memory computing accelerators typically include multiple compute nodes, on-chip memory, external data interfaces, and on-chip interconnect buses. Multiple compute nodes can process different data blocks separately or form a pipeline according to neural network layers.

[0003] Existing technologies have the following problems in terms of control architecture and data flow organization: The global control node directly issues fine-grained control information such as data loading, computation startup, array configuration, and result reading to the in-memory computing unit inside the computing node. Since in-memory computing involves a large number of micro-operations, these micro-operations consume a large amount of on-chip bus bandwidth and increase the software scheduling complexity of the global control node. At the same time, when transferring intermediate results between multiple computing nodes, the intermediate results of the front-end computing node need to be written back to the global memory first and then read out by the back-end computing node, resulting in redundant handling. When multiple computing nodes share a unified on-chip memory, arbitration conflicts and bandwidth bottlenecks are likely to occur. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention provides an in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control.

[0005] Therefore, the technical problem solved by this invention is: how to reduce the communication overhead between the global control node and the computing node while reducing the overhead of transporting intermediate results between multiple computing nodes.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a memory-based computing accelerator device based on shared ring buffer and multi-node collaborative control, comprising, It features a shared ring buffer, multiple compute nodes, and a global control node. The multiple computing nodes are connected in sequence to form a ring structure, and each computing node includes a local controller and a memory computing unit. The shared ring buffer memory includes multiple sets of shared buffers. Each set of shared buffers is set between two adjacent computing nodes. The global control node uses each set of shared buffers as an input buffer, intermediate buffer, output buffer, or inter-node transfer buffer. The local controller is used to generate the control timing for the in-memory computing unit to perform in-memory computing based on the task description, and to control the data transfer between the in-memory computing unit and the shared buffer based on the state of the shared buffer. The global control node is used to send the task description to the plurality of computing nodes, write the input feature map into the shared ring buffer, receive the completion status sent by the plurality of computing nodes, and read the calculation result from the shared ring buffer according to the completion status.

[0007] As a preferred embodiment of the in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control described in this invention, each shared buffer includes a first storage partition and a second storage partition. The first storage partition and the second storage partition are respectively set with a writable status flag, a readable status flag, and a task number. The task number identifies the task to which the data block in the storage partition belongs. The compute node that writes the data block writes the data block to the storage partition when the writable status flag of the storage partition is set, and sets the readable status flag of the storage partition after the writing is completed. The compute node that reads the data block reads the data block when the readable status flag of the storage partition is set, and sets the writable status flag of the storage partition after the reading is completed.

[0008] As a preferred embodiment of the in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control described in this invention, wherein: the computing node that writes data blocks writes the current data block to the first storage partition, and the computing node that reads data blocks reads the previous data block from the second storage partition; After the current data block is written and the previous data block is read, the computing node that wrote the data block switches to the second storage partition to write the subsequent data block, and the computing node that reads the data block switches to the first storage partition to read the current data block.

[0009] As a preferred embodiment of the in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control described in this invention, each computing node is connected to both the input / intermediate buffer and the output / intermediate buffer. The input and intermediate buffer stores the input data input to the computing node, as well as the intermediate results stored back by the computing node; The output, which also serves as an intermediate buffer, stores the computation results output by the computing node, as well as the intermediate results stored back by the computing node. The input data is either the input feature map or the intermediate result of the previous calculation node.

[0010] As a preferred embodiment of the in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control described in this invention, wherein: the global control node controls the multiple computing nodes to work in parallel mode or pipelined mode; In the parallel mode, the same set of shared buffers serves as local extended buffers for two adjacent computing nodes, and data blocks are not transferred between adjacent computing nodes. In the pipelined mode, the computing node that writes data blocks writes the calculation results to a shared buffer, and the computing node that reads data blocks reads the calculation results from the shared buffer as input data.

[0011] As a preferred embodiment of the in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control described in this invention, the global control node further divides the multiple computing nodes into multiple computing node groups and distributes different data blocks to different computing node groups. Adjacent compute nodes within the same compute node group transfer data blocks via a shared buffer; The global control node isolates the shared buffer between different computing node groups, and multiple computing node groups process their respective data blocks in parallel.

[0012] This invention provides a control method for an in-memory computing accelerator that supports multi-node pipelined collaboration.

[0013] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a control method for an in-memory computing accelerator that supports multi-computing node pipelined collaboration, comprising: The global control node sends task descriptions to multiple computing nodes that are connected in a ring structure, and writes the input feature map into a shared buffer set between two adjacent computing nodes. Each computing node's local controller generates a control sequence based on the task description. The control sequence controls the in-memory computing unit of the computing node to perform in-memory computing. The local controller controls the data transfer between the in-memory computing unit and the shared buffer based on the state of the shared buffer. Two adjacent computing nodes pass data blocks through the shared buffer; The global control node receives the completion status sent by the multiple computing nodes and reads the computing results based on the completion status.

[0014] As a preferred embodiment of the in-memory computing accelerator control method supporting multi-compute node pipelined collaboration described in this invention, each shared buffer includes a first storage partition and a second storage partition, wherein the first storage partition and the second storage partition are respectively set with a writable status flag and a readable status flag. The compute node that writes the data block writes the data block to the storage partition when the writable status flag of the storage partition is set, and sets the readable status flag of the storage partition after the writing is completed. The compute node that reads the data block reads the data block when the readable status flag of the storage partition is set, and sets the writable status flag of the storage partition after the reading is completed.

[0015] As a preferred embodiment of the in-memory computing accelerator control method supporting multi-compute node pipelined collaboration described in this invention, wherein: the computing node writing data blocks writes the current data block to the first storage partition, and the computing node reading data blocks reads the previous data block from the second storage partition; After the current data block is written and the previous data block is read, the computing node that wrote the data block switches to the second storage partition to write the subsequent data block, and the computing node that reads the data block switches to the first storage partition to read the current data block.

[0016] As a preferred embodiment of the in-memory computing accelerator control method supporting multi-computing node pipelined collaboration described in this invention, wherein: the global control node divides the multiple computing nodes into multiple computing node groups, isolates the shared buffer between different computing node groups, and distributes different data blocks to the primary computing node in each computing node group; In each compute node group, compute nodes pass data blocks cascading through a shared buffer. The last-level computing node in each computing node group writes the processing result to the shared buffer and sends the completion status to the global control node.

[0017] The beneficial effects of this invention are as follows: This invention employs a two-level hierarchical control system between the global control node and the computing nodes. The global control node only sends task descriptions, input data, and synchronization information to the computing nodes, without sending cycle-by-cycle micro-control signals to the in-memory computing units. The control timing of the in-memory computing units is autonomously generated by the local controller within the computing nodes based on the task description. This prevents the on-chip bus from being occupied by a large number of fine-grained control commands, reducing global control communication overhead and software scheduling complexity. Furthermore, this invention establishes a shared buffer between adjacent computing nodes, forming a shared ring buffer memory. Adjacent computing nodes directly pass intermediate results through the shared buffer, and the data output by the preceding computing node is directly used as the input to the following computing node, avoiding intermediate results being relayed through the global memory or global bus, thus reducing memory access power consumption and transmission latency. Finally, this invention utilizes a ping-pong mechanism between the two storage partitions of the shared buffer. The process of the upstream computing node writing the current data block overlaps with the process of the downstream computing node reading the previous data block, reducing waiting time between upstream and downstream computing nodes and improving pipeline continuity and computing node utilization. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the control architecture of an in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control, provided as an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of a computing node in an in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control, provided as an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the data flow of an in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control in parallel mode, as provided in an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of the data flow of an in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control in pipeline mode, as provided in an embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram of the data flow of an in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control in a group pipeline mode, as provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0025] Example 1, referring to Figures 1-5 This is one embodiment of the present invention, which provides a memory computing accelerator device based on a shared ring buffer and multi-node collaborative control, comprising: It features a shared ring buffer, multiple compute nodes, and a global control node.

[0026] It should be noted that the in-memory computing accelerator system in this embodiment is used for neural network inference, which includes convolution calculations, matrix multiplication calculations, or fully connected calculations in image recognition, object detection, natural language processing, or edge intelligence scenarios. The global control node is a chip-level task scheduling and external data interaction control module. Figure 1 The node shown is the master control node. Figures 3 to 5 The diagram shows the main control node. The computing node is a computing execution module containing a memory computing unit, a local controller, a shared buffer, and a data transceiver interface. Figures 1 to 5 The node shown is the storage core computing node.

[0027] Reference Figure 1 The plurality of computing nodes are sequentially connected to form a ring structure. The shared ring buffer memory includes multiple sets of shared buffers disposed between two adjacent computing nodes. The global control node is connected to the plurality of computing nodes via an on-chip bus. The global control node and the plurality of computing nodes transmit data through a data transmission path and control information through a control path.

[0028] Specifically, Figure 1 Taking four computing nodes as an example, the four computing nodes are storage core computing node 1, storage core computing node 2, storage core computing node 3 and storage core computing node 4. The four computing nodes are connected in sequence and form a ring structure.

[0029] It is understood that the number of computing nodes is arbitrary, and those skilled in the art can increase or decrease the number of computing nodes based on four computing nodes, all of which are considered to be part of the ring structure of this embodiment.

[0030] The shared ring buffer memory is described below.

[0031] Reference Figure 1 The shared ring buffer memory includes multiple sets of shared buffers, each set of shared buffers is set between two adjacent computing nodes, and the global control node uses each set of shared buffers as an input buffer, intermediate buffer, output buffer or inter-node transfer buffer.

[0032] Specifically, taking four computing nodes as an example, a first set of shared buffers is set between in-store computing node 1 and in-store computing node 2, a second set of shared buffers is set between in-store computing node 2 and in-store computing node 3, a third set of shared buffers is set between in-store computing node 3 and in-store computing node 4, and a fourth set of shared buffers is set between in-store computing node 4 and in-store computing node 1. The four sets of shared buffers together form a ring structure, forming the shared ring buffer memory.

[0033] Furthermore, the global control node uses each set of shared buffers as an input buffer, intermediate buffer, output buffer, or inter-node transfer buffer. This means that the global control node configures the read / write direction, partition role, and upstream / downstream relationship of the computing nodes of the shared buffers, so that the same set of shared buffers can assume different buffer roles in different execution modes.

[0034] For example, in pipelined mode, the shared buffer is used as a buffer for passing data between nodes, and the output of the preceding computing node is directly used as the input of the following computing node via the shared buffer; in parallel mode, the shared buffer is used as an input buffer or an output buffer for each computing node.

[0035] The following reference Figure 2 The computing nodes are described below.

[0036] Reference Figure 2 Each computing node includes a local controller, a storage unit, a shared buffer, a program buffer, a data interface, a program parsing module, a shared buffer access controller, and local memory.

[0037] The in-memory computing unit is used to perform matrix multiplication, convolution mapping, fully connected computation, or other in-memory computation supported computations. Figure 2 The image shows a memristor-based in-memory computing core. The shared buffer is a shared data buffer for the computing node, used to store the input data, local intermediate results, and output results required for the operation of this computing node. The local memory is used to store the weight data required for the in-memory computing unit to perform calculations.

[0038] Reference Figure 2The computing node is equipped with input data processing and output data processing, forming the following data path: after the feature map input or the previous calculation result input is input through the shared buffer, it is preprocessed by the input data processing and then sent to the in-memory computing unit; after the in-memory computing unit completes the in-memory computing, the calculation result is postprocessed by the output data processing and written into the shared buffer, and then output as the inference result output or the intermediate result output.

[0039] Reference Figure 2 The computing node is equipped with a PE control module, which acts as the local controller to control the data path within the computing node, as well as the input data processing, the storage and computing unit, and the output data processing.

[0040] Reference Figure 2 The computing node is equipped with a program buffer and a program parser. The program buffer stores programs issued by the global control node. The program parsing is performed by the program parsing module, which converts the task description issued by the global control node into a locally executable micro-operation control sequence. The micro-operation control sequence is a microinstruction or a custom register control. The data interface is used for data transmission and reception of this computing node, and the shared buffer access controller is used to control the read and write access of this computing node to adjacent shared buffers.

[0041] The local controller is used to generate the control timing for the in-memory computing unit to perform in-memory computing based on the task description, and to control the data transfer between the in-memory computing unit and the shared buffer based on the state of the shared buffer.

[0042] It should be noted that the task description is a task-descriptive program issued by the global control node, at a higher level than micro-operation abstraction. The task description includes operator type, data address, block size, weight index, execution mode, and upstream and downstream relationships. The control timing is generated by the local controller within the computing node based on the task description. The control timing is used to control the in-memory computing unit to complete input loading, computation initiation, computation result reading, and result writing back.

[0043] It is understood that the global control node does not send cycle-by-cycle micro-control signals to the in-memory computing unit, and the cycle-by-cycle operation timing of the in-memory computing unit is generated by the local controller inside the computing node.

[0044] Furthermore, the local controller controls data transfer based on the state of the shared buffer, including determining whether to wait for upstream data or downstream buffer release based on the buffer state of the shared buffer; when data is valid in the shared buffer, the local controller controls the in-memory processing unit to read data from the shared buffer; when the shared buffer is idle, the local controller controls the in-memory processing unit to write data to the shared buffer.

[0045] In this embodiment, the local controller generates control timing within the computing node based on the task description. The global control node does not issue cycle-by-cycle micro-control signals, so that the on-chip bus is not occupied by a large number of fine-grained control commands, reducing the global control communication overhead. At the same time, the global control node does not need to manage the internal details of the in-memory computing unit, reducing the software scheduling complexity.

[0046] The following reference Figure 1 The global control node is described below.

[0047] Reference Figure 1 The global control node includes a main controller, an external data interface, a program loading interface, and an on-chip bus interface. The external data interface is used for feature map input. Figure 1 The interface shown is the external data interface; the program loading interface is used for program input. Figure 1 The interface shown is the external program interface; the on-chip bus interface is connected to the on-chip bus.

[0048] In some embodiments, the main controller employs a reduced instruction set processor.

[0049] In some embodiments, the external data interface is a gigabit Ethernet interface, a peripheral component interconnect high-speed interface, or other high-speed data input interface.

[0050] In some embodiments, the program loading interface adopts a serial peripheral interface, a general asynchronous transceiver interface, or other low-speed configuration interfaces.

[0051] In some embodiments, the on-chip bus employs an Advanced Scalable Interface Bus or other on-chip interconnect protocols.

[0052] The global control node is used to send the task description to the plurality of computing nodes, write the input feature map into the shared ring buffer, receive the completion status sent by the plurality of computing nodes, and read the calculation result from the shared ring buffer according to the completion status.

[0053] It should be noted that the global control node and the computing node form a two-level heterogeneous hierarchical control system. The first level is the global control layer, which is responsible for coarse-grained task allocation, data distribution, and state synchronization. The second level is the local control layer, where the local controller within the computing node autonomously controls the local in-memory computing unit. The global control node does not transmit cycle-by-cycle micro-control signals to the in-memory computing unit; it only transmits the task description and synchronization information.

[0054] Specifically, the global control node performs the following functions in the global control layer: receiving input feature maps or other data to be processed from the external data interface; reading the program and task description generated by the compiler from the program loading interface; determining the computational blocks, computational order, and dependencies responsible for each computing node according to the program; writing the task description, program fragment, and input data into the corresponding computing node; configuring the execution mode of the multiple computing nodes; updating the global task status according to the status signals fed back by the computing nodes; and reading the computation results from the final output computing node or the output buffer of each computing node.

[0055] The local controller performs the following functions at the local control layer: receiving and saving the task description issued by the global control node; parsing the operator type, data address, block size, weight index, execution mode, and upstream and downstream relationships in the task description; generating a microinstruction sequence to control the in-memory computing unit and the local data path; controlling the data transfer between the shared ring buffer and the in-memory computing unit; controlling the in-memory computing unit to complete input loading, computation initiation, computation result reading, and result writing back; determining whether to wait for upstream data or downstream buffer release based on the buffer status; sending a completion status to the global control node after the task is completed, and automatically switching to the next task when there is a pre-loaded task.

[0056] In this embodiment, a task description-level interface is used between the global control node and the computing nodes. The specific timing of the in-memory computing unit is managed by the local controller. When the number of computing nodes increases, the size of the in-memory computing unit changes, or the local data path is optimized, the global control node still schedules the data through a unified task description, thereby improving the scalability of the architecture.

[0057] The storage partitions and status flags for each shared buffer group are described below.

[0058] Reference Figure 1 and Figure 2Each shared buffer includes a first storage partition and a second storage partition. The first storage partition and the second storage partition are respectively set with a writable status flag, a readable status flag, and a task number. The task number identifies the task to which the data block in the storage partition belongs.

[0059] It should be noted that the first storage partition and the second storage partition have independent read and write status flags. The status flags include at least: an idle flag, indicating that the storage partition can be written to; a valid flag, indicating that the data stored in the storage partition can be read; a read complete flag, indicating that the downstream computing node has completed reading; a write complete flag, indicating that the upstream computing node has completed writing; and the task number, used to distinguish different tasks or different data blocks.

[0060] It is understood that the writable status flag corresponds to the idle flag, the readable status flag corresponds to the valid flag, and the read completion flag and the write completion flag are used to indicate the completion status of reading and writing.

[0061] Specifically, the computing node that writes the data block writes the data block to the storage partition when the writable status flag of the storage partition is set, and sets the readable status flag of the storage partition after the writing is completed.

[0062] The compute node that reads the data block reads the data block when the readable status flag of the storage partition is set, and sets the writable status flag of the storage partition after the reading is completed.

[0063] Furthermore, after the computing node that writes the data block completes the write operation, it also sets the write completion flag of the storage partition; after the computing node that reads the data block completes the read operation, it also sets the read completion flag of the storage partition.

[0064] It should be noted that the compute node writing the data block and the compute node reading the data block are one of two adjacent compute nodes connected to the same set of shared buffers. For a set of shared buffers, the compute node writing data to the shared buffer is the upstream compute node, and the compute node reading data from the shared buffer is the downstream compute node.

[0065] It is understandable that in a ring structure, the same computing node is relative to two sets of shared buffers on its two sides, namely the upstream computing node and the downstream computing node.

[0066] In this embodiment, status flags are set for the first storage partition and the second storage partition respectively. The computing nodes that write and read control the reading and writing according to the setting of the status flags, so that the writing of the upstream computing node and the reading of the downstream computing node are coordinated with each other, avoiding arbitration conflicts when concurrently accessing the unified on-chip memory.

[0067] The following describes the ping-pong working mode of the first storage partition and the second storage partition.

[0068] Reference Figure 4 The computing node that writes the data block writes the current data block to the first storage partition, and the computing node that reads the data block reads the previous data block from the second storage partition. After the current data block is written and the previous data block is read, the computing node that writes the data block switches to the second storage partition to write the subsequent data block, and the computing node that reads the data block switches to the first storage partition to read the current data block.

[0069] It should be noted that the preceding data block is the data block before the current data block, and the subsequent data block is the data block after the current data block; the first storage partition and the second storage partition adopt a ping-pong working mode, at a certain moment, the upstream computing node writes the current data block to the first storage partition, and the downstream computing node reads the previous data block from the second storage partition; at the next moment, the upstream computing node writes the next data block to the second storage partition, and the downstream computing node reads the current data block from the first storage partition.

[0070] In this embodiment, the first and second storage partitions adopt a ping-pong working mode, where the process of the upstream computing node writing the current data block overlaps with the process of the downstream computing node reading the previous data block. This reduces the waiting time between upstream and downstream computing nodes, improves pipeline continuity, reduces pipeline bubbles, and improves the utilization rate of computing nodes.

[0071] The following section explains the input / intermediate buffer and output / intermediate buffer for each computing node.

[0072] Reference Figure 2 Each computing node is connected to both an input / intermediate buffer and an output / intermediate buffer. It should be noted that the input / intermediate buffer and the output / intermediate buffer are two sets of shared buffers on either side of the computing node. For a single computing node, the shared buffer located on the input side is the input / intermediate buffer, and the shared buffer located on the output side is the output / intermediate buffer.

[0073] The input / intermediate buffer stores the input data of the computing node and the intermediate results stored back by the computing node; the output / intermediate buffer stores the computation results output by the computing node and the intermediate results stored back by the computing node; the input data is the input feature map or the intermediate result of the previous computing node.

[0074] It is understood that the intermediate result stored back is the intermediate result generated by the computing node and returned to the computing node for calculation; the calculation result output by the computing node is output to the next computing node or the global control node.

[0075] For example, the input / intermediate buffer is used to store the input data for inference, which is an input feature map or the result of an intermediate layer of a neural network calculated by the previous computing node. The input / intermediate buffer is also used to store the intermediate results of the current computing node. The output / intermediate buffer is used to store the calculation results output by the current computing node and also to store the intermediate results of the current computing node.

[0076] This embodiment uses the input / intermediate buffer and output / intermediate buffer of each computing node to perform multiple storage functions. The same set of shared buffers is reused among storing input data, intermediate results and output results, avoiding the redundant handling caused by configuring completely private input and output memory for each computing node.

[0077] The execution mode of the global control node controlling the multiple computing nodes is described below.

[0078] The global control node controls the multiple computing nodes to work in parallel or pipelined modes.

[0079] Reference Figure 3 In the parallel mode, the same set of shared buffers serves as local extended buffers for two adjacent computing nodes, and data blocks are not transferred between adjacent computing nodes.

[0080] Specifically, taking two computing nodes as an example, the global control node divides the input image into two feature map blocks and writes them into the input buffers corresponding to the two computing nodes respectively; each computing node independently completes the calculation according to its local weights and task description, and the calculation results are written into its respective allocated output buffer; at this time, the shared circular buffer memory does not serve as an upstream and downstream transmission path, and the first storage partition and the second storage partition of the same set of shared buffers serve as the input and intermediate buffers or the output and intermediate buffers of two adjacent computing nodes respectively; the global control node reads the calculation results from the two output buffers and splices or summarizes them.

[0081] It is understood that the parallel mode is applicable to scenarios involving output channel partitioning, spatial partitioning, or batch data parallel processing.

[0082] Reference Figure 4 In the pipelined mode, the computing node that writes data blocks writes the calculation results to a shared buffer, and the computing node that reads data blocks reads the calculation results from the shared buffer as input data.

[0083] Specifically, taking two computing nodes as an example, the global control node writes the input feature map into the in-store computing node 1; the in-store computing node 1 performs the first stage of computation and writes the intermediate results into the shared buffer between the in-store computing node 1 and the in-store computing node 2; the in-store computing node 2 reads the intermediate results from the shared buffer, performs the second stage of computation, and writes the final result into the output buffer; the global control node reads the final result from the output buffer of the in-store computing node 2.

[0084] It is understandable that in the pipelined mode, the global control node does not need to read the intermediate results of the in-memory computing node 1 and then write them to the in-memory computing node 2, thus reducing the number of global bus transmissions.

[0085] In this embodiment, the global control node controls multiple computing nodes to work in parallel or pipelining mode, and the same hardware architecture can be adapted to different network structures and different inference tasks. In pipelining mode, the data output by the front-end computing node directly enters the input buffer of the back-end computing node, reducing memory access power consumption and transmission latency.

[0086] The grouping flow pattern is described below.

[0087] Reference Figure 5 The global control node further divides the multiple computing nodes into multiple computing node groups and distributes different data blocks to different computing node groups; adjacent computing nodes within the same computing node group pass data blocks through a shared buffer; the global control node isolates the shared buffer between different computing node groups, and multiple computing node groups process their respective data blocks in parallel.

[0088] Specifically, taking four computing nodes as an example, the global control node divides in-memory computing node 1 and in-memory computing node 2 into a first computing node group, and in-memory computing node 3 and in-memory computing node 4 into a second computing node group; the global control node inputs different data blocks to in-memory computing node 1 and in-memory computing node 3 respectively; in-memory computing node 1 and in-memory computing node 2 form a two-stage pipeline, and in-memory computing node 3 and in-memory computing node 4 form another two-stage pipeline, with the two pipelines working in parallel.

[0089] It should be noted that the isolation of shared buffers between different computing node groups refers to configuring the boundaries of shared buffers between different computing node groups to be in an isolated state, so as to avoid data crosstalk between groups.

[0090] It is understood that the grouped pipeline mode is suitable for neural network tasks that require multi-stage processing and have multiple independent data blocks; compared with a single long pipeline, the grouped pipeline mode reduces the impact of pipeline stage imbalance; compared with the fully parallel mode, the grouped pipeline mode reduces the global transport of intermediate results within a group.

[0091] Furthermore, the global control node also controls the multiple computing nodes to work in a hybrid mode, which is a combination of two or more of the following: parallel mode, pipelined mode, and grouped pipelined mode.

[0092] For example, in the hybrid mode, some computing nodes transmit data blocks in a pipelined mode, while other computing nodes process independent data blocks in a parallel mode. It is understood that the global control node, by configuring the read / write direction of the shared buffer, partition roles, and upstream / downstream relationships of computing nodes, implements the data flow organization of the parallel mode, the pipelined mode, the grouped pipelined mode, and the hybrid mode, thereby improving the hardware resource reuse rate.

[0093] Furthermore, the computing node also includes a shadow task register group, which is a register group that temporarily stores the task description of the next task. The capacity of the shadow task register group is determined according to the task description required to store one task. The task description of the next task stored in the shadow task register group includes operator type, data address, block size, weight index, execution mode, and upstream and downstream relationships.

[0094] During the execution of the current task by the computing node, the global control node writes the task description of the next task into the shadow task register group; when the current task ends and there is a preloaded task description in the shadow task register group, the local controller reads the task description of the next task from the shadow task register group, switches to the next task and continues to execute.

[0095] It should be noted that the shadow task register group is used for task preloading, so that the computing node automatically switches to the next task after the current task ends, without waiting for the global control node to reissue the task description; when the current task ends and there is no preloaded task description in the shadow task register group, the computing node sends a completion status to the global control node and waits for the global control node to issue the task description for the next task.

[0096] In this embodiment, multiple computing nodes are divided into multiple computing node groups by the global control node and the shared buffer between groups is isolated. Multiple computing node groups process their respective data blocks in parallel, and the load organization takes into account both multi-stage processing and parallel processing of multiple data blocks. The next task is preloaded by the shadow task register group, which reduces the waiting time of computing nodes when switching tasks.

[0097] The following example, using four computing nodes executing a two-level grouped pipeline task, illustrates the working process of the system in this embodiment.

[0098] During initialization, the global control node reads the program generated by the compiler through the program loading interface. The program includes computing node allocation information, weight index, block size, and execution mode. Weight data is pre-deployed in the storage unit or local memory of each computing node.

[0099] In the task configuration, the global control node configures in-memory computing node 1 and in-memory computing node 2 as the first computing node group, and in-memory computing node 3 and in-memory computing node 4 as the second computing node group, and configures the read / write direction and storage partition status of the corresponding shared buffer.

[0100] Input data distribution: The global control node receives the input feature map from the external data interface, writes the first data block into the input buffer of the in-memory computing node 1, and writes the second data block into the input buffer of the in-memory computing node 3.

[0101] After receiving the task description, the in-memory computing node 1 and the in-memory computing node 3 respectively generate local microinstruction sequences by the program parsing module. The local controller then controls the in-memory computing unit, the input data processing unit, and the output data processing unit to perform the first stage of computation.

[0102] Intermediate result transmission: In-store computing node 1 writes the intermediate result of the first data block into the shared buffer between itself and in-store computing node 2; in-store computing node 3 writes the intermediate result of the second data block into the shared buffer between itself and in-store computing node 4; and sets the valid flag of the shared buffer.

[0103] Downstream computation begins. After detecting that the valid flag of the corresponding shared buffer is set, in-store computing node 2 and in-store computing node 4 read the intermediate results and start the second stage of computation.

[0104] Ping-pong buffer switching: when the in-memory computing node 1 writes to the first storage partition, the in-memory computing node 2 reads the previous data block from the second storage partition; when the next data block arrives, the reading and writing storage partitions are swapped; the same applies between in-memory computing node 3 and in-memory computing node 4.

[0105] The output results are read, and the storage-core computing nodes 2 and 4 write the final results into the output buffer and send the completion status to the global control node. The global control node then reads the output results.

[0106] The task switching is automatic. During the above calculation process, the global control node has written the task description of the next batch of tasks into the shadow task register group of the computing node. After the current task ends, the computing node automatically switches and continues to execute the next task.

[0107] Example 2, refer to Figures 1 to 5 As one embodiment of the present invention, this embodiment provides a control method for an in-memory computing accelerator that supports multi-compute node pipelined collaboration, comprising: S1, the global control node sends task descriptions to multiple computing nodes that are connected in a ring structure, and writes the input feature map into a shared buffer set between two adjacent computing nodes.

[0108] It should be noted that the task description is a task description program issued by the global control node at a higher level than the micro-operation abstraction level. The task description includes operator type, data address, block size, weight index, execution mode, and upstream and downstream relationships.

[0109] Specifically, in S1, the global control node issues task descriptions to multiple computing nodes. This includes the global control node reading the program and task description generated by the compiler from the program loading interface, determining the computing blocks, computing order, and dependencies for each computing node based on the program, and writing the task descriptions to the corresponding computing nodes.

[0110] Furthermore, in S1, the global control node writes the input feature map into the shared buffer, which includes the global control node receiving the input feature map from an external data interface and writing the input feature map into the shared buffer as an input buffer.

[0111] S2, the local controller of each computing node generates a control sequence according to the task description. The control sequence controls the in-memory computing unit of the computing node to perform in-memory computing. The local controller controls the data transfer between the in-memory computing unit and the shared buffer according to the state of the shared buffer.

[0112] Furthermore, in S2, the local controller generates a control timing sequence based on the task description, including the program parsing module of the computing node converting the task description into a locally executable micro-operation control sequence. The micro-operation control sequence is a micro-instruction or a custom register control. The control timing sequence controls the in-memory computing unit to complete input loading, computation initiation, computation result reading, and result writing back.

[0113] It is understood that the global control node does not send cycle-by-cycle micro-control signals to the in-memory computing unit, and the cycle-by-cycle operation timing of the in-memory computing unit is generated by the local controller inside the computing node.

[0114] Furthermore, in S2, the local controller controls data transfer based on the state of the shared buffer, including determining whether to wait for upstream data or downstream buffer release based on the buffer state of the shared buffer; when the data in the shared buffer is valid, the local controller controls the storage and processing unit to read data from the shared buffer; when the shared buffer is idle, the local controller controls the storage and processing unit to write data to the shared buffer.

[0115] S3, two adjacent computing nodes pass data blocks through the shared buffer.

[0116] It should be noted that, for a set of shared buffers, the computing node that writes data to the shared buffer is the upstream computing node, and the computing node that reads data from the shared buffer is the downstream computing node; in S3, two adjacent computing nodes pass data blocks through the shared buffer, meaning that the upstream computing node writes the data block to the shared buffer, and the downstream computing node reads the data block from the shared buffer.

[0117] Understandably, in pipelined mode, the output of the upstream computing node is directly used as the input of the downstream computing node through the shared buffer. The global control node does not need to read the intermediate results of the upstream computing node and write them to the downstream computing node, thus reducing the number of global bus transmissions.

[0118] S4, the global control node receives the completion status sent by the multiple computing nodes and reads the computing results according to the completion status.

[0119] Furthermore, in S4, the global control node reads the calculation results based on the completion status, including updating the global task status based on the completion status fed back by the computing node, and reading the calculation results from the final output computing node or the output buffer of each computing node.

[0120] In this embodiment, the global control node issues task descriptions to the multiple computing nodes, and the local controller generates control timing sequences within the computing nodes based on the task descriptions. The global control node does not issue cycle-by-cycle micro-control signals, so that the on-chip bus is not occupied by a large number of fine-grained control commands, thereby reducing the global control communication overhead and software scheduling complexity.

[0121] S5, each shared buffer includes a first storage partition and a second storage partition, the first storage partition and the second storage partition are respectively set with a writable status flag and a readable status flag; the computing node that writes data blocks writes data blocks to the storage partition when the writable status flag of the storage partition is set, and sets the readable status flag of the storage partition after writing is completed; the computing node that reads data blocks reads data blocks when the readable status flag of the storage partition is set, and sets the writable status flag of the storage partition after reading is completed.

[0122] It should be noted that the first storage partition and the second storage partition have independent read and write status flags. The status flags include at least: an idle flag, indicating that the storage partition can be written to; a valid flag, indicating that the data stored in the storage partition can be read; a read complete flag, indicating that the downstream computing node has completed reading; a write complete flag, indicating that the upstream computing node has completed writing; and the task number, used to distinguish different tasks or different data blocks.

[0123] It is understood that the writable status flag corresponds to the idle flag, and the readable status flag corresponds to the valid flag.

[0124] Specifically, in S5, after the computing node that writes the data block completes the write operation, it also sets the write completion flag of the storage partition; after the computing node that reads the data block completes the read operation, it also sets the read completion flag of the storage partition.

[0125] This embodiment controls the reading and writing of data blocks by setting status flags between the computing nodes that write data blocks and the computing nodes that read data blocks, thereby coordinating the writing of upstream computing nodes with the reading of downstream computing nodes and avoiding arbitration conflicts when concurrently accessing the unified on-chip memory.

[0126] S6, the computing node writing the data block writes the current data block to the first storage partition, and the computing node reading the data block reads the prior data block from the second storage partition; after the current data block is written and the prior data block is read, the computing node writing the data block switches to the second storage partition to write the subsequent data block, and the computing node reading the data block switches to the first storage partition to read the current data block.

[0127] It should be noted that, referring to Figure 4The preceding data block is the data block preceding the current data block, and the subsequent data block is the data block following the current data block. The first storage partition and the second storage partition adopt a ping-pong working mode. At a certain moment, the upstream computing node writes the current data block to the first storage partition, and the downstream computing node reads the previous data block from the second storage partition. At the next moment, the upstream computing node writes the next data block to the second storage partition, and the downstream computing node reads the current data block from the first storage partition.

[0128] In this embodiment, the first and second storage partitions adopt a ping-pong working mode, where the process of the upstream computing node writing the current data block overlaps with the process of the downstream computing node reading the previous data block. This reduces the waiting time between upstream and downstream computing nodes, improves pipeline continuity, reduces pipeline bubbles, and improves the utilization rate of computing nodes.

[0129] S7, the global control node divides the multiple computing nodes into multiple computing node groups, isolates the shared buffer between different computing node groups, and distributes different data blocks to the first-level computing node in each computing node group; the computing nodes in each computing node group pass the data blocks level by level through the shared buffer; the last-level computing node in each computing node group writes the processing result into the shared buffer and sends the completion status to the global control node.

[0130] It should be noted that, referring to Figure 5 The isolation of shared buffers between different computing node groups refers to configuring the boundaries of shared buffers between different computing node groups to be in an isolated state to avoid data crosstalk between groups; the first-level computing node is the computing node in the computing node group that receives data blocks distributed by the global control node, and the last-level computing node is the computing node in the computing node group that outputs the final processing result.

[0131] The following example, using four computing nodes executing a two-level grouped pipelined task, illustrates the working process of the control method in this embodiment.

[0132] During initialization, the global control node reads the program generated by the compiler through the program loading interface. The program includes computing node allocation information, weight index, block size, and execution mode. Weight data is pre-deployed in the storage unit or local memory of each computing node.

[0133] In the task configuration, the global control node divides the in-memory computing node 1 and the in-memory computing node 2 into a first computing node group, and divides the in-memory computing node 3 and the in-memory computing node 4 into a second computing node group. It configures the read / write direction and storage partition status of the corresponding shared buffer, and isolates the shared buffer between the first computing node group and the second computing node group.

[0134] Input data distribution: The global control node receives the input feature map from the external data interface, writes the first data block into the input buffer of the in-memory computing node 1, and writes the second data block into the input buffer of the in-memory computing node 3.

[0135] After receiving the task description, the in-memory computing node 1 and the in-memory computing node 3 generate local microinstruction sequences by the program parsing module. The local controller then controls the in-memory computing unit, the input data processing unit, and the output data processing unit to perform the first stage of computation.

[0136] Intermediate result transmission: In-store computing node 1 writes the intermediate result of the first data block into the shared buffer between itself and in-store computing node 2; in-store computing node 3 writes the intermediate result of the second data block into the shared buffer between itself and in-store computing node 4; and sets the valid flag of the shared buffer.

[0137] Downstream computation begins. After detecting that the valid flag of the corresponding shared buffer is set, in-store computing node 2 and in-store computing node 4 read the intermediate results and start the second stage of computation.

[0138] Ping-pong buffer switching: when the in-memory computing node 1 writes to the first storage partition, the in-memory computing node 2 reads the previous data block from the second storage partition; when the next data block arrives, the reading and writing storage partitions are swapped; the same applies between in-memory computing node 3 and in-memory computing node 4.

[0139] The output results are read. As the last-level computing nodes, the storage-core computing node 2 and the storage-core computing node 4 write the final processing results into the output buffer and send the completion status to the global control node. The global control node then reads the output results.

[0140] The task switching is automatic. During the above calculation process, the global control node has written the task description of the next batch of tasks into the shadow task register group of the computing node. After the current task ends, the computing node automatically switches and continues to execute the next task.

[0141] This embodiment divides the multiple computing nodes into multiple computing node groups through the global control node, isolates shared buffers between groups, and distributes different data blocks to each computing node group. Multiple computing node groups process their respective data blocks in parallel, balancing multi-stage processing and multi-data block parallelism in terms of load organization. When neural network tasks require both multi-stage processing and multiple independent data blocks, it reduces the impact of pipeline stage imbalance compared to a single long pipeline, and reduces the global transport of intermediate results within a group compared to a fully parallel mode.

[0142] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A memory computing accelerator device based on shared ring buffer and multi-node collaborative control, characterized in that, include: It features a shared ring buffer, multiple compute nodes, and a global control node. The multiple computing nodes are connected in sequence to form a ring structure, and each computing node includes a local controller and a memory computing unit. The shared ring buffer memory includes multiple sets of shared buffers. Each set of shared buffers is set between two adjacent computing nodes. The global control node uses each set of shared buffers as an input buffer, intermediate buffer, output buffer, or inter-node transfer buffer. The local controller is used to generate the control timing for the in-memory computing unit to perform in-memory computing based on the task description, and to control the data transfer between the in-memory computing unit and the shared buffer based on the state of the shared buffer. The global control node is used to send the task description to the plurality of computing nodes, write the input feature map into the shared ring buffer, receive the completion status sent by the plurality of computing nodes, and read the calculation result from the shared ring buffer according to the completion status.

2. The in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control as described in claim 1, characterized in that, Each shared buffer includes a first storage partition and a second storage partition. The first storage partition and the second storage partition are respectively set with a writable status flag, a readable status flag, and a task number. The task number identifies the task to which the data block in the storage partition belongs. The compute node that writes the data block writes the data block to the storage partition when the writable status flag of the storage partition is set, and sets the readable status flag of the storage partition after the writing is completed. The compute node that reads the data block reads the data block when the readable status flag of the storage partition is set, and sets the writable status flag of the storage partition after the reading is completed.

3. The in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control as described in claim 2, characterized in that, The compute node that writes the data block writes the current data block to the first storage partition, and the compute node that reads the data block reads the previous data block from the second storage partition; After the current data block is written and the previous data block is read, the computing node that wrote the data block switches to the second storage partition to write the subsequent data block, and the computing node that reads the data block switches to the first storage partition to read the current data block.

4. The in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control as described in claim 3, characterized in that, Each computing node connects to both the input / intermediate buffer and the output / intermediate buffer. The input and intermediate buffer stores the input data input to the computing node, as well as the intermediate results stored back by the computing node; The output, which also serves as an intermediate buffer, stores the computation results output by the computing node, as well as the intermediate results stored back by the computing node. The input data is either the input feature map or the intermediate result of the previous calculation node.

5. The in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control as described in claim 4, characterized in that, The global control node controls the multiple computing nodes to work in parallel or pipelined mode. In the parallel mode, the same set of shared buffers serves as local extended buffers for two adjacent computing nodes, and data blocks are not transferred between adjacent computing nodes. In the pipelined mode, the computing node that writes data blocks writes the calculation results to a shared buffer, and the computing node that reads data blocks reads the calculation results from the shared buffer as input data.

6. The in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control as described in claim 5, characterized in that, The global control node also divides the multiple computing nodes into multiple computing node groups and distributes different data blocks to different computing node groups. Adjacent compute nodes within the same compute node group transfer data blocks via a shared buffer; The global control node isolates the shared buffer between different computing node groups, and multiple computing node groups process their respective data blocks in parallel.

7. A control method for an in-memory computing accelerator supporting multi-node pipelined collaboration, employing an in-memory computing accelerator device based on shared ring buffer and multi-node collaborative control as described in any one of claims 1 to 6, characterized in that, include: The global control node sends task descriptions to multiple computing nodes that are connected in a ring structure, and writes the input feature map into a shared buffer set between two adjacent computing nodes. Each computing node's local controller generates a control sequence based on the task description. The control sequence controls the in-memory computing unit of the computing node to perform in-memory computing. The local controller controls the data transfer between the in-memory computing unit and the shared buffer based on the state of the shared buffer. Two adjacent computing nodes pass data blocks through the shared buffer; The global control node receives the completion status sent by the multiple computing nodes and reads the computing results based on the completion status.

8. The in-memory computing accelerator control method supporting multi-computing node pipelined collaboration as described in claim 7, characterized in that, Each shared buffer includes a first storage partition and a second storage partition, with the first storage partition and the second storage partition respectively set to a writable status flag and a readable status flag; The compute node that writes the data block writes the data block to the storage partition when the writable status flag of the storage partition is set, and sets the readable status flag of the storage partition after the writing is completed. The compute node that reads the data block reads the data block when the readable status flag of the storage partition is set, and sets the writable status flag of the storage partition after the reading is completed.

9. A control method for an in-memory computing accelerator supporting multi-node pipelined collaboration as described in claim 8, characterized in that, The compute node that writes the data block writes the current data block to the first storage partition, and the compute node that reads the data block reads the previous data block from the second storage partition; After the current data block is written and the previous data block is read, the computing node that wrote the data block switches to the second storage partition to write the subsequent data block, and the computing node that reads the data block switches to the first storage partition to read the current data block.

10. A control method for an in-memory computing accelerator supporting multi-node pipelined collaboration as described in claim 9, characterized in that, The global control node divides the multiple computing nodes into multiple computing node groups, isolates the shared buffer between different computing node groups, and distributes different data blocks to the first-level computing node in each computing node group. In each compute node group, compute nodes pass data blocks cascading through a shared buffer. The last-level computing node in each computing node group writes the processing result to the shared buffer and sends the completion status to the global control node.