Topology of an accelerator

Abstract

The application provides a topology of an accelerator, comprising a plurality of accelerators and a broadcast buffer, each of the plurality of accelerators corresponding to a first memory and respectively acquiring input data from an external second memory, wherein the accelerator can only directly access the first memory corresponding thereto, and the broadcast buffer is coupled between one of the plurality of accelerators and the corresponding first memory. The broadcast buffer is used to write the input data into the corresponding first memory according to a write command from the coupled accelerator when the write command and the input data are received, and broadcast the weight data in the write command and the input data when the broadcast is enabled. The application can improve the access performance of the accelerator and reduce the access delay.

Description

Technical Field

[0001] This application relates to the field of integrated circuits, and more particularly to a topology device for an accelerator. Background Technology

[0002] The Artificial Intelligence Core (AI core) is a hardware accelerator that speeds up deep learning inference algorithms. This accelerator can be configured and scheduled by an external CPU to perform operations on various layers of operators in convolutional neural networks, such as convolutional layers, deconvolutional layers, fully connected layers, activation layers, pooling layers, and local response normalization.

[0003] The input data for the AI ​​core includes feature data and weight data, which are used to generate output data. To improve access performance, a relatively large static random access memory (SRAM) (e.g., 2M to 4M bytes) is typically placed outside the AI ​​core. This SRAM stores the following data: the input data for the first layer (including feature data and weight data), and the data generated from the calculations of the first to the last layer. The input data for the first layer needs to be transferred from external memory (such as DDR-RAM) to the AI ​​core, and the data generated from the calculations of the last layer needs to be transferred from the SRAM to external memory.

[0004] A single chip can contain multiple AI cores to support applications with varying performance, power consumption, and cost requirements. The expansion of multiple AI cores allows for different topologies to be chosen, with common topologies including mesh structures and crossbar interconnect structures.

[0005] However, using a Crossbar fully interconnected structure or a Mesh structure has the following drawbacks:

[0006] (1) When multiple (e.g. n) AI cores access the same address range of data in SRAM in parallel, multiple access requests need to be arbitrated. At this time, only one AI core arbitrates to the SRAM interface, which results in the bandwidth being reduced to 1 / n.

[0007] (2) When the bus width is large (e.g., 1024 bits), the Crossbar or Mesh structure is prone to timing and routing congestion problems in the back end.

[0008] (3) Due to limitations in performance and backend implementation, the scalability of AI cores is poor;

[0009] (4) The uneven running time of multiple AI cores is detrimental to the performance and power consumption of the entire AI system.

[0010] Therefore, it is necessary to propose a topology for the accelerator that can improve scalability and access performance. Summary of the Invention

[0011] The purpose of this application is to provide a topology for a hardware accelerator to improve the access performance of the accelerator and reduce access latency.

[0012] One embodiment of this application discloses a topology arrangement for a hardware accelerator, comprising:

[0013] Multiple accelerators, each corresponding to a first memory and respectively acquiring input data from an external second memory, wherein each accelerator can only directly access its own corresponding first memory; and

[0014] A broadcast buffer, coupled between one of the plurality of accelerators and a corresponding first memory, is used to write the input data to the corresponding first memory according to the write command when a write command and input data are received from the coupled accelerator, and to broadcast the write command and weight data in the input data when broadcasting is enabled.

[0015] In a preferred embodiment, the topology further includes:

[0016] At least one broadcast arbitrator is coupled between each of the other accelerators in the plurality of accelerators and a corresponding first memory. The broadcast arbitrator is configured to write the input data to the corresponding first memory when it receives a write command and input data from the accelerator it is coupled to, and when broadcast reception is enabled, also receive a broadcast write command and weight data. Furthermore, when it simultaneously receives a write command and input data from the accelerator and a broadcast write command and weight data, it arbitrates the priority of the write command and input data from the accelerator and the broadcast write command and weight data, and performs a write operation based on the arbitration result.

[0017] In a preferred embodiment, the priority of the broadcast write commands and weighted data is higher than the priority of the write commands and input data from the accelerator.

[0018] In a preferred embodiment, the broadcast buffer further includes:

[0019] A command buffer is configured to receive write commands from the coupled accelerator when broadcasting is enabled, and to broadcast the write commands to instruct the broadcast weight data to be written to the same offset address of another first memory; and

[0020] A data buffer is configured to receive weight data from the coupled accelerator and broadcast the weight data when broadcasting is enabled.

[0021] In a preferred embodiment, the broadcast buffer further includes:

[0022] The first identifier is configured to obtain identifier information from the received write command of the accelerator, determine whether the identifier information is target identifier information, and notify the data buffer to receive the weight data when the identifier information is determined to be target identifier information.

[0023] In a preferred embodiment, the broadcast buffer further includes:

[0024] A synchronizer is configured to receive and synchronize write-complete response signals from the broadcast arbitrator, and output synchronized write-complete response signals; and

[0025] The response arbitrator is configured to receive a write-complete response signal from the first memory corresponding to the broadcast buffer and a synchronized write-complete response signal from the synchronizer, arbitrate the priority of the received write-complete response signals, and output a write-complete response signal according to the arbitration result.

[0026] In a preferred embodiment, the broadcast arbitrator includes:

[0027] A write command selector is configured to receive write commands from a coupled accelerator, output the write commands to a corresponding first memory, and also receive broadcast write commands when broadcasting is enabled. When both write commands from the accelerator and broadcast write commands are received simultaneously, the selector arbitrates the priorities of the broadcast write commands and the accelerator write commands, and outputs the result based on the arbitration.

[0028] A write data selector is configured to receive input data from the coupled accelerator, write the input data to a corresponding first memory, and also receive broadcast weight data when broadcast reception is enabled. When both broadcast weight data and accelerator input data are received simultaneously, the selector arbitrates the priority of the broadcast weight data and the accelerator input data and outputs the result based on the arbitration.

[0029] In a preferred embodiment, the broadcast arbitrator further includes:

[0030] The second identifier is configured to obtain identifier information from the write command from the accelerator, determine whether the write operation of the input data corresponding to the write command of the accelerator is completed based on the identifier information, and when the write operation is determined to be completed, notify the write data selector to write the broadcast weight data into the corresponding first memory.

[0031] In a preferred embodiment, the broadcast arbitrator further includes:

[0032] The response selector is configured to receive a write-complete response signal from the corresponding first memory and, depending on the data source corresponding to the write-complete response signal, selectively send the write-complete response signal to the broadcast buffer or the coupled accelerator.

[0033] In a preferred embodiment, the accelerator includes a direct memory accessor;

[0034] The direct memory accessor is configured to retrieve input data from an external second memory and write the input data to the corresponding first memory via the broadcast buffer.

[0035] Compared to traditional technical solutions, in this application's implementation, each accelerator has its own SRAM. In scenarios where multiple accelerators access the same address range of data in parallel, each accelerator accesses its own SRAM interface in parallel without arbitration, thus preventing bandwidth reduction. Even with a large bus bandwidth (e.g., 1024 bits), it will not cause backend timing congestion. There are no limitations on the number of accelerator cores as long as the chip's DDR bandwidth and power consumption / area specifications are met.

[0036] Furthermore, it can increase the access bandwidth of accelerators, reduce access latency, and reduce the synchronization and waiting time of each accelerator, thereby achieving a balance in the synchronous running time of multiple accelerators, which is beneficial to the performance and power consumption of the entire deep learning system.

[0037] In addition, the accelerator has two modes: batch processing and non-batch processing. In batch processing mode, the input weight data can be reused. The input weight data only needs to be moved from DDR to the corresponding SRAM by one hardware accelerator, and at the same time, the input weight data is broadcast to the corresponding SRAM of other hardware accelerators. This can save the time and power consumption caused by moving data from DDR to local SRAM. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of a point-to-point topology structure for multiple accelerators in one embodiment of this application.

[0039] Figure 2 This is a hardware block diagram of an accelerator broadcast mechanism in one embodiment of this application.

[0040] Figure 3 This is a schematic diagram of a broadcast buffer in one embodiment of this application.

[0041] Figure 4 This is a schematic diagram of a broadcast arbitrator in one embodiment of this application. Detailed Implementation

[0042] In the following description, many technical details are presented to help the reader better understand this application. However, those skilled in the art will understand that the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0044] One embodiment of this application discloses an accelerator topology, which is a point-to-point structure comprising multiple accelerators, each accelerator (AI core) corresponding to a first memory (SRAM), and each accelerator can only directly access its own corresponding first memory. Therefore, there is no scenario where multiple accelerators access the same address range of data in the SRAM in parallel, thus improving the accelerator's access bandwidth.

[0045] Figure 1 The diagram illustrates a point-to-point topology of four accelerators according to an embodiment of this application. However, this is not a limitation. The number of accelerators is not limited, provided that the chip's DDR bandwidth and power consumption area specifications are met. For example, the number of accelerators can be expanded to 8, 16, 32, etc. (See reference) Figure 1 As shown, this topology includes four accelerators, AI core0 to AI core3. Each accelerator corresponds to a first memory (i.e., AI core0 to AI core3 correspond to SRAM 0 to SRAM 3, respectively), and each accelerator can only access its own corresponding first memory. The four accelerators access an external second memory (DDR) (not shown in the figure) through an arbitrator. Each accelerator includes a Direct Memory Access (DMA), a first interface SRAMIF, and a second interface DDRIF. The DMA accesses the corresponding SRAM through the first interface and the external DDR through the second interface. The DMA can move input data (including feature data, weight data, etc.) from the external DDR to the corresponding SRAM and move output data (data calculated based on the input data) from the corresponding SRAM to the external DDR.

[0046] In one embodiment, accelerator computation exists in two modes: batch mode and non-batch mode. Batch mode corresponds to a scenario where the weight data input to the accelerator is reused by the feature data input to multiple accelerators; that is, the feature data input to multiple (at least two) accelerators corresponds to the same weight data. Non-batch mode corresponds to a scenario where the weight data input to the accelerator is not reused by the feature data input to multiple accelerators; that is, the feature data input to each accelerator corresponds to different weight data.

[0047] Clearly, in non-batch processing mode, the DMA within each accelerator only needs to move input and output data between the external DDR and the corresponding SRAM. In batch processing mode, since multiple accelerators need to input the same weight data, one accelerator can obtain the weight data from the external second memory DDR. During the process of writing the weight data into the corresponding first memory SRAM, the weight data can be broadcast so that other accelerators in need can obtain it.

[0048] Specifically, the accelerator topology also includes a broadcast buffer. The broadcast buffer is coupled between one of the plurality of accelerators and its corresponding first memory, and is used to write the input data into the corresponding first memory according to the write command when a write command and input data are received from the coupled accelerator, and to broadcast the write command and the weight data in the input data when broadcasting is enabled.

[0049] In one embodiment, the accelerator topology may further include at least one broadcast arbitrator. This at least one broadcast arbitrator is coupled between each of the other accelerators in the plurality of accelerators and its corresponding first memory. Upon receiving a write command and input data from its coupled accelerator, it writes the input data into the corresponding first memory. When broadcast reception is enabled, it also receives broadcast write commands and weight data. Furthermore, when simultaneously receiving write commands and input data from the accelerator and broadcast write commands and weight data, it arbitrates the priority of the write commands and input data from the accelerator and the broadcast write commands and weight data, and performs the write operation based on the arbitration result. In one embodiment, the priority of the broadcast write commands and weight data may be set higher than the priority of the write commands and input data from the accelerator. Thus, when the broadcast arbitrator simultaneously receives write commands and input data from the accelerator and broadcast write commands and weight data, it will prioritize writing the weight data into the first memory according to the broadcast write command, and then write the input data into the first memory according to the write command from the accelerator.

[0050] like Figure 2 As shown, taking the accelerator AI core 0 coupled to the broadcast buffer as an example, the broadcast buffer 501 is coupled between the accelerator AI core 0 and the corresponding SRAM 0. The accelerator AI core 0 realizes the write operation with the first memory SRAM 0 through the broadcast buffer 501. There is a write channel (AW / W / B) between the accelerator AI core 0 and the broadcast buffer 501. There is a write command channel (AW), a write data channel (W) and a write return channel (B) between the broadcast buffer 501 and the SRAM 0. There is also a read channel (AR / R) between the accelerator AI core 0 and the first memory SRAM 0. Each broadcast arbiter 502 is coupled between AI core 1-AI core 3 and its corresponding SRAM 1-SRAM 3. Taking AI core 1 as an example, accelerator AI core 1 performs write operations with SRAM 1 through broadcast arbiter 502. There is a write channel (AW / W / B) between accelerator AI core 1 and broadcast arbiter 502, a write command channel (AW), a write data channel (W), and a write return channel (B) between broadcast arbiter 502 and SRAM 1, and a read channel (AR / R) between accelerator AI core 1 and SRAM 1. Broadcast buffer 501 has broadcast write channels (AW / W / B bc1-AW / W / B bc3) between it and each broadcast arbiter 502.

[0051] In one embodiment, the broadcast function of the broadcast buffer 501 is controlled by the broadcast enable signal Broadcast. For example, when the broadcast enable signal Broadcast is high, the broadcast function is enabled, and the broadcast buffer can broadcast to the outside world; when the broadcast enable signal Broadcast is low, the broadcast function is disabled, and the broadcast buffer cannot broadcast to the outside world, and vice versa.

[0052] In one embodiment, the broadcast arbitrator 502's broadcast reception function is controlled by the arbitration enable signal Grp_en. For example, when the arbitration enable signal Grp_en is high, the broadcast arbitrator 502's broadcast reception function is enabled and it receives broadcasts from the broadcast buffer 501; when the arbitration enable signal Grp_en is low, the broadcast arbitrator 502's broadcast reception function is disabled and it cannot receive broadcasts from the broadcast buffer 501, and vice versa.

[0053] refer to Figure 2As shown, accelerator AI cores 0-3 correspond to first memory SRAM 0-first memory 3, respectively. Each SRAM can only be accessed by its corresponding accelerator AI core, so there is no performance bandwidth loss during the read process. When the broadcast buffer 501 receives a write command and input data from accelerator AI core 0, it writes the input data into the corresponding first memory SRAM 0 according to the write command. At the same time, when the broadcast function is enabled (e.g., Broadcast is high), it broadcasts the write command and the weight data in the input data. When each broadcast arbitrator 502 receives a write command and input data from the coupled accelerator, it writes the input data into the corresponding first memory according to the write command. When broadcast reception is enabled, it receives the write command and weight data broadcast by the broadcast buffer 501. When simultaneously receiving the accelerator's write command, input data, and the broadcast's write command and weight data, it arbitrates the priority of the accelerator's write command, input data, and the broadcast's write command and weight data, and performs the write operation according to the arbitration result.

[0054] It should be noted that the convolutional neural network consists of multiple layers. The input data of the first layer (including feature data and weight data, etc.) is transferred from the external second memory (DDR) to the corresponding first memory (SRAM) by the accelerator AI core. The input data of each subsequent layer is the output data of the layer above it. Therefore, from the perspective of application scenarios, this broadcast mechanism is only triggered when the weight data input to the first layer needs to be transferred from the external second memory (DDR) to the corresponding first memory (SRAM) by accelerator AI core 0, and this weight data can be used by multiple accelerators simultaneously. At this time, accelerators AI core 1-AI core 3 will not perform write operations on the first memory (SRAM 1-SRAM 3), and the broadcast arbitrator will not receive write requests from accelerators AI core 1-AI core 3, so there is no performance bandwidth loss for write operations. SRAM 0 only receives write requests from accelerator AI core 0, so there is no performance bandwidth loss for the write operation of accelerator AI core 0. In one embodiment, hardware synchronization logic or CPU control can be used to prevent accelerators AI core 1-AI core 3 from performing write operations on the first memory (SRAM 1-SRAM 3) during the broadcast process of broadcast buffer 501. Therefore, the arbitration function of the broadcast arbitrator is not necessary for the embodiments of this application.

[0055] Figure 3 This is a schematic diagram of a broadcast buffer 600 in one embodiment of this application. The broadcast buffer 600 is... Figure 2An example of a broadcast buffer 501. The broadcast buffer 600 includes a command buffer 601 and a data buffer 602. Reference Figure 2 and Figure 3 As shown, the broadcast buffer 600 receives the write command aw_0 and write data Wdata_0 (i.e., input data) generated by accelerator AI core 0. When the broadcast function is disabled, it outputs the write command aw_o and write data Wdata_o to SRAM 0 via the write command channel AW and the write data channel W, thereby writing the input data into SRAM 0. When the broadcast function is enabled (e.g., the broadcast enable signal Broadcast is high), the command buffer 601 receives and buffers the write command from the coupled accelerator AI core 0 and broadcasts it to instruct the weight data to be written to the same offset address of other first memories. At the same time, the data buffer 602 receives and buffers the weight data from AI core 0 and broadcasts it. The command buffer 601 broadcasts to each broadcast arbitrator. For example, the command buffer 601 broadcasts the write command Awbroadcast1 to the broadcast arbitrator corresponding to accelerator AI core 1, broadcasts the write command Awbroadcast2 to the broadcast arbitrator corresponding to accelerator AI core 2, and so on. Data buffer 602 also broadcasts to each broadcast arbitrator. For example, data buffer 602 broadcasts write data Wdata_broadcast1 to the broadcast arbitrator corresponding to accelerator AI core 1, broadcasts write data Wdata_broadcast2 to the broadcast arbitrator corresponding to accelerator AI core 2, and so on.

[0056] In one embodiment, the broadcast buffer 600 further includes a first identifier 605. The first identifier 605 is configured to obtain identifier information from a received write command from the accelerator, determine whether the identifier information is target identifier information, and, if the identifier information is determined to be target identifier information, notify the data buffer to receive the weight data. The target identifier information can be the ID of the Direct Memory Access (DMA) in accelerator AI core 0. Figure 3 As shown, the first identifier 605 obtains the identifier information from the received write command aw_0 and determines whether the identifier information is the ID of the direct memory accessor (DMA) in the corresponding accelerator. If so, the data buffer 602 is allowed to receive and cache the write data Wdata_0.

[0057] In one embodiment, the broadcast buffer 600 further includes a synchronizer 603 and a response arbitrator 604. The synchronizer 603 is configured to receive and synchronize write-complete response signals from each broadcast arbitrator, and output synchronized write-complete response signals to the response arbitrator 604. Specifically, after each broadcast arbitrator writes the broadcast weight data into its corresponding SRAM, it sends a corresponding write-complete response signal to the synchronizer 603 of the broadcast buffer. For example, if each broadcast arbitrator corresponding to accelerator AI core 1-AI core 3 writes the broadcast weight data into its corresponding SRAM 1-SRAM 3, the synchronizer 603 will receive write-complete response signals Bvld_1, Bvld_2, and Bvld_3 from each broadcast arbitrator (e.g., ...). Figure 3 (As shown) and synchronizes these write complete response signals Bvld_1, Bvld_2, and Bvld_3, outputting a synchronized write complete response signal Bvld_syn. The response arbitrator 604 is configured to receive write complete response signals from the first memory corresponding to the broadcast buffer and synchronized write complete response signals from the synchronizer, arbitrate the priority of the received write complete response signals, and output these write complete response signals according to the arbitration result. For example... Figure 3 As shown, after completing the operation on SRAM 0 via the broadcast buffer, SRAM 0 sends a write completion signal Bvld_i to the reply arbitrator 604. The reply arbitrator 604 also receives a synchronized write completion response signal Bvld_syn output by the synchronizer 603. The reply arbitrator 604 arbitrates the priority of the received write completion response signal Bvld_i and the synchronized write completion response signal Bvld_syn, and outputs these write response signals to the corresponding accelerator AI core 0 according to the arbitration result.

[0058] Figure 4 This is a schematic diagram of a broadcast arbitrator 700 in one embodiment of this application. The broadcast arbitrator 700 is... Figure 2An example of a broadcast arbitrator 502. The broadcast arbitrator 700 includes a write command selector 701 and a write data selector 702. The write command selector 701 is configured to output a write command to a corresponding first memory upon receiving a write command from a coupled accelerator, and also receive a broadcast write command when broadcast reception is enabled. When both a broadcast write command and an accelerator write command are received simultaneously, the selector arbitrates the priorities of the broadcast write command and the accelerator write command, and outputs the result based on the arbitration. The write data selector 702 is configured to write input data to a corresponding first memory upon receiving input data from a coupled accelerator, and receive broadcast weight data when broadcast reception is enabled. When both broadcast weight data and accelerator input data are received simultaneously, the selector arbitrates the priorities of the broadcast weight data and the accelerator input data, and outputs the result based on the arbitration. Figure 2 and Figure 4 As shown, the Direct Memory Access (DMA) of any AI core i (i=1, 2, 3) in accelerators AI core 1-AI core 3 obtains input data from the external secondary memory DDR and generates a write command aw_i and write data Wdata_i (i.e., input data). The write command selector 701 in the broadcast arbitrator 700 receives the write command aw_i, and the write data selector 702 receives the write data Wdata_i. If the broadcast arbitrator's broadcast reception is not enabled, the write command selector 701 and the write data selector 702 can directly output the received write command aw_i and write data Wdata_i to the corresponding SRAM i. If the broadcast arbitrator's broadcast reception is enabled (e.g., Grp_en is high), the write command selector 701 will also receive the write command Aw_broadcast i broadcast from the broadcast buffer. When both the broadcast write command Aw_broadcast i and the accelerator write command aw_i are received simultaneously, the priority of the accelerator write command aw_i and the broadcast write command Aw_broadcast i will be arbitrated, and the result will be output based on the arbitration result. Similarly, the write data selector 702 will also receive the write data Wdata_broadcast i broadcast from the broadcast buffer. When both the broadcast write data Wdata_broadcast i and the accelerator write data Wdata_i are received simultaneously, the priority of the accelerator write data Wdata_i and the broadcast write data Wdata_broadcast i will be arbitrated, and the result will be output based on the arbitration result.

[0059] In one embodiment, the priority of the broadcast write command Aw_broadcast i and the broadcast write data Wdata_broadcast i can be set to be higher than that of the accelerator write command aw_i and the accelerator write data Wdata_i. That is, when the write command selector 701 receives both the broadcast write command Aw_broadcast i and the accelerator write command aw_i, it will output the broadcast write command Aw_broadcast i to the first memory first, and then output the accelerator write command aw_i to the first memory. Similarly, when the write data selector 702 receives both the broadcast write data Wdata_broadcast i and the accelerator write data Wdata_i, it will also output the broadcast write data Wdata_broadcast i to the first memory first, and then output the accelerator write data Wdata_i to the first memory.

[0060] If the broadcast arbitrator is writing the accelerator's input data into the first memory according to the accelerator's write command when it receives the broadcast write command Aw_broadcast i and the broadcast write data Wdata_broadcast i, it will wait for the write operation to complete before writing the broadcast write data Wdata_broadcast i into the first memory according to the broadcast write command Aw_broadcast i.

[0061] In one embodiment, the broadcast arbitrator 700 further includes a second identifier 704. The second identifier 704 is configured to obtain identifier information from a write command from the accelerator, determine based on the identifier information whether the write operation of the input data corresponding to the write command sent by the accelerator is complete, and, if the write operation of the input data is determined to be complete, notify the write data selector to write the broadcast weight data into the corresponding first memory. Figure 4 As shown, the second identifier 704 obtains the identifier information awid from the write command aw_i sent by the accelerator, and determines whether an empty_nxt is generated based on the awid. If an empty_nxt is generated, it means that the write data Wdata_i corresponding to the write command aw_i has completed the write operation. At this time, the second identifier 704 will allow the write data selector 702 to select the broadcast write data Wdata_broadcast i to write to the corresponding SRAM.

[0062] In one embodiment, the broadcast arbitrator 700 further includes a reply selector 703. The reply selector 703 is configured to receive a write-complete response signal from the corresponding first memory and selectively send a write-complete response signal to the coupled accelerator or broadcast buffer based on the write-complete response signal. As previously described, the data written to SRAM i (i=1, 2, 3) may be input data from the coupled accelerator AI core i or weight data broadcast by the broadcast buffer. Therefore, when SRAM i completes the write operation and sends the corresponding write-complete response signal Bvld_axi4sram to the reply selector 703, the reply selector 703 needs to select the reply object based on the data source corresponding to the write-complete response signal Bvld_axi4sram. If the write-complete response signal Bvld_axi4sram corresponds to input data sent by accelerator AI core i, then a write-complete signal bvld_o is sent to accelerator AI core i; if the write-complete response signal Bvld_axi4sram corresponds to weight data broadcast by the broadcast buffer, then a write-complete signal bvld_bc is sent to the broadcast buffer. The response selector 703 can determine the data source based on the identification information carried in the write-complete response signal Bvld_axi4sram. It can be understood that the response selector 703 only needs to determine the data source corresponding to the write-complete response signal Bvld_axi4sram and select the response object based on the data source when the broadcast arbitrator's broadcast reception is enabled. When the broadcast arbitrator's broadcast reception is not enabled, the response selector 703, upon receiving the write-complete response signal Bvld_axi4sram, does not need to determine the data source corresponding to the write-complete response signal Bvld_axi4sram and can directly send the write-complete response signal bvld_o to the coupled accelerator.

[0063] It should be noted that in this patent application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this patent application, if it refers to performing an action according to an element, it means performing the action at least according to that element, including two cases: performing the action only according to that element, and performing the action according to that element and other elements. Expressions such as "multiple," "repeatedly," and "various" include two, two times, two kinds, and more than two, more than two times, and more than two kinds.

[0064] This specification includes combinations of various embodiments described herein. Individual references to embodiments (e.g., “one embodiment”, “some embodiments”, or “preferred embodiments”) do not necessarily refer to the same embodiment; however, these embodiments are not mutually exclusive unless indicated to be mutually exclusive or are readily apparent to those skilled in the art. It should be noted that the word “or” is used in a non-exclusive sense throughout this specification unless the context explicitly indicates or requires it.

[0065] All references to this specification are considered to be incorporated integrally into the disclosure herein so that they can serve as a basis for modifications where necessary. Furthermore, it should be understood that the above descriptions are merely preferred embodiments of this specification and are not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the scope of protection of one or more embodiments of this specification.

Claims

1. A topology device for an accelerator, characterized in that, include: Multiple accelerators, each corresponding to a first memory and respectively obtaining input data from an external second memory, wherein the accelerator can only directly access its own corresponding first memory; and A broadcast buffer is coupled between one of the plurality of accelerators and a corresponding first memory. The broadcast buffer is used to write the input data to the corresponding first memory according to the write command when a write command and input data are received from the coupled accelerator, and to broadcast the write command and the weight data in the input data when broadcasting is enabled. At least one broadcast arbitrator is coupled between each of the other accelerators in the plurality of accelerators and a corresponding first memory. The broadcast arbitrator is configured to write the input data to the corresponding first memory when it receives a write command and input data from the accelerator it is coupled to, and when broadcast reception is enabled, also receive a broadcast write command and weight data. Furthermore, when it simultaneously receives a write command and input data from the accelerator and a broadcast write command and weight data, it arbitrates the priority of the write command and input data from the accelerator and the broadcast write command and weight data, and performs a write operation based on the arbitration result.

2. The apparatus according to claim 1, characterized in that, The priority of the broadcast write commands and weighted data is higher than the priority of the write commands and input data from the accelerator.

3. The apparatus according to claim 1, characterized in that, The broadcast buffer includes: The command buffer is configured to receive a write command from the coupled accelerator when broadcasting is enabled, and to broadcast the write command to instruct the broadcast weight data to be written to the same offset address of another first memory. and A data buffer is configured to receive weight data from the coupled accelerator and broadcast the weight data when broadcasting is enabled.

4. The apparatus according to claim 3, characterized in that, The broadcast buffer also includes; The first identifier is configured to obtain identifier information from the received write command of the accelerator, determine whether the identifier information is target identifier information, and notify the data buffer to receive the weight data when the identifier information is determined to be target identifier information.

5. The apparatus according to claim 3, characterized in that, The broadcast buffer also includes: A synchronizer is configured to receive and synchronize write-complete response signals from the broadcast arbitrator, and output synchronized write-complete response signals; and The response arbitrator is configured to receive a write-complete response signal from the first memory corresponding to the broadcast buffer and a synchronized write-complete response signal from the synchronizer, arbitrate the priority of the received write-complete response signals, and output a write-complete response signal according to the arbitration result.

6. The apparatus according to claim 1, characterized in that, The broadcast arbitrator includes: A write command selector is configured to receive write commands from a coupled accelerator, output the write commands to a corresponding first memory, and also receive broadcast write commands when broadcasting is enabled. When both write commands from the accelerator and broadcast write commands are received simultaneously, the selector arbitrates the priorities of the broadcast write commands and the accelerator write commands, and outputs the result based on the arbitration. A write data selector is configured to receive input data from the coupled accelerator, write the input data to a corresponding first memory, and also receive broadcast weight data when broadcast reception is enabled. When both broadcast weight data and accelerator input data are received simultaneously, the selector arbitrates the priority of the broadcast weight data and the accelerator input data and outputs the result based on the arbitration.

7. The apparatus according to claim 6, characterized in that, The broadcast arbitrator also includes: The second identifier is configured to obtain identifier information from the write command from the accelerator, determine whether the write operation of the input data corresponding to the write command of the accelerator is completed based on the identifier information, and when the write operation is determined to be completed, notify the write data selector to write the broadcast weight data into the corresponding first memory.

8. The apparatus according to claim 6, characterized in that, The broadcast arbitrator also includes: The response selector is configured to receive a write-complete response signal from the corresponding first memory and, depending on the data source corresponding to the write-complete response signal, selectively send the write-complete response signal to the broadcast buffer or the coupled accelerator.

9. The apparatus according to claim 1, characterized in that, The accelerator includes a direct memory accessor; The direct memory accessor is configured to retrieve input data from an external second memory and write the input data to the corresponding first memory via the broadcast buffer.

Images