Data loading apparatus and electronic device

Abstract

A data loading device and an electronic device. The data loading device comprises a crossbar switch circuit, N storage units in a storage component, N being a positive integer greater than 1; the storage component is configured to store data; the crossbar switch circuit is configured to load the data in the storage component to a corresponding processing core for calculation or processing; the crossbar switch circuit comprises N demultiplexers and N multiplexers; wherein the N storage units are configured to send request data to the respective coupled demultiplexers in the same preset return order; each demultiplexer further has N first-in-first-out cache modules corresponding to the N processing cores, and each first-in-first-out cache module is configured to cache the data information of each data packet sent by the storage unit coupled to the demultiplexer in the receiving order. The data loading device can effectively alleviate the arbitration conflict of multiple storage units returning to the same processing core at the same time, and effectively improve the data transmission performance.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of electric digital data processing, in particular to a data loading device and an electronic device. BACKGROUND

[0002] In a multi-core processor architecture, data as the core input of a computing task needs to be transmitted from a storage unit to each processor core through a network on chip (NoC). The storage unit includes not only a near-end memory (such as an on-chip cache) close to the processor core and having a low access delay, but also a far-end memory (such as an off-chip DRAM) having a larger capacity but a higher delay. Finally, the computation units of different processor cores, such as central processor cores and graphics processor cores, complete the computation and data processing. When all the memories and processor cores are directly connected by the NoC, with the increase of the number of cores and memories, the physical wiring of the NoC will exponentially increase, which not only occupies a large chip silicon area, but also causes signal attenuation, transmission delay rise and power consumption increase due to the excessively long wiring, thereby seriously restricting the overall performance and integration of the chip. SUMMARY

[0003] The present application provides a data loading device, including a crossbar switch circuit, N storage units in a storage component, N is a positive integer greater than 1; the storage component is configured to store data; the crossbar switch circuit is configured to load the data in the storage component to the corresponding processing core for computation or processing; wherein the crossbar switch circuit includes: N de-multiplexers, wherein each de-multiplexer includes one input end and N output ends, the input ends of the N de-multiplexers are one-to-one coupled with the N storage units; N multiplexers, wherein each multiplexer includes N input ends and one output end, the output ends of the N multiplexers are one-to-one coupled with N processing cores; the N output ends of each de-multiplexer are coupled with one input end of each multiplexer in the N multiplexers, respectively; the N storage units are configured to send request data to the respective coupled de-multiplexers in turn according to the same preset return order, wherein the request data includes at least one data packet, each request data returns a destination processing core, and the preset return order indicates the arrangement order of the destination processing cores returned by the request data sent by the N storage units in turn; in the crossbar switch circuit, each de-multiplexer is provided with N first-in-first-out cache modules corresponding to the N processing cores one-to-one, and each first-in-first-out cache module is configured to cache the data information of each data packet sent by the storage unit coupled with the de-multiplexer in the receiving order, and the destination processing core of each data packet cached by the first-in-first-out cache module is the processing core corresponding to the first-in-first-out cache module.

[0004] For example, in at least one embodiment of the present application, a data loading device is provided, each storage unit is provided with a cache area for caching requested data read from the storage unit but not sent, and the requested data cached in the cache area is arranged in reading order; the storage unit is configured to select requested data from the cache area in the preset return order and sequentially send to the coupled multiplexer.

[0005] For example, in at least one embodiment of the present application, a data loading device is provided, the crossbar circuit is further provided with N shared cache areas corresponding to the N multiplexers; each shared cache area is configured to cache data packets returned by the storage unit coupled to the corresponding multiplexer, wherein the shared cache area supports non-continuous storage between multiple returned data packets.

[0006] For example, in at least one embodiment of the present application, a data loading device is provided, in response to the M data packets cached in the shared cache area being sent to different processing cores, the M data packets sent to the M different processing cores participate in arbitration of data returned to the M processing cores in parallel, and M is a positive integer less than N.

[0007] For example, in at least one embodiment of the present application, a data loading device is provided, the M data packets include a first data packet and a second data packet, the M processing cores include a first processing core and a second processing core, the first data packet and the second data packet are returned from the storage unit in sequence and cached in the shared cache area, the first data packet is returned before the second data packet, the destination processing core of the first data packet is the first processing core, and the destination processing core of the second data packet is the second processing core, after the first data packet is cached in the shared cache area, the first data packet participates in arbitration of data returned to the first processing core, and after the second data packet is cached in the shared cache area, the second data packet directly participates in arbitration of data returned to the second processing core.

[0008] For example, in at least one embodiment of the present application, a data loading device is provided, the crossbar circuit is configured to, in response to the multiplexer receiving a data packet sent by the coupled storage unit, store the received data packet in the shared cache area corresponding to the multiplexer, and store the storage address of the received data packet in the shared cache area as data information of the data packet to the first-in-first-out cache module corresponding to the destination processing core of the received data packet.

[0009] For example, in at least one embodiment of the present application, a data loading device is provided, each multiplexer is configured to select one arbitration candidate from at least one arbitration candidate based on an arbitration mechanism, and output the indication information corresponding to the arbitration candidate, the indication information indicates the selected arbitration candidate, wherein the storage unit meeting the preset condition is selected as the arbitration candidate to participate in arbitration, and the preset condition includes that the non-empty first-in-first-out cache module corresponding to the processing core coupled to the demultiplexer of the storage unit.

[0010] For example, in at least one embodiment of the present application, a data loading device is provided, the data loading device is further configured to: based on the indication information output by the multiplexer, perform p times of read operation to send the requested data requested by one data loading request to the cache area corresponding to the processing core coupled to the multiplexer, wherein the requested data requested by the one data loading request includes p data packets, each read operation is configured to transmit one data packet in the p data packets to the cache area corresponding to the processing core coupled to the multiplexer, and p is a positive integer.

[0011] For example, in at least one embodiment of the present application, a data loading device is provided, the crossbar circuit is further provided with N shared cache areas corresponding to the N demultiplexers; when the data loading device performs one read operation, the following operations are performed: based on the indication information output by the multiplexer, determine the selected target storage unit; from the N first-in-first-out cache modules provided by the demultiplexer coupled to the target storage unit, determine the target first-in-first-out cache module corresponding to the processing core coupled to the multiplexer; read the read address first entered from the target first-in-first-out cache module; according to the read address, read the target data packet from the shared cache area corresponding to the demultiplexer coupled to the target storage unit, and send the target data packet to the cache area.

[0012] For example, in at least one embodiment of the present application, a data loading device is provided, each multiplexer is further provided with a corresponding credit counter, the current value of the credit counter is used to indicate the number of data packets that can be cached in the cache area corresponding to the processing core coupled to the multiplexer, and the preset condition further includes that the number of data packets included in the target requested data is less than or equal to the current value of the credit counter, wherein the data packet corresponding to the first entered data information in the first-in-first-out cache module corresponding to the processing core coupled to the multiplexer belongs to the target requested data.

[0013] For example, in at least one embodiment of the present application, the number of data packets included in the target request data is determined by: determining the number of data packets included in the target request data based on data length information of the target request data carried by the data packets; or buffering a group of received data packets in a shared buffer area corresponding to a multiplexer coupled to the storage unit that transmits the target request data, and determining the number of data packets included in the target request data based on identification information carried by each data packet in the group of received data packets, wherein the identification information is used to indicate whether the current data packet is the last data packet in the target request data.

[0014] For example, in at least one embodiment of the present application, the data length of the data packet is the bandwidth of the data bus from the storage unit to the N multiplexers; the target request data includes at least one data packet, and the at least one data packet is transmitted continuously; in response to the at least one data packet carrying data length information of the target request data, the data information of the first data packet of the target request data has entered the corresponding first-in-first-out buffer module, and the storage unit that transmits the first data packet satisfies the preset condition, the storage unit that transmits the target request data participates in arbitration of the target request data destination processing core as an arbitration candidate.

[0015] For example, in at least one embodiment of the present application, each multiplexer is further provided with a corresponding credit counter, the current value of the credit counter is used to indicate the number of data packets that can be buffered in the buffer area corresponding to the processing core coupled to the multiplexer, and the credit counter adjusts the count value in units of data transmission units of the data bus, and the data length of the data transmission unit is the data length of a single data packet.

[0016] For example, in at least one embodiment of the present application, the credit counter is configured to: in response to a data packet being sent to the buffer area corresponding to the processing core coupled to the multiplexer, the value of the credit counter is decremented by 1; and in response to receiving a reception response returned by the buffer area, the value of the credit counter is incremented by 1; wherein the requested request data of a data loading request includes p data packets, after the p data packets are all returned to the buffer area corresponding to the processing core coupled to the multiplexer, the multiplexer performs the next round of arbitration, and after the requested request data is returned, the value of the credit counter is decremented by p, and p is a positive integer.

[0017] For example, in at least one embodiment of the present application, a data loading device is provided, wherein the outputs of the N multiplexers are coupled one-to-one with N ports of the network-on-chip bus, and the network-on-chip bus is coupled with the N processing cores, and the network-on-chip bus is used to transmit data received from the storage component through the crossbar circuit to the corresponding processing core.

[0018] In at least one embodiment of the present application, an electronic device is provided, which comprises the data loading device as described in any embodiment of the present application.

[0019] For example, in at least one embodiment of the present application, an electronic device is provided, which comprises a graphics processor or a general-purpose graphics processor, the storage component comprises a memory or a second cache in the graphics processor or the general-purpose graphics processor, and the processing core comprises a computing core in the graphics processor or the general-purpose graphics processor.

[0020] In the data loading device provided in at least one embodiment of the present application, the plurality of storage units return data according to the same preset return order, and the data returned to the same processing core is sent in sequence according to the sending order, so that after a period of time, there is a greater possibility that the data returned by different storage units is returned to different processing cores, which maximally reduces arbitration conflicts, effectively alleviates arbitration conflicts when the plurality of storage units return data to the same processing core at the same time, and effectively improves data transmission performance. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present disclosure, and are not a limitation on the present disclosure.

[0022] Figure 1 A schematic diagram of data returned by a plurality of slave devices to a master device;

[0023] Figure 2 A structural schematic diagram of the data loading device provided in at least one embodiment of the present application;

[0024] Figure 3 A schematic diagram of the data loading device provided in an embodiment of the present application;

[0025] Figure 4 A schematic structural diagram of the electronic device provided in at least one embodiment of the present application;

[0026] Figure 5 A specific structural schematic diagram of the electronic device provided in at least one embodiment of the present application;

[0027] Figure 6An illustrative block diagram of a general purpose graphics processor (GPGPU). DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present disclosure.

[0029] Unless otherwise defined, technical terms or scientific terms used in the present disclosure shall have the ordinary meaning of the terms to a person of ordinary skill in the art to which the present disclosure belongs. The terms “first”, “second” and similar terms used in the present disclosure do not denote any order, quantity or importance, but are used to distinguish different components. The terms “include” or “contain” and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms “connect” or “connected” and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “up”, “down”, “left”, “right” and the like are only used to represent relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships may also change accordingly.

[0030] In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits the detailed description of some known functions and known components.

[0031] In a multi-core processor, data is often loaded from near / far memory units to different processor cores for computation / processing through an on-chip network bus. As described above, if all memory units and processor cores are directly connected by an on-chip network, the physical wiring will increase exponentially.

[0032] In order to optimize the problem of excessive area of on-chip network wiring, an optimization scheme of adding a CrossBar circuit between a multi-memory bank (each memory bank can be read and written in parallel to improve memory bandwidth) and an on-chip network bus can be used: through the CrossBar circuit, the routing scheduling is completed in advance before the data returns to the on-chip network bus, and the output data of multiple memory banks are allocated to the target transmission path as needed, greatly reducing the number of direct wirings between the on-chip network and the memory units (such as memory banks), thereby achieving a balance among area, delay and power consumption.

[0033] Figure 1This is a schematic diagram illustrating a method where data is returned from multiple slave devices to multiple master devices.

[0034] like Figure 1 As shown, slave0 to slave2 are three slave devices, and master0 to master2 are three master devices. When data is transmitted from the slave devices to the master devices, slave0 returns data packets of varying sizes to different master devices, such as master0 and master1, according to the source index of the data load request. Slave1 also returns data packets of varying sizes to different master devices according to the source index of the data load request. In one clock cycle, a master device will only receive data packets returned by one slave device. If slave0 and slave1 simultaneously return data packets to a master device, arbitration is required between these two data packets from different sources. The data packet that wins the arbitration can continue to be returned to the corresponding master device.

[0035] This approach will introduce new performance challenges.

[0036] When multiple slave devices simultaneously return data to multiple master devices, resource contention can occur between the output ports of the crossbar switch circuit and the access ports of the on-chip network, i.e., arbitration conflict. Since each output port of the crossbar switch can only transmit one data path at a time, if the return data from multiple memory blocks needs to share the same port, the hardware arbitrator must determine the data transmission order according to preset rules (such as priority and request timing), thus triggering an arbitration conflict.

[0037] For burst transfers, each request can send multiple data packets with consecutive storage addresses. These packets must be sent consecutively, otherwise parsing errors may occur. Therefore, once a burst transfer is initiated, the data channel will be occupied by that transfer until all data packets have been transmitted (or terminated abnormally). During this period, other devices cannot use the bus resources, so the transfer will occupy the bus for a certain period of time, leading to severe arbitration conflicts.

[0038] Memory blocks that fail to obtain transmission rights will be unable to receive new read / write requests due to the limited capacity of their output buffers (hardware units that temporarily store data to be transmitted). This triggers a "backpressure mechanism"—which suspends subsequent data output from the memory block through signal feedback to prevent buffer overflow and data loss. While the backpressure mechanism can control traffic, it can also cause other slave devices to be unable to return data in a timely manner during this period, increasing latency, reducing throughput, and leading to performance degradation.

[0039] Even if arbitration conflicts are temporarily alleviated through backpressure, if the return data from multiple memory blocks floods the on-chip network bus in a short period of time, it will exceed the instantaneous bandwidth capacity of the on-chip network, causing bus data traffic congestion. Congestion will not only lead to a significant increase in data transmission latency, but will also cause other data transmission tasks on the on-chip network to queue up, further reducing the throughput of the entire on-chip data transmission link. Ultimately, this will cause the processor cores to idle while waiting for data, dragging down the overall computing efficiency of the multi-core processor.

[0040] At least one embodiment of this disclosure provides a data loading device and an electronic device. The data loading device includes a crossbar switch circuit, N storage units in a storage component, and N processing cores, where N is a positive integer greater than 1. The crossbar switch circuit is configured to load data from the storage component into the corresponding processing cores for calculation or processing. The crossbar switch circuit includes: N multiplexers, each multiplexer including one input terminal and N output terminals, the input terminals of the N multiplexers being coupled one-to-one with the N storage units; and N multiplexers, each multiplexer including N input terminals and one output terminal, the output terminals of the N multiplexers being coupled one-to-one with the N processing cores; the N output terminals of each multiplexer are respectively coupled to one input terminal of each of the N multiplexers. The N storage units are configured to send request data to their respective coupled multiplexers in the same preset return order. The request data includes at least one data packet, and each request data returns to a destination processing core. The preset return order indicates the order in which the requested data sent by the N storage units returns to the destination processing cores. In the cross-connect circuit, each multiplexer is provided with N first-in-first-out (FIFO) buffer modules corresponding one-to-one with the N processing cores. Each FIFO buffer module is configured to buffer the data information of each data packet sent by the storage unit coupled to the multiplexer in the receiving order. The destination processing core of each data packet buffered by the FIFO buffer module is the processing core corresponding to the FIFO buffer module.

[0041] In the data loading apparatus provided in at least one embodiment of this disclosure, multiple storage units return data in the same preset return order, and the data returning to the same processing core are sent sequentially in the sending order. As a result, after a period of time, there is a high probability that data returned by different storage units will be returned to different processing cores, thereby minimizing arbitration conflicts, effectively alleviating arbitration conflicts when multiple storage units return to the same processing core at the same time, and effectively improving data transmission performance.

[0042] The circuit structure of the data loading device provided in at least one embodiment of this disclosure will be described in detail below with reference to the accompanying drawings.

[0043] Figure 2A structural schematic diagram of a data loading apparatus provided by at least one embodiment of the present disclosure is shown.

[0044] As shown in Figure 2 , the data loading apparatus 100 includes a crossbar circuit 101, N storage units 102 in a storage component, such as a storage unit 102_1, a storage unit 102_2, a storage unit 102_n, and the like, in Figure 2 , where N is a positive integer greater than 1.

[0045] The storage component is configured to store data. For example, the storage component can be a memory (such as a high bandwidth memory HBM), a cache (such as a level 2 cache), and the like, which is not specifically limited in the present disclosure.

[0046] For example, the memory can be divided into multiple memory blocks, each of which can independently respond to read / write requests, and multiple memory blocks can work in parallel to improve overall throughput. For example, each memory block can be used as a storage unit in the data loading apparatus provided by at least one embodiment of the present disclosure.

[0047] The crossbar circuit 101 is configured to load data in the storage component to corresponding processing cores for computation or processing. For example, the crossbar circuit can send data in the N storage units to corresponding processing cores in the N processing cores according to a source index, for subsequent computation or processing.

[0048] The N processing cores can be GPU cores, CPU cores, i.e., multi-processor cores; or the N processing cores can be computation cores in GPU cores, and the like, which is not specifically limited in the present disclosure. The processing core can be a component that needs to use data in the storage component for processing and computation, and its structure and function can be set as needed.

[0049] For example, the crossbar circuit can transmit data to the processing core through a bus, such as an on-chip network bus, and the like.

[0050] As shown in Figure 2 , the crossbar circuit 101 includes N demultiplexers (DMUXs), such as a demultiplexer 103_1, a demultiplexer 103_2,..., and a demultiplexer 103_N in Figure 2 . Each demultiplexer includes one input end and N output ends, and the input ends of the N demultiplexers are one-to-one coupled with the N storage units.

[0051] As shown in Figure 2 , the crossbar circuit 101 further includes N multiplexers (MUXs), such as a multiplexer 104_1, a multiplexer 104_2,..., and a multiplexer 104_N in Figure 2The N multiplexers are coupled to the N storage units respectively, and each of the N multiplexers includes N inputs and one output. The outputs of the N multiplexers are coupled to the N processing cores respectively.

[0052] As shown in FIG. 1, each of the N multiplexers includes N inputs and one output. The outputs of the N multiplexers are coupled to the N processing cores respectively. Figure 2 As shown in FIG. 1, each of the N multiplexers includes N inputs and one output. The outputs of the N multiplexers are coupled to the N processing cores respectively.

[0053] The N storage units are configured to send the request data to the respective multiplexers in the same preset return order. The request data includes at least one data packet, each request data returns to a target processing core, and the preset return order indicates the arrangement order of the target processing cores to which the request data sent by the N storage units returns in sequence.

[0054] The N storage units return the request data in the same preset return order. For example, the preset return order can be processing core 1 -> processing core 2 -> processing core 3 ->... -> processing core N, that is, each storage unit first sends data to processing core 1, then sends data to processing core 2, then sends data to processing core 3, and so on. Of course, the preset return order can be set as needed, and the present disclosure does not make specific limitations in this regard.

[0055] For example, the current request data return order is the reading order, that is, which request data is read first and ready first. In the data loading device provided by at least one embodiment of the present disclosure, the request data returned from the storage unit returns in sequence according to the preset return order. For example, taking the preset return order as processing core 1 -> processing core 2 -> processing core 3 ->... -> processing core N as an example, the storage unit can first return the request data to processing core 1, then return the request data to processing core 2, then return the request data to processing core 3, and so on. Assuming that the storage unit returns the request data to processing core 1, and there is currently no request data to processing core 2, then processing core 2 can be skipped, and the request data to processing core 3 is returned first, and then the request data to processing core 4 is sent according to the preset return order, and so on.

[0056] For example, the data length of a data packet is the bandwidth of the data bus from the storage unit to the N multiplexers.

[0057] AXI bus is widely used in data transmission of high-performance processors, and is an on-chip bus for high performance, high bandwidth and low latency. In order to meet the data interaction / transmission requirements between multi-host devices and multi-slave devices, CrossBar based on AXI bus is widely used. For example, the data bus can be an AXI bus, of course, the present disclosure is not limited thereto.

[0058] For example, Figure 2 The connection line from the storage unit to the multiplexer shown in FIG. 1 can be a read data channel of the AXI bus, which is a unidirectional channel specially used for transmitting read operation return data and related state information, and is responsible for returning the request data from the slave device (such as the storage unit) to the master device (such as the processing core) in response to the data load request of the master device.

[0059] Taking the AXI bus protocol as an example, in a burst transmission, a data load request return request data can be transmitted by continuously sending multiple data packets. In an example, the data bandwidth of the AXI bus is 1KB (kilobyte), and the data length of a data packet is 1KB. The data length of the request data requested by a data load request can be 2KB, 4KB, etc., including 2 data packets or 4 data packets. Taking the data length of the request data as an example, 4 data packets need to be sent to the multiplexer in succession, of course, 4 data packets also need to be sent to the processing core in succession. The returned data cannot be interleaved, for example, data packets to different processing cores are mixed together, and interleaving will cause the upstream module (such as the processing core) to process incorrectly.

[0060] In the crossbar circuit, N first-in-first-out cache modules corresponding to the N processing cores are provided for each multiplexer. Each first-in-first-out cache module is configured to cache data information of data packets sent by a storage unit coupled to the multiplexer in the order of reception. The destination processing core of the data packet cached by the first-in-first-out cache module is the processing core corresponding to the first-in-first-out cache module.

[0061] Figure 2 It is shown that the multiplexer 103_1 has N first-in-first-out cache modules, that is, Figure 2FIFO0_1, FIFO0_2,..., FIFO0_N, where FIFO0_1 corresponds to processing core 1, FIFO0_2 corresponds to processing core 2,..., and FIFO0_N corresponds to processing core N. FIFO0_1 is configured to buffer data information of data packets sent by storage unit 102_1 in the order of reception, and FIFO0_1 only buffers data information of data packets whose destination processing core (e.g. indicated by the source index) is processing core 1. Similarly, FIFO0_2 is configured to buffer data information of data packets sent by storage unit 102_2 in the order of reception, and FIFO0_2 only buffers data information of data packets whose destination processing core is processing core 2. Other FIFO modules are configured in the same way, and will not be described here.

[0062] Of course, multiplexer 103_2 also has N FIFO modules, e.g. FIFO1_1, FIFO1_2,..., FIFO1_N, Figure 2 which are not shown in FIG. 3. Other multiplexers are also provided with their own N FIFO modules, and will not be described here.

[0063] For example, each storage unit is provided with a buffer area for buffering request data that has been read from the storage unit but not yet sent. The request data buffered in the buffer area is arranged in the order of reading, i.e. the buffer area buffers data packets to be sent. The buffer area can reuse the buffer area originally provided by the storage unit, or an additional buffer area can be provided in the crossbar circuit, and the present disclosure does not make specific limitations in this regard.

[0064] The storage unit is configured to select request data from the buffer area in a predetermined return order and send them to the coupled multiplexer in turn.

[0065] For example, still taking the predetermined return order of processing core 1 -> processing core 2 -> processing core 3 ->... -> processing core N as an example, the storage unit first selects request data whose destination processing core is processing core 1 from the buffer area, and sends it to the coupled multiplexer if any; then the storage unit selects request data whose destination processing core is processing core 2 from the buffer area, and sends it to the coupled multiplexer if any; then the storage unit selects request data whose destination processing core is processing core 3 from the buffer area, and sends it to the coupled multiplexer if any; and so on.

[0066] The plurality of storage units return data according to the preset return order. After a period of time, it is more likely that the request data return processing core N of the storage unit 102_1, the request data return processing core N-1 of the storage unit 102_2, the request data return processing core N-2 of the storage unit 102_3,..., and the request data return processing core 1 of the storage unit 102_N are returned to different processing cores in the same clock cycle. The arbitration conflict is minimized, and the data transmission performance and the processing performance of the processing core are improved.

[0067] Referring to Table 1 below, taking N=3 as an example, in clock cycle 0, the storage unit 1, the storage unit 2, and the storage unit 3 return data at the same time. In the time sequence in the horizontal direction of the table, the storage units return data in the order of processing core 1, 2, and 3, and the preset return order is processing core 1 -> processing core 2 -> processing core 3. That is, the storage unit 1 returns the request data to the processing core 1 first, then returns the request data to the processing core 2, then returns the request data to the processing core 3, and then returns the request data to the processing core 1, and so on.

[0068] In clock cycle 0, the storage units all participate in the arbitration of the multiplexer mux1 coupled to the processing core 1. For example, the storage unit 1 obtains arbitration first, and the storage units 2 and 3 continue to wait. The continued waiting here means waiting for an arbitration opportunity, but does not affect the storage units 2 and 3 to continue to send request data to the multiplexer according to the preset return order. Moreover, since the data information of the data packets of the request data is buffered in the first-in-first-out buffer module according to the receiving order, the data returned to the same processing core is still processed in the crossbar circuit according to the sending order.

[0069] In clock cycle 1, the storage unit 1 returns the request data to the processing core 2. At this time, only the request data of the storage unit 1 needs to be returned to the processing core 2, so the request data of the storage unit 1 can be selected to be returned to the processing core 2. The storage unit 2 obtains arbitration to return the request data to the processing core 1, and the request data of the storage unit 2 can be selected to be returned to the processing core 1. The storage unit 3 continues to wait.

[0070] In clock cycle 2, the storage unit 1 returns the data to the processing core 3. At this time, only the request data of the storage unit 1 needs to be returned to the processing core 3, so the request data of the storage unit 1 can be selected to be returned to the processing core 3. The storage unit 2 returns the request data to the processing core 2. At this time, only the request data of the storage unit 2 needs to be returned to the processing core 2, so the request data of the storage unit 2 can be selected to be returned to the processing core 2. The storage unit 3 obtains arbitration to return the request data to the processing core 1, and the request data of the storage unit 3 can be selected to be returned to the processing core 1.

[0071] The following clock cycles are similar. In this example, from clock cycle 2, the data of 3 storage units return to different processing cores, and there is no arbitration conflict, effectively improving the data transmission performance.

[0072]

[0073] As shown in Figure 2 A register slice (Regslice) is also provided between each demultiplexer and the coupled storage unit, and a register slice is also provided between each multiplexer and the coupled processing core. The register slice is a hardware unit in chip design for optimizing timing and improving signal transmission reliability. Essentially, it is a set of flip-flops inserted in the data path or control path, which solves the timing convergence problem in high-speed circuits by dividing the long path into multiple short paths. Timing convergence can be achieved by inserting a register slice.

[0074] For example, in some embodiments, the data information is the data in the data packet, that is, the data in the data packet is stored in the first-in-first-out cache module.

[0075] For example, in some embodiments, the crossbar circuit is also provided with N shared cache areas corresponding to the N demultiplexers; each shared cache area is configured to cache the data packet returned by the storage unit coupled to the corresponding demultiplexer, and the shared cache area supports non-continuous storage between the returned multiple data packets.

[0076] For example, the shared cache area can reuse the shared cache in a graphics processing unit or a general-purpose graphics processing unit.

[0077] For example, Figure 2 The demultiplexer 103_1 in the crossbar circuit has a corresponding shared cache area 1, the demultiplexer 103_2 has a corresponding shared cache area 2, and so on.

[0078] The shared cache area is different from the first-in-first-out cache module, and the shared cache area supports non-continuous storage between the returned multiple data packets. For example, the sequentially returned data packets can be sequentially stored in the shared cache area in the order of reception, or the sequentially returned data packets can also be non-continuously stored in the shared cache area. When accessing the shared cache area, the data packet at any position can be extracted by reading the address.

[0079] For example, the shared cache area 1 is configured to cache the data packet included in the requested data returned by the storage unit 102_1, the shared cache area 2 is configured to cache the data packet included in the requested data returned by the storage unit 102_2, and so on.

[0080] For example, in some embodiments, the crossbar circuit 101 is configured to, in response to the multiplexer receiving a data packet sent by a coupled storage unit, store the received data packet into a shared buffer area corresponding to the multiplexer, and store a storage address of the received data packet in the shared buffer area as data information of the data packet into a first-in-first-out cache module corresponding to a destination processing core of the data packet.

[0081] In this embodiment, data is stored into a shared buffer area, and since the shared buffer area does not require first-in-first-out, data packets in the shared buffer area destined for different processing cores can directly participate in arbitration for returning to the processing cores without waiting for the transmission of previous data packets to complete before participating in arbitration, effectively improving data transmission efficiency and realizing parallel arbitration processing of serial data; and only the storage address of the data packet in the shared buffer area is stored as data information in the first-in-first-out cache module, so the storage capacity of the first-in-first-out cache module can be set smaller, effectively saving chip area.

[0082] For example, in some embodiments, in response to M data packets cached in the shared buffer area being sent to different processing cores, the M data packets sent to the M different processing cores participate in arbitration for returning to the M processing cores in parallel, and M is a positive integer less than N.

[0083] The arbitration of data returning to the processing core means that the storage unit sending the data packet can participate in arbitration of the multiplexer coupled to the processing core. Of course, in other embodiments, the storage unit as an arbitration candidate also needs to meet other conditions, which can be referred to later.

[0084] For example, taking shared buffer area 1 as an example, M data packets are cached in the shared buffer area 1, and the destination processing cores of the M data packets are different from each other. Since the shared buffer area can not need to access data according to the first-in-first-out rule, the M data packets can participate in arbitration for returning to the M processing cores in parallel, and after obtaining arbitration, the data can be directly read from the shared buffer area and sent to the processing core, thereby allowing the serial data returned by the same storage unit to participate in arbitration for returning to different processing cores in parallel, and relieving arbitration conflicts between returning to each processing core.

[0085] For example, the M data packets include a first data packet and a second data packet, and the M processing cores include a first processing core and a second processing core; the first data packet and the second data packet are returned from the storage unit in sequence and cached in the shared cache area, the first data packet is returned earlier than the second data packet, the destination processing core of the first data packet is the first processing core, and the destination processing core of the second data packet is the second processing core; after the first data packet is cached in the shared cache area, the first data packet participates in arbitration of returning data to the first processing core, and after the second data packet is cached in the shared cache area, the second data packet directly participates in arbitration of returning data to the second processing core without waiting for the first data packet to be transmitted before arbitrating the second data packet.

[0086] In one specific example, if the same storage unit (for example, the storage unit 102_1) returns two requested data, namely requested data 0 and requested data 1, and it is assumed that the requested data 0 includes one data packet and the requested data 1 includes one data packet. The destination processing core of the requested data 0 is the processing core 1, the destination processing core of the requested data 1 is the processing core 2, the requested data is cached in the shared cache area 1, and the requested data 0 is returned earlier than the requested data 1. The data packet of the requested data 0 can be arbitrated with the data packet returned by the processing core 1 of the other storage unit; and the subsequent requested data 1 does not have to wait for the requested data 0 to be transmitted before participating in arbitration, but can directly participate in arbitration of returning the processing core 2, thereby greatly reducing the time waiting for arbitration and effectively improving the efficiency of data transmission.

[0087] Here, direct participation in arbitration means that, in some embodiments, the storage unit sending the second data packet can directly participate in arbitration as an arbitration candidate at this time, and in other embodiments, the storage unit sending the second data packet can participate in arbitration as an arbitration candidate at this time only if a comparison condition with the current value of the credit counter is met, which will be described below and will not be described here again.

[0088] For example, in some embodiments, each multiplexer is configured to select one arbitration candidate from at least one arbitration candidate based on an arbitration mechanism, and output indication information corresponding to the arbitration candidate, the indication information indicating the selected arbitration candidate, for example, the indication information is an index value of the storage unit.

[0089] For example, the storage unit satisfying the preset condition is selected as an arbitration candidate to participate in arbitration, and the preset condition includes that the storage unit is coupled to a plurality of multiplexers, and the plurality of multiplexers are coupled to a plurality of first-in-first-out cache modules, and the processing core corresponding to the multiplexer is coupled to a first-in-first-out cache module that is not empty.

[0090] For example, in one embodiment, taking storage unit 102_1 and processing core 1 as an example, when a data packet is sent to storage unit 102_1 and the data information is cached in the first-in-first-out cache module FIFO0_1 corresponding to processing core 1 in the N first-in-first-out cache modules of the N-way distributor 103_1, the storage unit 102_1 can participate in arbitration for returning data to processing core 1 as an arbitration candidate.

[0091] For example, in some embodiments, the data loading apparatus is further configured to perform p read operations based on the indication information output by the multiplexer, wherein the requested data requested by one data loading request includes p data packets, each read operation is configured to transmit one data packet in the p data packets to the cache area corresponding to the processing core to which the multiplexer is coupled, and p is a positive integer.

[0092] For example, in some embodiments, the output ends of the N multiplexers are one-to-one coupled with the N ports of the network-on-chip bus, and are coupled with the N processing cores through the network-on-chip bus, and the network-on-chip bus is used to transmit data received from the storage component through the crossbar circuit to the corresponding processing core. That is, in this embodiment, the multiplexer is first coupled with the port of the network-on-chip bus, and the cache area corresponding to the processing core is the place where the network-on-chip caches the data before sending the data through the network-on-chip bus. If the cache area is set to be large, it will cause the bus area to be too large.

[0093] For example, in some embodiments, the data stored in the first-in-first-out cache module is data in a data packet, and one data packet can be read out (POP operation) at a time through a read operation and returned to the cache area corresponding to the processing core.

[0094] For example, in some embodiments, in the case where the crossbar circuit is provided with N shared cache areas corresponding to the N multi-way distributors, when the data loading apparatus performs one read operation, the following operations are included: based on the indication information output by the multiplexer, determining a selected target storage unit; from the N first-in-first-out cache modules of the multi-way distributor coupled to the target storage unit, determining a target first-in-first-out cache module corresponding to the processing core to which the multiplexer is coupled; reading out (POP operation) the read address that enters first from the target first-in-first-out cache module; and according to the read address, reading a target data packet from the shared cache area corresponding to the multi-way distributor coupled to the target storage unit, and sending the target data packet to the cache area.

[0095] For example, in one embodiment, taking multiplexer 104_1 as an example to specifically illustrate the above content, and the other multiplexers are the same, which will not be described here.

[0096] For the multiplexer 104_1, at least one arbitration candidate participating in arbitration can be arbitrated according to an arbitration mechanism, and the multiplexer 104_1 outputs indication information of the arbitration candidate (selected) that obtains arbitration. The arbitration mechanism can be set according to predetermined rules, such as priority, request timing, etc., and the present disclosure does not make specific limitations thereto.

[0097] After outputting the indication information, based on the indication information output by the multiplexer, p times of read operations are performed to send the request data requested by one data loading request to the cache area corresponding to the processing core coupled to the multiplexer. The request data includes p data packets, and each request data transmits one data packet to the cache area.

[0098] In each read operation, first, the target storage unit that obtains arbitration and is selected is determined according to the indication information, for example, the target storage unit is the storage unit 102_1; then, from the N first-in-first-out cache modules set in the demultiplexer 103_1 coupled to the target storage unit 102_1, the target first-in-first-out cache module corresponding to the processing core 1 coupled to the multiplexer 104_1, i.e., FIFO0_1, is determined; then, the read address that enters first is read from the target first-in-first-out cache module FIFO0_1; finally, according to the read address, the target data packet is read from the shared cache area 1 corresponding to the demultiplexer 103_1 coupled to the target storage unit 102_1, and the target data packet is sent to the cache area corresponding to the processing core 1. After the transmission of the target data packet, the shared cache area 1 also releases the storage space of the target data packet to receive a new data packet.

[0099] In the above embodiment, by storing data packets in the shared cache area, arbitration of data packets to different processing cores can be performed in parallel, arbitration of some data packets is performed in advance, serial data is processed in parallel, transmission time of data from the storage component to the processing core is compressed, and data transmission performance is effectively improved; the first-in-first-out cache module only needs to store the storage address of the data packet, so the occupied area in the chip is smaller, the chip area is effectively saved, and the hardware resource overhead is reduced.

[0100] In the data loading device provided in at least one embodiment of the present disclosure, a credit mechanism between the crossbar circuit and the processing core is established, and bus congestion is alleviated by limiting the number of data returns.

[0101] For example, in some embodiments, each multiplexer is also provided with a corresponding credit counter, and the current value of the credit counter is used to indicate the number of data packets that can be buffered in the cache area of the processing core coupled to the multiplexer.

[0102] As mentioned above, the storage unit satisfying the preset condition is involved in arbitration as an arbitration candidate, and the preset condition includes that the corresponding N first-in first-out cache modules of the multi-plexer-coupled multi-plexer are not empty; in addition, in this embodiment, the preset condition further includes that the number of data packets included in the target request data is less than or equal to the current value of the credit counter, wherein the data packet corresponding to the first-arriving data information in the corresponding first-in first-out cache module belongs to the target request data.

[0103] For example, assuming that a data packet sent by the storage unit 102_1 and destined for the processing core 1 is cached in the shared cache 1, at this time, the FIFO0_1 is not empty; if the total number a of data packets included in the request data to which the data packet belongs is less than or equal to the current value of the credit counter 1 corresponding to the multi-plexer 104_1, the storage unit 102_1 can be involved in arbitration as an arbitration candidate, and a is a positive integer.

[0104] If the number a of data packets included in the request data to which the data packet belongs is greater than the current value of the credit counter 1 corresponding to the multi-plexer 104_1, the storage unit 102_1 cannot be involved in arbitration as an arbitration candidate, and the storage unit 102_1 can be involved in arbitration as an arbitration candidate only after the current value of the credit counter 1 is greater than or equal to a.

[0105] Therefore, in at least one embodiment of the present disclosure, the credit mechanism can effectively control the amount of data returned to the processing core, that is, only when there is enough idle space in the cache area corresponding to the port of the network-on-chip bus to place a complete request data, the storage unit sending the request data can be involved in arbitration as an arbitration candidate, and when arbitration is obtained, the complete request data can be sent, and the data transmission can be completed without interleaving. Especially for burst transmission, this embodiment can effectively reduce the arbitration occupancy time and improve the data transmission performance.

[0106] Therefore, the amount of data in the cache area can be effectively reduced, the situation that a large amount of data waits for transmission on the network-on-chip bus can be avoided, and the data congestion of the network-on-chip bus can be effectively alleviated. Moreover, since, for example, the burst transmission does not support the interleaving mode, the data packets in a request data must be transmitted continuously, so the burst transmission usually occupies arbitration until the burst transmission is completed. The credit mechanism in the above embodiment is used to control arbitration, and the arbitration candidate satisfying the credit value (the number of data packets is greater than the current value of the credit counter) is involved in arbitration, thereby effectively reducing the arbitration occupancy time, avoiding the situation that the request data cannot be completely transmitted due to the lack of idle space in the cache area, and occupying arbitration, effectively reducing the arbitration occupancy time. Moreover, the number of arbitration candidates involved in arbitration is reasonably reduced, and the arbitration efficiency is improved.

[0107] For example, the number of data packets included in the target request data is determined in the following manner: the number of data packets included in the target request data is determined based on data length information carried by the data packets; or, a group of received data packets are buffered in a shared buffer area corresponding to a multiplexer to which the storage unit sending the target request data is coupled, and the number of data packets included in the target request data is determined based on identification information carried by each data packet in the group of received data packets, wherein the identification information is used to indicate whether the current data packet is the last data packet in the target request data.

[0108] In some embodiments, the length information of the request data is not carried in the data packets, in which case the data loading device initializes a count value (for example, 0), and then adds 1 to the count value each time a data packet is received, and determines whether the identification information carried by the data packet indicates that the data packet is the last data packet in the request data. If so, the count value (which is 1 at this time) is taken as the number of data packets included in the request data; if not, the next data packet is received, the count value is added by 1 again, and it is determined whether the identification information carried by the data packet indicates that the data packet is the last data packet in the request data.

[0109] In the above embodiments, the number of data packets included in the request data cannot be determined until all the data packets arrive at the shared memory, and therefore all the data packets included in the request data cannot participate in arbitration until they arrive at the shared memory, otherwise the credit value comparison cannot be performed, which can occupy the cache space of the parallel cache spaces of the plurality of processing cores.

[0110] In other embodiments, length information (for example, burst length) is added in the data packets, and the data buffered in the shared buffer area does not need to be packaged through the above process, and can participate in arbitration as soon as the first data packet is received, or can participate in arbitration as an arbitration candidate when the first data packet is received and the credit value condition is met.

[0111] For example, in some embodiments, the data length of the data packet is the bandwidth of the data bus from the storage unit to the N multiplexers; the target request data includes at least one data packet, and the at least one data packet is transmitted continuously. In response to the fact that the at least one data packet carries the data length information of the target request data, the data information of the first data packet of the target request data has entered the corresponding first-in-first-out buffer module, and the storage unit sending the first data packet satisfies a preset condition, the storage unit sending the target request data participates in arbitration of the target request data by the destination processing core as an arbitration candidate.

[0112] In this embodiment, the arbitration can be started after the first data packet arrives without waiting for all the requested data of the data packet to arrive in the shared cache. When the first data packet is selected, the storage unit occupies the arbitration. After the transmission of the requested data is completed, the next round of arbitration is performed, which improves the data transmission performance in the crossbar circuit, reduces the occupancy time of the data in the shared cache area, improves the utilization of the shared cache area, and effectively improves the data parallel processing performance. The length information carried can avoid long-time occupation of the shared cache area, effectively relieve the data back pressure at the storage unit, and improve the transmission performance of the data returned by the storage unit.

[0113] In some embodiments, to further save the area of the cache corresponding to the processing core, a credit mechanism based on beat granularity can be optimized. The credit mechanism based on request granularity requires caching according to the maximum number of data packets for transmitting data, and a large amount of storage overhead can be caused by one request data. The credit mechanism based on request granularity requires a large cache space for the processing core, which can cause excessive pressure on the bus area.

[0114] For example, in some embodiments, the credit counter adjusts the count value in the granularity of a data bus data transmission unit, and the data length transmitted by the data transmission unit is the data length of a single data packet.

[0115] In the AXI bus protocol, the beat granularity refers to the size of each data transmission unit (i.e., a "beat") in one burst transfer, which is usually measured in bytes and determined by the bit width of the AXI bus. The minimum unit of burst transfer is composed of multiple consecutive data packets, and each data packet is the smallest indivisible unit of transmission. For example, a burst transfer length of "4 beats" means that 4 data packets of the above fixed size are transmitted continuously.

[0116] In at least one embodiment of the present disclosure, a credit mechanism based on beat granularity is adopted to effectively save the area of the cache corresponding to the processing core.

[0117] For example, the credit counter is configured to decrease by 1 in response to one data packet being sent to the cache of the processing core coupled to the multiplexer, and to increase by 1 in response to receiving a return response of the processing core coupled to the multiplexer.

[0118] For example, the requested request data of one data loading request includes p data packets. After the p data packets are all returned to the cache of the processing core coupled to the multiplexer, the multiplexer performs the next round of arbitration, and the value of the credit counter decreases by p after the requested request data is returned, where p is a positive integer.

[0119] In combination with the above embodiments,Figure 3 For example, storage units 102_1, 102_2, and 102_3 simultaneously return data packets to processing core 1. At this point, round-robin arbitration is required. If the current count value of credit counter 1 corresponding to multiplexer 104_1 is 'b', due to a clock cycle delay in receiving the buffer response, to reduce the arbitration occupancy / locking time, data packets with a number of packets less than or equal to 'b' are included in the request data for arbitration, while data packets greater than 'b' continue to wait. Each time arbitration data is returned to the processing core, a credit value is consumed; for example, the credit counter value is decremented by 1 for each data packet sent, and decremented by 'p' after the transmission of a request data is completed. When the processing core's buffer can continue receiving data, it returns a receive response (ack) signal, at which point the credit counter value is increased, where 'b' is a positive integer.

[0120] For example, the processing core's buffer may send a receive response signal for each data packet sent to the on-chip network; or, the processing core's buffer may directly send a value indicating the amount of free storage space to the credit counter, without any specific limitation in this disclosure.

[0121] In this embodiment, the credit value adjustment process is optimized to the beats granularity based on the credit mechanism, which effectively reduces the buffer area of ​​the upstream processing core or on-chip network.

[0122] Figure 3 This is a schematic diagram of a data loading device provided in an embodiment of the present disclosure.

[0123] The following is combined Figure 3 This fully describes the overall structure and function of the data loading apparatus provided in at least one embodiment of the present disclosure. Figure 3 In the example provided, for convenience, N=3 is used as an example for illustration, but this disclosure is not limited to this.

[0124] For example, a storage unit can also be used as a slave device, and a port that processes the core / on-chip network can also be called a master device. The cross switch circuit is a cross switch circuit based on the AXI bus, which meets the data interaction / transmission needs between multiple master devices and multiple slave devices.

[0125] like Figure 3 As shown, the data loading device 100 includes three storage units, namely storage unit 102_1, storage unit 102_2, and storage unit 102_3. The storage units are, for example, memory banks in memory.

[0126] The crossbar circuit 101 is configured to load data in the storage components to corresponding processing cores for computation or processing. For example, the crossbar circuit 101 allocates the output data of the plurality of memory blocks to the target transmission path on demand through routing scheduling to send to the corresponding processing core.

[0127] In Figure 3 the example, the data output by the crossbar circuit first reaches the port of the network-on-chip, and then is transmitted to the corresponding processing core through the network-on-chip bus. Of course, in other examples, it can also be transmitted to the corresponding processing core through other buses, or directly transmitted to the corresponding processing core, and the like, which will not be described here.

[0128] As Figure 3 shown, the crossbar circuit 101 includes three demultiplexers, namely demultiplexer 103_1, demultiplexer 103_2, and demultiplexer 103_3. The input ends of the three demultiplexers are one-to-one coupled with the three storage units.

[0129] The crossbar circuit 101 further includes three multiplexers, namely multiplexer 104_1, multiplexer 104_2, and multiplexer 104_3. The output ends of the three multiplexers are one-to-one coupled with the three ports of the network-on-chip, and then coupled to the corresponding processing core through the network-on-chip.

[0130] The three output ends of each demultiplexer are coupled with one input end of each multiplexer.

[0131] The bus used by the data loading device is an AXI bus, which is an on-chip bus oriented to high performance, high bandwidth, and low latency. The crossbar circuit 101 is an AXI bus-based crossbar circuit.

[0132] The three storage units are configured to sequentially send request data to the respective coupled demultiplexers in the same preset return order. The request data includes at least one data packet, and each request data returns a destination processing core. The preset return order indicates the arrangement order of the destination processing cores to which the sequentially sent request data returns when the N storage units send the request data.

[0133] Each storage unit is provided with a cache area for temporarily storing the request data that has been read but not yet sent. Each request data includes at least one data packet, and the data packets of each request data need to be sent continuously without interleaving.

[0134] For example, taking the preset return order as processing core 1 -> processing core 2 -> processing core 3, the storage units all return data according to the same preset return order. That is, they first select the request data destined for processing core 1 from the storage unit's cache and send it, then select the request data destined for processing core 2 and send it, and so on. Referring to Table 1, after a period of time, the request data of each storage unit can be returned to different ports within the same clock cycle, minimizing arbitration conflicts and effectively improving data transmission performance.

[0135] Each multiplexer also has N first-in-first-out cache modules corresponding to N processing cores. Figure 3 (not shown in the figure) For more information about the first-in-first-out (FIFO) cache module, please refer to the description of the foregoing embodiments, which will not be repeated here.

[0136] like Figure 3 As shown, the cross switch circuit also has three shared buffers corresponding to the three multiplexers: shared buffer 1 corresponding to multiplexer 103_1, shared buffer 2 corresponding to multiplexer 103_2, and shared buffer 3 corresponding to multiplexer 103_3.

[0137] The shared buffer is configured to cache data packets returned by the storage unit coupled to the corresponding multiplexer. For example, shared buffer 1 is used to cache data packets returned by storage unit 102_1, shared buffer 2 is used to cache data packets returned by storage unit 102_2, and shared buffer 3 is used to cache data packets returned by storage unit 102_3.

[0138] For example, the shared buffer can be a shared buffer in a graphics processor or a general-purpose graphics processor, and it is not limited to a first-in, first-out (FIFO) mode. Returned data packets can be stored in the shared buffer in any order.

[0139] Taking the multiplexer 103_1 as an example, when the multiplexer 103_1 receives the data packet sent by the storage unit 102_1, it stores the received data packet in the shared buffer 1, and stores the storage address of the received data packet in the shared buffer 1 as the data information of the data packet in the first-in-first-out buffer module FIFO0_0 (assuming that the destination processing core is processing core 1).

[0140] Since the shared buffer is not a first-in-first-out mechanism, if two data packets are stored in the shared buffer in sequence and are sent to different processing cores, the two data packets can participate in arbitration for returning to the respective processing cores in parallel. The data packet that arrives later does not need to wait for the transmission of the data packet that arrives earlier to complete before participating in arbitration. Thus, the serial data returned from the same storage unit can participate in arbitration for returning to different processing cores in parallel, thereby alleviating arbitration conflicts between the returns to the respective processing cores.

[0141] In some embodiments, when the first-in-first-out cache module is not empty, the data packet corresponding to the data information that enters the first-in-first-out cache module first can participate in arbitration, that is, the storage unit that sends the data packet can participate in arbitration as an arbitration candidate.

[0142] In other embodiments, as shown in Figure 4 the data loading device also provides a credit mechanism to control arbitration.

[0143] As shown in Figure 4 each multiplexer is also provided with a corresponding credit counter. The current value of the credit counter is used to indicate the number of data packets that can be buffered in the buffer area of the port to which the multiplexer is coupled. For example, the multiplexer 104_1 corresponds to the credit counter 1, the multiplexer 104_2 corresponds to the credit counter 2, and the multiplexer 104_3 corresponds to the credit counter 3.

[0144] The credit counter adjusts the count value in units of data bus data transmission units. For example, the credit counter is decremented by 1 for each data packet transmitted, and the credit counter is incremented by 1 for each receive response returned by the buffer area of the port.

[0145] Taking the multiplexer 104_1 as an example, assume that the current three storage units all have data packets that need to participate in arbitration, and the three first-in-first-out cache modules corresponding to the processing core 1, i.e., FIFO0_1, FIFO1_1, and FIFO2_1, are not empty. In addition, the current value of the credit counter 1 corresponding to the multiplexer 104_1 is 3, the data length information carried by the data packet sent by the storage unit 102_1 indicates that the requested data length is 2, the data length information carried by the data packet sent by the storage unit 102_2 indicates that the requested data length is 1, and the data length information carried by the data packet sent by the storage unit 102_3 indicates that the requested data length is 4.

[0146] Therefore, since the number of data packets included in the requested data of the storage unit 102_1 and the storage unit 102_2 is less than 4, the storage unit 102_1 and the storage unit 102_2 can be arbitration candidates for final arbitration, while the storage unit 102_3 continues to wait.

[0147] For example, the multiplexer 104_1 is configured to select one arbitration candidate from the two arbitration candidates based on an arbitration mechanism, and output the indication information corresponding to the arbitration candidate, for example, output the index value of the storage unit 102_1, indicating that the storage unit 102_1 is selected.

[0148] The data loading apparatus performs two read operations according to the index value, reads and sends the request data including two data packets from the shared buffer area to the buffer area of the port.

[0149] Specifically, in the first read operation, the data information currently entered first in the first-in-first-out buffer module FIFO0_1 corresponding to the processing core 1 / port 1 is read out (POP) from the three first-in-first-out buffer modules corresponding to the multiplexer 103_1, the data packet 1 is read from the shared buffer area 1 according to the data information, and sent to the buffer area corresponding to the port 1, and the value of the credit counter 1 is reduced by 1, becoming 3.

[0150] In the second read operation, the data information currently entered first in the first-in-first-out buffer module FIFO0_1 corresponding to the processing core 1 / port 1 is read out (POP), the data packet 2 is read from the shared buffer area 1 according to the data information, and sent to the buffer area corresponding to the port 1, and the value of the credit counter 1 is reduced by 1, becoming 2.

[0151] After the port sends the data packet 1 or the data packet 2 through the on-chip bus, a receive response is returned to the crossbar circuit, and the value of the credit counter 1 is increased by 1 after the crossbar circuit receives one receive response.

[0152] In this embodiment, a crossbar circuit architecture based on a credit mechanism is established, which effectively limits the data return flow of the on-chip network bus and relieves the congestion of the on-chip network bus; on this basis, the credit value addition and subtraction process based on the beat granularity is optimized, which effectively reduces the cache area of the upstream processing core / on-chip network bus; through the optimization of the transmission order of the storage unit returned to different ports, the arbitration conflict of multiple storage units returned to the same port / processing core at the same time is effectively relieved, and the data transmission performance in the crossbar circuit is improved; the shared cache is used to realize the parallel processing of serial data, the transmission time of the data from the storage unit to the processing core is compressed, and the data length information is added in the data packet, which reduces the occupancy time of the data in the shared buffer area, and effectively improves the performance of the data parallel processing.

[0153] The electronic device provided in at least one embodiment of the present disclosure has the advantages that the data loading apparatus provided in at least one embodiment of the present disclosure can effectively limit the data return flow of the on-chip network bus and relieve the congestion of the on-chip network bus, effectively reduce the cache area of the upstream processing core / on-chip network bus, effectively relieve the arbitration conflict of multiple storage units returned to the same port / processing core at the same time, and improve the data transmission performance in the crossbar circuit. Figure 5 An illustrative structural diagram of the electronic device provided in at least one embodiment of the present disclosure is shown in FIG. 2.

[0154] As shown in FIG. 2, the electronic device 200 includes the data loading apparatus 100 as described in any of the above embodiments. Figure 5 The electronic device 200 includes the data loading apparatus 100 as described in any of the above embodiments.

[0155] The specific content of the data loading device 100 can refer to the related description of the foregoing embodiments, which will not be repeated here.

[0156] For example, the electronic device can be implemented as any circuit requiring a data loading device. For example, the circuit can include one circuit board or a combination of multiple circuit boards. In the embodiments of the present application, the one circuit board or the combination of multiple circuit boards can include: (1) one or more processors, which can be general-purpose processors (such as ARM architecture processors or x86 architecture processors) or special-purpose processors (such as digital signal processors DSP or microcontrollers MCU); (2) one or more non-transitory memories connected to the processor, which can include volatile memories (such as dynamic random access memories DRAM or static random access memories SRAM) and non-volatile memories (such as flash memories Flash or read-only memories ROM), wherein the non-volatile memories can store configuration parameters (such as an interrupt priority table and an interrupt mask) of the data loading device in addition to storing firmware executable by the processor, and when the electronic device is powered on, the processor can first read the configuration parameters from the non-volatile memories and issue them to the data loading device to complete the initialization setting of the circuit; (3) firmware stored in the memory and executable by the processor, which can include control logic of core functions of the device and can also integrate processing subprograms (such as interrupt service functions ISR) of interrupt events, and when the processor receives an interrupt event transmitted by the data loading device through an interrupt controller, the corresponding interrupt service function can be quickly called to perform a preset operation (such as starting an alarm, recording an interrupt log, or adjusting a module working state).

[0157] For example, the digital circuit can include a large-scale integrated circuit. The data loading device can be integrated as an independent functional module inside the large-scale integrated circuit — it can be packaged together with processors, memories, and other modules as a multi-chip module (MCM), or it can be monolithically integrated with other functional modules (such as data acquisition modules or communication modules).

[0158] For example, the electronic device can include a system on chip (SOC) and the like, which can integrate the data loading device, the processor core, the memory, the peripheral interface (such as a UART interface, an SPI interface, and a GPIO interface), and other modules into a single chip to form a highly integrated hardware architecture.

[0159] For example, the electronic device can include a central processor, a graphics processor, a tensor processor, etc. large-scale integrated circuits with data processing capability and / or instruction execution capability, or the electronic device can also include a digital circuit or a digital circuit unit that implements a specific function, such as a field programmable gate array (FPGA), a complex programmable logic device (CPLD).

[0160] In addition, the electronic device can also include auxiliary circuit modules to ensure the stable operation of the data loading device. For example, a power management module can provide a stable working voltage for the data loading device; a clock module can provide an accurate clock signal for the data loading device; a reset module can generate a reset signal, and after receiving the reset signal, the data loading device can first clear the internal cache and configuration parameters, and then cooperate with the processor to complete the reset and restart of the device, to ensure that the interrupt reporting function can work normally after the device restarts.

[0161] The electronic device of the present disclosure is not limited in function, form, or use, and can include the data loading device of any embodiment of the present disclosure.

[0162] Figure 5 The specific structure diagram of the electronic device provided by at least one embodiment of the present disclosure is shown.

[0163] The following refers to Figure 5 which shows a specific structure diagram of an electronic device (for example, a terminal device or a server) 300 suitable for implementing a data loading device including embodiments of the present disclosure.

[0164] The electronic device in the embodiments of the present disclosure can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, as well as fixed terminals such as digital TVs, desktop computers, and the like. For example, the electronic device can be in the form of a server, which is used for deep learning and artificial intelligence, scientific computing, graphics rendering and video editing, virtual reality and game development, cloud services, and various application scenarios, for example, the electronic device can be a data center, cloud computing, etc. Special server deployed with deep learning training, large-scale data analysis, high-performance computing, etc.

[0165] Figure 5 The electronic device shown is only an example and should not limit the function and use range of the embodiments of the present disclosure.

[0166] As Figure 5As shown, the electronic device 300 may include a processing unit 301, such as the aforementioned processor 200, which can execute various appropriate actions and processes according to non-transitory computer-readable instructions stored in memory to achieve various functions. The processing unit 301 may also include devices with instruction optimization capabilities and / or program execution capabilities, such as a central processing unit (CPU) or a tensor processor (TPU). The CPU can be based on x86, ARM, or RISC-V architectures. The GPU can be directly integrated into the SOC, directly integrated onto the motherboard, or built into the northbridge chip of the motherboard.

[0167] like Figure 6 As shown, for example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory, etc., for example, computer-readable instructions may be loaded from storage device 308 into RAM 303 to execute computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0168] For example, the processing device 301, the read-only memory (ROM) 302, and the random access memory (RAM) 303 are interconnected via a bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.

[0169] Typically, the following devices can be connected to the input / output (I / O) interface 305: input devices 306 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 308 including, for example, magnetic tape, hard disk, flash memory, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 6 An electronic device 300 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 300 may alternatively implement or have more or fewer devices. For example, the processing device 301 may control other components in the electronic device 300 to perform desired functions.

[0170] For example, the electronic device provided by at least one embodiment of the present disclosure can be a general-purpose graphics processor. Figure 6 An exemplary structure diagram of a general-purpose graphics processor (GPGPU).

[0171] As shown in Figure 6 , the general-purpose graphics processor is actually an array of programmable multi-processors, for example, the programmable multi-processors can be streaming processor clusters (SPC), for example, including Figure 6 streaming processor cluster 1,..., streaming processor cluster M as shown in , M is a positive integer greater than 1. In the general-purpose graphics processor, one streaming processor cluster processes a computing task, or multiple streaming processor clusters process a computing task. Multiple streaming processor clusters share data through global cache or global memory.

[0172] As shown in Figure 6 , taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, for example Figure 6 computing unit 1, computing unit 2,..., computing unit N in , N is a positive integer. Each computing unit (CU) is used to perform arithmetic logic operations, such as accumulation, reduction, general addition, subtraction, multiplication, division, etc. A computing unit includes multiple computing cores, each computing core includes an arithmetic logic unit (ALU), a floating point computing unit, etc., and the computing core is used to perform specific computing tasks. In addition, the computing unit also includes a register (for example Figure 6 register stack) and shared memory, which are used to store source data and destination data related to the computing task in layers, and the shared memory in one computing unit is used to share data between the cores in the computing unit.

[0173] As shown in Figure 6 , each computing unit is also provided with a tensor core, which is used to perform tensor-related calculations, for example, related tensor operations such as tensor contraction operations, and the tensor core can accelerate tensor operations such as matrix multiplication. The tensor cores in multiple computing units can be uniformly scheduled and controlled.

[0174] As shown in Figure 6 , each streaming processor cluster is also provided with a buffer (Buffer) for data caching of N computing units in the streaming processor cluster.

[0175] In parallel computing, a computing task is generally executed by multiple threads. These threads are divided into multiple thread blocks before being executed in a general-purpose graphics processor (or parallel computing processor), and then the multiple thread blocks are distributed to each computing unit via a thread block distribution module (not shown in Figure 6 ). All threads in a thread block must be assigned to the same computing unit for execution. Meanwhile, a thread block is split into minimum execution thread bundles (or simply referred to as thread bundles, warp), each of which contains a fixed number (or less than the fixed number) of threads, for example, 32 threads. Multiple thread blocks can be executed in the same computing unit or in different computing units.

[0176] In each computing unit, a thread bundle scheduling / distribution module (not shown in Figure 6 ) schedules and assigns thread bundles so that multiple computing cores of the computing unit run the thread bundles. According to the number of computing cores in the computing unit, multiple thread bundles in a thread block can be executed simultaneously or in time-sharing manner. Multiple threads in each thread bundle execute the same instruction. Memory execution instructions are transmitted to a shared memory in the computing unit or further transmitted to an intermediate-level cache or a global cache or a global memory (for example, a high bandwidth memory, HBM, in Figure 6 ) for read / write operations, etc.

[0177] In this embodiment, the storage component in the data loading device includes a memory (for example, an HBM in ​ ) in a general-purpose graphics processor or a second-level cache, and the processing core includes a computing core, a tensor core, etc. in the general-purpose graphics processor that uses data for processing or computing.

[0178] For example, if the crossbar circuit is further provided with N shared cache areas corresponding to the N demultipulators one by one, the shared cache areas can be implemented using a shared memory in ​ .

[0179] For the present disclosure, the following points need to be explained:

[0180] The above description is only the preferred embodiment of the present disclosure and the explanation of the applied technical principles. Those skilled in the art should understand that the disclosure range involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions.

[0181] Moreover, while operations are depicted in a particular, chronological sequence in this disclosure, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, at all. To the extent that specific steps are described in a particular order, alternate implementations can perform the operations in different orders, or employ the steps in parallel or concurrently. Additionally, certain features can also be implemented as processes - in pieces of code - which execute inside one or more physical hardware devices (e.g., servers, computers, etc.). Yet, moreover, while specific implementation details are contained in the above discussion, these should not be construed as limiting the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0182] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0183] For the disclosure, the following points also need to be explained:

[0184] (1) The drawings of the embodiments of the disclosure only involve the structures related to the embodiments of the disclosure, and other structures can refer to the general design.

[0185] (2) In the case of no conflict, the embodiments of the disclosure and the features in the embodiments can be combined to obtain new embodiments.

[0186] The above merely provides the specific implementation of the disclosure, but the protection scope of the disclosure is not limited thereto, and the protection scope of the disclosure should be subject to the protection scope of the claims.

Claims

1. A data loading apparatus, characterized by comprising: The data loading device comprises a crossbar circuit, N storage units in a storage component, N being a positive integer greater than 1; The storage component is configured to store data; The crossbar circuit is configured to load data in the storage component to corresponding processing cores for calculation or processing; The crossbar circuit comprises: N demultiplexers, each demultiplexer comprising one input end and N output ends, the input ends of the N demultiplexers being coupled to the N storage units one by one; N multiplexers, each multiplexer comprising N input ends and one output end, the output ends of the N multiplexers being coupled to N processing cores one by one; The N output ends of each demultiplexer are coupled to one input end of each multiplexer in the N multiplexers; Each storage unit is configured to send request data to the demultiplexer coupled to the storage unit in the same preset return order, wherein the request data comprises at least one data packet, each request data returns a destination processing core, and the preset return order indicates the arrangement order of the destination processing cores of the request data sent by the N storage units in turn; In the crossbar circuit, N first-in-first-out cache modules corresponding to the N processing cores are provided for each demultiplexer, and each first-in-first-out cache module is configured to cache data information of each data packet sent by the storage unit coupled to the demultiplexer in the received order, and the destination processing core of each data packet cached by the first-in-first-out cache module is the processing core corresponding to the first-in-first-out cache module.

2. The data loading device of claim 1, wherein, Each storage unit is provided with a cache area for caching request data read from the storage unit but not sent, and the request data cached in the cache area is arranged in the read order; The storage unit is configured to select request data from the cache area in the preset return order and send to the coupled demultiplexer in turn.

3. The data loading device of claim 1, wherein The crossbar circuit is further provided with N shared cache areas corresponding to the N demultiplexers; Each shared cache area is configured to cache data packets returned by the storage unit coupled to the corresponding demultiplexer, and the shared cache area supports non-continuous storage between the returned data packets.

4. The data loading device of claim 3, wherein, In response to M data packets cached in the shared cache area being sent to different processing cores, the M data packets sent to different M processing cores participate in arbitration of data returned to the M processing cores in parallel, M being a positive integer less than N.

5. The data loading device of claim 4, wherein, The M data packets comprise a first data packet and a second data packet, and the M processing cores comprise a first processing core and a second processing core, The first data packet and the second data packet are returned from the storage unit in turn and cached in the shared cache area, the first data packet being returned before the second data packet, The destination processing core of the first data packet is the first processing core, and the destination processing core of the second data packet is the second processing core, After the first data packet is cached to the shared buffer area, the first data packet participates in arbitration of data returning to the first processing core, and after the second data packet is cached to the shared buffer area, the second data packet directly participates in arbitration of data returning to the second processing core.

6. The data loading device of claim 3, wherein, The crossbar circuit is configured to, in response to the multiplexer receiving a data packet sent by a coupled storage unit, store the received data packet to a shared buffer area corresponding to the multiplexer, and store a storage address of the received data packet in the shared buffer area as data information of the received data packet to a first-in-first-out buffer module corresponding to a destination processing core of the received data packet.

7. The data loading device of claim 1, wherein Each multiplexer is configured to select one arbitration candidate from at least one arbitration candidate based on an arbitration mechanism, and output indication information corresponding to the arbitration candidate, the indication information indicating the selected arbitration candidate, Wherein, the storage unit satisfying the preset condition is taken as an arbitration candidate to participate in arbitration, and the preset condition includes that, in N first-in-first-out buffer modules corresponding to a multiplexer of a multiplexer coupled to the storage unit, a first-in-first-out buffer module corresponding to a processing core coupled to the multiplexer is not empty.

8. The data loading device of claim 7, wherein, The data loading device is further configured to: Based on the indication information output by the multiplexer, perform p times of reading operations to send requested data requested by one data loading request to a buffer area corresponding to a processing core coupled to the multiplexer, wherein the requested data requested by the one data loading request includes p data packets, each reading operation is configured to transmit one data packet in the p data packets to the buffer area corresponding to the processing core coupled to the multiplexer, and p is a positive integer.

9. The data loading device of claim 8, wherein, The crossbar circuit is further provided with N shared buffer areas corresponding to the N multiplexers; The data loading device includes the following operations when performing one reading operation: Based on the indication information output by the multiplexer, determine a target storage unit selected; From N first-in-first-out buffer modules set by a multiplexer coupled to the target storage unit, determine a target first-in-first-out buffer module corresponding to a processing core coupled to the multiplexer; Read a read address first entered from the target first-in-first-out buffer module; According to the read address, read a target data packet from a shared buffer area corresponding to a multiplexer coupled to the target storage unit, and send the target data packet to the buffer area.

10. The data loading device of claim 7, wherein, Each multiplexer is further provided with a corresponding credit counter, and a current value of the credit counter is used to indicate a number of data packets that can be cached in a buffer area corresponding to a processing core coupled to the multiplexer, The preset condition further includes: The number of data packets included in the target requested data is less than or equal to the current value of the credit counter, wherein a data packet corresponding to first-entered data information in the first-in-first-out buffer module corresponding to the processing core coupled to the multiplexer belongs to the target requested data.

11. The data loading device of claim 10, wherein, The crossbar circuit is further provided with N shared buffer areas corresponding to the N multiplexers, The number of data packets included in the target request data is determined in the following manner: The number of data packets included in the target request data is determined based on data length information of the target request data carried by the data packets; or A plurality of data packets received are buffered in a shared buffer area corresponding to a multiplexer to which the storage unit sending the target request data is coupled, and the number of data packets included in the target request data is determined based on identification information carried by each data packet in the received plurality of data packets, wherein the identification information is used to indicate whether the current data packet is the last data packet in the target request data.

12. The data loading device of claim 10, wherein, The data length of the data packet is the bandwidth of a data bus from the storage unit to the N multiplexers; The target request data includes at least one data packet, and the at least one data packet is transmitted continuously; In response to the at least one data packet carrying the data length information of the target request data, the data information of the first data packet of the target request data having entered a corresponding first-in-first-out buffer module, and the storage unit sending the first data packet satisfying the preset condition, the storage unit sending the target request data participates in arbitration of a target processing core of the target request data as an arbitration candidate.

13. The data loading device of claim 1, wherein, Each multiplexer is further provided with a corresponding credit counter, a current value of the credit counter being used to indicate the number of data packets that can be buffered in a buffer area corresponding to a processing core to which the multiplexer is coupled, The credit counter adjusts the count value in units of data transmission units of a data bus, and the data length transmitted by the data transmission unit is the data length of a single data packet.

14. The data loading device of claim 13, wherein, The credit counter is configured to: in response to one data packet being sent to a buffer area corresponding to a processing core to which the multiplexer is coupled, the value of the credit counter is decremented by 1, and in response to receiving a reception response returned by the buffer area, the value of the credit counter is incremented by 1. Wherein, the requested request data requested by one data loading request includes p data packets, after the p data packets are all returned to the buffer area corresponding to the processing core to which the multiplexer is coupled, the multiplexer performs the next round of arbitration, and after the requested request data is returned, the value of the credit counter is decremented by p, and p is a positive integer.

15. The data loading device of any of claims 1-14, wherein, The output ends of the N multiplexers are one-to-one coupled with N ports of a network-on-chip bus, and the N multiplexers are coupled with the N processing cores through the network-on-chip bus, and the network-on-chip bus is used to transmit data received from the storage component through the crossbar circuit to the corresponding processing core.

16. An electronic device, comprising: The data loading device includes the data loading device of any one of claims 1-15.

17. The electronic device of claim 16, wherein, The electronic device includes a graphics processor or a general-purpose graphics processor, the storage component includes a memory or a second cache in the graphics processor or the general-purpose graphics processor, and the processing core includes computation in the graphics processor or the general-purpose graphics processor.

Images