Data processing device, data processing method, and electronic device

Abstract

A data processing apparatus, a data processing method, and an electronic device are disclosed. The data processing apparatus includes multiple computing units, an arbitration unit, a first daisy chain, a second daisy chain, and a third daisy chain. The command queues of each computing unit are cascaded through the first daisy chain and connected to the arbitration unit, and are also cascaded through the second daisy chain and connected to the arbitration unit. The first daisy chain is used to send data migration requests issued by each of the multiple computing units to the arbitration unit. The arbitration unit receives the data migration requests and arbitrates them to select the data migration request to be processed, thereby generating an arbitration result. The second daisy chain is used to return the arbitration result to the command queue of the computing unit that issued the selected data migration request. Vector memories in the multiple computing units are cascaded through the third daisy chain, and the third daisy chain is used to transfer data between the vector memories in the multiple computing units. This data processing apparatus can save bandwidth consumption and improve the performance of parallel processors.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a data processing apparatus, a data processing method, and an electronic device. Background Technology

[0002] With the development of artificial intelligence technology, higher demands are being placed on the bandwidth and computing power of parallel processors. However, due to limitations in current processes and materials, bandwidth and computing power have significant physical limits. For tasks with high data repetition rates, such as matrix operations, increasing data reuse can effectively reduce bandwidth usage and thus significantly improve computational efficiency.

[0003] A typical parallel processor consists of multiple computing units, each containing multiple processing elements. Each processing element performs computations on data, and these elements interact with each other via vector memory. However, there is no direct data exchange path between computing units. Therefore, if interaction is required, one computing unit must first store the data in a cache, and then the other unit must read it from the cache. The cache, as the primary medium for interaction, handles a large amount of data transfer. Therefore, if data interaction between computing units, especially between the vector memories within each computing unit, could be achieved without going through the cache, the degree of data reuse could be improved, thereby enhancing the computational performance of the parallel processor without increasing the size of the vector memory. Summary of the Invention

[0004] At least one embodiment of this disclosure provides a data processing apparatus, including a plurality of computing units, an arbitration unit, a first daisy chain, a second daisy chain, and a third daisy chain. Each of the plurality of computing units includes a command queue and a vector memory. The command queues of each computing unit are cascaded through the first daisy chain and connected to the arbitration unit, and are also cascaded through the second daisy chain and connected to the arbitration unit. The first daisy chain is used to send data migration requests issued by each of the plurality of computing units to the arbitration unit. The arbitration unit receives data migration requests issued by each of the plurality of computing units and arbitrates one or more pending data migration requests to select a data migration request for processing, thereby generating an arbitration result. The second daisy chain is used to return the arbitration result generated by the arbitration unit to the command queue of the computing unit that issued the selected data migration request. The vector memories in the plurality of computing units are cascaded through the third daisy chain, and the third daisy chain is used to transfer data between the vector memories in the plurality of computing units.

[0005] For example, in the data processing apparatus provided in at least one embodiment of this disclosure, each of the multiple computing units issues a data migration request including a source identifier, a corresponding workgroup identifier, the size of the data volume, a destination identifier, and data bias information.

[0006] For example, in the data processing apparatus provided in at least one embodiment of this disclosure, the arbitration result includes a source identifier, a destination identifier, the size of the data volume, and a corresponding working group identifier.

[0007] For example, at least one embodiment of the data processing apparatus provided in this disclosure further includes a repeater, and the vector memory in the plurality of computing units is connected to the repeater in a loop through a third daisy chain, wherein the third daisy chain is a unidirectional daisy chain.

[0008] For example, in a data processing apparatus provided in at least one embodiment of this disclosure, the arbitration unit includes: a buffer queue configured to receive and cache data migration requests issued by each of multiple computing units via a first daisy chain, and output arbitration results via a second daisy chain; a first submodule configured to convert each data migration request issued by the multiple computing units into a daisy chain loop; a second submodule configured to predict the current and future changes of the daisy chain loop to obtain a first prediction matrix; and multiple chain loop processing modules, each corresponding to an operable daisy chain loop in a third daisy chain, each configured to use a corresponding daisy chain... The system stores the current state of the corresponding data migration operation in the form of a ring, predicts the current and future changes of the corresponding daisy chain ring to obtain the corresponding second prediction matrix; and a third submodule is configured to receive the first prediction matrix, the current state of the daisy chain ring stored by each of the multiple chain ring processing modules, and the corresponding second prediction matrix, determine the occupancy status of the multiple chain ring processing modules based on the current state of the daisy chain ring stored by each of the multiple chain ring processing modules, determine whether the multiple chain ring processing modules can receive the daisy chain ring provided by the first submodule based on the first prediction matrix and the second prediction matrix of each of the multiple chain ring processing modules, and output the arbitration result.

[0009] For example, in the data processing apparatus provided in at least one embodiment of this disclosure, the daisy chain includes p registers, where p = the number of nodes in the third daisy chain and p is a positive integer. Each register corresponds to a node in the third daisy chain and is used to represent the working state of the corresponding node.

[0010] For example, in the data processing apparatus provided in at least one embodiment of this disclosure, the size of the first prediction matrix and the corresponding second prediction matrix are both s×p, where s represents the number of prediction periods, and the s rows of the first prediction matrix and the corresponding second prediction matrix are used to represent the data transmission status of p nodes within s periods starting from the current point.

[0011] For example, in the data processing device provided by at least one embodiment of the present disclosure, the first sub-module is further configured to convert the data migration requests issued by each of the multiple computing units into the form of a daisy chain based on the source identifier, destination identifier, and data volume size in the data migration requests issued by each of the multiple computing units.

[0012] For example, in the data processing device provided by at least one embodiment of the present disclosure, the source identifier = m, the destination identifier = n, the data volume size = k, and each node in the daisy chain is sequentially assigned a different node serial number CU_ID. When m < n, for m ≤ CU_ID ≤ n, the register corresponding to the node CU_ID is given an initial value of k + CU_ID; when m > n, for m ≤ CU_ID < p, the register corresponding to the node CU_ID is given an initial value of k + CU_ID, and for CU_ID ≤ n, the register corresponding to the node CU_ID is given an initial value of k + CU_ID + p.

[0013] For example, in the data processing device provided by at least one embodiment of the present disclosure, the value v in each register in the daisy chain is used to determine whether there is data in the corresponding node. When 0 < v ≤ k, there is data in the corresponding node, and when v > k, there is no data in the corresponding node.

[0014] For example, in the data processing device provided by at least one embodiment of the present disclosure, each of the multiple link processing modules includes: a storage sub-module configured to store the status of the corresponding data migration operation in the form of the corresponding daisy chain, and a prediction sub-module configured to predict the current and future changes of the corresponding daisy chain to obtain the corresponding second prediction matrix.

[0015] For example, in the data processing device provided by at least one embodiment of the present disclosure, the number of the multiple link processing modules is less than or equal to the maximum number of data migration operations that can be performed simultaneously in the third daisy chain.

[0016] For example, in the data processing device provided by at least one embodiment of the present disclosure, the third sub-module is further configured to determine the link processing modules in an idle state among the multiple link processing modules according to the occupancy of each of the multiple link processing modules.

[0017] For example, in the data processing apparatus provided in at least one embodiment of this disclosure, the third submodule is further configured to match the first prediction matrix and the multiple second prediction matrices corresponding to the multiple chain link processing modules respectively, based on the matching result indicating that the first prediction matrix does not conflict with the multiple second prediction matrices, and in the case that it is determined that there is an idle chain link processing module among the multiple chain link processing modules, the idle chain link processing module that can receive the daisy chain converted by the first submodule is determined, and an enable signal is sent to the determined idle chain link processing module. The multiple chain link processing modules are further configured to, based on the enable signal from the third submodule, enable the determined idle chain link processing module to receive and store the daisy chain from the first submodule.

[0018] At least one embodiment of this disclosure provides a data processing method applied to a data processing apparatus provided in at least one embodiment of this disclosure. Multiple computing units include a first computing unit and a second computing unit. The data processing method includes: a command queue in the first computing unit sending a first data migration request to an arbitration unit via a first daisy chain, and sending a first command to a vector memory in the first computing unit to notify the vector memory in the first computing unit to be ready to receive first data from a second computing unit; the arbitration unit, in response to receiving the first data migration request, arbitrating one or more pending data migration requests to select a data migration request for processing, generating a first arbitration result, and transmitting the first arbitration result to the command queue of the second computing unit via a second daisy chain; the command queue of the second computing unit receiving the first arbitration result and sending a second command to the vector memory of the second computing unit based on the first arbitration result; and the vector memory of the second computing unit, in response to receiving the command, transmitting the first data to the vector memory of the first computing unit via a third daisy chain.

[0019] At least one embodiment of this disclosure provides an electronic device, including a data processing device provided in at least one embodiment of this disclosure. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0021] Figure 1 A schematic diagram of a data processing device for parallel computing is shown.

[0022] Figure 2 A schematic diagram of the internal structure of a computing unit is shown;

[0023] Figure 3A schematic diagram of a data processing apparatus provided in at least one embodiment of the present disclosure is shown;

[0024] Figure 4A A schematic diagram of a data migration process is shown;

[0025] Figure 4B A schematic diagram of another data migration process is shown;

[0026] Figure 5 A schematic diagram of an arbitration unit provided in at least one embodiment of this disclosure is shown;

[0027] Figure 6 A schematic diagram of the structure of a daisy chain provided in at least one embodiment of the present disclosure is shown;

[0028] Figure 7 The graph shows the relationship between the number of simultaneous data migration operations and the maximum amount of data transferred.

[0029] Figure 8 This diagram illustrates the storage format of the data migration operation status within the storage submodule.

[0030] Figure 9 A schematic flowchart of a data processing method provided by at least one embodiment of the present disclosure is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0032] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0033] Figure 1 A schematic diagram of a data processing device for parallel computing is shown. For example, this data processing device is a parallel processor, such as a general-purpose graphics processing unit (GPGPU).

[0034] like Figure 1 As shown, the data processing apparatus includes multiple computing units (CUs) CU0 to CU15 and a cache (not shown). Each computing unit interacts with the cache via a cache input / output interface (cache_IO) (for example, CU0 interacts with the cache via cache_IO0). In this data processing apparatus, there is no direct data interaction path between one computing unit and another. Therefore, if interaction is required, one computing unit must first store the data in the cache, and then another computing unit must read it from the cache.

[0035] For example, in the aforementioned data processing apparatus, the smallest unit for a predetermined task to be run by the computing unit is called a thread bundle. For example, each thread bundle (also known as a "work group") contains 32 threads or 64 threads (also known as "work items"), and the thread bundles running within the computing unit are independent of each other.

[0036] Figure 2 A method for Figure 1 A schematic diagram of the internal structure of the computing unit 200 of the data processing device. (See diagram below.) Figure 2As shown, the computing unit 200 includes processing elements 201 and 202, a vector memory 203, and a command queue 204. Processing elements 201 and 202 each perform calculations on vector data, and each contains numerous vector registers for calculations (e.g., floating-point multiplication and division, complex calculations using trigonometric functions such as cosine and sin). The vector memory 203 handles data storage and interaction between processing elements 201 and 202, for example, through interfaces PE0idx / data and PE1idx / data. The command queue 204 controls the operation of processing elements 201 and 202 and the vector memory 203 based on specific programs, for example, through interfaces PE0_cmd, VM_cmd, and PE1_cmd to facilitate interaction and communication between the command queue 204 and corresponding devices for control. The vector data of the computing unit 200 interacts with the outside world through cache_IO0.

[0037] Figure 1 The structure of the data processing device shown can handle the computation of independent vector data very well. In this data processing device, the computation between computing units is highly independent. However, with the increasing bandwidth requirements of existing applications, such completely independent computing units are not conducive to the reuse of data.

[0038] At least one embodiment of this disclosure provides a data processing apparatus, including a plurality of computing units, an arbitration unit, a first daisy chain, a second daisy chain, and a third daisy chain; each of the plurality of computing units includes a command queue and a vector memory; the command queues of each computing unit are cascaded through the first daisy chain and connected to the arbitration unit, and are also cascaded through the second daisy chain and connected to the arbitration unit; the first daisy chain is used to send data migration requests issued by each of the plurality of computing units to the arbitration unit; the arbitration unit receives data migration requests issued by each of the plurality of computing units and arbitrates one or more pending data migration requests to select a data migration request for processing, thereby generating an arbitration result; the second daisy chain is used to return the arbitration result generated by the arbitration unit to the command queue of the computing unit that issued the selected data migration request; the vector memories in the plurality of computing units are cascaded through the third daisy chain, and the third daisy chain is used to transfer data between the vector memories in the plurality of computing units.

[0039] The data processing apparatus provided in the above embodiments of this disclosure can effectively expand the storage space for a workgroup by sharing the storage resources of multiple computing units without increasing the storage area of ​​the vector memory. For parallel computing with high spatial dependence, it can effectively save bandwidth consumption and improve the performance of parallel processors.

[0040] In this disclosure, a daisy chain is a method of transmitting signals along a bus, in which devices in the chain are cascaded, and signals are transmitted from one device to the next; for example, the daisy chain can be connected end to end to form a daisy chain loop.

[0041] At least one embodiment of this disclosure also provides a data processing method applied to the above-described data processing apparatus and an electronic device including the above-described data processing apparatus.

[0042] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0043] Figure 3 A schematic diagram of a data processing apparatus 300 provided in at least one embodiment of the present disclosure is shown. For example, the data processing apparatus is a parallel processor, such as a general-purpose graphics processing unit (GPGPU), a data processor, a tensor processor, etc.

[0044] like Figure 3 As shown, the data processing device 300 includes multiple computing units CU0 to CU15 (16 units are used as an example here), an arbitration unit 301, a first daisy chain cq_req_daisy, a second daisy chain cq_arb_daisy, and a third daisy chain VM_DMA_daisy. Each of the multiple computing units CU0 to CU15 includes a command queue and a vector memory (for example, computing unit CU0 includes a command queue CQ0 and a vector memory VM0). The command queues of each computing unit in the multiple computing units CU0 to CU15 are cascaded through the first daisy chain cq_req_daisy and connected to the arbitration unit 301, and are also cascaded through the second daisy chain cq_arb_daisy and connected to the arbitration unit 301.

[0045] The first daisy chain, cq_req_daisy, is used to send data migration requests from multiple computing units to arbitration unit 301. For example... Figure 3 In the example shown, the first daisy chain cq_req_daisy is a unidirectional chain from computing unit CU15 to computing unit CU0. Therefore, data migration requests issued by the nodes (computing units) on the first daisy chain will also be passed along this direction one by one, and eventually reach arbitration unit 301.

[0046] For example, in some embodiments of this disclosure, each data migration request issued by multiple computing units includes a source identifier (e.g., the serial number of the computing unit sending data during data migration, which is also the serial number of the computing unit issuing the data migration request), a corresponding workgroup identifier, the size of the data volume, a destination identifier (e.g., the serial number of the computing unit receiving data during data migration), and data bias information.

[0047] Arbitration unit 301 receives data migration requests from each of multiple computing units and arbitrates one or more pending data migration requests to select the data migration request to be processed, thereby generating an arbitration result. Arbitration unit 301 may cache the received one or more data migration requests based on its processing capacity and system resources (described below), and then, for example, arbitrate them sequentially to produce an arbitration result.

[0048] For example, in some embodiments of this disclosure, the arbitration result includes the source identifier, the destination identifier, the size of the data volume, and the corresponding working group identifier.

[0049] The second daisy chain, cq_arb_daisy, is used to return the arbitration result generated by arbitration unit 301 to the command queue of the computing unit that issued the selected data migration request. For example... Figure 3 In the example shown, the second daisy chain cq_arb_daisy is a unidirectional chain from computing unit CU0 to computing unit CU15. Therefore, when arbitration unit 301 issues an arbitration result, the arbitration results received by the nodes (computing units) on the second daisy chain will also be passed along this direction one by one, and eventually reach the computing unit that issued the selected data migration request.

[0050] For example, the computing unit that issues the selected data migration request is computing unit CU3. The source identifier in the arbitration result issued by arbitration unit 301 points to computing unit CU3. When the arbitration result arrives in computing units CU0, CU1, and CU2 in sequence, the command queues in computing units CU0, CU1, and CU2 can know from the source identifier in the arbitration result that their own computing unit has not issued the corresponding data migration request. Therefore, they forward the arbitration result directly. When the arbitration result arrives in computing unit CU3, the command queue in computing unit CU3 can know from the source identifier in the arbitration result that its own computing unit has issued the corresponding data migration request. Therefore, it responds to the arbitration result and no longer forwards the arbitration result along the second daisy chain.

[0051] In addition, the same computing unit may issue different data migration requests at different times. Although these different data migration requests have the same source identifier, they may have different destination identifiers (e.g., sending data to different computing units), different corresponding workgroup identifiers (e.g., corresponding to different computing tasks), different data volumes (e.g., corresponding to different batches of data), or different data bias information (i.e., corresponding to different positions in the same batch).

[0052] Vector memories in multiple computing units are cascaded through a third daisy chain VM_DMA_daisy, and the third daisy chain VM_DMA_daisy is used to transfer data between vector memories in multiple computing units.

[0053] For example, such as Figure 3 As shown, the data processing device 300 also includes a repeater 302. The vector memory in the multiple computing units is connected to the repeater 302 through a third daisy chain VM_DMA_daisy to form a daisy chain loop. The third daisy chain is a unidirectional daisy chain.

[0054] Since the third daisy chain VM_DMA_daisy is circular, a repeater 302 is added to account for the physical distance between compute unit CU0 and compute unit CU15. For example, the repeater 302 includes one or more register units (e.g., shift registers). The number of register units in the repeater 302 may be equal to the number of compute units (16 in this example), but this disclosure does not limit the type and number of repeaters 302; for example, it may be less than or more than the number of compute units.

[0055] Because the third daisy chain is a unidirectional daisy chain, if the source identifier in the data migration request is less than the destination identifier, the migrated data does not need to pass through the relay. Figure 4A As shown in the diagram, when the source identifier in a data migration request is greater than the destination identifier, the migrated data needs to pass through a repeater. This situation occurs in... Figure 4B As shown in the image.

[0056] Figure 4A A schematic diagram of data migration from vector memory VM0 to vector memory VM3 is shown.

[0057] like Figure 4A As shown, the migrated data travels from vector memory VM0 along the third daisy chain VM_DMA_daisy through vector memories VM1 and VM2 to VM3. Nodes with gray shading represent the nodes through which the data passes.

[0058] Figure 4B A schematic diagram of data migration from vector memory VM3 to vector memory VM0 is shown.

[0059] like Figure 4BAs shown, the migrated data travels from vector memory VM3 along the third daisy chain VM_DMA_daisy, passing through vector memories VM4, VM5, ..., VM15, and the repeater before reaching VM0. The data migration from VM3 to VM0 involves a large number of nodes and repeaters, far exceeding the number of nodes traversed by the data migration from VM0 to VM3. Nodes with gray shading represent the nodes through which the data has passed.

[0060] The data processing apparatus provided in the above embodiments can effectively expand the storage space for the workgroup by sharing the storage resources of multiple computing units without increasing the storage area of ​​the vector memory. For parallel computing with high spatial dependence, it can effectively save bandwidth consumption and improve the performance of parallel processors.

[0061] Figure 5 A schematic diagram of an arbitration unit 500 provided in at least one embodiment of the present disclosure is shown.

[0062] like Figure 5 As shown, the arbitration unit 500 includes a buffer queue 501, a first submodule 502, a second submodule 503, multiple chain processing modules ring0 to ring7, and a third submodule 504. These queues or modules can be implemented using digital or analog circuits.

[0063] Buffer queue 501 is configured to receive and cache data migration requests issued by multiple computing units via a first daisy chain, and output arbitration results via a second daisy chain. For example, buffer queue 501 is implemented as a first-in-first-out (FIFO) queue.

[0064] For example, buffer queue 501 receives data migration requests from the first daisy chain cq_req_daisy and caches them sequentially, and outputs the arbitration result through the second daisy chain cq_arb_daisy.

[0065] The first submodule 502 is configured to convert data migration requests issued by each of the multiple computing units into a daisy chain.

[0066] For example, such as Figure 5 As shown, the first submodule 502 transforms each data migration request issued by multiple computing units into a daisy-chain req_ring form based on the source identifier src_id, destination identifier dst_id, and data size in each data migration request (see, for example, below). Figure 6 ).

[0067] For example, in some embodiments of the present disclosure, the daisy chain includes p registers (p is a positive integer), where p = the number of nodes in the third daisy chain. The third daisy chain includes actual nodes, i.e., computing units CU0 - CU15, and also includes virtual nodes represented by repeaters. The number of virtual nodes can be, for example, greater than or equal to the number of actual nodes. Each register corresponds to a node (i.e., an actual node or a virtual node) in the third daisy chain and is used to represent the working state of the corresponding node. Each node in the daisy chain is sequentially assigned a different node serial number CU_ID. For example, the computing unit CU0 in the daisy chain is assigned the node serial number CU_ID = 0.

[0068] Figure 6 FIG. shows a schematic structural diagram of a daisy chain provided by at least one embodiment of the present disclosure.

[0069] As Figure 6 shown, in this embodiment, the daisy chain includes 32 registers. Each register corresponds to a node in the third daisy chain and represents the working state of the corresponding node. For example, the registers with node serial numbers CU_ID in the range of 0 to 15 correspond to the actual nodes represented by 16 computing units, and the registers with node serial numbers in the range of 16 to 31 correspond to 16 virtual nodes represented by repeaters. The initial values of all registers inside the daisy chain are 0, which indicates that the corresponding nodes have not started working.

[0070] It should be noted that the present disclosure is not limited to Figure 6 the daisy chain structure shown. The number of registers included in the daisy chain and the corresponding relationship between each register and the node can be determined according to actual situations.

[0071] Returning to Figure 5 , in the process of converting the data migration request into the form of a daisy chain, the first sub-module 502 assigns initial values to the registers in the daisy chain.

[0072] For example, in some embodiments of the present disclosure, the source identifier = m, the destination identifier = n, and the size of the data volume = k. In the case of m < n, for m ≤ CU_ID ≤ n, the register corresponding to the node CU_ID is assigned the initial value k + CU_ID; in the case of m > n, for m ≤ CU_ID < p, the register corresponding to the node CU_ID is assigned the initial value k + CU_ID, and for CU_ID ≤ n, the register corresponding to the node CU_ID is assigned the initial value k + CU_ID + p. The case of m = n represents data transmission within the computing unit and does not involve data migration between computing units, so it does not need to be considered at this time.

[0073] For example, in some embodiments of this disclosure, the value v in each register of the daisy chain is used to determine whether there is data in the corresponding node. If 0 < v ≤ k, there is data in the corresponding node. If v > k, it indicates that the data is still in the process of transmission and has not yet reached the corresponding node, and there is no data in the corresponding node.

[0074] The second submodule 503 is configured to predict the current and future changes of the daisy chain to obtain the first prediction matrix.

[0075] For example, in some embodiments of this disclosure, the size of the first prediction matrix is ​​s×p (i.e., s rows and p columns, where p is the number of nodes), s represents the number of prediction periods, and the s rows of the first prediction matrix are used to represent the data transmission situation in p nodes within s periods starting from the current time, i.e., each row corresponds to one period.

[0076] For the value of s, for example, it can be considered as follows: When the arbitration result is sent from the arbitration unit to the corresponding computing unit along the second daisy chain, without considering the internal latency of the computing unit, the data then travels from the vector memory inside that computing unit to the vector memory inside the destination computing unit. For Figure 3 The data processing device shown has a maximum transmission time of 16 + 32 = 48 cycles, where the maximum transmission time for the arbitration result is 16 cycles and the maximum transmission time for the data is 32 cycles. Therefore, in this example, s takes the value 48.

[0077] For example, the first prediction matrix req_array is obtained by predicting the current and future changes of the daisy chain over 48 cycles. The first prediction matrix req_array is a 48×32 matrix, which represents the data transmission status of 32 nodes within 48 cycles starting from the current time.

[0078] Figure 5 The multiple chain processing modules ring0 to ring7 shown correspond to the operable chain links in the third daisy chain. Each of the multiple chain processing modules is configured to store the current state of the corresponding data migration operation in the form of the corresponding daisy chain, and to predict the current and future changes of the corresponding daisy chain to obtain the corresponding second prediction matrix.

[0079] In this example, there are 8 chain processing modules. It should be noted that the number of chain processing modules is determined by the maximum number of data migration operations that can be performed simultaneously in the third daisy chain, and this disclosure does not impose a limit on the number of chain processing modules.

[0080] For example, in some embodiments of this disclosure, the number of multiple chain processing modules is less than or equal to the maximum number of data migration operations that can be performed simultaneously in the third daisy chain.

[0081] For example, for Figure 3 The data processing device shown has a total of 16 computing units CU0-CU15. When data transmission occurs between computing units, in extreme cases, a maximum of 8 data migration operations can be performed simultaneously, such as from computing unit CU0 to computing unit CU1, from computing unit CU2 to computing unit CU3, from computing unit CU4 to computing unit CU5, ..., from computing unit CU14 to computing unit CU15.

[0082] When the amount of data transmitted in the data processing device is large, the number of data migration operations that can be performed simultaneously is usually less than the maximum value, and this decreases rapidly as the amount of data transmitted increases, as detailed below. Figure 7 As shown.

[0083] like Figure 7 As shown, the horizontal axis "time" represents the number of cycles, the vertical axis "ring" represents the number of daisy-chain loops, and the parameter "transmit_data" refers to the number of cycles required to transmit the data in one data packet. Figure 7 It can be seen that only when the maximum amount of data to be transmitted is small enough can more simultaneous data migration operations occur.

[0084] Back Figure 5 Each of the multiple chain processing modules ring0 to ring7 includes a storage submodule 505 and a prediction submodule 506.

[0085] The storage submodule 505 of each of the chain processing modules ring0 to ring7 is configured to store the status of the corresponding data migration operation in the form of a daisy chain corresponding to that chain processing module.

[0086] For example, the storage submodule 505 is used to store the status of ongoing data migration operations, and the storage format is as follows: Figure 8 As shown.

[0087] like Figure 8 As shown, each line stores a daisy chain, a parameter `ring_size` indicating the size of the data corresponding to that daisy chain, and a parameter `ring_valid` indicating whether the daisy chain is idle. For example, in this disclosure, 8 daisy chains are used, and there are 8 lines to store the status of the data migration operations corresponding to the 8 daisy chains. In other examples of this disclosure, there may be fewer than 8 daisy chains.

[0088] Back Figure 5 Each prediction submodule 506 of the chain processing module ring0 to ring7 is configured to predict the current and future changes of the daisy chain corresponding to that daisy chain to obtain the corresponding second prediction matrix.

[0089] For example, the size of the second prediction matrix is ​​the same as that of the first prediction matrix, which is s×p. Similarly, s represents the number of prediction periods, and the s rows of the second prediction matrix are used to represent the data transmission status of p nodes within s periods starting from the current point.

[0090] In each prediction cycle, the 32 registers in the daisy chain are evaluated and decremented by 1 until the data in the register becomes 0. When all the data in the daisy chain becomes 0, the parameter ring_valid becomes 0, indicating that the daisy chain is idle, and the parameter ring_size becomes 0.

[0091] The third submodule 504 is configured to receive a first prediction matrix, the current state of the daisy chain storage corresponding to each of the multiple chain processing modules, and the corresponding second prediction matrix, and to determine the occupancy status of the multiple chain processing modules based on the current state of the daisy chain storage corresponding to each of the multiple chain processing modules, and to determine whether the multiple chain processing modules can receive the daisy chain provided by the first submodule based on the first prediction matrix and the second prediction matrix corresponding to each of the multiple chain processing modules, thereby outputting an arbitration result.

[0092] For example, such as Figure 5 As shown, the third submodule 504 receives the first prediction matrix req_array provided by the second submodule 503, and the current state (ring0_valid to ring7_valid) and corresponding second prediction matrix (ring0_array to ring7_array) stored in each corresponding daisy chain provided by the daisy chain processing modules ring0 to ring7. Based on the current state ring0_valid to ring7_valid, the occupancy status of the daisy chains ring0 to ring7 is determined, and the first prediction matrix req_array and multiple second prediction matrices ring_array to ring7_array are matched respectively to determine whether the daisy chains ring0 to ring7 can receive the daisy chain req_ring provided by the first submodule 502, thereby outputting the arbitration result.

[0093] For example, in some embodiments of this disclosure, the third submodule 504 is further configured to determine which chain processing module is in an idle state among the multiple chain processing modules based on the occupancy status of each of the multiple chain processing modules.

[0094] For example, if the current status parameter ring0_valid received by the third submodule is equal to 0, it indicates that the chain processing module corresponding to the current status parameter ring0_valid is in an idle state.

[0095] For example, in some embodiments of this disclosure, the third submodule 504 is further configured to match the first prediction matrix and the multiple second prediction matrices corresponding to the multiple chain link processing modules respectively. Based on the matching result, it is determined that the first prediction matrix does not conflict with the multiple second prediction matrices. In the case that it is determined that there is an idle chain link processing module among the multiple chain link processing modules, the idle chain link processing module that can receive the daisy chain converted by the first submodule is determined, and an enable signal is sent to the determined idle chain link processing module. The multiple chain link processing modules are further configured to receive and store the daisy chain from the first submodule based on the enable signal from the third submodule.

[0096] For example, the third submodule 504 performs an OR operation on the received multiple second prediction matrices ring0_array to ring7_array (each second prediction matrix has a size of 48×32), thereby obtaining a 48×32 matrix. This matrix indicates the data transmission status of the entire third daisy chain within 48 prediction periods. This matrix is ​​then ANDed with the first prediction matrix req_array. If the result of the AND operation contains a 1, it indicates that the first prediction matrix req_array conflicts with at least one of the multiple second prediction matrices ring0_array to ring7_array, and the data migration request cannot be sent. If the result of the AND operation is 0 for all matrices, it indicates that the first prediction matrix req_array does not conflict with any of the multiple second prediction matrices ring0_array to ring7_array, and the data migration request is allowed to be sent.

[0097] For example, the third submodule 504 performs an AND operation on the received current states ring0_valid to ring7_valid. If the result is 1, it means that all 8 daisy chain links have been used and no new data migration request can be sent. If the result is 0, it means that there are daisy chain processing modules in an idle state.

[0098] The following is combined with Figure 5 The flowchart of the arbitration method of the arbitration unit provided in at least one embodiment of this disclosure is described. The workflow of the arbitration unit includes the following steps S1 to S6.

[0099] S1: Based on the source identifier src_id, destination identifier dst_id, and data size in each data migration request issued by multiple computing units, the first submodule 502 converts the data migration request output by the buffer queue 501 into a daisy chain req_ring and sends it to the second submodule 503.

[0100] S2: The second submodule 503 predicts the daisy chain req_ring within q loops to obtain the first prediction matrix req_array, and provides the first prediction matrix req_array to the third submodule 504.

[0101] S3: The third submodule 504 determines whether to receive the new first prediction matrix req_array. If not, return to step S1; otherwise, continue to receive the current state ring0_valid to ring7_valid stored in each corresponding daisy chain of multiple chain processing modules ring0 to ring7, and perform an AND operation on the current state ring0_valid to ring7_valid. If the result is 1, jump back to step S1. If the result is 0, it is determined that there is a chain processing module in an idle state, and jump to step S4.

[0102] S4: Receive multiple second prediction matrices ring0_array to ring7_array through the third submodule 504 and perform an OR operation on ring0_array to ring7_array. Perform an AND operation between the matrix generated by the OR operation and the first prediction matrix req_array. If there is a 1 in the result, it indicates that there is a data transmission conflict, and jump back to step S1. If the result is 0, it indicates that a data migration request can be issued, and jump to step S5.

[0103] S5: Based on the enable signals ring0_update to ring7_update issued by the third submodule 504, the daisy chain req_ring, the data size of the daisy chain req_ring req_ring req_size, and the current state stored in the daisy chain req_ring req_ring req_valid are stored in the chain processing module that is currently determined to be in an idle state. The output of the buffer queue 501 is controlled by the read and write control signal. After the read and write control signal (re) is enabled, the buffer queue 501 outputs the arbitration result through the second daisy chain cq_arb_daisy.

[0104] The embodiments described above utilize the bandwidth of the vector memory transmission channel effectively through a robust and sound arbitration method.

[0105] Figure 9 A schematic flowchart illustrating a data processing method according to at least one embodiment of this disclosure is shown. This data processing method is applied, for example, to the data processing apparatus described above. For instance, the data processing apparatus includes a first computing unit and a second computing unit.

[0106] like Figure 9As shown, the data processing method includes the following steps S901 to S903. In this processing method, the first calculation unit and the second calculation unit are used as examples for description. The first calculation unit and the second calculation unit can be any calculation unit among multiple calculation units. That is, "first" and "second" here are not description numbers.

[0107] Step S901: The command queue in the first computing unit sends a first data migration request to the arbitration unit through the first daisy chain, and sends a first command to the vector memory in the first computing unit to notify the vector memory in the first computing unit to be ready to receive the first data from the second computing unit.

[0108] For example, while the command queue in the first computing unit sends the first data migration request through the first daisy chain, it also sends the first command to the vector memory in the same computing unit. The first command will be stored in a first-in-first-out queue of the vector memory in the same computing unit and will not affect the processing of other commands. When the first data arrives from the second computing unit, the storage of the first data will begin.

[0109] Step S902: In response to receiving the first data migration request, the arbitration unit arbitrates one or more pending data migration requests to select the data migration request to be processed, generates a first arbitration result, and transmits the first arbitration result to the command queue of the second computing unit through a second daisy chain.

[0110] For example, upon receiving a first data migration request, the arbitration unit first stores the request in a buffer queue and then begins arbitration sequentially. The arbitration criteria include the source identifier, destination identifier, data size, and the unit's own resources, ensuring that no contention for daisy-chain resources occurs during the data migration process. At an appropriate time, the arbitration unit transmits the first arbitration result to the command queue of the second computing unit via the second daisy chain.

[0111] Step S903: The command queue of the second computing unit receives the first arbitration result and sends a second command to the vector memory of the second computing unit based on the first arbitration result. In response to receiving the command, the vector memory of the second computing unit transmits the first data to the vector memory of the first computing unit through a third daisy chain.

[0112] It should be noted that in this embodiment, the first computing unit receives data from the second computing unit and issues a data migration request. Alternatively, a computing unit other than the first and second computing units may send a data migration request, so that the first computing unit receives data from the second computing unit. This disclosure does not limit this.

[0113] The technical effects of the above data processing methods and Figure 3 The data processing device shown has the same technical effect, and will not be described again here.

[0114] According to at least one embodiment of this disclosure, an electronic device is also provided, which includes the data processing device of any of the above embodiments. For example, the data processing device is a parallel processor, such as a general-purpose graphics processing unit (GPGPU), a data processor, or a tensor processor. For example, the electronic device may also include, for example, a central processing unit (CPU), which, together with the data processing device, forms a heterogeneous computing system via a system bus. For example, the bus may be a Peripheral Component Interconnect Standard (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc.

[0115] For example, in at least one embodiment of this disclosure, the electronic device may further include input devices such as a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices such as a liquid crystal display, speaker, vibrator, etc.; storage devices such as magnetic tape, hard disk (HDD or SDD), etc.; and communication devices such as network interface cards such as LAN cards, modems, etc. The communication device can allow the electronic device to communicate wirelessly or wiredly with other devices to exchange data and perform communication processing via a network such as the Internet. A drive is connected to the I / O interface as needed. A removable storage medium, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive as needed so that computer programs read from it can be installed into the storage device as needed.

[0116] For example, the electronic device may further include a peripheral interface. This peripheral interface can be of various types, such as a USB interface, a Lightning interface, etc. The communication device can communicate wirelessly with networks and other devices, such as the Internet, intranets and / or wireless networks such as cellular telephone networks, wireless local area networks (LANs) and / or metropolitan area networks (MANs). Wireless communication can use any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email, instant messaging, and / or Short Message Service (SMS), or any other suitable communication protocol.

[0117] The electronic device may be, for example, a system-on-a-chip (SOC) or a device that includes the SOC. For example, it may be any device such as a mobile phone, tablet computer, laptop computer, e-book reader, game console, television, digital photo frame, navigator, home appliance, communication base station, industrial controller, server, etc. It may also be any combination of data processing device and hardware. The embodiments disclosed herein do not limit this.

[0118] The following points should be noted regarding this disclosure:

[0119] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0120] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0121] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A data processing apparatus, comprising multiple computing units, an arbitration unit, a first daisy chain, a second daisy chain, and a third daisy chain. in, Each of the plurality of computing units includes a command queue and a vector memory. The command queues of each of the plurality of computing units are cascaded through the first daisy chain and connected to the arbitration unit, and are also cascaded through the second daisy chain and connected to the arbitration unit. The first daisy chain is used to send the data migration request issued by each of the plurality of computing units to the arbitration unit. The arbitration unit receives the data migration request from each of the plurality of computing units, and arbitrates one or more pending data migration requests to select the data migration request to be processed, thereby generating an arbitration result. The second daisy chain is used to return the arbitration result generated by the arbitration unit to the command queue of the computing unit that issued the selected data migration request. The vector memories in the plurality of computing units are cascaded through the third daisy chain, and the third daisy chain is used to transfer data between the vector memories in the plurality of computing units.

2. The data processing apparatus according to claim 1, wherein, Each data migration request issued by the multiple computing units includes a source identifier, a corresponding workgroup identifier, the size of the data, a destination identifier, and data bias information.

3. The data processing apparatus according to claim 1, wherein, The arbitration result includes the source identifier, destination identifier, data volume, and corresponding working group identifier.

4. The data processing apparatus according to claim 1 further includes a repeater, wherein, The vector memory in the plurality of computing units is connected to the repeater in a loop through the third daisy chain, which is a unidirectional daisy chain.

5. The data processing apparatus according to claim 1 or 4, wherein, The arbitration unit includes: A buffer queue is configured to receive and cache data migration requests issued by each of the plurality of computing units through the first daisy chain, and to output the arbitration result through the second daisy chain; The first submodule is configured to convert the data migration request issued by each of the plurality of computing units into a daisy chain form; The second submodule is configured to predict the current and later changes of the daisy chain to obtain a first prediction matrix; Multiple chain-link processing modules, wherein each of the multiple chain-link processing modules corresponds to an operable chain link in the third daisy chain, and each of the multiple chain-link processing modules is configured to store the current state of the corresponding data migration operation in the form of the corresponding daisy chain link, and to predict the current and future changes of the corresponding daisy chain link to obtain a corresponding second prediction matrix; and The third submodule is configured to receive the first prediction matrix, the current state of the daisy chain storage corresponding to each of the plurality of chain processing modules, and the corresponding second prediction matrix; determine the occupancy status of the plurality of chain processing modules based on the current state of the daisy chain storage corresponding to each of the plurality of chain processing modules; determine whether the plurality of chain processing modules can receive the daisy chain provided by the first submodule based on the first prediction matrix and the second prediction matrix corresponding to each of the plurality of chain processing modules; and output the arbitration result.

6. The data processing apparatus according to claim 5, wherein, The daisy chain includes p registers, where p = the number of nodes in the third daisy chain, and p is a positive integer. Each register corresponds to a node in the third daisy chain and is used to represent the working state of the corresponding node.

7. The data processing apparatus according to claim 6, wherein, The sizes of the first prediction matrix and the corresponding second prediction matrix are both s×p. s represents the number of predicted cycles. The s rows of the first prediction matrix and the corresponding second prediction matrix are respectively used to represent the data transmission conditions of the p nodes within s cycles starting from the current time.

8. The data processing apparatus according to claim 5, wherein, The first sub-module is further configured to convert the data migration requests sent by each of the multiple computing units into the form of a daisy chain loop based on the source identifier, destination identifier, and data volume size in the data migration requests sent by each of the multiple computing units.

9. The data processing apparatus according to claim 8, wherein, The source identifier = m, the destination identifier = n, the data volume size = k. Each node in the daisy chain loop is sequentially assigned a different node serial number CU_ID. In the case of m < n, for m ≤ CU_ID ≤ n, the register corresponding to node CU_ID is given an initial value of k + CU_ID. In the case of m > n, for m ≤ CU_ID < p, the register corresponding to node CU_ID is given an initial value of k + CU_ID, and for CU_ID ≤ n, the register corresponding to node CU_ID is given an initial value of k + CU_ID + p.

10. The data processing apparatus according to claim 9, wherein, The value v in each register in the daisy chain loop is used to determine whether there is data in the corresponding node. In the case of 0 < v ≤ k, there is data in the corresponding node. In the case of v > k, there is no data in the corresponding node.

11. The data processing apparatus according to claim 5, wherein, Each of the multiple link processing modules includes: A storage sub-module configured to store the status of the corresponding data migration operation in the form of the corresponding daisy chain loop. A prediction sub-module configured to predict the current and future changes of the corresponding daisy chain loop to obtain the corresponding second prediction matrix.

12. The data processing apparatus according to claim 10, wherein, The number of the multiple link processing modules is less than or equal to the maximum number of data migration operations that can be performed simultaneously in the third daisy chain.

13. The data processing apparatus according to claim 5, wherein, The third sub-module is further configured to determine the link processing modules in an idle state among the multiple link processing modules according to the occupancy situation of each of the multiple link processing modules.

14. The data processing apparatus according to claim 5, wherein, The third sub-module is further configured to match the first prediction matrix and the multiple second prediction matrices respectively corresponding to the multiple link processing modules. Based on the matching result indicating that the first prediction matrix does not conflict with the multiple second prediction matrices, and in the case of determining that there are link processing modules in an idle state among the multiple link processing modules, determine the link processing module in an idle state that can receive the daisy chain loop converted by the first sub-module, and send an enable signal to the determined link processing module in an idle state. The multiple link processing modules are further configured to, based on the enable signal from the third sub-module, cause the determined link processing module in an idle state to receive and store the daisy chain loop from the first sub-module.

15. A data processing method applied to the data processing apparatus according to any one of claims 1-14, wherein, The multiple computing units include a first computing unit and a second computing unit. The data processing method includes: The command queue in the first computing unit sends a first data migration request to the arbitration unit through the first daisy chain, and sends a first command to the vector memory in the first computing unit to notify the vector memory in the first computing unit to be ready to receive the first data from the second computing unit. In response to receiving the first data migration request, the arbitration unit arbitrates the one or more pending data migration requests to select the data migration request to be processed, generates a first arbitration result, and transmits the first arbitration result to the command queue of the second computing unit through the second daisy chain. The command queue of the second computing unit receives the first arbitration result and sends a second command to the vector memory of the second computing unit based on the first arbitration result. In response to receiving the command, the vector memory of the second computing unit transmits the first data to the vector memory of the first computing unit through the third daisy chain.

16. An electronic device comprising: The data processing apparatus as claimed in any one of claims 1 to 14.

Images