Data transmission method and device of network-on-chip, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a data transmission method and device of a network-on-chip, electronic equipment and a storage medium. The method comprises the following steps: splitting a calculation task into at least two sub-tasks; according to the processing stages to which the sub-tasks belong and the dependency relationship among the sub-tasks, determining data transmission requirements among the sub-tasks, memory access requirements corresponding to each sub-task and external interface access requirements; based on the data transmission requirement, the memory access requirement and the external interface access requirement, an unloading strategy corresponding to the subtask is determined; and unloading the subtask to the target computing node based on the unloading strategy. According to the embodiment of the invention, the utilization rate of the computing unit is improved, and the overall computing efficiency of the network-on-chip is improved.

Description

Technical Field

[0001] The present invention relates to the field of network-on-chip technology, and in particular to a data transmission method, device, electronic device and storage medium of a network-on-chip. Background Art

[0002] According to the workload of the deep learning acceleration chip, there is randomness in the spatial scheduling of computing units; however, when the computing cluster performs inference tasks, the tasks performed by a certain computing unit, or a specific computing unit in a computing unit, are relatively fixed. Frequent and complex resource scheduling will lead to a decrease in computing resource utilization and a waste of on-chip network bandwidth. Summary of the invention

[0003] The embodiments of the present invention provide a data transmission method, device, electronic device and storage medium of an on-chip network, which can solve the problem in related technologies that frequent and complex resource scheduling will lead to a decrease in computing resource utilization and a waste of on-chip network bandwidth.

[0004] In order to solve the above problem, an embodiment of the present invention discloses a data transmission method of a network on chip, which is applied to the network on chip. The method includes: Split the computational task into at least two subtasks; Determine the data transmission requirements between the subtasks, the memory access requirements and the external interface access requirements corresponding to each subtask according to the processing stage to which the subtasks belong and the dependencies between the subtasks; Determining an offloading strategy corresponding to the subtask based on the data transmission requirement, the memory access requirement, and the external interface access requirement; Based on the offloading strategy, the subtask is offloaded to a target computing node.

[0005] Optionally, determining the offloading strategy corresponding to the subtask based on the data transmission requirement, the memory access requirement, and the external interface access requirement includes: If there is data to be transmitted between at least two first subtasks that meets the first preset condition, offloading the at least two first subtasks to at least two adjacent first computing nodes; If there is data dependency between at least two second subtasks, offloading the at least two second subtasks to at least two adjacent second computing nodes; If there is at least one third subtask that exchanges data with a storage module or an external interface of the network on chip, the third subtask is unloaded to an edge computing node of the network on chip.

[0006] Optionally, the step of offloading the subtask to a target computing node based on the offloading strategy includes: When the computing capacity of the on-chip network is insufficient, cascading at least two on-chip networks; Based on the offloading strategy, offloading the subtask to the target computing nodes of the at least two on-chip networks; or, The at least two subtasks are time-sharedly offloaded to a target computing node of the on-chip network according to the data correlation between the subtasks.

[0007] Optionally, the computing task includes a deep learning reasoning task; and splitting the computing task into at least two subtasks includes: According to the deep learning reasoning task, determining an input layer, at least one intermediate layer, and an output layer; Based on the data dependency between the input layer, the at least one intermediate layer and the output layer, and a preset granularity, each processing stage of the deep learning reasoning task is divided into at least one subtask.

[0008] Optionally, the method further comprises: After the third computing node obtains the first data after executing the subtask of the computing task, adding identification information to the first data to obtain second data; the identification information is used to indicate the order in which the first data is used in subsequent computing; The second data is sent to a fourth computing node, so that the fourth computing node reorders each received second data according to the identification information in the second data, and continues to execute the computing task.

[0009] Optionally, the method further comprises: Determine a target multicast domain to which the target computing node belongs; According to the target multicast domain to which the target computing node belongs, a multicast control field is added to the first data to be transmitted to obtain the target data; the multicast control field includes a multicast domain indication field and an intra-domain mask field, the multicast domain indication field is used to indicate the target multicast domain, and the intra-domain mask field is used to indicate the target computing node in the target multicast domain; Determining a first routing path outside the target multicast domain according to the multicast control field and the routing algorithm; transmitting the target data to the target multicast domain based on the first routing path; Determining a second routing path within the target multicast domain according to the intra-domain mask field; The target data is sent to the target computing node based on the second routing path, so that the target computing node executes the subtask based on the target data.

[0010] On the other hand, an embodiment of the present invention discloses a data transmission device of a network on chip, which is applied to the network on chip, and the device includes: A splitting module, used for splitting a computing task into at least two subtasks; A determination module, used to determine the data transmission requirements between the subtasks, the memory access requirements and the external interface access requirements corresponding to each subtask according to the processing stage to which the subtasks belong and the dependencies between the subtasks; and determine the unloading strategy corresponding to the subtask based on the data transmission requirements, the memory access requirements and the external interface access requirements; The unloading module is used to unload the subtask to the target computing node based on the unloading strategy.

[0011] Optionally, the determining module includes: An unloading submodule is used to unload the at least two first subtasks to at least two adjacent first computing nodes if the data to be transmitted between at least two first subtasks meets a first preset condition; to unload the at least two second subtasks to at least two adjacent second computing nodes if there is data dependency between at least two second subtasks; and to unload the third subtask to an edge computing node of the on-chip network if at least one third subtask exchanges data with a storage module or an external interface of the on-chip network.

[0012] On the other hand, an embodiment of the present invention further discloses an electronic device, which includes a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store executable instructions, and the executable instructions enable the processor to execute the aforementioned on-chip network data transmission method.

[0013] The embodiment of the present invention further discloses a readable storage medium. When instructions in the readable storage medium are executed by a processor of an electronic device, the electronic device can execute the aforementioned data transmission method of the on-chip network.

[0014] The embodiments of the present invention include the following advantages: The embodiment of the present invention provides a data transmission method for a network on chip, which splits a computing task into at least two subtasks; determines the data transmission requirements between the subtasks, the memory access requirements and the external interface access requirements corresponding to each subtask according to the processing stage to which the subtasks belong and the dependency relationship between the subtasks; determines the unloading strategy corresponding to the subtask based on the data transmission requirements, the memory access requirements and the external interface access requirements; and unloads the subtasks to a target computing node based on the unloading strategy. The embodiment of the present invention can first split the computing task; then determine the unloading strategy based on the processing stage to which the subtasks after the split belong and the dependency relationship between the subtasks, and then can unload the subtasks to the target computing node according to the unloading strategy, thereby reducing the design complexity and time overhead of the computing unit, improving the utilization rate of the computing unit, and being conducive to improving the overall computing efficiency of the network on chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0016] Figure 1 It is a flowchart of a data transmission method embodiment of a network on chip of the present invention; Figure 2 It is a structural frame of a network on chip of the present invention. Figure 1 ; Figure 3 It is a structural frame of a network on chip of the present invention. Figure 2 ; Figure 4 It is a structural block diagram of an embodiment of a data transmission device of a network on chip of the present invention; Figure 5 It is a structural block diagram of an electronic device provided by an example of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] The terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable when appropriate, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, the term "and / or" in the specification and claims is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In the embodiments of the present invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0019] Method Embodiment Reference Figure 1 , shows a flowchart of a data transmission method of a network on chip embodiment of the present invention, the method may specifically include the following steps: Step S101, splitting a computing task into at least two subtasks; Step S102: Determine the data transmission requirements between the subtasks, the memory access requirements and the external interface access requirements corresponding to each subtask according to the processing stage to which the subtasks belong and the dependencies between the subtasks; Step S103: Determine the offloading strategy corresponding to the subtask based on data transmission requirements, memory access requirements, and external interface access requirements; Step S104: Unload the subtask to the target computing node based on the unloading strategy.

[0020] In the embodiment of the present invention, the computing task may include but is not limited to a summing or averaging computing task, which is not limited in the embodiment of the present invention.

[0021] In an embodiment of the present invention, at least two subtasks may be obtained by dividing or abstracting the computing task into an input layer, several intermediate layers, an output layer, etc., or may be obtained by dividing the computing task into finer granularity subtasks, which are not limited in the embodiment of the present invention.

[0022] In the embodiment of the present invention, the processing stage to which the subtask belongs may include but is not limited to an input stage, or an output stage, etc., which is not limited in the embodiment of the present invention.

[0023] For example, in an embodiment of the present invention, when the inference task Task can be divided into multiple (task) Operations, that is, OP0-OP15, a total of 16 operations, each operation depends on the calculation results of the first two levels (OP0 only depends on the input, and OP1 only depends on the result of OP0), a single chip can run at most 8 of the 16 OPs at the same time. If two chips are available, these 16 operations can be completely performed in parallel, and the operations performed by each CU (Compute Unit) do not need to change at all. This is static unloading. However, in the case of only one chip, these 16 OPs need to be unloaded to the CU of the chip in a time-sharing manner. Then, according to the stage of T, there may be simultaneous running of OP0-OP7, OP8-OP15 and intermediate situations on the chip (such as OP0 can completely release resources, then the instructions of OP8 can be unloaded to the CU to wait for its data dependency, and the data generated by OP0 to be used for subsequent OPs can be stored at the CU or written back to the memory), until a complete T is completed, that is, semi-static unloading.

[0024] Exemplarily, the dependency relationship between the subtasks may include the case where the computing task Task can be divided into tasks OP0-OP15, totaling 16 operations, and each operation depends on the calculation results of the previous two levels (OP0 only depends on the input, and OP1 only depends on the result of OP0), which is not limited in the embodiments of the present invention.

[0025] In the embodiment of the present invention, the data transmission requirement corresponding to each subtask may include a transmission path with the fewest number of computing nodes, which is not limited in the embodiment of the present invention.

[0026] In an embodiment of the present invention, the memory access requirement corresponding to each subtask may include a mapping and conversion process from a logical address (or virtual address) to a physical address when a CPU (Central Processing Unit) or other processor accesses the memory.

[0027] In the embodiment of the present invention, the offloading strategy corresponding to the subtask may include determining based on the node with the least hops of the multicast message, or determining based on the shortest path selected by the routing protocol to reach any node in the multicast domain, which is not limited in the embodiment of the present invention.

[0028] In the embodiment of the present invention, offloading the subtask to the target computing node may include determining the computing node according to the node with the least hop count of the multicast message or determining the computing node according to the shortest path, which is not limited in the embodiment of the present invention.

[0029] The embodiment of the present invention provides a data transmission method for a network on chip, which splits a computing task into at least two subtasks; according to the processing stage to which the subtasks belong and the dependency relationship between the subtasks, the data transmission requirements between the subtasks, the memory access requirements and the external interface access requirements corresponding to each subtask are determined; based on the data transmission requirements, the memory access requirements and the external interface access requirements, the unloading strategy corresponding to the subtask is determined; based on the unloading strategy, the subtask is unloaded to a target computing node. The embodiment of the present invention can first split the computing task; then determine the unloading strategy according to the processing stage to which the subtasks after the split belong and the dependency relationship between the subtasks, and then the subtasks can be unloaded to the target computing node according to the unloading strategy, thereby improving the data transmission efficiency of the network on chip.

[0030] In an embodiment of the present invention, in the process of determining the offloading strategy corresponding to the subtask based on the data transmission requirement, the memory access requirement and the external interface access requirement, the method may specifically include the following steps: Step S1031: If there is data to be transmitted between at least two first subtasks that meets a first preset condition, the at least two first subtasks are unloaded to at least two adjacent first computing nodes.

[0031] In an embodiment of the present invention, in the process of determining the unloading strategy corresponding to the subtask based on data transmission requirements, memory access requirements and external interface access requirements, if there is data to be transmitted between at least two first subtasks that meets the first preset condition, then the at least two first subtasks are unloaded to at least two adjacent first computing nodes.

[0032] Exemplarily, in an embodiment of the present invention, when the data to be transmitted between at least two first subtasks meets the transmission capacity that exceeds the preset capacity, the at least two first subtasks can be offloaded to at least two adjacent first computing nodes, which is not limited in the embodiment of the present invention.

[0033] Exemplarily, in an embodiment of the present invention, when the data to be transmitted between at least two first subtasks meets the transmission rate exceeding a preset rate, the at least two first subtasks can be offloaded to at least two adjacent first computing nodes, which is not limited in the embodiment of the present invention.

[0034] In the embodiment of the present invention, the first preset condition may include a preset transmission capacity or a preset transmission rate, which is not limited in the embodiment of the present invention.

[0035] Illustratively, in the embodiment of the present invention, the preset transmission capacity may be 500 bytes, or 100 bytes, etc., which is not limited in the embodiment of the present invention.

[0036] Illustratively, in the embodiment of the present invention, the preset transmission rate may be 100 Mbps, or 50 Mbps, etc., which is not limited in the embodiment of the present invention.

[0037] Exemplarily, in an embodiment of the present invention, when the rate of data to be transmitted between at least two first subtasks is 52Mbps, and the first preset condition is that the rate of data to be transmitted is greater than 50Mbps, then at least two first subtasks can be offloaded to at least two adjacent first computing nodes, which is not limited in the embodiment of the present invention.

[0038] Exemplarily, in an embodiment of the present invention, when the capacity of data to be transmitted between at least two first subtasks is 501 bytes and the first preset condition is that the capacity of data to be transmitted is greater than 500 bytes, at least two first subtasks can be offloaded to at least two adjacent first computing nodes, which is not limited in the embodiment of the present invention.

[0039] Step S1032: If there is data dependency between at least two second subtasks, offload the at least two second subtasks to at least two adjacent second computing nodes.

[0040] In an embodiment of the present invention, in the process of determining the unloading strategy corresponding to the subtask based on data transmission requirements, memory access requirements and external interface access requirements, if there is data dependency between at least two second subtasks, the at least two second subtasks are offloaded to at least two adjacent second computing nodes.

[0041] Exemplarily, in an embodiment of the present invention, data dependency can be when the reasoning task T can be divided into 16 operations OP0-OP15, each operation depends on the calculation results of the first two levels (OP0 only depends on the input, OP1 only depends on the result of OP0).

[0042] Illustratively, in the embodiment of the present invention, the subtasks containing previous and next data dependencies are offloaded to adjacent computing units, which is not limited in the embodiment of the present invention.

[0043] Step S1033: If there is at least one third subtask that exchanges data with a storage module or an external interface of the on-chip network, the third subtask is unloaded to an edge computing node of the on-chip network.

[0044] In an embodiment of the present invention, in the process of determining the unloading strategy corresponding to the subtask based on data transmission requirements, memory access requirements and external interface access requirements, if there is at least one third subtask that exchanges data with the storage module or external interface of the on-chip network, the third subtask is unloaded to the edge computing node of the on-chip network.

[0045] Exemplarily, in an embodiment of the present invention, primary and final tasks that exchange data with a memory and an external interface are offloaded to an edge computing unit that is closer to a memory controller and an external interface, which is not limited in the embodiment of the present invention.

[0046] In an embodiment of the present invention, offloading the third subtask to an edge computing node of the on-chip network can make the path for offloading the task shortest or the offloading process pass through fewer nodes, which is not limited in the embodiment of the present invention.

[0047] In an embodiment of the present invention, in the process of offloading a subtask to a target computing node based on an offloading strategy, the method may specifically include the following steps: Step S1041: When the computing capacity of the on-chip network is insufficient, at least two on-chip networks are cascaded.

[0048] In an embodiment of the present invention, in the process of offloading a subtask to a target computing node based on an offloading strategy, when the computing capacity of the on-chip network is insufficient, at least two on-chip networks are cascaded.

[0049] Exemplarily, in an embodiment of the present invention, a certain reasoning task T can be divided into 16 operations, OP0-OP15, each operation depends on the calculation results of the previous two levels (OP0 only depends on the input, and OP1 only depends on the result of OP0). A single chip can run at most 8 of the 16 OPs simultaneously. However, if there is only one chip, that is, the computing capacity of the on-chip network is insufficient, this is not limited in the embodiment of the present invention.

[0050] For example, in an embodiment of the present invention, when the computing capacity of the on-chip network is insufficient, at least two on-chip networks can be cascaded, thereby enabling 16 operations to be performed completely in parallel, and the operations performed by each CU do not need to be changed at all.

[0051] Step S1042: based on the offloading strategy, offload the subtask to target computing nodes of at least two on-chip networks.

[0052] In an embodiment of the present invention, when the computing capacity of the on-chip network is insufficient, after cascading at least two on-chip networks, the on-chip network can also offload subtasks to target computing nodes of the at least two on-chip networks based on an offloading strategy.

[0053] In an embodiment of the present invention, the unloading strategy may include: if there is data to be transmitted between at least two first subtasks that meets a first preset condition, then the at least two first subtasks are unloaded to at least two adjacent first computing nodes; if there is data dependency between at least two second subtasks, then the at least two second subtasks are unloaded to at least two adjacent second computing nodes; if there is at least one third subtask that exchanges data with a storage module or an external interface of the on-chip network, then the third subtask is unloaded to an edge computing node of the on-chip network.

[0054] In an embodiment of the present invention, the target computing nodes of the at least two on-chip networks may include a computing node determined according to a node with the least hop count of a multicast message or a computing node determined according to a shortest path, which is not limited in the embodiment of the present invention.

[0055] Step S1043 : according to the data correlation between the subtasks, offload at least two subtasks to the target computing node of the on-chip network in a time-sharing manner.

[0056] It should be noted that, in the embodiment of the present invention, executing step S1042 or executing step S1043 is not limited in the embodiment of the present invention.

[0057] In an embodiment of the present invention, after offloading the subtasks to at least two target computing nodes of the on-chip network based on the offloading strategy, the on-chip network can also offload at least two subtasks to the target computing nodes of the on-chip network in a time-sharing manner according to the data correlation between the subtasks.

[0058] For example, in an embodiment of the present invention, in the case of one chip, 16 OPs need to be unloaded to the CU of the chip in a time-sharing manner. Then, according to the stage of T, the chip may run OP0-OP7, OP8-OP15 and intermediate situations therebetween (for example, if OP0 can completely release resources, the instructions of OP8 can be unloaded to the CU to wait for its data dependencies, and the data generated by OP0 to be used for subsequent OPs can be stored in the CU or written back to the memory) until a complete T is completed, which is not limited in the embodiment of the present invention.

[0059] In an embodiment of the present invention, when the computing task includes a deep learning reasoning task; in the process of splitting the computing task into at least two subtasks, the method may specifically include the following steps: Step S1011: Determine an input layer, at least one intermediate layer, and an output layer according to a deep learning reasoning task.

[0060] In an embodiment of the present invention, the computing task includes a deep learning reasoning task; in the process of splitting the computing task into at least two subtasks, the on-chip network can also determine an input layer, at least one intermediate layer and an output layer according to the deep learning reasoning task.

[0061] In an embodiment of the present invention, according to the deep learning reasoning task, subtasks are obtained by dividing or abstracting it into an input layer, several intermediate layers, an output layer, etc., or it is divided into finer granularity, and the subtasks obtained are not limited in the embodiment of the present invention.

[0062] Step S1012: Based on the data dependency between the input layer, at least one intermediate layer, and the output layer, and a preset granularity, each processing stage of the deep learning reasoning task is divided into at least one subtask.

[0063] In an embodiment of the present invention, after determining the input layer, at least one intermediate layer and the output layer according to the deep learning reasoning task, the on-chip network can also divide each processing stage of the deep learning reasoning task into at least one subtask based on the data dependency relationship between the input layer, at least one intermediate layer and the output layer, and a preset granularity.

[0064] In the embodiment of the present invention, the preset granularity may include a finer division level, that is, the input layer or at least one intermediate layer and the output layer may be divided into finer levels, which is not limited in the embodiment of the present invention.

[0065] In an embodiment of the present invention, the data transmission method of the network on chip may further include the following steps: Step S105: After the third computing node obtains the first data after executing the subtask of the computing task, identification information is added to the first data to obtain second data; the identification information is used to indicate the order in which the first data is used in subsequent computing.

[0066] In the embodiment of the present invention, the network on chip may obtain first data obtained after at least one third computing node executes a subtask of the computing task.

[0067] In the embodiment of the present invention, the first data may be obtained after at least one third computing node executes a subtask of the computing task.

[0068] In an embodiment of the present invention, after obtaining the first data obtained after at least one third computing node executes a subtask of the computing task, the on-chip network can also add identification information to the first data to obtain second data; the identification information is used to indicate the order in which the first data is used in subsequent calculations.

[0069] Illustratively, in the embodiment of the present invention, the first data may include Data0, Data1, Date8, and Data9, which is not limited in the embodiment of the present invention.

[0070] In the embodiment of the present invention, the identification information can be used to sort the first data.

[0071] Exemplarily, in an embodiment of the present invention, the second data may include adding ID (Identifier, identification information) to Data0, Data1, Date8, and Data9, that is, obtaining data Data0 (ID0), Data1 (ID1), Data8 (ID2), and Data9 (ID3), which is not limited in the embodiment of the present invention.

[0072] In the embodiment of the present invention, the second data may be data obtained after adding identification information to the first data.

[0073] In the embodiment of the present invention, after adding identification information to the first data to obtain the second data, the on-chip network may further sort the second data according to the identification information.

[0074] In the embodiment of the present invention, the second data may be data obtained after adding identification information to the first data.

[0075] In an embodiment of the present invention, the process of sorting the second data may include sorting according to the order of the identification information as required, which may be sorting from high to low or from low to high, etc., which is not limited in the embodiment of the present invention.

[0076] Step S106: Send the second data to the fourth computing node, so that the fourth computing node reorders the received second data according to the identification information in the second data, and continues to execute the computing task.

[0077] In an embodiment of the present invention, after sorting the second data according to the identification information, the on-chip network can also send the second data to the fourth computing node, so that the fourth computing node reorders the received second data according to the identification information in the second data and continues to execute the computing task.

[0078] In the embodiment of the present invention, the sorting result may be a sorting result from high to low, or a sorting result from low to high, which is not limited in the embodiment of the present invention.

[0079] Illustratively, in an embodiment of the present invention, after sorting the second data, the sorting results that may be obtained are Data9 (ID3), Data8 (ID2), Data1 (ID1), Data0 (ID0), or Data0 (ID0), Data1 (ID1), Data8 (ID3), Data9 (ID3), etc., which are not limited in the embodiment of the present invention.

[0080] In the embodiment of the present invention, the fourth computing node may include a node that performs calculations on the sorted data. The computing node may perform calculations such as summing or averaging, which is not limited in the embodiment of the present invention.

[0081] In an embodiment of the present invention, the data transmission method of the network on chip may further include the following steps: Step S107: Determine the target multicast domain to which the target computing node belongs.

[0082] In the embodiment of the present invention, after offloading the subtask to the target computing node based on the offloading policy, the on-chip network may also determine the target multicast domain to which the target computing node belongs.

[0083] In the embodiment of the present invention, the target multicast domain is used to indicate a target computing area for multicast transmission, which is not limited in the embodiment of the present invention.

[0084] Step S108: According to the target multicast domain to which the target computing node belongs, a multicast control field is added to the first data to be transmitted to obtain the target data; the multicast control field includes a multicast domain indication field and an intra-domain mask field, the multicast domain indication field is used to indicate the target multicast domain, and the intra-domain mask field is used to indicate the target computing node within the target multicast domain.

[0085] In an embodiment of the present invention, after determining the target multicast domain to which the target computing node belongs, the on-chip network can also add a multicast control field to the first data to be transmitted according to the target multicast domain to which the target computing node belongs to obtain the target data; the multicast control field includes a multicast domain indication field and an intra-domain mask field, the multicast domain indication field is used to indicate the target multicast domain, and the intra-domain mask field is used to indicate the target computing node within the target multicast domain.

[0086] In an embodiment of the present invention, the multicast control field may include a multicast domain indication field and an intra-domain mask field. The multicast domain indication field is used to indicate a target multicast domain, and the intra-domain mask field is used to indicate a target computing node within the target multicast domain.

[0087] In the embodiment of the present invention, the target data can be obtained by adding a multicast control field to the first data to be transmitted.

[0088] In the embodiment of the present invention, the multicast domain indication field is used to indicate the target multicast domain, that is, to determine the area for multicast transmission, which is not limited in the embodiment of the present invention.

[0089] In the embodiment of the present invention, the intra-domain mask field includes a subnet mask used to distinguish a network address part and a host address part in an IP (Internet Protocol), which is not limited in the embodiment of the present invention.

[0090] In the embodiment of the present invention, the target computing node in the target multicast domain may be a computing node in the multicast domain, and the number of the computing nodes may include one or more, which is not limited in the embodiment of the present invention.

[0091] Step S109: Determine a first routing path outside the target multicast domain according to the multicast control field and the routing algorithm.

[0092] In an embodiment of the present invention, after adding a multicast control field to the first data to be transmitted according to the target multicast domain to which the target computing node belongs and obtaining the target data, the on-chip network can also determine a first routing path outside the target multicast domain according to the multicast control field and the routing algorithm.

[0093] In the embodiment of the present invention, the core function of the routing algorithm is to determine the transmission path of the data packet in the network. When the data packet is transmitted in the network, the router will forward the data packet to the next node according to the best path calculated by the routing algorithm until the data packet reaches the destination node.

[0094] Exemplarily, in an embodiment of the present invention, common routing algorithm types may include but are not limited to distance vector algorithm, link state algorithm, path vector algorithm, etc. Routing algorithms can be classified according to their adaptability and non-adaptability, which is not limited in the embodiment of the present invention.

[0095] In the embodiment of the present invention, the first routing path outside the target multicast domain may be a path composed of routing nodes involved in calculation outside the target multicast domain, which is not limited in the embodiment of the present invention.

[0096] Step S110: Transmit target data to a target multicast domain based on the first routing path.

[0097] In the embodiment of the present invention, after determining a first routing path outside the target multicast domain according to the multicast control field and the routing algorithm, the on-chip network may also transmit the target data to the target multicast domain based on the first routing path.

[0098] In the embodiment of the present invention, the on-chip network may transmit the target data to the target multicast domain according to the first routing path, and then perform multicast transmission on the target multicast domain.

[0099] Step S111: Determine a second routing path within the target multicast domain according to the intra-domain mask field.

[0100] In the embodiment of the present invention, after transmitting the target data to the target multicast domain based on the first routing path, the on-chip network may also determine a second routing path within the target multicast domain according to the intra-domain mask field.

[0101] In the embodiment of the present invention, the second routing path within the target multicast domain may be a routing path composed of routing nodes within the target multicast domain, which is not limited in the embodiment of the present invention.

[0102] Step S112: Send the target data to the target computing node based on the second routing path, so that the target computing node executes the subtask based on the target data.

[0103] In an embodiment of the present invention, after determining the second routing path within the target multicast domain according to the intra-domain mask field, the on-chip network can also send the target data to the target computing node based on the second routing path so that the target computing node can execute the subtask based on the target data.

[0104] In the embodiment of the present invention, reordering the target data may be a process of ordering the target data according to identification information of the target data, which is not limited in the embodiment of the present invention.

[0105] Reference Figure 2 , showing a composition structure frame of a network on chip of the present invention Figure 1 , which may include the following: The on-chip network includes PCIe (Peripheral Component Interconnect Express), MC (Memory Controller) and CU, which are not limited in the embodiments of the present invention.

[0106] Divide the front-end operations (automatic software stack / manual optimization) into subtasks that can be processed within similar cycles of a single computing unit and contain dependencies between the previous and next data, and offload them to adjacent computing units. Also, try to offload the primary and final tasks that exchange data with the memory and external interface to the edge computing units that are closer to the memory controller and external interface, and reuse the data paths for accessing memory and entering and exiting the chip with the data transmission paths between computing units.

[0107] Exemplarily, the input layer may include 12 computing units, the normalization layer may include 4 computing units, the convolutional neural network layer may include 8 computing units, an embodiment of the present invention may include multiple convolutional layers, the dense layer may include 16 computing units, the output layer may include 8 computing units, etc., which are not limited in the embodiments of the present invention.

[0108] Reference Figure 3 , showing a composition structure frame of a network on chip of the present invention Figure 2 , the on-chip network can include the following: Access devices and routers, access devices include processor cores and cache.

[0109] In practical applications, although the multiple processor cores in a multi-core processor chip each execute their own code, data needs to be shared and synchronized between the processor cores. Therefore, the hardware structure of the multi-core processor must support efficient inter-core communication, and the performance of the on-chip communication structure will directly affect the performance of the processor.

[0110] Therefore, the on-chip network aims to solve the communication problem within the multi-core processor. The on-chip network is an integrated circuit on a network-based communication subsystem. The on-chip network is a packet switching network based on routers between on-chip systems. The on-chip network technology draws on the interconnection network structure of parallel computers, integrating a large number of computing resources and an on-chip communication network connecting these resources on a single chip. Each processor core has an independent router and its private cache, and is connected together through an on-chip communication network. The processor cores use a message communication mechanism, and use routers and packet switching technology to replace the on-chip bus of related technologies to complete the communication task, thereby solving various bottleneck problems caused by bus interconnection in related technologies.

[0111] Depending on the workload of the deep learning acceleration chip, there is randomness in the spatial scheduling of computing units; however, when the computing cluster performs inference tasks, the tasks performed by a certain computing unit, or a specific computing unit in a computing unit, are relatively fixed. Frequent and complex resource scheduling will lead to a decrease in computing resource utilization and a waste of NoC (Network on Chip) bandwidth.

[0112] Deep learning reasoning acceleration chips are usually composed of several identical computing units, each of which is responsible for executing a part of the reasoning task. This solution proposes a method for statically or semi-statically offloading subtasks of deep learning reasoning tasks to multiple computing nodes. Before using the deep learning reasoning acceleration chip to accelerate the reasoning task, the hardware-bound deep learning software stack (or the software personnel manually) divides the specific deep learning reasoning task into several subtasks that can be executed by a single computing unit, and statically offloads the specific subtasks to the computing nodes at fixed spatial locations according to the spatial correlation of the memory, computing unit, and off-chip interface. When the computing capacity of a single chip is insufficient, according to business needs and cost constraints, multiple chips are cascaded and the tasks are statically offloaded to different chips, or some performance is sacrificed to offload the time-sharing (semi-static) of the reasoning subtasks to a smaller number of reasoning chips according to data correlation.

[0113] For a deep learning inference accelerator chip in which computing units are distributed in two dimensions and storage controllers and external interfaces are placed at the edge of a two-dimensional network, this technical solution divides the front-end operations (automatic software stack / manual optimization) into subtasks containing previous and next data dependencies that can be processed within similar cycles of a single computing unit, and offloads them to adjacent computing units. It also offloads the primary and final tasks that exchange data with the memory and external interface to the edge computing units that are closer to the memory controller and external interface as much as possible, and multiplexes the data paths for accessing memory and entering and exiting the chip with the data transmission paths between computing units.

[0114] By statically or semi-statically scheduling computing tasks on the accelerator computing unit, the spatial relationship between subtasks offloaded to the computing units is utilized to reduce the data traffic and average latency of the on-chip network, making the computing units closer to the theoretical performance in actual tasks and reducing the overall power consumption of the chip; in addition, when multiple tasks are offloaded to the same chip, static offloading can ensure that multiple tasks are allocated to reasonable hardware resources on demand.

[0115] When performing semi-static unloading of subtasks on a single computing chip, task switching of an upstream computing node can be performed during the execution of downstream tasks, thereby saving time overhead on the computing node caused by task switching.

[0116] By leveraging the spatial correlation of components inside the accelerator chip, the hardware architecture supports static or semi-static offloading of deep learning inference tasks, improves computing performance and computing unit utilization, and reduces the traffic and latency of data packets on the on-chip network at the system level, reducing power consumption and improving performance.

[0117] In summary, the embodiment of the present invention provides a data transmission method for a network on chip, which splits a computing task into at least two subtasks; determines the data transmission requirements between the subtasks, the memory access requirements and the external interface access requirements corresponding to each subtask according to the processing stage to which the subtasks belong and the dependency relationship between the subtasks; determines the unloading strategy corresponding to the subtask based on the data transmission requirements, memory access requirements and external interface access requirements; and unloads the subtasks to a target computing node based on the unloading strategy. The embodiment of the present invention can first split the computing task; then determine the unloading strategy based on the processing stage to which the subtasks after the split belong and the dependency relationship between the subtasks, and then can unload the subtasks to the target computing node according to the unloading strategy, thereby reducing the design complexity and time overhead of the computing unit, improving the utilization rate of the computing unit, and helping to improve the overall computing efficiency of the network on chip.

[0118] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0119] Device Embodiment Reference Figure 4 , shows a structural block diagram of a data transmission device of a network on chip of the present invention, the device is applied to the network on chip, and the device may specifically include: A splitting module 401 is used to split the computing task into at least two subtasks; The determination module 402 is used to determine the data transmission requirements between the subtasks, the memory access requirements and the external interface access requirements corresponding to each subtask according to the processing stage to which the subtasks belong and the dependency relationship between the subtasks; and determine the unloading strategy corresponding to the subtask based on the data transmission requirements, the memory access requirements and the external interface access requirements; The unloading module 403 is used to unload the subtask to a target computing node based on the unloading policy.

[0120] Optionally, the determining module includes: An unloading submodule is used to unload the at least two first subtasks to at least two adjacent first computing nodes if the data to be transmitted between at least two first subtasks meets a first preset condition; to unload the at least two second subtasks to at least two adjacent second computing nodes if there is data dependency between at least two second subtasks; and to unload the third subtask to an edge computing node of the on-chip network if at least one third subtask exchanges data with a storage module or an external interface of the on-chip network.

[0121] Optionally, the step of offloading the subtask to a target computing node based on the offloading strategy includes: A cascading module, used for cascading at least two on-chip networks when the computing capacity of the on-chip network is insufficient; The first unloading submodule is used to unload the subtask to the target computing nodes of the at least two on-chip networks based on the unloading strategy; or, according to the data correlation between the subtasks, unload the at least two subtasks to the target computing nodes of the on-chip network in a time-sharing manner.

[0122] Optionally, the computing task includes a deep learning reasoning task; and splitting the computing task into at least two subtasks includes: A determination submodule, used to determine an input layer, at least one intermediate layer, and an output layer according to the deep learning reasoning task; A partitioning module is used to divide each processing stage of the deep learning reasoning task into at least one subtask based on the data dependency between the input layer, the at least one intermediate layer and the output layer, and a preset granularity.

[0123] Optionally, the method further comprises: A first obtaining submodule, configured to add identification information to the first data to obtain second data after the third computing node obtains the first data by executing the subtask of the computing task, wherein the identification information is used to indicate the order in which the first data is used in subsequent computing; The first sending submodule is used to send the second data to the fourth computing node, so that the fourth computing node reorders the received second data according to the identification information in the second data and continues to execute the computing task.

[0124] Optionally, after the subtask is unloaded to the target computing node based on the unloading strategy, the method further includes: A third determination submodule is used to determine the target multicast domain to which the target computing node belongs; The second obtaining submodule is used to add a multicast control field to the first data to be transmitted according to the target multicast domain to which the target computing node belongs, so as to obtain the target data; the multicast control field includes a multicast domain indication field and an intra-domain mask field, the multicast domain indication field is used to indicate the target multicast domain, and the intra-domain mask field is used to indicate the target computing node in the target multicast domain; according to the multicast control field and the routing algorithm, determine a first routing path outside the target multicast domain; a transmission submodule, configured to transmit the target data to the target multicast domain based on the first routing path; A fourth determination submodule, configured to determine a second routing path within the target multicast domain according to the intra-domain mask field; The second sending submodule is used to send the target data to the target computing node based on the second routing path, so that the target computing node executes the subtask based on the target data.

[0125] In summary, the embodiment of the present invention provides a data transmission method for a network on chip, which splits a computing task into at least two subtasks; determines the data transmission requirements between the subtasks, the memory access requirements and the external interface access requirements corresponding to each subtask according to the processing stage to which the subtasks belong and the dependency relationship between the subtasks; determines the unloading strategy corresponding to the subtask based on the data transmission requirements, memory access requirements and external interface access requirements; and unloads the subtasks to a target computing node based on the unloading strategy. The embodiment of the present invention can first split the computing task; then determine the unloading strategy based on the processing stage to which the subtasks after the split belong and the dependency relationship between the subtasks, and then can unload the subtasks to the target computing node according to the unloading strategy, thereby reducing the design complexity and time overhead of the computing unit, improving the utilization rate of the computing unit, and helping to improve the overall computing efficiency of the network on chip.

[0126] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0127] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0128] Regarding the processor in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0129] Reference Figure 5, is a structural block diagram of an electronic device for data transmission in a network on chip provided by an embodiment of the present invention. Figure 5 As shown, the electronic device includes: a processor, a memory, a communication interface and a communication bus, and the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store executable instructions, and the executable instructions enable the processor to execute the data transmission method of the on-chip network of the aforementioned embodiment.

[0130] The processor may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other editable devices, transistor logic devices, hardware components or any combination thereof. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0131] The communication bus may include a path for transmitting information between the memory and the communication interface. The communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The fact that only one line is used in the diagram does not mean that there is only one bus or only one type of bus.

[0132] The memory may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0133] The embodiment of the present invention also provides a non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device (server or terminal), the processor can execute Figure 1 The data transmission method of the on-chip network is shown.

[0134] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0135] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, devices or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a predictable manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0139] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0140] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0141] The above is a detailed introduction to the data transmission method, device, electronic device and storage medium of a network on chip provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data transmission method for a network on chip, characterized in that: Applied to a network on chip, the method comprises: Split the computational task into at least two subtasks; Determine the data transmission requirements between the subtasks, the memory access requirements and the external interface access requirements corresponding to each subtask according to the processing stage to which the subtasks belong and the dependencies between the subtasks; Determining an offloading strategy corresponding to the subtask based on the data transmission requirement, the memory access requirement, and the external interface access requirement; Based on the offloading strategy, the subtask is offloaded to a target computing node.

2. The method according to claim 1, characterized in that: The determining the offloading strategy corresponding to the subtask based on the data transmission requirement, the memory access requirement, and the external interface access requirement includes: If there is data to be transmitted between at least two first subtasks that meets the first preset condition, offloading the at least two first subtasks to at least two adjacent first computing nodes; If there is data dependency between at least two second subtasks, offloading the at least two second subtasks to at least two adjacent second computing nodes; If there is at least one third subtask that exchanges data with a storage module or an external interface of the network on chip, the third subtask is unloaded to an edge computing node of the network on chip.

3. The method according to claim 1, characterized in that The step of offloading the subtask to a target computing node based on the offloading strategy includes: When the computing capacity of the on-chip network is insufficient, cascading at least two on-chip networks; Based on the offloading strategy, offloading the subtask to the target computing nodes of the at least two on-chip networks; or, The at least two subtasks are time-sharedly offloaded to a target computing node of the on-chip network according to the data correlation between the subtasks.

4. The method according to claim 1, characterized in that: The computing tasks include deep learning reasoning tasks; The step of dividing the computing task into at least two subtasks includes: According to the deep learning reasoning task, determining an input layer, at least one intermediate layer, and an output layer; Based on the data dependency between the input layer, the at least one intermediate layer and the output layer, and a preset granularity, each processing stage of the deep learning reasoning task is divided into at least one subtask.

5. The method according to claim 1, characterized in that The method further comprises: After the third computing node obtains the first data after executing the subtask of the computing task, adding identification information to the first data to obtain second data; the identification information is used to indicate the order in which the first data is used in subsequent computing; The second data is sent to a fourth computing node, so that the fourth computing node reorders each received second data according to the identification information in the second data, and continues to execute the computing task.

6. The method according to claim 1, characterized in that The method further comprises: Determine a target multicast domain to which the target computing node belongs; According to the target multicast domain to which the target computing node belongs, a multicast control field is added to the first data to be transmitted to obtain the target data; the multicast control field includes a multicast domain indication field and an intra-domain mask field, the multicast domain indication field is used to indicate the target multicast domain, and the intra-domain mask field is used to indicate the target computing node in the target multicast domain; Determining a first routing path outside the target multicast domain according to the multicast control field and the routing algorithm; transmitting the target data to the target multicast domain based on the first routing path; Determining a second routing path within the target multicast domain according to the intra-domain mask field; The target data is sent to the target computing node based on the second routing path, so that the target computing node executes the subtask based on the target data.

7. A data transmission device for a network on chip, characterized in that: Applied to a network on chip, the device comprises: A splitting module, used for splitting a computing task into at least two subtasks; A determination module, used to determine the data transmission requirements between the subtasks, the memory access requirements and the external interface access requirements corresponding to each subtask according to the processing stage to which the subtasks belong and the dependencies between the subtasks; and determine the unloading strategy corresponding to the subtask based on the data transmission requirements, the memory access requirements and the external interface access requirements; The unloading module is used to unload the subtask to the target computing node based on the unloading strategy.

8. The device according to claim 7, characterized in that The determining module comprises: An unloading submodule is used to unload the at least two first subtasks to at least two adjacent first computing nodes if the data to be transmitted between at least two first subtasks meets a first preset condition; to unload the at least two second subtasks to at least two adjacent second computing nodes if there is data dependency between at least two second subtasks; and to unload the third subtask to an edge computing node of the on-chip network if at least one third subtask exchanges data with a storage module or an external interface of the on-chip network.

9. An electronic device, characterized in that: The electronic device includes a processor, a memory, a communication interface and a communication bus, and the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store executable instructions, and the executable instructions enable the processor to execute the data transmission method of the on-chip network as described in any one of claims 1 to 6.

10. A readable storage medium, characterized in that: When the instructions in the readable storage medium are executed by a processor of an electronic device, the processor is enabled to execute the data transmission method of the on-chip network as claimed in any one of claims 1 to 6.

Images