Method, device and electronic device for determining data transmission path

Abstract

The present invention discloses a method, device and electronic device for determining a data transmission path, and relates to the field of artificial intelligence technology. The method includes: obtaining at least one data to be transmitted, wherein at least one data to be transmitted is data for communication between different processing units contained in a target computing system; inputting at least one data to be transmitted into a target path planning model in sequence according to a target calculation order, and outputting a target path for each data to be transmitted, wherein the target path planning model is obtained based on training of a preset algorithm, and the target path is a path with the shortest link contention time within a preset range, and the link contention time is the time required to wait when the data to be transmitted is transmitted along the link of the current routing set. The present invention solves the technical problem of communication link contention in a computing system based on an on-chip network in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device and electronic device for determining a data transmission path. Background Art

[0002] Due to their high performance and low energy consumption, heterogeneous computing systems (HCSs) based on network-on-chip (NoC) are widely used to process computing tasks. NoC-based HCSs are composed of processing elements (PEs) with different performance. As more and more cores are integrated onto chips, the contention problem of communication links becomes more and more serious during on-chip communication between different PEs, which increases unnecessary link waiting time and reduces the computing efficiency of the entire HCSs.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide a method, device and electronic device for determining a data transmission path, so as to at least solve the technical problem of communication link contention in a computing system based on an on-chip network in the prior art.

[0005] According to one aspect of an embodiment of the present invention, a method for determining a data transmission path is provided, comprising: obtaining at least one data to be transmitted, wherein at least one data to be transmitted is data for communication between different processing units included in a target computing system; inputting at least one data to be transmitted into a target path planning model in sequence according to a target calculation order, and outputting a target path for each data to be transmitted, wherein the target path planning model is obtained by training based on a preset algorithm, the target path is a path with the shortest link contention time within a preset range, and the link contention time is the waiting time required for the data to be transmitted to be transmitted along the links of the current routing set.

[0006] Furthermore, the target path planning model is generated by the following method: obtaining at least one sample data to be transmitted; training the first neural network and the second neural network of the preset algorithm according to the at least one sample data to be transmitted to obtain the target path planning model.

[0007] Furthermore, the method for determining the data transmission path also includes: selecting a first sample data to be transmitted from at least one sample data to be transmitted; obtaining a current position of the first sample data to be transmitted in the transmission link to obtain a first router, wherein the first router is used to transmit the first sample data to be transmitted; determining at least one transmission direction of the first sample data to be transmitted according to the first neural network, wherein at least one transmission direction is used to indicate that the first sample data to be transmitted is transmitted from the first router to the target router; and recording at least one transmission direction to obtain a routing set of the first sample data to be transmitted.

[0008] Furthermore, the method for determining the data transmission path also includes: evaluating the route set according to the second neural network to obtain an evaluation result, wherein the evaluation result is used to characterize the length of the link contention time of the route set; based on the evaluation result, determining the target path of the first sample data to be transmitted.

[0009] Furthermore, the method for determining the data transmission path also includes: obtaining a link state set occupied by the routes in the current route set, wherein the link state set is used to characterize the usage of the link; based on the link state set, calculating the link contention time of the current route set to obtain an evaluation result.

[0010] Furthermore, the method for determining the data transmission path also includes: comparing the link contention time of the current route set with the link contention time of the target route set; updating the target route set when the link contention time of the current route set is less than the link contention time of the target route set; and generating a target path based on the target route set.

[0011] Furthermore, the method for determining the data transmission path also includes: evaluating at least one transmission direction according to the second neural network to obtain a reward value for at least one transmission direction, wherein the reward value is used to measure the pros and cons of at least one transmission direction; calculating an advantage value for at least one transmission direction according to the reward value, wherein the advantage value represents the degree of superiority of a current state of the first sample data to be transmitted compared to a target state, and the target state represents the average value of all states of the first sample data to be transmitted; updating the first neural network and the second neural network based on the advantage value to obtain an updated first neural network and an updated second neural network; generating a target path planning model when the updated first neural network and the updated second neural network meet preset conditions.

[0012] Furthermore, the method for determining the data transmission path also includes: based on a preset principle, sorting at least one data to be transmitted to obtain a sorting result; and according to the sorting result, determining the order of performing routing calculation on at least one data to be transmitted to obtain a target calculation order.

[0013] According to another aspect of an embodiment of the present invention, a device for determining a data transmission path is also provided, including: an acquisition module, used to acquire at least one data to be transmitted, wherein at least one data to be transmitted is data for communication between different processing units included in a target computing system; a determination module, used to input at least one data to be transmitted into a target path planning model in sequence according to a target calculation order, and output a target path for each data to be transmitted, wherein the target path planning model is obtained through training based on a preset algorithm, and the target path is a path with the shortest link contention time within a preset range, and the link contention time is the waiting time required for the data to be transmitted to be transmitted along the links of the current routing set.

[0014] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for determining a data transmission path when running.

[0015] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, which includes one or more processors; a memory for storing one or more programs, which, when the one or more programs are executed by the one or more processors, enables the one or more processors to run the programs, wherein the programs are configured to execute the above-mentioned method for determining the data transmission path when running.

[0016] In an embodiment of the present invention, a method of planning a data transmission path by using a reinforcement learning algorithm is adopted, firstly, at least one data to be transmitted is obtained, and then at least one data to be transmitted is sequentially input into a target path planning model according to a target calculation order, and a target path of each data to be transmitted is output. Among them, the target path planning model is obtained by training based on a preset algorithm, the target path is a path with the shortest link contention time within a preset range, the link contention time is the waiting time required for the data to be transmitted to be transmitted along the link of the current routing set, and at least one data to be transmitted is data for communication between different processing units included in the target computing system.

[0017] In the above process, by obtaining at least one data to be transmitted, a data basis is provided for the subsequent transmission route calculation; based on the target path planning model, the transmission link of the data to be transmitted can be calculated, so that the planning of the transmission link can be realized, and then the data transmission path with the shortest link contention time can be obtained, and the process of transmitting the data to be transmitted along the transmission link with the shortest link contention time is realized, which reduces the communication conflicts in the data transmission process, thereby reducing the waiting time for data transmission, thereby improving the computing efficiency of the computing system and ensuring the high-speed operation of the computing system.

[0018] It can be seen that through the technical solution of the present invention, the purpose of reducing the communication link contention in the computing system based on the on-chip network is achieved, thereby realizing the technical effect of reducing the link waiting time and improving the computing efficiency of the computing system, and further solving the technical problem of communication link contention in the computing system based on the on-chip network in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 is a flow chart of an optional method for determining a data transmission path according to an embodiment of the present invention;

[0021] Figure 2 is an optional computing task graph according to an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of an optional NoC-based HCSs architecture according to an embodiment of the present invention;

[0023] Figure 4 is a flow chart of an optional route planning based on a dominant actor critic algorithm according to an embodiment of the present invention;

[0024] Figure 5 is a flowchart of an optional method for determining the operation of a computing task according to an embodiment of the present invention;

[0025] Figure 6 is a schematic diagram of an optional device for determining a data transmission path according to an embodiment of the present invention;

[0026] Figure 7 is a schematic diagram of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set between the system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0030] Example 1

[0031] According to an embodiment of the present invention, a method embodiment of a method for determining a data transmission path is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 1 is a flow chart of an optional method for determining a data transmission path according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0033] Step S101 , obtaining at least one piece of data to be transmitted, wherein the at least one piece of data to be transmitted is data for communication between different processing units included in a target computing system.

[0034] In the above steps, at least one piece of data to be transmitted may be obtained through an application system, a processor, an electronic device, or the like. Figure 2 is an optional computing task graph according to an embodiment of the present invention, such as Figure 2As shown, it includes 4 circular nodes v0~v3, each of which represents a computing task. Each computing task needs to be handed over to the processing unit PE for processing. The unidirectional edge between the nodes represents the pre-relationship between the tasks, that is, Figure 2 In the example, v0, v1, and v2 are the pre-tasks of v3. Task v3 needs to wait for v0, v1, and v2 to complete their calculations and transfer the calculation result data to v3 before it can start executing. Optionally, the data to be transferred can be the above calculation result data.

[0035] It should be noted that, in the above process, by obtaining at least one piece of data to be transmitted, a data basis is provided for subsequent transmission route calculation.

[0036] Step S102, input at least one data to be transmitted into the target path planning model in sequence according to the target calculation order, and output the target path of each data to be transmitted, wherein the target path planning model is obtained based on the training of the preset algorithm, and the target path is the path with the shortest link contention time within the preset range, and the link contention time is the waiting time required for the data to be transmitted to be transmitted along the link of the current routing set.

[0037] In the above steps, the default algorithm is the Advantage Actor-Critic (A2C) algorithm. A2C is a reinforcement learning algorithm composed of an "actor" and a "critic". Reinforcement learning is a field under machine learning that focuses on training the subject to act based on the current environment to maximize the expected benefits. The training process of reinforcement learning is generally as follows: the subject (agent) observes the environment (environment) at time t and learns that it is in state (state) S t , then the subject selects an action a from the action space A t Execution, the environment will follow the state transition function From state S t Transition to S t+1 . Then the reward function R(S t ) gives a new state S t+1 The reward value is used to train the subject to learn the value of this action. This process will continue until the current state has caused the environment to end (for example, when the environment is a maze, the subject has walked out of the maze) or the developer terminates it early.

[0038] Specifically, the Actor and Critic can be regarded as two neural networks. The Actor receives the state S of the subject at time t. t As input, and output the subject in state S t The action that should be performed nextt , Critic will comment on action a t , that is, output a value Q(S t , a t ) is used to measure the t Take action a t The value of, then, through Q(S t , a t ) to further calculate the advantage value A(S t , a t ), the advantage value is calculated by A(S t , a t )=Q(S t , a t )-V(S t ), where V(S t ) is state S t The average value is given by t The state-action value Q(S, a) corresponding to all possible actions is multiplied by the probability of taking the action and then summed.

[0039] Optionally, Figure 2 The computing tasks in the mapping are transformed into PE as follows Figure 3 The NoC-based HCS shown s The architecture diagram shows the data transmission routing more intuitively, such as Figure 3 As shown in the figure, the circular nodes are PEs with computing capabilities. The computing tasks are assigned to PEs for execution. After the execution is completed, the computing result data is transmitted to the next computing task (if any) through the router and the communication link. The square nodes are routers connected to PEs, which are responsible for the on-chip communication of NoC. Each router and the router and PE can be connected together through bidirectional transmission links for data transmission and interaction. Optionally, the link from router m to router n is denoted as l m,n .

[0040] Optionally, since each router and the router and PE can be connected together through bidirectional transmission links for data transmission and interaction, there will be link contention. Figure 3 As shown in the figure, suppose that the three tasks v0, v1, and v2 are calculated at the same time and are about to transmit data to v3. At this time, the data transmission from task v0 to v3 needs to pass through link l 0,1 ,l 1,4 , and the data transmission from task v1 to v3 also needs to pass through link l 1,4 The data transmission from task v2 to v3 is the same, so the three calculation result data are transmitted on link l 1,4 Contention occurs, and you need to wait for other data transmissions to complete before you can use the link l1,4 , which reduces the computing efficiency of the entire HCSs and increases unnecessary waiting time. Therefore, in the present invention, after obtaining at least one data to be transmitted, based on the dominant actor-critic algorithm, the routing calculation is performed on at least one data to be transmitted respectively, and a routing set of each data to be transmitted can be obtained, thereby providing a data basis for path planning.

[0041] Optionally, the routing set may be the link through which the calculation result data is transmitted from the current router to the target router, for example, Figure 3 The data transmission from task v0 to v3 must pass through link l 0,1 ,l 1,4 , link l 0,1 and link l 1,4 Constitutes a routing set {l 0,1 ,l 1,4 The preset range can be the number of training times set by the developer according to the needs. Figure 3 Taking the data transmission from task v0 to v3 as an example, the number of training times is set to 10 when the routing calculation starts. The routing set obtained in the first training is {l 0,1 ,l 1,4}, the routing set corresponds to a link contention time, and the routing set obtained by the second training is {l 0,3 ,l 3,4}, this routing set also corresponds to a link contention time, and so on, until the end of the training, the target path is the path with the shortest link contention time in these 10 trainings. Among them, since different PEs take different times to process different computing tasks, and for complex task graphs, there are multiple data transmissions in parallel, so there is link contention time.

[0042] It should be noted that in the above process, by outputting the target path of each data to be transmitted, the data to be transmitted can be transmitted along the transmission link with the shortest link contention time, thereby reducing communication conflicts during data transmission, thereby reducing the waiting time for data transmission, and further improving the computing efficiency of the computing system.

[0043] Based on the scheme defined by the above steps S101 to S102, it can be known that in an embodiment of the present invention, a method of planning a data transmission path by using a reinforcement learning algorithm is adopted, firstly, at least one data to be transmitted is obtained, and then at least one data to be transmitted is sequentially input into a target path planning model according to a target calculation order, and a target path of each data to be transmitted is output. Among them, the target path planning model is obtained based on training of a preset algorithm, the target path is a path with the shortest link contention time within a preset range, the link contention time is the waiting time required for the data to be transmitted to be transmitted along the link of the current routing set, and at least one data to be transmitted is data for communication between different processing units included in the target computing system.

[0044] It is easy to notice that in the above process, by obtaining at least one data to be transmitted, a data basis is provided for the subsequent transmission route calculation; based on the target path planning model, the transmission link of the data to be transmitted can be calculated, so that the planning of the transmission link can be realized, and then the data transmission path with the shortest link contention time can be obtained, and the process of transmitting the data to be transmitted along the transmission link with the shortest link contention time is realized, which reduces the communication conflicts in the data transmission process, thereby reducing the waiting time for data transmission, thereby improving the computing efficiency of the computing system and ensuring the high-speed operation of the computing system.

[0045] It can be seen that through the technical solution of the present invention, the purpose of reducing the communication link contention in the computing system based on the on-chip network is achieved, thereby realizing the technical effect of reducing the link waiting time and improving the computing efficiency of the computing system, and further solving the technical problem of communication link contention in the computing system based on the on-chip network in the prior art.

[0046] In an optional embodiment, the target path planning model is generated by the following method: obtaining at least one sample data to be transmitted; training the first neural network and the second neural network of the preset algorithm according to the at least one sample data to be transmitted to obtain the target path planning model.

[0047] In an optional embodiment, after obtaining at least one sample data to be transmitted, a first sample data to be transmitted is selected from the at least one sample data to be transmitted, and then the current position of the first sample data to be transmitted in the transmission link is obtained to obtain a first router, and then at least one transmission direction of the first sample data to be transmitted is determined according to the first neural network, and then at least one transmission direction is recorded to obtain a routing set of the first sample data to be transmitted. The first router is used to transmit the first sample data to be transmitted, and at least one transmission direction is used to indicate that the first sample data to be transmitted is transmitted from the first router to the target router.

[0048] Alternatively, although reinforcement learning is widely known as a relatively mature technology in the field of artificial intelligence, due to the complexity of the task graphs currently used, each task graph usually has dozens or even hundreds of data transmissions that require routing planning. Therefore, it is still difficult to apply reinforcement learning to routing planning for on-chip network communications.

[0049] Optionally, firstly model the environment, the training agent, the action space and the reward, so that the reinforcement learning algorithm can be applied to the routing planning of the on-chip network communication. Specifically, the concepts in the reinforcement learning corresponding to the data in the NoC are defined: the environment is defined as the occupancy of the communication link in the NoC. It should be noted that the communication link may be occupied by other data to be transmitted in a certain period of time. The training agent is defined as a certain piece of data m to be transmitted (i.e., the sample data to be transmitted). Although the data is a continuous process during transmission, in the process of routing planning, the data can be regarded as hop-by-hop transmission between each router. When the agent reaches a router, it needs to take another action to reach the next router. In order to ensure that the data can eventually reach the destination router, it is stipulated that the data is not transmitted in the reverse direction. Therefore, the actions of the training agent are defined as two types, namely, transmission along the X axis to the destination router and transmission along the Y axis to the destination router. The state is defined as the router that has been passed during the data transmission. The reward of a certain state is defined as the inverse of the link contention time when data is transmitted along the route corresponding to the state, that is, the shorter the contention time, the higher the reward.

[0050] Optional, with Figure 2 The task graph and Figure 3 Take the NoC architecture in as an example, assuming that task v0 needs to transmit data to v3, that is, it needs to calculate the route from router 0 to router 4. The initial state of this data transmission is S0 = {0}, that is, the data initially stays at router 0. In the next time step, the data is transmitted one hop downward and reaches router 3. At this time, the state is S1 = {0,3}, and it has not yet reached the destination router 4; in the next time step, the data is transmitted one hop to the right and reaches router 4. At this time, the state is S2 = {0,3,4}, and the data reaches the destination router. Among them, the state of each time step can correspond to a route. For example, the route represented by state S2 is that the data is transmitted from router 0 to router 3, and then from router 3 to router 4.

[0051] Optionally, after defining the concepts in reinforcement learning corresponding to the data in the NoC, a reinforcement learning algorithm can be used for training. Optionally, in the embodiment of the present invention, the dominant actor-critic algorithm (i.e., the preset algorithm) is used for training. Specifically, at time step t, the training subject has reached a router but has not yet reached the final destination, waiting for the "actor" to tell the training subject which direction it should transmit in the next time step. The "actor" gives an action A t , train the subject to perform action A t and transmitted to the next router, the current state is determined by S t Transfer to S t+1 After that, the "Critic" will give a response to the state S based on its current understanding. t and S t+1 , and then calculates the advantage value based on the given reward, and then uses the advantage value to update the strategies of the "actor" and "critic" so that they can make smarter choices next time. Optional, Figure 4 is an optional flowchart of performing route planning based on a dominant actor critic algorithm according to an embodiment of the present invention, such as Figure 4 As shown in the figure, given a data to be transmitted m and NoC communication link usage u, and initialized, As the best state, then determine whether the maximum number of training times has been reached. If so, return S b ; If the maximum number of training times is not reached, the state of the training subject is initialized to S = S0. Then, determine whether the state S has ended. If so, compare the current state with the optimal state S b , if S is better, update the state to S b =S; if state S is not ended, the actor takes action A t , the subject performs action A t , the current state is S t Transfer to S t+1 , then the "critic" evaluates the state S according to u t and S t+1 , then calculates the advantage value and updates the strategies of the "actor" and "critic".

[0052] Optionally, for a computing task graph that needs to transmit data multiple times, each outgoing edge in the task graph represents a data transmission. During the calculation process, the routing planning method based on the dominant actor-critic algorithm needs to be called multiple times to calculate the route for each data transmission. Therefore, it is first necessary to sort the outgoing edges of the routes to be calculated, that is, the target calculation order, to decide which outgoing route to calculate first. The specific sorting method is described in detail later.

[0053] Optionally, after obtaining at least one sample data to be transmitted, the data to be first subjected to routing calculation is selected, i.e., the first sample data to be transmitted, and then the current position of the data in the transmission link is obtained. For example, the sample data to be transmitted initially stays at router No. 0, and the current position is router No. 0, i.e., the first router.

[0054] Furthermore, according to the first neural network of the preset algorithm, at least one transmission direction of the first sample data to be transmitted can be determined. Optionally, the transmission direction can be transmission along the X-axis to the destination router, or transmission along the Y-axis to the destination router. Specifically, assuming that the data to be transmitted currently stays at router No. 0, at this time, the initial state is S0, and the routing set in the routing planning scheme is empty. Then, the training subject, i.e., the data to be transmitted, asks the Actor, i.e., the first neural network, what action it should perform now, i.e., whether it should be transmitted along the X-axis to the destination router or along the Y-axis to the destination router. The Actor will select an action from all executable actions, assuming it is a0 (transmission along the X-axis to the destination router), and let the training subject, i.e., the data to be transmitted, execute it. For example, the Actor tells the training subject, i.e., the data to be transmitted, that is, it should be transmitted along the X-axis to the destination router, occupying link l. 0,1 to transfer.

[0055] Furthermore, at least one transmission direction is recorded to obtain a routing set of the first sample data to be transmitted. Specifically, after the training subject, i.e., the data to be transmitted, performs the above actions, it arrives at router No. 1, at which point its state changes from S0 to S1, and the routing set in the routing plan is added with l 0,1 , becomes {l 0,1 At the next moment, the above steps are repeated, and the training subject, i.e., the data to be transmitted, continues to query the Actor, i.e., the first neural network. At this time, the Actor tells the training subject, i.e., the data to be transmitted, that it should be transmitted along the Y axis to the destination router, i.e., occupying link l. 1,4 After the training subject has completed the above actions, the data to be transmitted arrives at router 4, and its state changes from S1 to S2. The routing set in the routing plan is added with l 1,4 , becomes {l 0,1 ,l 1,4}, since router 4 is the end point of the execution body, state S2 is the end, and the routing set {l 0,1 ,l 1,4} is the route planned for the data m to be transmitted in this training.

[0056] In an optional embodiment, after recording at least one transmission direction to obtain a route set of the first sample data to be transmitted, the route set is first evaluated according to the second neural network to obtain an evaluation result, and then based on the evaluation result, a target path of the first sample data to be transmitted is determined. The evaluation result is used to characterize the length of the link contention time of the route set.

[0057] Among them, in an optional embodiment, in the process of determining the target path of the first sample data to be transmitted based on the evaluation result, the link contention time of the current route set is compared with the link contention time of the target route set. When the link contention time of the current route set is less than the link contention time of the target route set, the target route set is updated, and then the target path is generated according to the target route set.

[0058] Optionally, the target route set can be a separately recorded best route set. After a new round of training and learning, the model gives the current route set, i.e., the current route set, and compares the link contention time of the current route set with the link contention time corresponding to the separately recorded best route set, i.e., the target route set. When the link contention time of the current route set is less than the link contention time of the target route set, the recorded best route set is updated, and then the target path is generated based on the best route set.

[0059] In an optional embodiment, in the process of evaluating the route set according to the second neural network to obtain the evaluation result, the link state set occupied by the routes in the current route set is obtained, and then the link contention time of the current route set is calculated based on the link state set to obtain the evaluation result. The link state set is used to characterize the usage of the link.

[0060] Optionally, in the process of reinforcement learning training, it is necessary to know the link contention time of waiting when transmitting data before training can be carried out. Generally speaking, the link contention time needs to be run on an actual chip or simulated on a platform such as GEM5 to monitor the clock cycle to obtain it. If each training needs to call GEM5 to completely simulate the task graph operation, this will bring a lot of unnecessary time overhead, and the interaction overhead between cross-platforms is also a problem. Therefore, the present invention provides a method for estimating link contention time, which can be run in all programming languages ​​after a given task graph.

[0061] Specifically, for a data m to be transmitted, given the planned route r and the link usage set U of each communication link in the current NoC, estimate the link contention time that the data needs to wait when transmitting along the route r. The transmission time required to transmit data m depends on the data size of m, and the transmission time of m is recorded as T. The end time of the source task of m is recorded as t1, which is also the earliest time when m can start transmission. Ideally, there is no contention for the link that route r needs to use, so m can start transmission at time t1 and end transmission at time t1+T, and time t1+T is recorded as t2, that is, in the ideal case, the transmission time interval of m is [t1, t2]. At this time, in the link usage set U, for all links in route r, the time interval [t1, t2] is marked as occupied by data m.

[0062] However, in actual scenarios, link contention is unavoidable. At time t1, query the current link usage set U and check whether the communication link required for transmission along route r is occupied by other data transmission. If there is a link in route r occupied by other data transmission, then m needs to wait for time t c Until all links in r are available. In this case, the transmission time interval of m becomes [t1+t c , t2+t c ],t c This is the link contention time that needs to be estimated.

[0063] Therefore, to calculate t c First, we need to query the link usage set U to find out which links in route r are occupied by other data transmissions in the time interval [t1, t2]. Specifically, we obtain the state set of links occupied by all planned routes. The link state set represents the occupancy of each link in all time periods.

[0064] Similarly, when other data occupies a link, it will also give a time interval in U to indicate which time period it needs to occupy the link. Therefore, among all the links occupied by other data transmissions, find the data that ends transmission last in the data occupying the time interval [t1, t2], and record its transmission end time as t′2. Then, t′2-t1 is used as t c The initial value of , and then check the time interval [t1+t c , t2+t c ], are all the links in route r available? If so, then t c is the final result of the estimation; if not, then gradually increase t c The value of (increases by 1 each time) and tries to check in the new time interval [t1+tc , t2+t c ] to check whether all links in route r are available until all links are available.

[0065] Furthermore, we can get the estimated t c After the value of , the link usage set U is updated. For the links used in route r, the time interval [t1+t c , t2+t c ] is marked as occupied by message m to facilitate the next calculation of the contention time of other routes.

[0066] In an optional embodiment, before evaluating the route set according to the second neural network and obtaining the evaluation result, at least one transmission direction is evaluated according to the second neural network to obtain a reward value of at least one transmission direction, and then the advantage value of at least one transmission direction is calculated according to the reward value, and then the first neural network and the second neural network are updated based on the advantage value to obtain an updated first neural network and an updated second neural network, and a target path planning model is generated when the updated first neural network and the updated second neural network meet preset conditions. The reward value is used to measure the quality of at least one transmission direction, the advantage value represents the degree of quality of the current state of the first sample data to be transmitted compared with the target state, and the target state represents the average value of all states of the first sample data to be transmitted.

[0067] Optionally, at time t1, the main agent is in state S(t1). It can reach different states at time t by performing different actions, so there are many possible states S(t). The meaning of advantage value is that after executing the action given by the Actor, its state S(t) has been determined at time t, and then the reward value of this determined S(t) is compared with the average reward value of all possible S(t). That is, the advantage value refers to the advantage of the current determined state S(t) relative to the average state at time t. It should be noted that the use of advantage value is to compare the current state value with the average state, which can speed up the convergence of training.

[0068] Optionally, in the aforementioned process, after the training subject, i.e. the data to be transmitted, executes action a0 informed by the Actor, the Critic, i.e. the second neural network, will evaluate the training subject, i.e. the data to be transmitted, executing action a0 in state S0, and give a reward value reward, and then the advantage value can be calculated based on the reward, and then the strategies of the Actor and the Critic are updated based on the advantage value, so that they become smarter and can make better judgments at the next moment. Optionally, the preset condition can be reaching a preset number of training times. For example, when the preset number of training times is set to 10 times at the beginning of training, after the strategies of the Actor and the Critic are updated 10 times based on the advantage value, the target path planning model is obtained.

[0069] In an optional embodiment, after obtaining at least one data to be transmitted, the at least one data to be transmitted is sorted based on a preset principle to obtain a sorting result, and then the order of routing calculation for the at least one data to be transmitted is determined based on the sorting result to obtain a target calculation order.

[0070] Optionally, as mentioned above, for a computational task graph that needs to transmit data multiple times, each outgoing edge in the task graph represents a data transmission. During the computation process, the routing planning method based on the dominant actor-critic algorithm needs to be called multiple times to calculate the route for each data transmission. Therefore, it is first necessary to sort out an order for the outgoing edges of the routes to be calculated, namely the target calculation order, to determine which outgoing edge route to calculate first. Specifically, first initialize an empty queue that follows the first-in-first-out principle. When the entire algorithm starts to execute, add the outgoing edges of the tasks that can be executed immediately (i.e., without waiting for other predecessor tasks to transmit data to it) to the queue, and arrange the order of the data to be transmitted according to the principle of "short task first", i.e., the preset principle. Take the outgoing edge at the head of the queue each time and calculate the route for it. When the route calculation of an edge is completed, delete it from the queue, and check whether there is a task that can be started after the data transmission corresponding to the outgoing edge is completed. If so, add all the outgoing edges of the task (also arranged in order according to the principle of "short task first") to the queue. When the queue is empty again, the routing planning of all data to be transmitted is completed.

[0071] It should be noted that, in the above process, by sorting at least one data to be transmitted, the calculation order of the data to be transmitted is reasonably planned, which further reduces the waiting time for data transmission and improves the calculation efficiency of the computing system.

[0072] In some other optional embodiments, the routing planning method proposed in the present invention can be applied to the calculation of NoC to improve the calculation speed of NoC. Specifically, in the embodiment of the present invention, a NoC (neural network on chip) based on a two-dimensional mesh structure 2D-Mesh is adopted.

[0073] Optional, Figure 5 is a flowchart of an optional method for determining the operation of a computing task according to an embodiment of the present invention, such as Figure 5 As shown, for a given task graph and the NoC architecture on which the task graph runs, it is first necessary to map each task node (computing task) in the task graph to each PE of the NoC, wherein the mapping scheme can be selected by the developer according to the needs of the existing scheme in the field, which is not limited here. After determining the mapping scheme, the method of routing planning based on the dominant actor critic algorithm in the present invention is used to perform routing planning between the PEs to which the tasks are mapped, calculate the optimal routing transmission scheme, and realize data transmission with minimized link contention time. After obtaining the mapping scheme and the routing planning results, the calculation method containing the mapping scheme and the routing planning results is returned.

[0074] Optionally, the returned calculation method can also be imported into the GEM5 simulator to simulate the operation of the task graph. Specifically, in order to verify the effect of the present invention, three common task graphs, Audiobeam, Autocor, and H.264, were selected as experimental objects. The detailed parameters of the task graph are shown in Table 1. Among them, the in-degree / out-degree represents the maximum in-degree / out-degree of the node in the task graph.

[0075] Table 1 Task graph parameters

[0076]

[0077] Specifically, a NoC based on a two-dimensional mesh structure 2D-Mesh was set up as an experimental platform on the GEM5 simulator platform, and experiments were conducted in two sizes of 8×8 and 16×16 and compared with other routing planning methods. The NoC infrastructure used in the experiment is ArSMART, which is an improved SMART NoC design that supports arbitrary turn transmission.

[0078] Specifically, in terms of algorithm time consumption, the method proposed in the present invention has significantly reduced time consumption and runs quickly compared with the current SOTA routing planning method. As shown in Table 2, the time in Table 2 is the time required for running the routing planning method once to perform routing planning for the task graph under fixed allocation.

[0079] Table 2 Algorithm time consumption details

[0080]

[0081] Specifically, in terms of reducing link contention time, the method proposed in the present invention is compared with the widely used XY routing algorithm and the current SOTA routing planning method, and both have achieved results that are not inferior to the prior art. As shown in Table 3, the time in Table 3 is the contention time waiting when transmitting data according to the route proposed by the routing planning method, and the unit of measurement is the number of clock cycles.

[0082] Table 3 Contention time details

[0083]

[0084]

[0085] It can be seen that through the technical solution of the present invention, the purpose of reducing the communication link contention in the computing system based on the on-chip network is achieved, thereby realizing the technical effect of reducing the link waiting time and improving the computing efficiency of the computing system, and further solving the technical problem of communication link contention in the computing system based on the on-chip network in the prior art.

[0086] Example 2

[0087] According to an embodiment of the present invention, an embodiment of a device for determining a data transmission path is provided, wherein: Figure 6 is a schematic diagram of an optional device for determining a data transmission path according to an embodiment of the present invention, such as Figure 6 As shown, the device includes: an acquisition module 601, used to acquire at least one data to be transmitted, wherein the at least one data to be transmitted is data for communication between different processing units included in the target computing system; a determination module 602, used to input the at least one data to be transmitted into the target path planning model in sequence according to the target calculation order, and output the target path of each data to be transmitted, wherein the target path planning model is obtained based on the training of the preset algorithm, the target path is the path with the shortest link contention time within the preset range, and the link contention time is the waiting time required for the data to be transmitted to be transmitted along the link of the current routing set.

[0088] It should be noted that the above-mentioned acquisition module 601 and determination module 602 correspond to steps S101 to S102 in the above-mentioned embodiment. The examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment 1.

[0089] Optionally, the device for determining the data transmission path also includes: a first acquisition module, used to acquire at least one sample data to be transmitted; and a first training module, used to train the first neural network and the second neural network of the preset algorithm according to the at least one sample data to be transmitted to obtain a target path planning model.

[0090] Optionally, the device for determining the data transmission path also includes: a second acquisition module, used to select the first sample data to be transmitted from at least one sample data to be transmitted; a third acquisition module, used to obtain the current position of the first sample data to be transmitted in the transmission link, and obtain the first router, wherein the first router is used to transmit the first sample data to be transmitted; a first determination module, used to determine at least one transmission direction of the first sample data to be transmitted according to the first neural network, wherein at least one transmission direction is used to indicate that the first sample data to be transmitted is transmitted from the first router to the target router; a first recording module, used to record at least one transmission direction, and obtain a routing set of the first sample data to be transmitted.

[0091] Optionally, the device for determining the data transmission path also includes: a first evaluation module, used to evaluate the route set according to the second neural network to obtain an evaluation result, wherein the evaluation result is used to characterize the length of the link contention time of the route set; a second determination module, used to determine the target path of the first sample data to be transmitted based on the evaluation result.

[0092] Optionally, the first evaluation module includes: a fourth acquisition module, used to obtain the link state set occupied by the routes in the current route set, wherein the link state set is used to characterize the usage of the link; a first calculation module, used to calculate the link contention time of the current route set based on the link state set to obtain an evaluation result.

[0093] Optionally, the second determination module includes: a first comparison module, used to compare the link contention time of the current route set with the link contention time of the target route set; a first update module, used to update the target route set when the link contention time of the current route set is less than the link contention time of the target route set; and a first generation module, used to generate a target path based on the target route set.

[0094] Optionally, the device for determining the data transmission path also includes: an evaluation module, used to evaluate at least one transmission direction according to the second neural network to obtain a reward value for at least one transmission direction, wherein the reward value is used to measure the pros and cons of at least one transmission direction; a second calculation module, used to calculate an advantage value of at least one transmission direction according to the reward value, wherein the advantage value represents the degree of superiority of a current state of the first sample data to be transmitted compared with a target state, and the target state represents the average value of all states of the first sample data to be transmitted; a second update module, used to update the first neural network and the second neural network based on the advantage value to obtain an updated first neural network and an updated second neural network; a second generation module, used to generate a target path planning model when the updated first neural network and the updated second neural network meet preset conditions.

[0095] Optionally, the device for determining the data transmission path also includes: a sorting module, which is used to sort at least one data to be transmitted based on a preset principle to obtain a sorting result; and a third determination module, which is used to determine the order of routing calculation for at least one data to be transmitted based on the sorting result to obtain a target calculation order.

[0096] Example 3

[0097] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for determining a data transmission path when running.

[0098] Example 4

[0099] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, wherein: Figure 7 is a schematic diagram of an optional electronic device according to an embodiment of the present invention, such as Figure 7 As shown, the electronic device includes one or more processors; a memory for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors are used to run the programs, wherein the programs are set to execute the above-mentioned method for determining the data transmission path when running. When the processor executes the program, the following steps are implemented: obtaining at least one data to be transmitted, wherein at least one data to be transmitted is data for communication between different processing units included in the target computing system; inputting at least one data to be transmitted into the target path planning model in sequence according to the target calculation order, and outputting the target path of each data to be transmitted, wherein the target path planning model is obtained based on the training of the preset algorithm, and the target path is the path with the shortest link contention time within the preset range, and the link contention time is the time required to wait when the data to be transmitted is transmitted along the link of the current routing set.

[0100] Optionally, when the processor executes the program, the following steps are also implemented: generating a target path planning model by the following method: obtaining at least one sample data to be transmitted; training a first neural network and a second neural network of a preset algorithm according to at least one sample data to be transmitted to obtain a target path planning model.

[0101] Optionally, when executing the program, the processor also implements the following steps: selecting a first sample data to be transmitted from at least one sample data to be transmitted; obtaining a current position of the first sample data to be transmitted in the transmission link, and obtaining a first router, wherein the first router is used to transmit the first sample data to be transmitted; determining at least one transmission direction of the first sample data to be transmitted according to the first neural network, wherein at least one transmission direction is used to indicate that the first sample data to be transmitted is transmitted from the first router to the target router; and recording at least one transmission direction to obtain a routing set of the first sample data to be transmitted.

[0102] Optionally, when executing the program, the processor also implements the following steps: evaluating the route set according to the second neural network to obtain an evaluation result, wherein the evaluation result is used to characterize the length of the link contention time of the route set; and determining the target path of the first sample data to be transmitted based on the evaluation result.

[0103] Optionally, when executing the program, the processor also implements the following steps: obtaining a link state set occupied by the routes in the current route set, wherein the link state set is used to characterize the usage of the link; based on the link state set, calculating the link contention time of the current route set to obtain an evaluation result.

[0104] Optionally, when the processor executes the program, the following steps are also implemented: comparing the link contention time of the current route set with the link contention time of the target route set; updating the target route set when the link contention time of the current route set is less than the link contention time of the target route set; generating a target path based on the target route set.

[0105] Optionally, when executing the program, the processor also implements the following steps: based on the second neural network, evaluating at least one transmission direction to obtain a reward value for at least one transmission direction, wherein the reward value is used to measure the pros and cons of at least one transmission direction; based on the reward value, calculating the advantage value of at least one transmission direction, wherein the advantage value represents the degree of superiority or inferiority of a current state of the first sample data to be transmitted compared to a target state, and the target state represents the average value of all states of the first sample data to be transmitted; based on the advantage value, updating the first neural network and the second neural network to obtain an updated first neural network and an updated second neural network; and generating a target path planning model when the updated first neural network and the updated second neural network meet preset conditions.

[0106] Optionally, when executing the program, the processor further implements the following steps: based on a preset principle, sorting at least one data to be transmitted to obtain a sorting result; according to the sorting result, determining the order of routing calculation for at least one data to be transmitted to obtain a target calculation order.

[0107] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0108] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0109] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0111] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0112] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0114] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for determining a data transmission path, characterized in that: include: Acquire at least one piece of data to be transmitted, wherein the at least one piece of data to be transmitted is data for communication between different processing units included in the target computing system; Inputting the at least one data to be transmitted into the target path planning model in sequence according to the target calculation order, and outputting the target path of each data to be transmitted, wherein the target path planning model is obtained by training based on a preset algorithm, and the target path is a path with the shortest link contention time within a preset range, and the link contention time is the waiting time required for the data to be transmitted to be transmitted along the link of the current routing set; The target path planning model is generated by the following method: Obtain at least one sample data to be transmitted; Training the first neural network and the second neural network of the preset algorithm according to the at least one sample data to be transmitted to obtain the target path planning model; After acquiring at least one sample data to be transmitted, the method further includes: Selecting first sample data to be transmitted from the at least one sample data to be transmitted; Acquire a current position of the first sample data to be transmitted in a transmission link, and obtain a first router, wherein the first router is used to transmit the first sample data to be transmitted; Determine, according to the first neural network, at least one transmission direction of the first sample data to be transmitted, wherein the at least one transmission direction is used to indicate that the first sample data to be transmitted is transmitted from the first router to a target router; The at least one transmission direction is recorded to obtain a route set of the first sample data to be transmitted.

2. The method according to claim 1, characterized in that After recording the at least one transmission direction to obtain a route set of the first sample data to be transmitted, the method further includes: According to the second neural network, the route set is evaluated to obtain an evaluation result, wherein the evaluation result is used to characterize the length of the link contention time of the route set; Based on the evaluation result, a target path of the first sample data to be transmitted is determined.

3. The method according to claim 2, characterized in that According to the second neural network, the routing set is evaluated to obtain an evaluation result, including: Obtaining a link state set occupied by routes in the current route set, wherein the link state set is used to characterize the usage of the link; Based on the link state set, the link contention time of the current route set is calculated to obtain the evaluation result.

4. The method according to claim 3, characterized in that Determining a target path of the first sample data to be transmitted based on the evaluation result includes: Comparing the link contention time of the current route set with the link contention time of the target route set; When the link contention time of the current route set is less than the link contention time of the target route set, updating the target route set; The target path is generated according to the target route set.

5. The method according to claim 2, characterized in that: Before evaluating the route set according to the second neural network to obtain an evaluation result, the method further includes: According to the second neural network, the at least one transmission direction is evaluated to obtain a reward value of the at least one transmission direction, wherein the reward value is used to measure the quality of the at least one transmission direction; Calculate the advantage value of at least one transmission direction according to the reward value, wherein the advantage value represents the degree of superiority of the current state of the first sample data to be transmitted compared with the target state, and the target state represents the average value of all states of the first sample data to be transmitted; Based on the advantage value, updating the first neural network and the second neural network to obtain an updated first neural network and an updated second neural network; When the updated first neural network and the updated second neural network meet preset conditions, the target path planning model is generated.

6. The method according to claim 1, characterized in that After obtaining at least one data to be transmitted, the method further includes: Based on a preset principle, sorting the at least one data to be transmitted to obtain a sorting result; According to the sorting result, the order of performing routing calculation on the at least one data to be transmitted is determined to obtain the target calculation order.

7. A device for determining a data transmission path, characterized in that: include: An acquisition module, used to acquire at least one piece of data to be transmitted, wherein the at least one piece of data to be transmitted is data for communication between different processing units included in the target computing system; A determination module, used to input the at least one data to be transmitted into a target path planning model in sequence according to a target calculation order, and output a target path for each data to be transmitted, wherein the target path planning model is obtained by training based on a preset algorithm, and the target path is a path with the shortest link contention time within a preset range, and the link contention time is the waiting time required for the data to be transmitted to be transmitted along the link of the current routing set; The device for determining the data transmission path further includes: a first acquisition module for acquiring at least one sample data to be transmitted; a first training module for training the first neural network and the second neural network of the preset algorithm according to the at least one sample data to be transmitted to obtain a target path planning model; The device for determining the data transmission path also includes: a second acquisition module, used to select the first sample data to be transmitted from at least one sample data to be transmitted; a third acquisition module, used to obtain the current position of the first sample data to be transmitted in the transmission link, and obtain the first router, wherein the first router is used to transmit the first sample data to be transmitted; a first determination module, used to determine at least one transmission direction of the first sample data to be transmitted according to the first neural network, wherein at least one transmission direction is used to indicate that the first sample data to be transmitted is transmitted from the first router to the target router; a first recording module, used to record at least one transmission direction, and obtain a routing set of the first sample data to be transmitted.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method for determining a data transmission path as claimed in any one of claims 1 to 6 when executed.

9. An electronic device, characterized in that: The electronic device includes one or more processors; A memory for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to run the programs, wherein the program is configured to execute the method for determining the data transmission path described in any one of claims 1 to 6 when running.

Images