Reconfigurable computing architecture mapping method and apparatus based on graph convolution and reinforcement learning

Abstract

The application provides a reconfigurable computing architecture mapping method based on graph convolution and reinforcement learning, comprising the following steps: acquiring a data dependency graph and a coarse-grained reconfigurable array, and calculating a minimum start interval of the data dependency graph; preprocessing the data dependency graph according to the minimum start interval; simplifying a hardware model resource graph according to the structure of the coarse-grained reconfigurable array; taking a node set of the hardware model resource graph as an action space; adopting a reinforcement learning model based on a graph convolution network to map the preprocessed data dependency graph, so as to obtain a mapping result of a data dependency graph node on the hardware model resource graph; calculating a reward value of the mapping result; and when it is judged according to the reward value that the mapping result meets a successful mapping condition, outputting the mapping result. The application optimizes the layout and wiring of a loop on a reconfigurable computing array by using graph convolution and reinforcement learning, improves the parallelism of loop execution, and thus increases the mapping quality and reduces the execution time of image denoising.

Description

Technical Field

[0001] This application relates to the field of reconfigurable computing architecture mapping technology, and in particular to a method and apparatus for reconfigurable computing architecture mapping based on graph convolution and reinforcement learning. Background Technology

[0002] Coarse-grained reconfigurable arrays (CGRAs) are characterized by the use of coarse-grained processing element arrays (PEAs) and data-path-oriented architectures. They offer lower programmability overhead compared to FPGAs (Field-Programmable Gate Arrays). Through efficient hardware resource organization, CGRAs can provide energy-efficient acceleration solutions for a variety of compute-intensive applications, such as image denoising and digital signal processing.

[0003] However, despite the high theoretical performance of CGRA, its actual performance is often limited by the compiler. The core issue in CGRA applications is to rationally plan the optimal mapping scheme—that is, to correctly map the Data Dependency Graph (DDG) onto the time-extended CGRA while minimizing the startup interval between two loop iterations. This mapping problem comprises three parts: scheduling, placement, and routing. The scheduling step determines the time step for executing computational units, the placement step determines the PE operator for executing computational units, and the routing step verifies the consistency between the connectivity between computational units and the connectivity between PE operators. When the difference between the scheduled time steps of two connected computational units is only 1, they need to be placed on the same or adjacent PE operators to ensure correct routing. When such a difference in time steps is greater than 1, i.e., when long dependencies occur, multiple routing strategies will be used, such as PE routing strategy, local register routing strategy, global register routing strategy, and data memory routing strategy. Figure 1 As shown, the loop core in image denoising processing applications ( Figure 1 (a) It needs to access the four surrounding array elements simultaneously. The original DDG after transformation is as follows: Figure 1 As shown in (b), it includes four memory access operators (L1, L2, L3, and L4). To alleviate the pressure of data memory access, data reuse techniques are typically used to reduce data memory access and transform data access into long data dependencies, such as... Figure 1 As indicated by the arrow in (c).

[0004] Since the DDG after data reuse will have multiple long dependencies, these can be routed in CGRA via Local Registers (LRF), Global Registers (GRF), Processing Units (PE), and data memory. Because there are various data routing methods, how to select appropriate resources in CGRA to route these long dependencies has a significant impact on application execution performance.

[0005] Most current work chooses to reduce the exploration of long-dependency parts to reduce problem complexity, sacrificing mapping quality for reduced compilation time. Because existing methods do not tightly couple the exploration of long-dependency routing strategies and mapping in the data dependency graph, CGRA mapping is prone to failure, thus reducing mapping quality and increasing the execution time of image denoising algorithms. Summary of the Invention

[0006] This application aims to at least partially address one of the technical problems in the related art.

[0007] Therefore, the first objective of this application is to propose a reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning. This method addresses the technical problem that existing methods fail to tightly couple the exploration and mapping of long-dependency routing strategies in the data dependency graph, leading to CGRA mapping failure, which in turn reduces mapping quality and increases the execution time of image denoising algorithms. By utilizing graph convolution and reinforcement learning to optimize the layout and routing of loops on the reconfigurable computational array, the parallelism of loop execution is improved, thereby increasing mapping quality and reducing the execution time of image denoising.

[0008] The second objective of this application is to propose a reconfigurable computational architecture mapping device based on graph convolution and reinforcement learning.

[0009] The third objective of this application is to propose a computer device.

[0010] The fourth objective of this application is to provide a non-transitory computer-readable storage medium.

[0011] To achieve the above objectives, the first aspect of this application proposes a reconfigurable computing architecture mapping method based on graph convolution and reinforcement learning, comprising: acquiring a data dependency graph and a coarse-grained reconfigurable array; calculating the minimum start interval of the data dependency graph based on the size of the data dependency graph and the coarse-grained reconfigurable array; preprocessing the data dependency graph according to the minimum start interval to obtain a preprocessed data dependency graph; simplifying the hardware model resource graph according to the structure of the coarse-grained reconfigurable array; using the node set of the hardware model resource graph as the action space, mapping the preprocessed data dependency graph using a reinforcement learning model based on graph convolutional networks to obtain the mapping result of the data dependency graph nodes on the hardware model resource graph; calculating the reward value of the mapping result, and outputting the mapping result when the mapping result meets the successful mapping condition based on the reward value.

[0012] Optionally, in one embodiment of this application, the data dependency graph is preprocessed according to the minimum start interval to obtain a preprocessed data dependency graph, including:

[0013] The data dependency graph is scheduled based on the minimum start interval, where scheduling the data dependency graph involves allocating a time control step to each node of the data dependency graph.

[0014] The nodes in the data dependency graph with a non-out-degree of 1 are split by recompiling nodes, and routing nodes are inserted between two nodes with long dependencies in the data dependency graph to obtain the preprocessed data dependency graph.

[0015] Optionally, in one embodiment of this application, a hardware model resource graph is simplified based on the structure of the coarse-grained reconfigurable array, including:

[0016] Based on the original number of processing units in the coarse-grained reconfigurable array, additional data loading nodes, data storage block nodes, data storage nodes, global register nodes, local register nodes, and interconnections between nodes are added to obtain the hardware model resource diagram.

[0017] Optionally, in one embodiment of this application, before the reinforcement learning model based on graph convolutional networks maps the preprocessed data dependency graph, the following steps are included:

[0018] Obtain layout samples. Based on the layout samples, estimate the gradient of the loss function with respect to the parameters using the action probability distribution. At the same time, introduce the reward value baseline to obtain the gradient of the loss function with respect to the parameters.

[0019] The parameters of a reinforcement learning model based on a graph convolutional network are learned by using the gradient of the loss function with respect to the parameters and an Adam optimizer based on the reinforcement equation.

[0020] The gradient of the loss function with respect to the parameters is expressed as:

[0021]

[0022] in, This represents the gradient of the loss function J with respect to θ, where K represents the number of layout samples. Represents the gradient operator, S represents the layout sample, and S represents the model state. Indicate layout Reward value, This represents the baseline reward value.

[0023] Optionally, in one embodiment of this application, the set of nodes in the hardware model resource graph is used as the action space, and a reinforcement learning model based on graph convolutional networks is used to map the preprocessed data dependency graph to obtain the mapping result of the data dependency graph nodes on the hardware model resource graph, including:

[0024] The preprocessed data dependency graph is input into a graph convolutional network. The preprocessed data dependency graph is encoded through a fully connected layer and then input into a multi-layer graph convolutional network to obtain node information. The node information is then input into a fully connected layer to transform the node information into the probability distribution of the node on the hardware model resource graph, thereby obtaining the position of the node in the preprocessed data dependency graph on the hardware model resource graph as the mapping result.

[0025] The state of the reinforcement learning model is to encode the features of the nodes in the preprocessed data dependency graph. The feature encoding includes the node number, time control step, and node type. The action of the reinforcement learning model is the corresponding position on the hardware model resource graph.

[0026] Optionally, in one embodiment of this application, calculating the reward value of the mapping result includes:

[0027] Based on the mapping results, obtain the number of edges of the invalid routes in the mapping results and the sum of the Manhattan distances of all edges of the invalid routes;

[0028] The reward value for the mapping result is calculated based on the number of edges in the invalid routes and the sum of the Manhattan distances of all edges in the invalid routes.

[0029] The formula for calculating the reward value of the mapping result is as follows:

[0030]

[0031] in, This represents the reward value of the mapping result. Let α represent the sum of Manhattan distances of all edges with invalid routes, and let α represent the empirical weighting factor. This indicates the number of edges on invalid routes.

[0032] Optionally, in one embodiment of this application, the successful mapping condition is a reward value of 0, and after calculating the reward value of the mapping result, it further includes:

[0033] If the reward value of the mapping result is not 0 within the preset time interval, the minimum start interval is increased by 1, the data dependency graph is reprocessed, and a reinforcement learning model based on graph convolutional networks is used to map the preprocessed data dependency graph until the mapping result meets the successful mapping condition.

[0034] To achieve the above objectives, a second aspect of the present invention proposes a reconfigurable computational architecture mapping device based on graph convolution and reinforcement learning, comprising:

[0035] The acquisition module is used to acquire the data dependency graph and the coarse-grained reconfigurable array, and calculate the minimum start interval of the data dependency graph based on the size of the data dependency graph and the coarse-grained reconfigurable array.

[0036] The preprocessing module is used to preprocess the data dependency graph according to the minimum start interval to obtain the preprocessed data dependency graph;

[0037] The hardware model resource graph module is used to simplify the hardware model resource graph based on the structure of the coarse-grained reconfigurable array.

[0038] The mapping module is used to take the set of nodes in the hardware model resource graph as the action space and use a reinforcement learning model based on graph convolutional networks to map the preprocessed data dependency graph to obtain the mapping result of the data dependency graph nodes on the hardware model resource graph.

[0039] The output module is used to calculate the reward value of the mapping result and output the mapping result when the mapping result meets the successful mapping condition based on the reward value.

[0040] To achieve the above objectives, a third aspect of the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the reconfigurable computing architecture mapping method based on graph convolution and reinforcement learning described in the above embodiment.

[0041] To achieve the above objectives, a fourth aspect of the present invention provides a non-transitory computer-readable storage medium that, when instructions in the storage medium are executed by a processor, enables the execution of a reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning.

[0042] The present application's embodiments of the reconfigurable computing architecture mapping method, apparatus, computer device, and non-transitory computer-readable storage medium based on graph convolution and reinforcement learning solve the technical problem that existing methods do not tightly couple the exploration and mapping of long-dependency routing strategies in the data dependency graph, leading to CGRA mapping failure, thereby reducing mapping quality and increasing the execution time of image denoising algorithms. By utilizing graph convolution and reinforcement learning to optimize the layout and routing of loops on the reconfigurable computing array, the parallelism of loop execution is improved, thereby increasing mapping quality and reducing the execution time of image denoising.

[0043] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0044] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0045] Figure 1 This is an example diagram of the image denoising algorithm and its data dependency graph according to an embodiment of this application;

[0046] Figure 2 This is a flowchart illustrating a reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning, provided in Embodiment 1 of this application.

[0047] Figure 3 This is an example diagram of module scheduling for CGRA, a reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning, according to an embodiment of this application.

[0048] Figure 4 The original structure diagram and hardware model resource diagram of the CGRA (Graph Convolutional and Reinforcement Learning-based Reconfigurable Computational Architecture Mapping) method in this application embodiment are shown below.

[0049] Figure 5 This is an example diagram showing the mapping of the hardware model resource graph to the actual location of LRF nodes in the reconfigurable computing architecture mapping method based on graph convolution and reinforcement learning according to an embodiment of this application.

[0050] Figure 6 This is an example diagram illustrating the mapping of the hardware model resource graph to the actual location of the LD node in the reconfigurable computing architecture mapping method based on graph convolution and reinforcement learning according to an embodiment of this application.

[0051] Figure 7 This is a schematic diagram of the overall graph convolutional network for the reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning, as described in an embodiment of this application.

[0052] Figure 8This is an example flowchart illustrating the implementation process of the reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning, as described in this application.

[0053] Figure 9 This is an example diagram of the data dependency graph preprocessing in the reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning according to an embodiment of this application.

[0054] Figure 10 This is a schematic diagram of the structure of a reconfigurable computing architecture mapping device based on graph convolution and reinforcement learning, provided in Embodiment 2 of this application. Detailed Implementation

[0055] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0056] Deep reinforcement learning is the core technology of current artificial intelligence. Its biggest advantage is that it does not require labels. The driving force for the agent to learn in reinforcement learning is the reward value given by the environment rather than the label in traditional supervised learning. Therefore, the agent will not only focus on the correct label once, as in supervised learning, but will try to improve the feedback score given by the environment, thereby gradually finding the optimal solution.

[0057] Graph convolutional neural networks (GCNNs) are a novel approach in artificial intelligence for processing graph-based data. Unlike traditional convolutional neural networks (CNNs), which can only handle grid-like data, GCNNs can process data with arbitrary topological shapes. They utilize the adjacency matrix of a graph to obtain the connections between different nodes, allowing each node to collect information about its neighbors. By stacking multiple layers of GCNNs, nodes can progressively collect information about their neighbors and their neighbors' neighbors, ultimately gaining a global view of the entire graph and aiding in solving subsequent problems.

[0058] The following describes, with reference to the accompanying drawings, a method and apparatus for mapping reconfigurable computational architectures based on graph convolution and reinforcement learning, according to embodiments of this application.

[0059] Figure 2 This is a flowchart illustrating a reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning, provided in Embodiment 1 of this application.

[0060] like Figure 2 As shown, this reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning includes the following steps:

[0061] Step 101: Obtain the data dependency graph and the coarse-grained reconfigurable array. Based on the size of the data dependency graph and the coarse-grained reconfigurable array, calculate the minimum start interval of the data dependency graph.

[0062] Step 102: Preprocess the data dependency graph according to the minimum start interval to obtain the preprocessed data dependency graph;

[0063] Step 103: Based on the structure of the coarse-grained reconfigurable array, the hardware model resource diagram is simplified to obtain the hardware model resource diagram.

[0064] Step 104: Using the set of nodes in the hardware model resource graph as the action space, a reinforcement learning model based on graph convolutional networks is used to map the preprocessed data dependency graph to obtain the mapping result of the data dependency graph nodes on the hardware model resource graph.

[0065] Step 105: Calculate the reward value of the mapping result, and output the mapping result when the mapping result meets the successful mapping condition based on the reward value.

[0066] The reconfigurable computing architecture mapping method based on graph convolution and reinforcement learning in this application obtains a data dependency graph and a coarse-grained reconfigurable array. Based on the size of the data dependency graph and the coarse-grained reconfigurable array, the minimum start interval of the data dependency graph is calculated. The data dependency graph is preprocessed according to the minimum start interval to obtain a preprocessed data dependency graph. Based on the structure of the coarse-grained reconfigurable array, a hardware model resource graph is simplified. The node set of the hardware model resource graph is used as the action space, and a reinforcement learning model based on graph convolutional networks is used to map the preprocessed data dependency graph to obtain the mapping result of the data dependency graph nodes on the hardware model resource graph. The reward value of the mapping result is calculated, and the mapping result is output when the mapping result meets the successful mapping condition based on the reward value. This solves the technical problem in existing methods where the exploration and mapping of long-dependency routing strategies in the data dependency graph are not tightly coupled, leading to CGRA mapping failure, which reduces mapping quality and increases the execution time of image denoising algorithms. By using graph convolution and reinforcement learning to optimize the layout and routing of loops on the reconfigurable computing array, the parallelism of loop execution is improved, thereby increasing mapping quality and reducing the execution time of image denoising.

[0067] This application first calculates the minimum start interval (minII) of the DDG graph. Next, the original DDG is processed according to the start interval. The next step is to feed the processed DDG graph into a reinforcement learning system based on a graph convolutional network to obtain the node mapping results on the hardware model resource graph (RRG). Then, the reward value for this mapping result is calculated. If the reward value is 0, the entire mapping process ends; if the reward value is not 0 and the network has not yet reached the preset maximum number of iterations, the network performs a reverse update; if the reward value is not 0 and the preset maximum number of iterations has been reached, the start interval is increased and the process returns to the step of processing the original DDG according to the start interval to re-run the process.

[0068] The goal of module scheduling on CGRA is to find an efficient layout and routing with the smallest possible startup interval (II). For example... Figure 3 As shown, the data dependency graph (DDG) in part (a) needs to be mapped onto a 1×2 CGRA structure with a start interval (II) of 2 in part (b). Part (c) shows a set of successful mapping results. Part (a) is a DDG graph with four nodes placed on four control steps (CS). There are two types of dependencies between connected nodes: short dependencies and long dependencies. Short dependencies are those that only cross one control step, such as... Figure 3 Dependencies like 1-2, 2-3, and 3-4 in the example. Long dependencies refer to dependencies that span more than one control step, such as... Figure 3 In (b), the startup interval (II) refers to the number of cycles between two consecutive iterations. II = 2 means that the time slot (TS) is 2. Each operational unit (PE) is connected to a PE whose Manhattan distance to itself is less than 2. Successful placement and routing requires that all nodes in the DDG have a PE that matches them in the CGRA, and that all dependencies between nodes also have a correct correspondence in the CGRA. A correct correspondence means that the mapping PEs of two nodes with dependencies in the CGRA still maintain the connection relationship. As shown in (c), in (a), all nodes have a corresponding PE and all dependencies still maintain the connection.

[0069] Furthermore, in this embodiment, the data dependency graph is preprocessed according to the minimum start interval to obtain a preprocessed data dependency graph, including:

[0070] The data dependency graph is scheduled based on the minimum start interval, where scheduling the data dependency graph involves allocating a time control step to each node of the data dependency graph.

[0071] The nodes in the data dependency graph with a non-out-degree of 1 are split by recompiling nodes, and routing nodes are inserted between two nodes with long dependencies in the data dependency graph to obtain the preprocessed data dependency graph.

[0072] In this embodiment, node scheduling is performed on the DDG. Since the mapped CGRA is based on time expansion, the CGRA resources in each time period are limited. This requires that the nodes on the DDG be reasonably distributed to each control step to ensure that the CGRA resources in each time period are not wasted or overflowed.

[0073] In this embodiment, the original DDG graph is processed, i.e., the data dependency graph is preprocessed, with two criteria: (1) all nodes with large out-degrees are split using recomputable nodes, and (2) routing nodes are inserted for long dependencies. For nodes with large out-degrees, the difficulty of successfully mapping that node will increase significantly because the number of nodes with dependencies on that node will increase. Recomputable nodes can split the dependencies of the original nodes and transfer some dependencies from the original nodes to the additional recomputable nodes, so that the newly generated nodes and the original nodes both have low out-degrees, thereby reducing the difficulty of successful mapping. Here, the out-degree of a node is the number of edges output by the node.

[0074] Furthermore, in this embodiment of the application, based on the structure of the coarse-grained reconfigurable array, a simplified hardware model resource graph is obtained, including:

[0075] Based on the original number of processing units in the coarse-grained reconfigurable array, additional data loading nodes, data storage block nodes, data storage nodes, global register nodes, local register nodes, and interconnections between nodes are added to obtain the hardware model resource diagram.

[0076] Because rich routing strategies are used in the mapping process, if the same CGRA model as in reality is still used, the action space that reinforcement learning needs to explore will be huge, which increases the difficulty of obtaining the correct result.

[0077] In this embodiment, a hardware model resource graph (RRG) is simplified based on the structure of the coarse-grained reconfigurable array. Using the RRG, while maintaining correct functionality, only five additional nodes (LD, BK, ST, GRF, LRF) are added to the original number of processing units (PE). Here, LD represents a data loading node, BK represents a data storage block node, ST represents a data storage node, GRF represents a global register node, and LRF represents a local register node. In the original N×N array, assuming each row contains one data memory access unit (LSU), the hardware resource graph needs to contain N... 2 There are N FU nodes. 2 There are 2N LRF nodes, N LSU nodes, and 1 GRF node. 2 +N+1 resource nodes. However, the RRG proposed in this application only contains N... 2Adding 5 resource nodes significantly reduces the size of the resource graph. For example... Figure 4 As shown, (a) is the original 1×2 CGRA diagram, and (b) is the RRG diagram used in this application. The original PE is split into independent operation units FU and additional shared local register (LRF) nodes. The other four additional operation nodes are shared global register (GRF) nodes and three shared memory nodes (LD, BK, ST). When a routing node is mapped to an LRF or GRF, it means that the node will use LRF and GRF resources. Due to the complexity of the memory resource model, three additional nodes, LD, BK, and ST, are introduced. When a node is mapped to LD or ST, it will choose to perform load and store operations, while if the node is mapped to BK, it will choose to maintain the operation in memory resources.

[0078] Furthermore, in this embodiment of the application, before the reinforcement learning model based on graph convolutional networks maps the preprocessed data dependency graph, the following steps are included:

[0079] Obtain layout samples. Based on the layout samples, estimate the gradient of the loss function with respect to the parameters using the action probability distribution. At the same time, introduce the reward value baseline to obtain the gradient of the loss function with respect to the parameters.

[0080] The parameters of a reinforcement learning model based on a graph convolutional network are learned by using the gradient of the loss function with respect to the parameters and an Adam optimizer based on the reinforcement equation.

[0081] The gradient of the loss function with respect to the parameters is expressed as:

[0082]

[0083] in, This represents the gradient of the loss function J with respect to θ, where K represents the number of layout samples. Represents the gradient operator, S represents the layout sample, and S represents the model state. Indicate layout Reward value, This represents the baseline reward value.

[0084] In this embodiment of the application, K layout samples are obtained, and the action probability distribution is used in the K layout samples. gradient An estimation is performed. To reduce the variance of the policy gradient, a baseline term is introduced.

[0085] The gradient of the loss function with respect to the parameters is expressed as:

[0086]

[0087] in, This represents the gradient of the loss function J with respect to θ, where K represents the number of layout samples. Represents the gradient operator, S represents the layout sample, and S represents the model state. Indicate layout Reward value, This represents the baseline reward value. It represents the average reward under distribution K. Based on experience, a moving average baseline is found. The experiment performed well.

[0088] The gradient of the loss function with respect to the parameters is used as the input to the Adam optimizer based on the reinforcement equation. The model parameters are then learned using the Adam optimizer based on the reinforcement equation, as follows:

[0089]

[0090] in, This indicates that the loss function J is the gradient with respect to θ. Represents the probability distribution of actions. Represents the gradient operator, S represents the layout sample, and S represents the model state. Indicate layout The reward value.

[0091] It is worth noting that the output method is specifically designed for a single DDG. II Once it finds a usable method, it will stop the training process. If DDG or Startup Interval II changes, training needs to be restarted from scratch.

[0092] Furthermore, in this embodiment, the set of nodes in the hardware model resource graph is used as the action space, and a reinforcement learning model based on graph convolutional networks is employed to map the preprocessed data dependency graph, obtaining the mapping result of the data dependency graph nodes on the hardware model resource graph, including:

[0093] The preprocessed data dependency graph is input into a graph convolutional network. The preprocessed data dependency graph is encoded through a fully connected layer and then input into a multi-layer graph convolutional network to obtain node information. The node information is then input into a fully connected layer to transform the node information into the probability distribution of the node on the hardware model resource graph, thereby obtaining the position of the node in the preprocessed data dependency graph on the hardware model resource graph as the mapping result.

[0094] The state of the reinforcement learning model is to encode the features of the nodes in the preprocessed data dependency graph. The feature encoding includes the node number, time control step, and node type. The action of the reinforcement learning model is the corresponding position on the hardware model resource graph.

[0095] In this embodiment, the reinforcement learning model is always learned around the preprocessed DDG. The state of each node in the DDG is defined by a state vector; the k-th node is represented by the state vector s. k Definition. s k The state vector contains three parts [k, t, h], where k is the node number represented by a one-hot code vector, t is the control step of the node represented by a one-hot code vector, and h is the node type represented by a one-hot code vector. For node type h, there are three types in the model: original node, additional recomputation node, and additional routing node. In a DDG with a total of 10 nodes and 5 control steps, the 8th node, placed in the 2nd control step as an additional routing node, has the following state vector:

[0096] [0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1]

[0097] In this embodiment of the application, in the reinforcement learning model, the actions generated by the network are the corresponding positions of each node in the DDG on the RRG. For the case where an RRG needs to be mapped to an n×ncGRA, the numbers 0 to n are used. 2 -1 indicates mapping to a regular arithmetic unit, numbered n. 2 Indicates mapping to a shared LRF, numbered n 2 +1 indicates mapping to a shared GRF, numbered n 2 +2 to n 2 +4 indicates mapping to the shared LD, BK, and ST in memory resources. If the node is mapped to 0 through n... 2 If i is in -1, then it means that the node is mapped to the PE at position i+1 in the original CGRA. For numbers greater than n... 2 In this case, the location of the routing node on the actual CGRA can be determined by examining the connectivity relationships before and after the routing node. If the node is mapped to n 2 The compiler can determine the actual location of the routing node in the CGRA location based on its predecessor nodes, such as... Figure 5As shown in (a), two routing nodes are mapped to the LRF on the RRG. Since the predecessor node 1 of routing node 1' is mapped to PE0, the actual location of routing node 1' in the CRGRA is in the local register of PE0; similarly, the actual location of routing node 2' in the CRGRA is in the local register of PE1. If a node is mapped to n 2 If +1 is added, then the point is mapped to the GRF. The GRF in the RRG is exactly the same as the GRF in the real CGRA, so the node is directly mapped to the GRF of the CGRA. If the node is mapped to n 2 +2 to n 2 On +4, the principle from shared LD, BK, and ST to the actual CGRA location is similar. For example... Figure 6 As shown in (a), two routing nodes are mapped to the LD on the RRG. Since the downstream node 1 of routing node 1′ is mapped to PE0, the real position of routing node 1′ in the CGRA is LSU0, which is in the same row as PE0. Similarly, the real position of routing node 2′ in the CGRA is LSU1, which is in the same row as PE1.

[0098] In this embodiment of the application, the preprocessed data dependency graph is input into a graph convolutional network to obtain the corresponding position of the node in the preprocessed data dependency graph on the hardware model resource graph as the mapping result. Figure 7 It is a global graph convolutional network model, such as Figure 7 As shown, the first layer is a fully connected layer used to encode features for each node in the processed DDG (Directed Graph). The last layer is also a fully connected layer, used to transform the information of each node in the last hidden layer into the probability distribution of that node in the RRG (Recurrent Graph), thus determining the mapping result of that node. Between the two fully connected layers, multiple layers of graph convolutional networks are stacked. In each graph convolutional layer, each node collects information about itself and its neighbors and propagates it to the next layer. Through the stacking of multiple graph convolutional layers, each node will collect information about its neighbors and their neighbors' neighbors, thus gradually forming a global view, which is beneficial for subsequent node mapping.

[0099] Furthermore, in this embodiment of the application, calculating the reward value of the mapping result includes:

[0100] Based on the mapping results, obtain the number of edges of the invalid routes in the mapping results and the sum of the Manhattan distances of all edges of the invalid routes;

[0101] The reward value for the mapping result is calculated based on the number of edges in the invalid routes and the sum of the Manhattan distances of all edges in the invalid routes.

[0102] The formula for calculating the reward value of the mapping result is as follows:

[0103]

[0104] in, This represents the reward value of the mapping result. Let α represent the sum of Manhattan distances of all edges with invalid routes, and let α represent the empirical weighting factor. This indicates the number of edges on invalid routes.

[0105] In this embodiment, since the placement and routing model ensures no location conflicts, only the number of invalid paths needs to be reduced to zero to obtain a valid placement and routing. Therefore, for a given layout P, ​​the number of edges of invalid routes can be used to determine the number of valid paths. The sum of Manhattan distances to all edges with invalid routes As a key reference for reward signals.

[0106] To reconcile the number of edges in invalid routes The sum of Manhattan distances to all edges with invalid routes The relationship between these factors introduces an influencing factor α, and the final reward value is calculated using the following formula:

[0107]

[0108] in, This represents the reward value of layout P. Let α represent the sum of Manhattan distances of all edges with invalid routes, and let α represent the empirical weighting factor. This represents the number of edges on invalid routes. The negative sign is used to increase the reward value of the agent network during reinforcement learning; the cube root is chosen because it makes the learning process more stable.

[0109] The sum of the Manhattan distances of all invalid routes is expressed as:

[0110]

[0111] in, x represents the sum of the Manhattan distances of all edges with invalid routes. i The x-coordinate of node i represents the x-coordinate of node i. j The x-coordinate of node j is represented by y. i The y-coordinate of node i represents the vertical coordinate of node i. j Represents the y-coordinate of node j. This represents the set of edges representing invalid routes.

[0112] Furthermore, in this embodiment of the application, the successful mapping condition is a reward value of 0. After calculating the reward value of the mapping result, the following is also included:

[0113] If the reward value of the mapping result is not 0 within the preset time interval, the minimum start interval is increased by 1, the data dependency graph is reprocessed, and a reinforcement learning model based on graph convolutional networks is used to map the preprocessed data dependency graph until the mapping result meets the successful mapping condition.

[0114] In this embodiment, the reward value of the mapping result is calculated, and it is determined whether the mapping result meets the successful mapping condition. If the successful mapping condition is met, the mapping result is output and the total compilation time is calculated. If the successful mapping condition is not met, the network performs a reverse update and then performs forward prediction for layout sampling. If the reward value of the mapping result is not 0 within a preset time interval, the start interval is incremented by 1, i.e., II = II + 1, and the loop restarts from the step of preprocessing the data dependency graph according to the start interval.

[0115] The following details the implementation of the reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning proposed in this application.

[0116] like Figure 8 As shown, the compiler in this application is divided into two parts. First, it processes the DDG, and then it uses the processed DDG to perform a mapping based on graph neural network and reinforcement learning (GNN-RL).

[0117] In the part of DDG processing:

[0118] The first step is that when the compiler receives a DDG diagram and a given CGRA scale, it will calculate the minimum startup interval MII value under this CGRA scale and set the current startup interval II as MII. The formula for calculating the minimum startup interval MII is as follows:

[0119]

[0120] Where, N o N represents the total number of nodes in the data dependency graph. pea This indicates the size of CGRA, and [·] indicates the rounding up operation.

[0121] The second step is to schedule nodes in the DDG. Since the mapped CGRA is based on time expansion, the CGRA resources in each time period are limited. This requires that the nodes on the DDG be reasonably distributed to each control step to ensure that the CGRA resources in each time period are not wasted or overflowed.

[0122] The third and fourth steps are inserting recalculated nodes and routing nodes, respectively. Recalculated nodes and routing nodes are inserted according to two criteria: (1) using recalculated nodes to split all nodes with large out-degrees, and (2) inserting routing nodes for long dependencies. Figure 9 As shown, criterion (1) causes node 1 to be split into node 1 and recomputed node 10, so that node 1 no longer has two out-degrees but only one out-degree; criterion (2) replaces the original long dependency 1→5 with additional routing nodes 11, 12 and 13.

[0123] The part processed by GNN-RL:

[0124] The first step is to perform an embedding operation on the input graph of the DDG according to the startup interval II. For a DDG with a total of 10 nodes and 5 control steps, the 8th node, which is placed in the 2nd control step as an additional routing node, is represented by the state vector as: [0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1].

[0125] The second step involves inputting the embedded data of DDG into a reinforcement learning network based on a graph convolution model, and then computing the network to provide the mapping result of DDG on RRG for this round.

[0126] The third step is to evaluate the reward value of the mapping result and determine whether the network result meets the conditions for successful mapping. If the conditions are met, the mapping result is output and the total compilation time is calculated. If the conditions are not met, the network performs a reverse update and then performs forward prediction for layout sampling. If the reward value of the mapping result is not 0 within a preset time interval, the start interval is incremented by 1, i.e., II = II + 1, and the loop restarts from the step of preprocessing the data dependency graph based on the start interval.

[0127] Figure 10 This is a schematic diagram of the structure of a reconfigurable computing architecture mapping device based on graph convolution and reinforcement learning, provided in Embodiment 2 of this application.

[0128] like Figure 10 As shown, the reconfigurable computational architecture mapping device based on graph convolution and reinforcement learning includes:

[0129] The acquisition module 10 is used to acquire the data dependency graph and the coarse-grained reconfigurable array, and calculate the minimum start interval of the data dependency graph based on the size of the data dependency graph and the coarse-grained reconfigurable array.

[0130] Preprocessing module 20 is used to preprocess the data dependency graph according to the minimum start interval to obtain the preprocessed data dependency graph;

[0131] Hardware model resource graph module 30 is used to simplify the hardware model resource graph based on the structure of the coarse-grained reconfigurable array.

[0132] The mapping module 40 is used to take the set of nodes of the hardware model resource graph as the action space, and use a reinforcement learning model based on graph convolutional network to map the preprocessed data dependency graph to obtain the mapping result of the data dependency graph nodes on the hardware model resource graph.

[0133] Output module 50 is used to calculate the reward value of the mapping result and output the mapping result when the mapping result meets the successful mapping condition based on the reward value.

[0134] The reconfigurable computing architecture mapping device based on graph convolution and reinforcement learning in this application includes: an acquisition module for acquiring a data dependency graph and a coarse-grained reconfigurable array, and calculating the minimum start interval of the data dependency graph based on the size of the data dependency graph and the coarse-grained reconfigurable array; a preprocessing module for preprocessing the data dependency graph according to the minimum start interval to obtain a preprocessed data dependency graph; a hardware model resource graph module for simplifying the hardware model resource graph according to the structure of the coarse-grained reconfigurable array; a mapping module for using the node set of the hardware model resource graph as the action space, and using a reinforcement learning model based on graph convolutional networks to map the preprocessed data dependency graph to obtain the mapping result of the data dependency graph nodes on the hardware model resource graph; and an output module for calculating the reward value of the mapping result, and outputting the mapping result when the mapping result meets the successful mapping condition based on the reward value. Therefore, this method can solve the technical problem that existing methods do not tightly couple the exploration and mapping of long-dependency routing strategies in the data dependency graph, which leads to CGRA mapping failure, thereby reducing mapping quality and increasing the execution time of image denoising algorithms. By using graph convolution and reinforcement learning to optimize the layout and routing of loops on a reconfigurable computation array, the parallelism of loop execution is improved, thereby increasing mapping quality and reducing the execution time of image denoising.

[0135] To implement the above embodiments, the present invention also proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the reconfigurable computing architecture mapping method based on graph convolution and reinforcement learning described in the above embodiments.

[0136] To implement the above embodiments, the present invention also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning as described in the above embodiments.

[0137] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0138] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0139] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0140] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0141] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0142] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0143] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0144] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A reconfigurable computational architecture mapping method based on graph convolution and reinforcement learning, characterized in that, Includes the following steps: Obtain the data dependency graph and the coarse-grained reconfigurable array, and calculate the minimum startup interval of the data dependency graph based on the size of the data dependency graph and the coarse-grained reconfigurable array. The data dependency graph is preprocessed according to the minimum start interval to obtain the preprocessed data dependency graph. Based on the structure of the coarse-grained reconfigurable array, a simplified hardware model resource diagram is obtained. Using the node set of the hardware model resource graph as the action space, a reinforcement learning model based on graph convolutional networks is employed to map the preprocessed data dependency graph, obtaining the mapping result of the data dependency graph nodes on the hardware model resource graph. Specifically, the preprocessed data dependency graph is input into a graph convolutional network, encoded through a fully connected layer, and then input into multiple layers of the graph convolutional network to obtain node information. This node information is then converted into a probability distribution corresponding to the nodes on the hardware model resource graph by the fully connected layers, thus obtaining the position of the node in the preprocessed data dependency graph on the hardware model resource graph as the mapping result. The state of the reinforcement learning model involves feature encoding of the nodes in the preprocessed data dependency graph, where the feature encoding includes node number, time control step, and node type. The action of the reinforcement learning model is the corresponding position on the hardware model resource graph. Calculate the reward value of the mapping result, and output the mapping result when the mapping result meets the successful mapping condition based on the reward value.

2. The method as described in claim 1, characterized in that, The step of preprocessing the data dependency graph according to the minimum start interval to obtain the preprocessed data dependency graph includes: The data dependency graph is scheduled according to the minimum start interval, wherein scheduling the data dependency graph involves allocating a time control step to each node of the data dependency graph. The nodes in the data dependency graph with a non-out-degree of 1 are split by recompiling nodes, and routing nodes are inserted between two nodes with long dependencies in the data dependency graph to obtain a preprocessed data dependency graph.

3. The method as described in claim 1, characterized in that, The process of refining the hardware model resource graph based on the structure of the coarse-grained reconfigurable array includes: Based on the original number of processing units in the coarse-grained reconfigurable array, additional data loading nodes, data storage block nodes, data storage nodes, global register nodes, local register nodes, and interconnections between nodes are added to obtain a hardware model resource diagram.

4. The method as described in claim 1, characterized in that, Before the reinforcement learning model based on graph convolutional networks maps the preprocessed data dependency graph, the following steps are included: Obtain layout samples, and based on the layout samples, estimate the gradient of the loss function with respect to the parameters using the action probability distribution, while introducing a reward value baseline to obtain the gradient of the loss function with respect to the parameters; The parameters of the reinforcement learning model based on the graph convolutional network are learned using the gradient of the loss function with respect to the parameters and the Adam optimizer based on the reinforcement equation. The gradient of the loss function with respect to the parameters is expressed as: in, Represents the loss function right Calculate the gradient, where K represents the number of layout samples. Represents the gradient operator. Indicates a layout sample. Indicates the model state. Indicate layout Reward value, This represents the baseline reward value.

5. The method as described in claim 1, characterized in that, The step of using the node set of the hardware model resource graph as the action space and employing a reinforcement learning model based on graph convolutional networks to map the preprocessed data dependency graph to obtain the mapping result of the data dependency graph nodes on the hardware model resource graph includes: The preprocessed data dependency graph is input into a graph convolutional network. The preprocessed data dependency graph is encoded through a fully connected layer and then input into a multi-layer graph convolutional network to obtain node information. The node information is then input into a fully connected layer to convert the node information into the probability distribution of the node on the hardware model resource graph, thereby obtaining the position of the node in the preprocessed data dependency graph on the hardware model resource graph as the mapping result. The state of the reinforcement learning model is to encode the features of the nodes in the preprocessed data dependency graph. The feature encoding includes the node number, time control step, and node type. The action of the reinforcement learning model is the corresponding position on the hardware model resource graph.

6. The method as described in claim 1, characterized in that, The calculation of the reward value of the mapping result includes: Based on the mapping result, the number of edges of the invalid routes in the mapping result and the sum of the Manhattan distances of all edges of the invalid routes are obtained; The reward value of the mapping result is calculated based on the number of edges in the invalid routes and the sum of the Manhattan distances of all edges in the invalid routes. The formula for calculating the reward value of the mapping result is expressed as follows: in, This represents the reward value of the mapping result. This represents the sum of the Manhattan distances of all edges with invalid routes. Represents the empirical weighting factor. This indicates the number of edges on invalid routes.

7. The method as described in claim 1, characterized in that, The successful mapping condition is that the reward value is 0. After calculating the reward value of the mapping result, the method further includes: If the reward value of the mapping result is not 0 within the preset time interval, the minimum start interval is increased by 1, the data dependency graph is reprocessed, and a reinforcement learning model based on graph convolutional networks is used to map the preprocessed data dependency graph until the mapping result meets the successful mapping condition.

8. A reconfigurable computational architecture mapping device based on graph convolution and reinforcement learning, characterized in that, include: An acquisition module is used to acquire a data dependency graph and a coarse-grained reconfigurable array, and calculate the minimum startup interval of the data dependency graph based on the size of the data dependency graph and the coarse-grained reconfigurable array. The preprocessing module is used to preprocess the data dependency graph according to the minimum start interval to obtain the preprocessed data dependency graph. The hardware model resource graph module is used to simplify the hardware model resource graph based on the structure of the coarse-grained reconfigurable array. The mapping module is used to take the node set of the hardware model resource graph as the action space, and use a reinforcement learning model based on graph convolutional networks to map the preprocessed data dependency graph to obtain the mapping result of the data dependency graph nodes on the hardware model resource graph. Specifically, the preprocessed data dependency graph is input into a graph convolutional network, encoded through a fully connected layer, and then input into a multi-layer graph convolutional network to obtain node information. This node information is then converted into a probability distribution corresponding to the node on the hardware model resource graph by a fully connected layer, thus obtaining the position of the node in the preprocessed data dependency graph on the hardware model resource graph as the mapping result. The state of the reinforcement learning model is to encode the features of the nodes in the preprocessed data dependency graph, where the feature encoding includes node number, time control step, and node type. The action of the reinforcement learning model is the corresponding position on the hardware model resource graph. The output module is used to calculate the reward value of the mapping result, and output the mapping result when it is determined that the mapping result meets the successful mapping condition based on the reward value.

9. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images