Fpga high-level synthesis scheduling method based on graph reinforcement learning

Abstract

The application discloses a FPGA high-level synthesis scheduling method based on graph reinforcement learning, which comprises the following steps: obtaining an initial schedule; a graph neural network receives data in a non-euclidean space, extracts features in the data in the non-euclidean space, embeds the features, obtains result information, and delivers the result information to a policy network of reinforcement learning; the policy network generates corresponding scheduling actions according to the result information and the initial schedule, and continuously interacts with an environment to complete a scheduling process; and an accurate clock cycle corresponding to an operation is obtained and output. The application has the advantages of improving scheduling quality and improving solving speed.

Description

Technical Field

[0001] This invention belongs to the field of graph reinforcement learning technology, and in particular relates to a high-level synthesis and scheduling method for FPGA based on graph reinforcement learning. Background Technology

[0002] To meet increasingly stringent performance and power consumption requirements, as well as the ever-growing demands of algorithms, Field Programmable Gate Arrays (FPGAs) have shown great promise due to their low power consumption and reconfigurability. In FPGA development, High Level Synthesis (HLS) methods have gained increasing attention by improving the level of abstraction to achieve lower circuit design complexity, significantly increasing productivity.

[0003] The HLS process is divided into three main phases: software compilation, scheduling and binding, and RTL generation. In the first phase, high-level software programs (such as C, C++, SystemC) are compiled, and intermediate variables (IRs) are transformed into a Control Data Flow Graph (CDFG) using a software compiler such as GCC or LLVM. In the second phase, scheduling analysis extracts parallelism from the CDFG to determine in which clock cycle each operation should execute under a set of time and resource constraints that must be satisfied. Binding interacts with scheduling to determine the resource instance utilized by each operation. In the final phase, the RTL model is generated based on the FSM to implement the functionality described by the input software program. This application primarily focuses on the scheduling phase in HLS, specifically: analyzing the parallelism extracted from the CDFG to determine in which clock cycle each operation should execute under a set of time and resource constraints that must be satisfied.

[0004] Graph Reinforcement Learning (GRL) is an algorithm that combines graph neural networks (GNNs) and reinforcement learning. GNNs can take non-Euclidean space data as input (such as the Control Data Flow Graph (CDFG) mentioned above) and automatically extract features to achieve encoding. Reinforcement learning is a machine learning paradigm centered on Markov decision processes, consisting of an environment and an agent. Without human intervention, the agent continuously interacts with the environment to automatically learn the optimal strategy for solving the problem. The combination of GNNs and reinforcement learning can solve problems based on non-Euclidean space data, achieving an end-to-end solution. This design uses a graph reinforcement learning algorithm to solve and optimize the scheduling problem in HLS (High-Speed ​​Rail).

[0005] Scheduling is a crucial component of High-Level Synthesis (HLS), influencing the resource consumption and frequency of the final generated circuit and determining the quality of the result. However, mainstream algorithms heavily rely on imprecise heuristics, which make ad-hoc decisions and fail to perform global optimization based on the characteristics and structural information of the task itself. This results in a significant difference between circuits generated by HLS methods and those optimized manually. Furthermore, scheduling algorithms are limited in both speed and quality; achieving accurate solutions for good scheduling quality often consumes considerable time, posing challenges to circuit design. Therefore, current technology lacks a new scheduling algorithm that can improve both scheduling quality and solution speed. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a high-level synthesis scheduling algorithm for FPGA based on graph reinforcement learning, which has the advantages of improving the quality of implementation scheduling and increasing the solution speed, thus solving the problems of poor circuit quality and slow solution speed in existing technologies.

[0007] A high-level synthesis scheduling method for FPGA based on graph reinforcement learning includes the following steps:

[0008] Get the initialization schedule;

[0009] Graph neural networks receive data from non-Euclidean space, extract features from the non-Euclidean space data, expand the expression range of graph neural networks by adding a multi-layer neural network, extract the feature data required for scheduling, embed the features, obtain the result information, and pass the result information to the policy network of reinforcement learning.

[0010] The reinforcement learning policy network generates corresponding scheduling actions based on the result information and the initialization schedule, and continuously interacts with the environment to complete the scheduling process;

[0011] Obtain and output the precise clock cycle corresponding to the operation.

[0012] As a preferred method, adding a multi-layer neural network to expand the expressive range of the graph neural network includes:

[0013]

[0014] in Represents node v iterating k times. The feature vector represents the initial input, where ∈ is any number that can be learned;

[0015] After k iterations, that is, a k-layer graph convolutional neural network (GCN);

[0016] Then, pooling operations are performed to obtain global, critical path information. This method parameterizes the GCN aggregation process and uses a multilayer perceptron to fit this process, demonstrating stronger discriminative capabilities.

[0017] Using this method, we can improve MLPs (Multilayer Neural Networks) through reinforcement learning, thereby learning convergence strategies for scheduling problems. However, this method only aggregates node information; scheduling tasks should also possess global information, such as a defined clock cycle and delay constraints. The extraction of these features can still employ the aforementioned approach, fusing the task's global information into the operator features.

[0018] Preferably, the method for a reinforcement learning policy network to generate corresponding scheduling actions based on the result information and continuously interact with the environment to complete the scheduling process includes:

[0019] (a) Setting up the state space:

[0020] Divide the entire scheduling task into multiple subtasks;

[0021] The entire scheduling process is completed by fulfilling the multiple sub-tasks.

[0022] (II) Setting up motion space:

[0023] The feature vectors of multiple nodes are decoded through a multi-layer neural network;

[0024] The output of the softmax function ensures that each node corresponds to only one probability.

[0025] Mapping probabilities to discrete actions;

[0026] (III) Setting up reward methods;

[0027] When the scheduling strategy does not meet the resource constraints, the reward is -1, and the solution that does not meet the constraints is quickly rejected.

[0028] When the scheduling policy satisfies resource constraints, the reward is related to the increased latency. The lower the increased latency in a given scheduling operation, the higher the reward for that operation, as shown in the following formula:

[0029]

[0030] λ is a hyperparameter that allows for adjustment of the reward to obtain a better scheduling strategy.

[0031] As a further preferred embodiment, the method for dividing the entire scheduling task into multiple subtasks in step (i) above includes:

[0032] A graph neural network with an added multilayer neural network is used to embed features into all nodes;

[0033] According to the initialization schedule, the re-encoded data will be divided into blocks;

[0034] Each block obtained from the block division is treated as a subgraph, and each subgraph corresponds to a feature matrix of appropriate size.

[0035] Set the hyperparameter N so that each scheduling operation only executes nodes within N cycles.

[0036] This partitioning greatly reduces the state space.

[0037] As a further preferred embodiment, in step (ii) above, the specific formula for mapping the probability to discrete actions is as follows:

[0038]

[0039] P is the output probability corresponding to a certain computing node, and N represents the number of cycles of the subtask. For the floor operation, the action space is mapped to integers in the range [0, N].

[0040] As a further preferred embodiment, in step (iii) above, the method for setting the reward method further includes:

[0041] Set a discount factor for the rewards;

[0042] After scheduling is completed, the scheduling is evaluated and a final reward is obtained;

[0043] The ultimate reward is broadcast to all strategies, updating all previous strategies and re-evaluating previous decisions.

[0044] Preferably, the reinforcement learning policy network is based on the REINFORCE algorithm, which constructs a neural network to directly learn the policy, the probability distribution of the input state, and the output action; the gradient of the neural network update is given by the following formula:

[0045]

[0046]

[0047] Preferably, the policy network of the reinforcement learning shares a multi-layer neural network and a softmax layer; the method for sharing a multi-layer neural network in the policy network of reinforcement learning includes:

[0048] Step S351: Input the features of multiple points, pass them through a shared multi-layer neural network, and map them onto a one-dimensional vector to complete parallel computation;

[0049] Step S352: When a resource-constrained scheduling task with only one adder and one multiplier per cycle arrives, the trained graph reinforcement learning model quickly obtains an accurate clock cycle table.

[0050] Reinforcement learning based on policy networks includes: convergence policy and decoding policy;

[0051] Convergence strategy, used to fuse structural information and attribute information;

[0052] Decoding strategies are used to distinguish rich feature information and provide corresponding scheduling actions.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0054] This invention utilizes a graph neural network (Graph NNN) to receive data, automatically extract features, and perform encoding. The Graph NNN can embed structural and global information to obtain the final result. It can perform global optimization based on the task's features and structural information, thereby improving the quality of circuits generated by the High-Level Synthesis (HLS) method.

[0055] The function of reinforcement learning is to automatically learn the optimal strategy for solving problems through continuous interaction between the agent and the environment. Combining these two approaches can solve the problem of control data flow graphs, achieving an end-to-end solution. End-to-end represents high efficiency, effectively reducing information asymmetry in intermediate links. Once a problem is detected at the terminal, every link in the entire system can be adjusted in a coordinated manner. Furthermore, through automatic learning, the optimal strategy for solving problems can be obtained, thereby improving the quality of scheduling and the quality of circuits generated by the HLS method. Comparing the scheduling results of the ILP algorithm and the algorithm of this invention, under the condition of satisfying resource constraints, the algorithm of this invention achieves near-exact solution accuracy and significantly improves solution speed. Attached Figure Description

[0056] Figure 1 This is a flowchart illustrating the FPGA high-level synthesis scheduling algorithm based on graph reinforcement learning provided in an embodiment of the present invention.

[0057] Figure 2 This is a flowchart illustrating the method for embedding features using a graph neural network, as provided in an embodiment of the present invention.

[0058] Figure 3 This is a flowchart illustrating a method provided in this embodiment of the invention for a policy network to generate corresponding scheduling actions based on the result information and continuously interact with the environment to complete the scheduling process.

[0059] Figure 4 This is a schematic diagram of the basic components of reinforcement learning provided in the embodiments of the present invention.

[0060] Figure 5 This is a flowchart illustrating a method for dividing a scheduling task into multiple subtasks, as provided in an embodiment of the present invention.

[0061] Figure 6 This is a flowchart illustrating a method for dividing a scheduling task into multiple subtasks, as provided in an embodiment of the present invention.

[0062] Figure 7 This is a flowchart illustrating the action space provided in an embodiment of the present invention.

[0063] Figure 8 This is a flowchart illustrating the reward method provided in an embodiment of the present invention.

[0064] Figure 9 This is a flowchart illustrating the reward setting method provided in an embodiment of the present invention.

[0065] Figure 10 This is a schematic diagram of a policy network in reinforcement learning provided in an embodiment of the present invention.

[0066] Figure 11 This is a flowchart illustrating the method for sharing the multilayer neural network in a strategy network according to an embodiment of the present invention.

[0067] Figure 12 This is a schematic diagram of the scheduling results of graph reinforcement learning provided in an embodiment of the present invention. Detailed Implementation

[0068] To further understand the invention's content, features, and effects, the following embodiments are provided, and detailed descriptions are given in conjunction with the accompanying drawings.

[0069] In existing technologies, there are two main techniques for addressing the scheduling problem in HLS (High-Level Synthesis):

[0070] ILP algorithm:

[0071] Since resource-constrained scheduling of a DFG (Data Flow Graph) is essentially a combinatorial optimization problem, it can be described as an exact Integer Linear Programming (ILP) model. Numerous ILP tools, such as CBC, CLP, and CPLEX, can provide exact solutions to the scheduling problem. It can also be solved quickly on small-scale graphs, and this method is integrated into the open-source HLS (High-Level Synthesis) tool. However, this exact solution method requires exploring a large space, resulting in a slow solution speed, making it unsuitable for large-scale circuit designs.

[0072] SDC algorithm:

[0073] The literature (, Cong J, Zhang Z. An efficient and versatile scheduling algorithm based on SDC formulation, [C] / / 2006 43rd ACM / IEEE Design Automation Conference.IEEE, 2006: 433-438.) assigns an integer variable to each operator based on the System of Difference Constraints (SDC), representing the clock cycle in which the operator is scheduled, and then uses this integer variable to achieve accurate modeling of a series of constraints.

[0074] Although SDC's modeling method is very accurate, it relies on heuristics to transform resource constraints into the integer difference form described above; that is, this linear ordering is heuristic. Such ordering involves a series of priority relationships. In this case, it may lead to a random, suboptimal solution, resulting in poor scheduling quality.

[0075] Therefore, this design aims to break through the bottleneck of existing algorithms and propose a scheduling algorithm based on graph reinforcement learning, which optimizes the scheduling algorithm in HLS (High-Level Synthesis) in both speed and quality dimensions.

[0076] The structure of the present invention will now be described in detail with reference to the accompanying drawings.

[0077] Please see Figure 1 The high-level FPGA synthesis scheduling algorithm based on graph reinforcement learning provided in this embodiment of the invention includes the following steps:

[0078] Step S1: Obtain the initialization schedule;

[0079] Step S2: The graph neural network receives data from non-Euclidean space, extracts features from the non-Euclidean space data, embeds the features to obtain the result information, and passes the result information to the policy network of reinforcement learning.

[0080] Step S3: The policy network generates corresponding scheduling actions based on the result information and the initialization schedule, and continuously interacts with the environment to complete the scheduling process;

[0081] Step S4: Obtain and output the precise clock cycle corresponding to the operation.

[0082] In this embodiment, the non-Euclidean space data is generated in the first stage (software compilation stage) of the High Level Synthesis (HLS) method. For example, software compilers such as GCC or LLVM transform intermediate variables IR into a Control Data Flow Graph (CDFG). The function of the graph neural network is to receive data, automatically extract features, and implement encoding. The graph neural network can embed structural and global information to obtain the result information. This process can perform global optimization based on the characteristics and structural information of the task itself, thereby improving the quality of the circuit generated by the HLS method.

[0083] Reinforcement learning enables agents to continuously interact with the environment and automatically learn the optimal strategy for solving problems. Combining reinforcement learning and reinforcement learning can address control data flow graph issues, achieving an end-to-end solution. End-to-end processing represents high efficiency, effectively reducing information asymmetry in intermediate stages. Once a problem is detected at the terminal, every component of the entire system can be adjusted in a coordinated manner. Furthermore, through automatic learning, the optimal strategy for solving problems can be obtained, thereby improving the quality of scheduling and the quality of circuits generated by the HLS (High-Level Synthesis) method.

[0084] Furthermore, for the graph neural network module, graph neural networks have the function of automatic graph embedding. However, the expressive power of ordinary graph convolutional neural networks is limited and cannot be well applied to scheduling problems. Therefore, it is necessary to learn the aggregation ability of graph convolutional neural networks (GCN).

[0085] Therefore, please refer to Figure 2 In step S2, the method for embedding features in the graph neural network includes:

[0086] Step S21: Add a multilayer neural network (MLP) to expand the expressive range of the graph neural network;

[0087] Step S22: Extract the feature data required for scheduling;

[0088] Step S23: Embed the feature data.

[0089] Specifically, in step S21, the method of adding a multi-layer neural network to expand the expressive range of the graph neural network includes:

[0090]

[0091] in Represents node v iterating k times. The feature vector represents the initial input, where ∈ is an arbitrary number that can be learned. After k iterations, i.e., a k-layer Graph Convolutional Neural Network (GCN), and then pooling operations, global and critical path information is obtained. This method parameterizes the GCN aggregation process and uses a multilayer perceptron to fit this process, demonstrating stronger discriminative ability. Using this method, we can improve multilayer neural networks (MLPs) through reinforcement learning, thereby learning convergence strategies for scheduling problems.

[0092] Meanwhile, this method only aggregates node information; however, scheduling tasks should also possess global information, such as a defined clock cycle and delay constraints. The extraction of these features can still employ the aforementioned approach, fusing the task's global information into the operator features. This can further improve the quality of circuits generated by the HLS (High-Level Synthesis) method.

[0093] Furthermore, the specific implementation of the reinforcement learning part in this invention is as follows:

[0094] In step S3, please refer to Figure 3 , Figure 4 and Figure 5 The policy network generates corresponding scheduling actions based on the result information and continuously interacts with the environment. The methods for completing the scheduling process include:

[0095] Step S31: Set up the state space (State):

[0096] This invention employs the divide-and-conquer approach, which includes the following steps:

[0097] Step S311: Divide the entire scheduling task into multiple subtasks;

[0098] Step S312: The entire scheduling process is completed by completing the multiple sub-tasks.

[0099] For details, please refer to the reference. Figure 6 In step S311, the method for dividing the entire scheduling task into multiple subtasks includes:

[0100] Step S3111: Use a graph neural network to embed features into all nodes;

[0101] Step S3112: Divide the re-encoded data into blocks according to the initialization schedule;

[0102] Step S3113: Each block obtained from the block division is treated as a subgraph, and each subgraph corresponds to a feature matrix of appropriate size;

[0103] Step S3114: Set the hyperparameter N, and execute only the nodes within N cycles in each scheduling.

[0104] The above division greatly reduces the state space, resulting in faster computation speed.

[0105] Furthermore, in step S3, the policy network generates corresponding scheduling actions based on the result information and continuously interacts with the environment. The method for completing the scheduling process also includes:

[0106] Step S32: Set up the action space:

[0107] Since the state is a subtask, the subtask has a moderate action space, avoiding both a large action space and long action sequences. For details, please refer to [link / reference]. Figure 7 The action space includes the following steps:

[0108] Step S321: The feature vectors of multiple nodes are decoded by a multi-layer neural network (MLP);

[0109] Step S322: The output is processed by the softmax function, so that each node corresponds to only one probability;

[0110] Step S323: Map the probability to discrete actions.

[0111] The specific formula is as follows:

[0112]

[0113] Where P is the output probability corresponding to a certain computing node, and N represents the number of cycles of the subtask. For the floor operation, the action space is mapped to integers in the range [0, N].

[0114] Furthermore, in step S3, the policy network generates corresponding scheduling actions based on the result information and continuously interacts with the environment. The method for completing the scheduling process includes:

[0115] Step S33: Set the reward method; that is, the reward function (Reward).

[0116] The design of rewards is crucial in reinforcement learning, as it affects whether the learning converges and directly impacts the effectiveness of the ultimately learned policy network.

[0117] Please see Figure 8 The specific rewards are as follows:

[0118] Step S331: When the scheduling strategy does not meet the resource constraints, the reward is -1, and the solution that does not meet the constraints is quickly rejected.

[0119] Step S332: When the scheduling policy meets the resource constraints, the reward is related to the increased latency. The lower the increased latency of a certain scheduling, the higher the reward obtained by that scheduling.

[0120] Corresponding to the following formula:

[0121] reward = λ / (lantency) new -lantency old )

[0122] λ is a hyperparameter that allows for adjustment of the reward to obtain a better scheduling strategy.

[0123] For further details, please refer to Figure 9 In step S33, the method for setting the reward method further includes:

[0124] Step S333: Set a discount factor for the reward;

[0125] Step S334: After the scheduling is completed, the scheduling is evaluated and judged to obtain a final reward;

[0126] Step S335: Broadcast the ultimate reward to all strategies, update all previous strategies, and re-evaluate previous decisions.

[0127] In the above settings, a discount factor is set for the rewards in steps S331 and S332. By setting the above settings, each decision is viewed holistically. Even if some decisions do not cause delay, the delay increases as a whole. Therefore, the impact of the previous scheduling needs to be ignored.

[0128] Furthermore, in step S3, the policy network generates corresponding scheduling actions based on the result information and continuously interacts with the environment. The method for completing the scheduling process also includes:

[0129] Step S34: The policy network of the reinforcement learning part is based on the REINFORCE algorithm. It constructs a neural network to directly learn the policy, input state, and probability distribution of output action. The gradient update of the neural network is given by the following formula:

[0130]

[0131]

[0132] Furthermore, step S35 includes the policy network sharing a multi-layer neural network and a softmax layer. See also... Figure 10This is a schematic diagram of the policy network in reinforcement learning. The softmax function is used for multi-class classification problems, yielding a probability distribution of [0, 1]. To improve inference speed, this design employs a shared multi-layer neural network.

[0133] Please see Figure 11 In step S35, the method for sharing a multi-layer neural network in the policy network includes:

[0134] Step S351: Input the features of multiple points, pass them through a shared multi-layer neural network, and map them onto a one-dimensional vector to complete parallel computation; this process does not need to be implemented through loops.

[0135] Step S352: When a resource-constrained scheduling task with only one adder and one multiplier per cycle arrives, the trained graph reinforcement learning model quickly obtains an accurate clock cycle table. Please refer to [link to details]. Figure 12 This is a schematic diagram of the scheduling results of graph reinforcement learning.

[0136] Furthermore, reinforcement learning includes convergence strategies and decoding strategies;

[0137] Convergence strategy, used to fuse structural information and attribute information;

[0138] Decoding strategies are used to distinguish rich feature information and provide corresponding scheduling actions.

[0139] To enrich the description of this application, we will describe it from another perspective. This algorithm mainly consists of two parts: a graph neural network to implement the encoding process, and reinforcement learning to abstract and solve the scheduling problem. The overall algorithm uses an end-to-end scheduling approach, converting the .dot file into a precise clock cycle table.

[0140] During the generation of IR information, software compilers (such as LLVM) produce an intermediate file DFG.dot, which records the attributes and connections of all operations. Basic node features, such as degree, are first extracted from this data structure. Then, structural and global information is incorporated through a graph neural network. For each node, after re-encoding, a multi-dimensional feature vector is obtained. This rich feature information can be used for scheduling decisions. Scheduling decisions are essentially priority determinations; based on the high-dimensional feature vectors of two nodes, it is determined which has higher priority. Therefore, node scheduling can be transformed into a node classification problem.

[0141] However, these nodes are not labeled, and we don't know how to classify them. At this point, we need to utilize the idea of ​​reinforcement learning, allowing the data to interact with the real environment. Through continuous learning, reinforcement learning gradually masters classification methods or strategies. The entire reinforcement learning process continuously improves the strategy: first, a convergence strategy, where we need to learn how to fuse structural and attribute information; second, a decoding strategy, where reinforcement learning learns how to distinguish rich feature information and provide corresponding scheduling actions.

[0142] Ultimately, this invention utilizes a trained graph reinforcement learning model to solve real-world scheduling problems.

[0143] We compared the scheduling results of the ILP algorithm and our proposed algorithm. Under the condition of satisfying resource constraints, for a 10-node task, the optimal scheduling delay for both the ILP algorithm and our proposed algorithm is 5 cycles, with solution times of 0.09s and 0.03s, respectively. For a 20-node task, the optimal scheduling delay for both the ILP algorithm and our proposed algorithm is 12 cycles, with solution times of 0.22s and 0.052s, respectively. This demonstrates that the algorithm presented in our paper achieves near-exact solution accuracy while significantly improving the solution speed.

[0144] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0145] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A FPGA high-level synthesis scheduling method based on graph reinforcement learning, characterized in that, The method comprises the following steps: obtaining an initialization schedule; The graph neural network receives data in a non-Euclidean space, extracts features from the data in the non-Euclidean space, expands the expression range of the graph neural network by adding a multi-layer neural network, extracts feature data required for scheduling, embeds the features, obtains result information, and delivers the result information to a policy network of reinforcement learning; The policy network of reinforcement learning generates corresponding scheduling actions according to the result information and the initialization schedule, and continuously interacts with the environment to complete the scheduling process; An accurate clock cycle corresponding to the operation is obtained and output; wherein: The method for the policy network of reinforcement learning to generate corresponding scheduling actions according to the result information and continuously interact with the environment to complete the scheduling process comprises: (1) setting a state space: divide the entire scheduling task into multiple subtasks; complete the multiple subtasks to complete the entire scheduling process; (2) setting an action space: the feature vectors of the multiple nodes are decoded through a multi-layer neural network; After the function outputs, so that each node corresponds to only one probability; map the probability into discrete actions; (3) setting a reward method; when the scheduling strategy does not satisfy the resource constraint, the reward is -1, and the solution that does not satisfy the constraint condition is quickly denied; when the scheduling strategy satisfies the resource constraint, the reward is related to the increased delay, and the lower the increased delay of a scheduling is, the higher the reward obtained by the scheduling is, corresponding to the following formula: ； As hyper-parameters, it is convenient to adjust the reward to obtain a better scheduling strategy.

2. The FPGA high-level synthesis scheduling method based on graph reinforcement learning of claim 1, wherein: The method for expanding the expression range of the graph neural network by adding a multi-layer neural network comprises: ； wherein representative node iterating k times, a feature vector representing an initial input, is an arbitrary number that can be learned; after k iterations, that is, k layers of graph convolutional neural network GCN; then, a pooling operation is performed to obtain global and critical path information.

3. The FPGA high-level synthesis scheduling method based on graph reinforcement learning of claim 1, wherein: In step (1), the method for dividing the entire scheduling task into multiple subtasks comprises: embedding features of all nodes by using a graph neural network; according to the initialization schedule, the re-encoded data is blocked; each block obtained by blocking is taken as a subgraph, and each subgraph corresponds to a feature matrix with a moderate size; set a hyperparameter N, and execute nodes in N cycles each time.

4. The FPGA high-level synthesis scheduling method based on graph reinforcement learning of claim 1, wherein: In step (3), the method for setting the reward method further comprises: set a discount factor for the reward; after the scheduling is completed, the scheduling is evaluated and judged to obtain a final reward; broadcast the final reward to all policies to update all previous policies and reevaluate the previous decisions.

5. The FPGA high-level synthesis scheduling method based on graph reinforcement learning of claim 1, wherein: The policy network of reinforcement learning is based on the REINFORCE algorithm, constructs a neural network to directly learn a strategy, inputs a state, and outputs a probability distribution of an action.

6. The FPGA high-level synthesis scheduling method based on graph reinforcement learning of claim 1, wherein: The policy network of reinforcement learning shares a multi-layer neural network and a layer; The method for the policy network of reinforcement learning to share the multi-layer neural network comprises: step S351, input the features of the multiple points, map them to a one-dimensional vector through the shared multi-layer neural network, and complete parallel operation; step S352, when a resource-constrained scheduling task with only one adder and multiplier per cycle arrives, the trained graph reinforcement learning model quickly obtains an accurate clock cycle table.

7. The FPGA high-level synthesis scheduling method based on graph reinforcement learning of claim 1, wherein: The reinforcement learning of the policy network based on reinforcement learning further comprises a convergence strategy and a decoding strategy; the convergence strategy is used for fusing structural information and attribute information; Decoding strategy is used to distinguish the rich feature information and give the corresponding scheduling action.

Images