FPGA-oriented DSP layout optimization method for CNN accelerator

Abstract

The invention discloses an FPGA-oriented DSP layout optimization method for a CNN accelerator, and the method comprises the following steps: S1, DSP path information extraction: converting a designed netlist into graph representation, and carrying out data path DSP node recognition and data path DSP graph construction; and S2, data path driven DSP layout: distributing data path DSP nodes to specific positions on the FPGA according to the extracted data path DSP diagram. According to the method, a data path DSP graph is automatically extracted and constructed through DSP node classification based on a graph neural network (GCN) and an optimization algorithm of a minimum cost flow (MCF) model, and compact layout and cascade constraint optimization are combined, so that the time sequence performance and the calculation efficiency of the layout are greatly improved, meanwhile, the clock frequency and the throughput are remarkably improved, and the time sequence performance and the calculation efficiency of the layout are greatly improved. And a universal and efficient FPGA layout solution is provided for various CNN accelerator architectures.

Description

Technical Field

[0001] The present invention relates to the field of integrated circuit design and hardware accelerator optimization, and in particular to a DSP layout optimization method for a CNN accelerator oriented to an FPGA. Background Art

[0002] In recent years, convolutional neural networks (CNNs) have been widely used in image recognition, object detection and other fields due to their excellent performance in computer vision tasks. As a highly flexible hardware platform, field programmable gate arrays (FPGAs) are ideal for deploying CNNs due to their reconfigurability and parallel computing capabilities. However, the layout stage in the FPGA design process has a significant impact on the final performance, and existing technologies still face many challenges in optimizing layout.

[0003] In FPGA design, the layout problem is a complex combinatorial optimization problem, which requires mapping heterogeneous components (such as lookup tables LUT, flip-flops FF, digital signal processing units DSP, etc.) to fixed positions on the FPGA chip. The layout results directly affect the wiring length and timing performance. However, due to the heterogeneity of resource distribution in FPGAs (such as DSP and RAM are distributed in columns) and the requirements of modern CNN accelerators for wide bit width and high parallelism, traditional layout methods are difficult to effectively solve these problems.

[0004] Existing layout algorithms are mainly divided into two categories: simulated annealing and analytical methods. Although simulated annealing can achieve high-quality layout in small-scale designs, its running time increases exponentially as the design scale increases, making it difficult to meet the needs of large-scale CNN accelerators. Analytical methods improve the scalability and efficiency of layout through mathematical models, but usually ignore the structural information of the data path, resulting in routing detours, which in turn affects timing performance. In addition, the lack of data path information is one of the main shortcomings of existing technologies. The data path consists of digital signal processing units (DSPs) responsible for calculations, and its layout determines the criticality of the timing path. However, many existing methods only focus on the spatial distribution of components and fail to integrate data path information into the layout process, which leads to increased routing complexity and decreased performance. Even though some studies have attempted to introduce data path rules, such as DSP grouping based on graph clustering or DSP allocation based on empirical rules, most of these methods lack versatility and automation capabilities and are difficult to adapt to different FPGA architectures and CNN designs. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a DSP layout optimization method for CNN accelerators for FPGA. Through the optimization algorithm of DSP node classification and minimum cost flow (MCF) model based on graph neural network (GCN), the data path DSP graph is automatically extracted and constructed. Combined with compact layout and cascade constraint optimization, the timing performance and computing efficiency of the layout are greatly improved, while the clock frequency and throughput are significantly improved, providing a general and efficient FPGA layout solution for various CNN accelerator architectures.

[0006] The object of the present invention is to achieve the following technical solution: a DSP layout optimization method for a CNN accelerator for FPGA, comprising the following steps: S1. DSP path information extraction: convert the design netlist into a graph representation, and perform data path DSP identification and data path DSP graph construction; S2. Datapath driven DSP placement: Datapath DSP nodes are assigned to specific locations on the FPGA based on the extracted datapath DSP graph.

[0007] The beneficial effects of the present invention are as follows: the present invention automatically extracts and constructs a data path DSP graph through an optimization algorithm based on DSP node classification and minimum cost flow (MCF) model of a graph neural network (GCN), and combines compact layout and cascade constraint optimization to greatly improve the timing performance and computing efficiency of the layout, while significantly improving the clock frequency and throughput, providing a general and efficient FPGA layout solution for a variety of CNN accelerator architectures. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a flow chart of the method of the present invention; Figure 2 A schematic diagram for legalizing a digital signal unit of a data path; Figure 3 Schematic diagram of the incremental data path DSP layout. DETAILED DESCRIPTION

[0009] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0010] The present invention uses the design netlist of the CNN accelerator as input, and then uses the existing FPGA layout tool to initialize the layout of all components to generate an initial layout plan, where the initial position is the initial solution calculated by the existing layout tool, and then iteratively optimizes the layout until the result converges, and finally outputs the optimized layout result. In each iteration, the layout of the DSP part of the current layout is first discarded, the layout of the non-DSP components is retained, and then the DSP layout module driven by the data path is used to optimize the positions of the DSP and non-DSP components in turn. This process involves the detailed layout of the DSP macro to ensure the rationality of the layout and the performance of the circuit; specifically: like Figure 1 As shown, a DSP layout optimization method for CNN accelerator for FPGA is characterized by comprising the following steps: S1. DSP path information extraction: convert the design netlist into a graph representation, and perform data path DSP identification and data path DSP graph construction; The step S1 comprises: S101. Graph representation and data path DSP node identification: convert the design netlist into a directed graph and identify the data path DSP nodes and control path DSP node classification, thereby realizing data path DSP node identification; The step S101 includes: A1. A design netlist is given, wherein the design netlist includes logic components and connection relationships between the logic components; A2. Convert the design netlist into a directed graph G=(V,E), where the nodes in the node set V represent the logic components in the design netlist, and the edges in the edge set E represent the connection relationship between the logic components; the logic components include DSP, LUT and RAM; A3. Extracting the features of each node from the design netlist, including local features and global features; the local features include in-degree and out-degree; the global features include closeness centrality, betweenness centrality, eccentricity and the number of feedback loops; For any node v in the node set V, the definitions of in-degree, out-degree, betweenness centrality, closeness centrality, eccentricity, and number of feedback loops are as follows: The in-degree of a node v refers to: the number of nodes connected to the input of node v; The out-degree of a node v refers to: the number of nodes connected to the output of node v; Betweenness centrality measures the importance of a node as an intermediary for the shortest path between other nodes. The betweenness centrality of a node v is defined as follows: in, Representation Node ui and u l The number of shortest paths between represents the number of paths passing through node v among these paths, represents the betweenness centrality of node v; Closeness centrality is used to measure the inverse of the average distance between a node and other nodes. The closeness centrality of node v is defined as follows: Among them, d(v,u) represents the shortest path distance between node v and node u, represents the closeness centrality of node v; The eccentricity measures the distance between a node and the farthest node. The eccentricity of a node v is defined as follows: Among them, d(v,u) represents the shortest path distance between node v and node u, represents the eccentricity of node v; The number of feedback loops of node v refers to: the number of paths starting from node v and returning to node v; A4. Model the DSP identification problem as a binary classification problem and label some DSP nodes in the node set V: the DSP on the data path is labeled as one category, and the DSP on the control path is labeled as another category; A5. Construct a graph neural network model, wherein the graph neural network model includes: a graph convolutional neural network GCN, a ReLU activation function, a dropout layer, a graph convolutional neural network GCN, a ReLU activation function, a dropout layer, a fully connected layer, and a Softmax classifier connected in sequence; A6. Use the labeled node features to train the graph neural network model, and classify the unlabeled DSP nodes in the node set V by training the mature graph neural network model.

[0011] In an embodiment of the present application, the architecture of the model includes two GCN layers, each followed by a ReLU activation function and a dropout layer to prevent overfitting. During the training process, the model optimizes parameters by minimizing the cross entropy loss function. Through the trained model, the DSP nodes in the design can be classified to accurately distinguish the data path DSP from the control path DSP. The classified data will be used in the subsequent layout optimization process to ensure that the layout of the DSP macro on the FPGA meets both the requirements of the data flow and the requirements of the control logic. Such a method not only improves the automation of the layout, but also enhances the performance and efficiency of the final hardware implementation. In an embodiment of the present application, in order to enhance the classification performance, the present invention adopts a weighted loss function to give higher penalty weights to nodes of a few categories. In addition, the accuracy of the model is evaluated by a cross-validation method to ensure its applicability on different design benchmarks.

[0012] During the training process, the input of the graph neural network model is the labeled node features, and the expected output is the category of the node label; after the training is completed, the node features of the unlabeled nodes are input into the graph neural network model to complete the classification.

[0013] Using graph neural networks (GCN) for classification, this method can effectively classify DSPs and guide the subsequent DSP graph construction process. Through the modeling of this binary classification problem, the present invention can accurately distinguish which DSPs belong to the data path and which belong to the control path when implementing a CNN accelerator on an FPGA. This distinction is crucial for subsequent layout optimization because it directly affects the compactness of the data flow. Through the powerful feature extraction capabilities of graph neural networks, the present invention can learn the hierarchical representation of nodes and integrate them into the layout optimization framework in an end-to-end manner, thereby achieving more efficient and accurate DSP macro layout.

[0014] S102. After completing the DSP classification, only the identified data path DSP nodes are retained, and a data path DSP graph is constructed based on them.

[0015] The step S102 includes: Shortest path calculation: Using iterative deepening depth-first search algorithm, starting from each data path DSP node, calculate the shortest path to other data path DSP nodes; A subgraph containing only data path DSP nodes and their connection relationships is constructed based on the shortest path information, and the subgraph describes the topological structure between the data path DSPs.

[0016] Exclusion of control path DSP When constructing the data path DSP graph, the present invention further removes the control path DSP nodes to avoid their negative impact on layout compactness. Specifically, the control path is usually associated with more storage elements (such as triggers, RAM), which helps signal retention and allows greater placement flexibility and less compactness, while the data path is associated with relatively fewer storage elements and has a higher compact layout requirement to improve performance. Therefore, only the data path DSP nodes related to calculations are retained in the data path DSP graph. Through the implementation of the data path DSP extraction stage, the present invention ensures the integrity and accuracy of the data path information in the layout optimization process, thereby providing a guarantee for the ultimate timing performance improvement.

[0017] S2. Datapath driven DSP placement: Datapath DSP nodes are assigned to specific locations on the FPGA based on the extracted datapath DSP graph.

[0018] Datapath-driven DSP placement is an important step in optimizing placement, which aims to allocate datapath-related DSP nodes to specific locations on the FPGA based on the extracted datapath DSP graph, while satisfying the cascade constraints on the datapath and optimizing the overall placement; Data path DSP layout is a complex allocation problem, which requires optimizing the distribution of data path DSP under the premise of satisfying constraints. To this end, the present invention formalizes the problem into an integer linear programming (ILP) problem, and improves the layout quality by optimizing distance and path characteristics. However, the computational complexity of directly solving ILP is high. Subsequently, the ILP problem is converted into a minimization cost flow (Min-Cost Flow, MCF) problem through linearization technology, which greatly reduces the computational complexity of the optimization problem; The step S2 comprises: In order to solve the DSP layout that realizes data path drive. The DSP layout problem separates the layout of the control path DSP from the layout of the data path DSP, where the DSP of the control path is optimized by commercial tools and the algorithm of the present invention. The data path DSP layout problem is formalized as a constrained optimization problem, the goal is to maximize the timing performance while satisfying all design rules and layout constraints. The following is a detailed description of the data path DSP layout problem modeling: S201. Convert the data path DSP layout problem into a constrained optimization problem and model the data path DSP layout problem: Assume that the datapath DSP graph contains N datapath DSP nodes that need to be laid out. Needs to be mapped to one of the M DSP locations in the FPGA; The coordinates of the jth DSP position are ; Introducing a binary decision variable x i，j Indicates the distribution relationship: The basic hard constraints that must be met for a given datapath DSP node layout are: Single-location constraint: Each datapath DSP node must be assigned to one location: Position uniqueness constraint: Each position can be assigned at most one datapath DSP node: S202. In this part of the constraints, we use the previously converted data path DSP graph. Cascade constraints are key constraints for DSP layout optimization in FPGAs, which aim to ensure that multiple data path DSP units are closely arranged in layout, thereby achieving efficient signal transmission and calculation in hardware.

[0019] In order to maintain the compactness and directionality of the data link, the data path DSP nodes can be arranged in a cascaded form in the layout, and the data path DSP node cascade constraints are introduced: The cascaded Datapath DSP nodes on the Datapath DSP diagram must be arranged in adjacent positions in the same column: The cascade relationship between multiple data path DSP nodes is decomposed into the cascade relationship between multiple DSP pairs. The cascade relationship between each DSP pair is a cascade pair. Assume that C is the set of all cascade pairs. Each cascade pair (c p ,c s ) represents the predecessor and successor nodes, the following constraints must be met: p and c s They must be adjacent in layout to ensure continuity and efficiency of signal transmission, which means they need to meet the following constraints: x cp,j Represents the predecessor node c p The binary decision variable at the jth DSP position, x cs,j+1 represents the successor node c s The binary decision variable at the j+1th DSP position; S203. In this part of the constraints, we also used the previously converted data path DSP diagram. In FPGA, the data path is composed of DSPs, which directly participate in the calculation and determine the key timing path of the design. Therefore, in the DSP layout, it is necessary not only to meet the cascade constraints, but also to further optimize the characteristics of the data path to improve the compactness and timing performance of the layout. The cascade constraints ensure that the interconnected DSP nodes in the data path are closely arranged in the physical layout, reducing the delay of signal transmission. The overall layout of the data path DSP aims to reduce the wiring length of the DSP nodes by minimizing the path, and maintain the directionality of the path, reduce detours in signal transmission, and avoid unnecessary path bending. Here we incorporate the cascade constraints and data path constraints into the overall optimization goals, convert them into soft constraints, and then unify the modeling. The overall mathematical model combines the directionality of the data path and the compactness of the layout, and is expressed as follows: (1) in, Represents the edge set between any datapath DSP node and other component nodes connected to them in the directed graph after netlist conversion. e for The edge of represents the edge set in the datapath DSP graph, eD for The edge in ep and es They represent the predecessor node and successor node of edge e respectively. x ep and x es They represent the assignment variables of the predecessor node and the successor node respectively, and are both vectors of length S; where S represents the number of layout positions of all components on the FPGA; if ep is assigned to the pth position, then x ep The pth element of is 1, and the rest are 0; if es is assigned to the qth position, then x es The pth element of is 1, and the rest are 0; where p=1,2,…,S; q=1,2,…,S; and p is not equal to q; p x A vector consisting of the x-coordinates of the layout positions of all components on the FPGA; p y is a vector of the y coordinates of the placement positions of all components on the FPGA; since the number of placement positions is S, p x , p xThe length is S; the superscript T indicates transposition, A distance matrix is ​​constructed; x ep - x es is a vector x ep With vector x es The difference vector has one element of 1 and one element of -1. The other elements are 0. The positions of 1 and -1 represent the positions of the predecessor and successor nodes. Multiply the transpose of the difference vector with the distance matrix and then multiply it by the difference vector to get the distance between the predecessor node and the successor node. θ eDp Represents edge eD Predecessor node eD The angle between the line connecting to the ARM core in the FPGA and the horizontal line; θ eDs Represents edge eD The successor node eD The angle between the line to the ARM core in the FPGA and the horizontal line; the ARM core is located at the lower left corner of the FPGA by default; λ is a hyperparameter used to control the trade-off between distance and data path effort, and η is the penalty factor; The hard constraints directly restrict the layout variables x i,j The legitimacy of the DSP node is to ensure that the resource allocation on the FPGA meets the basic rules: Single position constraint: Each data path DSP node i must and can only be assigned to one position j, that is: This constraint ensures that each datapath DSP node does not occupy multiple locations repeatedly; Resource limitation constraint: Each position j can only accommodate at most one datapath DSP node, expressed as: This constraint ensures that the limited resources on the FPGA are used reasonably and avoids resource conflicts; Variable validity constraints: x i,j is a binary variable indicating whether data path DSP node i is assigned to position j, that is: S204. The three penalty terms in the objective function together constitute soft constraints, which enable flexible layout adjustment through the weight parameters of the optimization process: Distance constraints between datapath DSP nodes and other components they are connected to: This term represents the sum of the squares of the distances between adjacent DSP nodes in the path. The optimization goal is to minimize the data path length to reduce routing delay and routing complexity; Data path directionality constraints: in represents the edge set in the data path DSP graph, θ eDp ,θ eDs Represents the angle between the predecessor, successor and horizontal line of edge eD in the data path DSP diagram; This item is measured by the difference of direction cosine values. The optimization goal is to maintain the directionality of the data path and avoid path detours. The weight parameter λ is used to adjust the intensity of directional optimization. Cascading constraints: , Where C is the set of all cascade relationships, x cp,j Represents the predecessor node c p The binary decision variable at the jth DSP position, x cs,j+1 represents the successor node c s The binary decision variables at the j+1th DSP position are converted into soft constraints by converting the cascade relationship and adding them into the objective function; This item ensures the close arrangement of cascaded DSP nodes in physical positions, and the weight parameter η controls the priority of the cascade constraint in the overall optimization. Different from traditional hard constraints, in soft constraints, the parameters λ and η of the soft constraint item provide the flexibility to further adjust the optimization strategy, so that the present invention can achieve the best balance between timing performance and layout compactness.

[0020] S204. Solve the constructed mathematical model of the optimization problem.

[0021] The step S204 includes: B1. After completing the linearization, further improve the optimization problem into the minimum cost flow (Min- The MCF model is a classic network optimization problem, which is very suitable for dealing with resource allocation problems. Its core idea is as follows: 1. Network modeling: The data path DSP nodes and FPGA resources are modeled as nodes of a directed graph, with the DSP nodes considered as "sources" and the available locations in the FPGA as "sinks". The weight of each edge represents the "cost" required to assign a DSP node to a location, which includes path length, directional deviation, and the degree of violation of the cascade relationship.

[0022] 2. Capacity restriction: The hard constraints are modeled by edge capacity restriction. For example, if the capacity of an edge is set to 1, it means that a certain position can accommodate at most one data path DSP node.

[0023] 3. Solution method: Taking advantage of the monotonicity and integer solution characteristics of the MCF model, use the above analytical algorithm to minimize the total allocation cost while satisfying the constraints. Compared with directly solving integer programming problems, the MCF model utilizes the characteristics of the network optimization algorithm and can quickly obtain integer solutions, significantly improving the solution efficiency.

[0024] B2. After the data path DSP layout optimization step is completed, because the cascade constraints were placed in the objective function to reduce the complexity of the algorithm, there will be a few conflicts that violate the DSP cascade constraints or exceed the resource limit, such as Figure 2 As shown, a legalization step is added as a post-processing: ensuring that the final result meets all design rules.

[0025] Perform inter-column legalization and adjust the horizontal position of the data path DSP to ensure that the path DSP nodes in the cascade relationship are located in the same column while minimizing the horizontal displacement. This goal is achieved through the hard constraints of integer linear programming, and the optimization problem is formalized as the following formula: t i,k is a binary variable indicating whether path DSP node i is assigned to column k: if yes, it takes 1, if not, it takes 0; D col (i, k) is the horizontal displacement of path DSP node i from the current column to column k, Ncol represents the number of path DSP node columns on the FPGA, and M k represents the number of available DSP positions in column k; C represents the set of all cascade relationships, c p and c s It is the front and back cascade nodes, that is, the predecessor node and the successor node; t cp,k and t cs,k A binary variable representing the successor node of a predecessor node; Objective function (2) minimizes the weighted sum of the horizontal displacements of all path DSP nodes. Constraint (2a) ensures that each path DSP node is assigned to and only to one column. Constraint (2b) ensures that the number of path DSP nodes in each column does not exceed the number of available positions. Constraint (2c) ensures that the path DSP nodes in the cascade relationship must be assigned to the same column. By solving this optimization problem, the horizontal position of the data path DSP can be effectively adjusted. In-column legalization: After obtaining the solution of inter-column legalization, intra-column legalization further optimizes the vertical position of the DSP nodes in each column to ensure that the DSP nodes in the cascade relationship are closely arranged while allowing gaps between non-cascade nodes. The optimization problem is formalized as the following formula: r i represents the vertical position of the path DSP node i; R col (i) is the vertical position of path DSP node i in the initial layout, M k is the total number of DSP positions in column k, N k is the number of DSP nodes in column k, C k represents the set of all cascade relationships in column k; Objective function (3) minimizes the deviation between the current vertical position of a DSP node and its initial position; constraint (3a) ensures that the DSP nodes in the cascade relationship are closely arranged; constraint (3b) allows gaps between non-cascade nodes to avoid overlap. By solving this optimization problem, the impact of vertical displacement on layout compactness is minimized while maintaining the cascade constraint.

[0026] like Figure 3 As shown, in order to further improve the compactness of the layout and satisfy the cascade constraints, the present invention adopts an incremental iterative optimization method: first, the initial positions of all logic components in the design netlist are given (which can be generated using an existing FPGA layout tool, such as vivado or AMF2.0); after completing the data path DSP identification and data path DSP graph construction according to step S1, the positions of other components except the path DSP nodes are fixed, and only the layout of the data path DSP is optimized according to step S2 to generate preliminary results; Then fix the position of the data path DSP node, and then use commercial (or open source) layout tools (such as vivado or AMF2.0, one of their rules is to minimize the distance between connected components so that the line length is shortened and data communication is fast) to adjust the positions of other components except the path DSP node; In subsequent iterations, the positions of the fixed data path DSP nodes and other components are alternately fixed to iterate and continuously optimize the overall layout; Convergence judgment: When the change of the objective function of formula (1) is lower than the preset threshold, the iteration is stopped and the final result is output.

[0027] The DSP layout optimization framework of the present invention was evaluated in multiple experimental environments. The experimental settings included target devices, benchmark designs, GCN training, and layout configurations, and the optimization effect of the framework was verified by comparative analysis. This experiment selected Xilinx Zynq UltraScale + MPSoC ZCU104 FPGA as the target device. The device has column-distributed heterogeneous resources (such as LUTs, DSPs, and RAM, etc.), which can fully verify the effectiveness of the present invention in complex designs. The benchmark design selected multiple test cases from the DAC system design competition, including iSmart DNN, SkyNet, and the expanded SkrSkr series design, with the number of DSPs ranging from 197 to 1431, covering CNN accelerators of different sizes. The experimental environment runs on a Linux workstation equipped with a 10-core CPU, and is combined with tools such as Vivado 2020.2 and RapidWright for synthesis and layout routing.

[0028] In the data path DSP extraction stage, the present invention uses a DSP node classification method based on graph neural network (GCN). By extracting features such as closeness centrality, betweenness centrality, in-degree, out-degree and average shortest path length from the design netlist, the global and local properties of the DSP node are fully reflected. The GCN model adopts a two-layer convolutional structure, combined with three layers of fully connected layers and Softmax classifier for classification, and uses a weighted loss function to deal with the class imbalance problem. The cross-validation results show that the classification accuracy of GCN reaches an average of 96%, which is significantly better than the 81% of the traditional support vector machine (SVM), which fully proves the accuracy of data path information extraction.

[0029] In the layout optimization stage, the initial layout is first generated by Vivado, and the clock frequency is gradually increased to test the performance limit of the tool. Subsequently, the DSP layout optimization framework of the present invention is applied to optimize based on the minimum cost flow (MCF) model, where the maximum number of iterations is set to 50 and the weight parameter λ of the data path constraint is set to 100. After the layout is completed, the layout is further adjusted through inter-column and intra-column legalization to strictly meet the cascade constraints and minimize the displacement deviation. The optimization results show that the framework of the present invention exhibits excellent performance on multiple benchmark designs. Compared with Vivado, the present invention improves the worst negative timing slack (WNS) by an average of 32%; compared with AMF-Placer, the improvement is 65%. In addition, although the compact layout may slightly increase the wiring length, its impact is minimal, and the layout remains efficient and controllable. In terms of running time, compared with AMF-Placer, DSPlacer reduces the time by about 50%, showing excellent efficiency.

[0030] Analysis of the layout results shows that the present invention is particularly outstanding in two aspects. First, the compact layout driven by the data path effectively reduces the routing detours; second, the cascade constraints are strictly followed, which improves the timing consistency of the layout. The visualization of the layout further verifies these advantages. Compared with Vivado and AMF-Placer, the layout generated by DSPlacer is more regular and compact in the data path, significantly reducing the number of cross-column connections.

[0031] In summary, the DSP layout optimization framework of the present invention outperforms existing tools in terms of timing performance, layout compactness, and operational efficiency by integrating data path information and efficient optimization algorithms, and provides a general and efficient solution for the design of CNN accelerators on FPGAs.

[0032] The above is a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art do not depart from the spirit and scope of the present invention, and should be within the scope of protection of the claims attached to the present invention.

Claims

1. A DSP layout optimization method for CNN accelerator for FPGA, characterized by: The following steps are involved: S1. DSP path information extraction: convert the design netlist into a graph representation, identify data path DSP nodes and construct the data path DSP graph; S2. Datapath driven DSP placement: Datapath DSP nodes are assigned to specific locations on the FPGA based on the extracted datapath DSP graph.

2. The DSP layout optimization method for CNN accelerator for FPGA according to claim 1, characterized in that: The step S1 comprises: S101. Graph representation and data path DSP node identification: convert the design netlist into a directed graph, and classify the data path DSP nodes and the control path DSP nodes, so as to realize data path DSP node identification; S102. After completing the DSP classification, only the identified data path DSP nodes are retained, and a data path DSP graph is constructed based on them.

3. The DSP layout optimization method for CNN accelerator for FPGA according to claim 2, characterized in that: The step S101 includes: A1. A design netlist is given, wherein the design netlist includes logic components and connection relationships between the logic components; A2. Convert the design netlist into a directed graph G=(V,E), where the nodes in the node set V represent the logic components in the design netlist, and the edges in the edge set E represent the connection relationship between the logic components; A3. Extracting the features of each node from the design netlist, including local features and global features; the local features include in-degree and out-degree; the global features include closeness centrality, betweenness centrality, eccentricity and the number of feedback loops; For any node v in the node set V, the definitions of in-degree, out-degree, betweenness centrality, closeness centrality, eccentricity, and number of feedback loops are as follows: The in-degree of a node v refers to: the number of nodes connected to the input of node v; The out-degree of a node v refers to: the number of nodes connected to the output of node v; Betweenness centrality measures the importance of a node as an intermediary for the shortest path between other nodes. The betweenness centrality of a node v is defined as follows: in, Representation Node u i and u l The number of shortest paths between Representation Node u i and u l The number of paths passing through node v in the shortest path between represents the betweenness centrality of node v; Closeness centrality is used to measure the inverse of the average distance between a node and other nodes. The closeness centrality of node v is defined as follows: Among them, d(v,u) represents the shortest path distance between node v and node u, represents the closeness centrality of node v; The eccentricity measures the distance between a node and the farthest node. The eccentricity of a node v is defined as follows: Among them, d(v,u) represents the shortest path distance between node v and node u, represents the eccentricity of node v; The number of feedback loops of node v refers to: the number of paths starting from node v and returning to node v; A4. Model the DSP identification problem as a binary classification problem and label some DSP nodes in the node set V: the data path DSP nodes are labeled as one category, and the control path DSP nodes are labeled as another category; A5. Construct a graph neural network model, wherein the graph neural network model includes: a graph convolutional neural network GCN, a ReLU activation function, a dropout layer, a graph convolutional neural network GCN, a ReLU activation function, a dropout layer, a fully connected layer, and a Softmax classifier connected in sequence; A6. Use the marked node features to train the graph neural network model, and classify the unlabeled DSP nodes in the node set V by training the mature graph neural network model, so as to realize data path DSP node identification.

4. The DSP layout optimization method for CNN accelerator for FPGA according to claim 3, characterized in that: The logic components include DSP, LUT and RAM.

5. The DSP layout optimization method for CNN accelerator for FPGA according to claim 3, characterized in that: The step S102 includes: Shortest path calculation: Using iterative deepening depth-first search algorithm, starting from each data path DSP node, calculate the shortest path to other data path DSP nodes; Based on the shortest path information, a subgraph containing only data path DSP nodes and their connection relationships is constructed. The subgraph describes the topological structure between data path DSPs and is recorded as a data path DSP graph. The connection relationship between data path DSP nodes is the edge of the data path DSP graph.

6. The DSP layout optimization method for CNN accelerator for FPGA according to claim 1, characterized in that: The step S2 comprises: S201. Convert the data path DSP node layout problem into a constrained optimization problem and model the data path DSP layout problem: Assume that the datapath DSP graph contains N datapath DSP nodes that need to be laid out. Needs to be mapped to one of the M DSP locations in the FPGA; The coordinates of the jth DSP position are ; Introducing a binary decision variable x i，j Indicates the distribution relationship: The basic hard constraints that must be met for a given datapath DSP node layout are: Single-location constraint: Each datapath DSP node must be assigned to one location: Position uniqueness constraint: Each position can be assigned at most one datapath DSP node: S202. In order to maintain the compactness and directionality of the data link, the data path DSP nodes can be arranged in a cascaded form in the layout, and the data path DSP node cascade constraint is introduced: The cascaded Datapath DSP nodes on the Datapath DSP diagram must be arranged in adjacent positions in the same column: The cascade relationship between multiple data path DSP nodes is decomposed into the cascade relationship between multiple DSP pairs. The cascade relationship between each DSP pair is a cascade pair. Assume that C is the set of all cascade pairs. Each cascade pair in the set C is denoted as (c p ,c s ), c p ,c s Represent the predecessor and successor nodes respectively, the following constraints must be met: p and c s They must be adjacent in layout to ensure continuity and efficiency of signal transmission, which means they need to meet the following constraints: x cp,j Represents the predecessor node c p The binary decision variable at the jth DSP position, x cs,j+1 represents the successor node c s The binary decision variable at the j+1th DSP position; S203. In DSP layout, not only the cascade constraints need to be met, but also the characteristics of the data path need to be further optimized to improve the compactness and timing performance of the layout. The cascade constraints and data path constraints are incorporated into the overall optimization goals and converted into soft constraints, thereby unified modeling: The overall mathematical model combines the directionality of the data path and the compactness of the layout, as shown below: (1) in, Represents the edge set between any datapath DSP node and other component nodes connected to them in the directed graph after netlist conversion. e for The edge of represents the edge set in the datapath DSP graph, eD for The edge in ep and es They represent the predecessor node and successor node of edge e respectively. x ep and x es They represent the assignment variables of the predecessor node and the successor node respectively, and are both vectors of length S; where S represents the number of layout positions of all components on the FPGA; if ep is assigned to the pth position, then x ep The pth element of is 1, and the rest are 0; if es is assigned to the qth position, then x es The pth element of is 1, and the rest are 0; where p=1,2,…,S; q=1,2,…,S; and p is not equal to q; p x A vector consisting of the x-coordinates of the layout positions of all components on the FPGA; p y is a vector of the y coordinates of the placement positions of all components on the FPGA; since the number of placement positions is S, p x , p x The length is S; the superscript T indicates transposition, A distance matrix is ​​constructed; x ep - x es is a vector x ep With vector x es The difference vector has one element of 1 and one element of -1; the other elements are 0. The positions of elements 1 and -1 represent the positions of predecessor and successor nodes. θ eDp Represents edge eD Predecessor node eD The angle between the line connecting to the ARM core in the FPGA and the horizontal line; θ eDs Represents edge eD The successor node eD The angle between the line to the ARM core in the FPGA and the horizontal line; the ARM core is located at the lower left corner of the FPGA by default; λ is a hyperparameter used to control the trade-off between distance and data path effort, and η is the penalty factor; The hard constraints directly restrict the layout variables x i,j The legitimacy of the DSP node is to ensure that the resource allocation on the FPGA meets the basic rules: Single position constraint: Each data path DSP node i must and can only be assigned to one position j, that is: This constraint ensures that each datapath DSP node does not occupy multiple locations repeatedly; Resource limitation constraint: Each position j can only accommodate at most one datapath DSP node, expressed as: This constraint ensures that the limited resources on the FPGA are used reasonably and avoids resource conflicts; Variable validity constraints: x i,j is a binary variable indicating whether data path DSP node i is assigned to position j, that is: S204. The three penalty terms in the objective function together constitute soft constraints, which enable flexible layout adjustment through the weight parameters of the optimization process: Distance constraints between datapath DSP nodes and other components they are connected to: This term represents the sum of the squares of the distances between the DSP node and other component nodes connected to it in the path. The optimization goal is to minimize the data path length to reduce routing delay and routing complexity; Data path directionality constraints: This item is measured by the difference of direction cosine values. The optimization goal is to maintain the directionality of the data path and avoid path detours. The weight parameter λ is used to adjust the intensity of directional optimization. Cascading constraints: Where C is the set of all cascade pairs, x cp,j Represents the predecessor node c p The binary decision variable at the jth DSP position, x cs,j+1 represents the successor node c s The binary decision variable at the j+1th DSP position; This term ensures the close arrangement of cascaded DSP nodes in physical locations, and the weight parameter η controls the priority of the cascade constraints in the overall optimization; S204. Solve the constructed mathematical model of the optimization problem.

7. The DSP layout optimization method for CNN accelerator for FPGA according to claim 6, characterized in that: The step S204 includes: B1. The objective function and constraints of the optimization problem contain nonlinear terms. The nonlinear problem is improved and transformed into a linear problem through linearization technology. After the linearization is completed, the optimization problem is further improved and transformed into a minimum cost flow model for solution: Network modeling: The data path DSP nodes and FPGA resources are modeled as nodes of a directed graph, and the FPGA resources are recorded as available positions in the FPGA. The path DSP nodes are regarded as sources, and the available positions in the FPGA are regarded as sinks. The weight of each edge represents the cost required to assign the path DSP nodes to the corresponding positions. The cost includes the path length, directional deviation, and the degree of violation of the cascade relationship. Capacity restriction: Modeling hard constraints through edge capacity restrictions, minimizing the total allocation cost while satisfying the constraints; The layout position of the DSP nodes in the data path is obtained by solving the minimum cost flow model; B2. After the data path DSP layout optimization step is completed, in order to reduce the complexity of the algorithm, the cascade constraints are placed in the objective function to be calculated as soft constraints. Because the constraints are converted into soft constraints, there will be a few conflicts that violate the DSP cascade constraints or exceed resource limits. Add a legalization step as post-processing: Perform inter-column legalization and adjust the horizontal position of the data path DSP to ensure that the path DSP nodes in the cascade relationship are located in the same column while minimizing the horizontal displacement. This goal is achieved through the hard constraints of integer linear programming, and the optimization problem is formalized as the following formula: t i,k is a binary variable indicating whether path DSP node i is assigned to column k: if yes, it takes 1, if not, it takes 0; D col (i, k) is the horizontal displacement of path DSP node i from the current column to column k, Ncol represents the number of path DSP node columns on the FPGA, and M k represents the number of available DSP positions in column k; C represents the set of all cascade relationships, c p and c s It is the front and back cascade nodes, that is, the predecessor node and the successor node; t cp,k and t cs,k A binary variable representing the successor node of a predecessor node; Objective function (2) minimizes the weighted sum of the horizontal displacements of all path DSP nodes. Constraint (2a) ensures that each path DSP node is assigned to and only to one column. Constraint (2b) ensures that the number of path DSP nodes in each column does not exceed the number of available positions. Constraint (2c) ensures that the path DSP nodes in the cascade relationship must be assigned to the same column. By solving this optimization problem, the horizontal position of the data path DSP can be effectively adjusted. In-column legalization: After obtaining the solution of inter-column legalization, intra-column legalization further optimizes the vertical position of the DSP nodes in each column to ensure that the DSP nodes in the cascade relationship are closely arranged while allowing gaps between non-cascade nodes. The optimization problem is formalized as the following formula: r i represents the vertical position of path DSP node i; Rcol ( i ) is the vertical position of path DSP node i in the initial layout, M k is the total number of DSP positions in column k, N k is the number of DSP nodes in column k, C k represents the set of all cascade relationships in column k; Objective function (3) minimizes the deviation between the current vertical position of a DSP node and its initial position; constraint (3a) ensures that the DSP nodes in the cascade relationship are closely arranged; constraint (3b) allows gaps between non-cascade nodes to avoid overlap. By solving this optimization problem, the impact of vertical displacement on layout compactness is minimized while maintaining the cascade constraint.

8. The DSP layout optimization method for CNN accelerator for FPGA according to claim 6, characterized in that: The method further comprises: First, the initial positions of all logic components of the design netlist are given; after the data path DSP identification and data path DSP graph construction are completed according to step S1, the positions of other components except the path DSP nodes are fixed, and only the layout of the data path DSP is optimized according to step S2 to generate preliminary results; Then, the position of the data path DSP node is fixed, and then the position of other components except the path DSP node is adjusted using a commercial layout tool; In subsequent iterations, the positions of the datapath DSP nodes and other components are alternately fixed to iterate and continuously optimize the overall layout; Convergence judgment: When the change of the objective function of formula (1) is lower than the preset threshold, the iteration is stopped and the final result is output.

Images