A DSP layout optimization method for CNN accelerators on FPGA

Abstract

This invention discloses a DSP layout optimization method for FPGA-based CNN accelerators, comprising the following steps: S1. DSP path information extraction: converting the designed netlist into a graph representation, identifying datapath DSP nodes, and constructing a datapath DSP graph; S2. Datapath-driven DSP layout: allocating datapath DSP nodes to specific locations on the FPGA based on the extracted datapath DSP graph. This method automatically extracts and constructs the datapath DSP graph through a graph neural network (GCN)-based DSP node classification and a minimum cost flow (MCF) model optimization algorithm. Combined with compact layout and cascade constraint optimization, this method significantly improves the timing performance and computational efficiency of the layout, while significantly increasing clock frequency and throughput. This provides a universal and efficient FPGA layout solution for various CNN accelerator architectures.

Description

Technical Field

[0001] The present invention relates to the fields of integrated circuit design and hardware accelerator optimization, and in particular to a DSP layout optimization method for a CNN accelerator oriented to an FPGA. Background Art

[0002] In recent years, convolutional neural networks (CNNs) have been widely used in image recognition, object detection, and other fields due to their outstanding performance in computer vision tasks. Field-programmable gate arrays (FPGAs), as a highly flexible hardware platform with its reconfigurability and parallel computing capabilities, are ideal for deploying CNNs. However, the layout stage of the FPGA design flow significantly impacts final performance, and existing technologies still face many challenges in optimizing layout.

[0003] In FPGA design, layout is a complex combinatorial optimization problem that requires mapping heterogeneous components (such as lookup tables (LUTs), flip-flops (FFs), and digital signal processing units (DSPs)) to fixed locations on the FPGA chip. The layout results directly affect the routing length and timing performance. However, due to the heterogeneous distribution of resources in FPGAs (such as the column-based distribution of DSPs and RAM) and the requirements of modern CNN accelerators for wide bit widths and high parallelism, traditional layout methods have difficulty effectively solving these problems.

[0004] Existing layout algorithms are primarily categorized into two types: simulated annealing and analytical methods. While simulated annealing can achieve high-quality layouts for small designs, its runtime increases exponentially with increasing design size, making it difficult to meet the requirements of large-scale CNN accelerators. Analytical methods improve layout scalability and efficiency through mathematical modeling, but they often ignore the structural information of data paths, leading to routing detours and, in turn, impacting timing performance. Furthermore, the lack of data path information is a major shortcoming of existing techniques. Data paths consist of computational digital signal processing units (DSPs), and their placement determines the criticality of timing paths. However, many existing methods focus solely on the spatial distribution of components and fail to integrate data path information into the layout process, resulting in increased routing complexity and reduced performance. While some research has attempted to introduce data path rules, such as graph clustering-based DSP grouping or empirical rules for DSP allocation, these methods lack generalizability and automation capabilities, making them difficult to adapt to diverse FPGA architectures and CNN designs. Summary of the Invention

[0005] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a DSP layout optimization method for CNN accelerators for FPGAs. Through the optimization algorithm of DSP node classification and minimum cost flow (MCF) model based on graph neural network (GCN), the data path DSP graph is automatically extracted and constructed. Combined with compact layout and cascade constraint optimization, the timing performance and computational efficiency of the layout are greatly improved, while the clock frequency and throughput are significantly increased. It provides a universal and efficient FPGA layout solution for various CNN accelerator architectures.

[0006] The object of the present invention is achieved through the following technical solution: a DSP layout optimization method for a CNN accelerator for FPGA, comprising the following steps:

[0007] S1. DSP path information extraction: Convert the design netlist into a graph representation, identify the data path DSP, and construct the data path DSP graph;

[0008] S2. Datapath-driven DSP placement: Datapath DSP nodes are assigned to specific locations on the FPGA based on the extracted datapath DSP graph.

[0009] The beneficial effects of the present invention are as follows: the present invention automatically extracts and constructs data path DSP graphs through the optimization algorithm of DSP node classification and minimum cost flow (MCF) model based on graph neural network (GCN), combines compact layout and cascade constraint optimization, greatly improves the timing performance and computing efficiency of the layout, and significantly improves the clock frequency and throughput, providing a universal and efficient FPGA layout solution for various CNN accelerator architectures. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a flow chart of the method of the present invention;

[0011] Figure 2 A schematic diagram for legalizing a digital signal unit in a data path;

[0012] Figure 3 Schematic diagram of the incremental data path DSP layout. DETAILED DESCRIPTION

[0013] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0014] The present invention uses the design netlist of the CNN accelerator as input, then uses existing FPGA layout tools to initialize the layout of all components and generate an initial layout solution, where the initial position is the initial solution calculated by the existing layout tool. The layout is then iteratively optimized until the results converge, and the optimized layout result is finally output. In each iteration, the layout of the DSP part of the current layout is first discarded, and the layout of non-DSP components is retained. Then, the data path-driven DSP layout module is used to optimize the positions of DSP and non-DSP components in turn. This process involves the detailed layout of the DSP macro to ensure the rationality of the layout and the performance of the circuit; specifically:

[0015] like Figure 1 As shown, a DSP layout optimization method for CNN accelerator for FPGA is characterized by comprising the following steps:

[0016] S1. DSP path information extraction: Convert the design netlist into a graph representation, identify the data path DSP, and construct the data path DSP graph;

[0017] The step S1 comprises:

[0018] S101. Graph representation and data path DSP node identification: Convert the design netlist into a directed graph and identify the data path DSP nodes and control path DSP node classification, thereby achieving data path DSP node identification;

[0019] The step S101 includes:

[0020] A1. Given a design netlist, the design netlist includes logic components and connection relationships between the logic components;

[0021] A2. Convert the design netlist into a directed graph G = (V, E), where the nodes in the node set V represent the logic components in the design netlist, and the edges in the edge set E represent the connection relationships between the logic components; the logic components include DSP, LUT, and RAM;

[0022] A3. Extracting features of each node from the design netlist, including local features and global features; the local features include in-degree and out-degree; the global features include closeness centrality, betweenness centrality, eccentricity, and the number of feedback loops;

[0023] For any node v in the node set V, the definitions of in-degree, out-degree, betweenness centrality, closeness centrality, eccentricity and number of feedback loops are as follows:

[0024] The in-degree of a node v is: the number of nodes connected to the input of node v;

[0025] The out-degree of a node v is: the number of nodes connected to the output of node v;

[0026] Betweenness centrality measures the importance of a node as an intermediary for the shortest path between other nodes. The betweenness centrality of a node v is defined as follows:

[0027]

[0028] in, Representation node u i and u l The number of shortest paths between represents the number of paths passing through node v among these paths, represents the betweenness centrality of node v;

[0029] Closeness centrality is used to measure the inverse of the average distance between a node and other nodes. The closeness centrality of node v is defined as follows:

[0030]

[0031] Among them, d(v,u) represents the shortest path distance between node v and node u, represents the closeness centrality of node v;

[0032] The eccentricity measures the distance between a node and the farthest node. The eccentricity of a node v is defined as follows:

[0033]

[0034] Among them, d(v,u) represents the shortest path distance between node v and node u, represents the eccentricity of node v;

[0035] The number of feedback loops of node v refers to the number of paths starting from node v and returning to node v;

[0036] A4. Model the DSP identification problem as a binary classification problem and label some DSP nodes in the node set V: DSPs on the data path are labeled as one category, while DSPs on the control path are labeled as another category.

[0037] A5. Construct a graph neural network model, which includes: a graph convolutional neural network (GCN), a ReLU activation function, a dropout layer, a graph convolutional neural network (GCN), a ReLU activation function, a dropout layer, a fully connected layer, and a Softmax classifier connected in sequence;

[0038] A6. Use the labeled node features to train the graph neural network model, and classify the unlabeled DSP nodes in the node set V through the trained mature graph neural network model.

[0039] In an embodiment of the present application, the architecture of the model includes two GCN layers, each followed by a ReLU activation function and a dropout layer to prevent overfitting. During the training process, the model optimizes parameters by minimizing the cross-entropy loss function. Through the trained model, the DSP nodes in the design can be classified and the data path DSP and the control path DSP can be accurately distinguished. The classified data will be used in the subsequent layout optimization process to ensure that the layout of the DSP macro on the FPGA meets both the data flow requirements and the control logic requirements. Such a method not only improves the degree of automation of the layout, but also enhances the performance and efficiency of the final hardware implementation. In order to enhance the classification performance in the embodiment of the present application, the present invention adopts a weighted loss function to give higher penalty weights to nodes in minority categories. In addition, the accuracy of the model is evaluated by a cross-validation method to ensure its applicability on different design benchmarks.

[0040] During the training process, the input of the graph neural network model is the labeled node features, and the expected output is the category of the node label; after training is completed, the node features of the unlabeled nodes are input into the graph neural network model to complete the classification.

[0041] Utilizing graph neural networks (GCNs) for classification, this method effectively categorizes DSPs, guiding the subsequent DSP graph construction process. By modeling this binary classification problem, the present invention can accurately distinguish which DSPs belong to the data path and which belong to the control path when implementing a CNN accelerator on an FPGA. This distinction is crucial for subsequent layout optimization, as it directly impacts the compactness of the data flow. Leveraging the powerful feature extraction capabilities of graph neural networks, the present invention learns a hierarchical representation of nodes and integrates it into an end-to-end layout optimization framework, achieving more efficient and accurate DSP macro layout.

[0042] S102. After completing the DSP classification, only the identified data path DSP nodes are retained, and a data path DSP graph is constructed based on them.

[0043] The step S102 includes:

[0044] Shortest path calculation: Using an iteratively deepened depth-first search algorithm, starting from each data path DSP node, calculate the shortest path to other data path DSP nodes;

[0045] A subgraph containing only data path DSP nodes and their connection relationships is constructed based on the shortest path information. The subgraph describes the topological structure between the data path DSPs.

[0046] Exclusion of Control Path DSP When constructing the data path DSP graph, the present invention further removes control path DSP nodes to avoid their negative impact on layout compactness. Specifically, control paths are usually associated with more storage elements (such as flip-flops, RAM), which helps signal retention and allows greater placement flexibility and less compactness, while data paths are associated with relatively fewer storage elements and have higher compact layout requirements to improve performance. Therefore, only data path DSP nodes related to calculations are retained in the data path DSP graph. Through the implementation of the data path DSP extraction stage, the present invention ensures the integrity and accuracy of data path information in the layout optimization process, thereby providing a guarantee for the ultimate timing performance improvement.

[0047] S2. Datapath-driven DSP placement: Datapath DSP nodes are assigned to specific locations on the FPGA based on the extracted datapath DSP graph.

[0048] Datapath-driven DSP placement is an important step in optimizing placement. It aims to allocate datapath-related DSP nodes to specific locations on the FPGA based on the extracted datapath DSP graph, while satisfying the cascade constraints on the datapath and optimizing the overall placement.

[0049] Datapath DSP placement is a complex allocation problem that requires optimizing the distribution of datapath DSPs while satisfying constraints. To this end, this paper formalizes the problem as an Integer Linear Programming (ILP) problem, improving placement quality by optimizing distance and path characteristics. However, directly solving ILP has high computational complexity. Therefore, linearization techniques are used to transform the ILP problem into a Minimization Cost Flow (MCF) problem, significantly reducing the computational complexity of the optimization problem.

[0050] The step S2 comprises:

[0051] To solve the datapath-driven DSP layout problem, the DSP layout problem separates the control path DSP layout from the datapath DSP layout. The control path DSP layout position is optimized by commercial tools and the algorithm of this invention. The datapath DSP layout problem is formalized as a constrained optimization problem, with the goal of maximizing timing performance while satisfying all design rules and layout constraints. The following is a detailed description of the datapath DSP layout problem modeling:

[0052] S201. Convert the data path DSP node layout problem into a constrained optimization problem and perform data path DSP layout problem modeling:

[0053] Assume that the data path DSP graph contains N data path DSP nodes that need to be laid out. Each data path DSP node Needs to be mapped to one of the M DSP locations in the FPGA;

[0054] The coordinates of the jth DSP position are ;

[0055] Introducing a binary decision variable x i，j Indicates the distribution relationship:

[0056]

[0057] The basic hard constraints that must be met for a given datapath DSP node layout are:

[0058] Single location constraint: Each datapath DSP node must be assigned to exactly one location:

[0059] Position uniqueness constraint: Each position can be assigned at most one datapath DSP node:

[0060] S202. To maintain the compactness and directionality of the data link, the data path DSP nodes can be arranged in a cascaded manner in the layout. The following data path DSP node cascade constraints are introduced:

[0061] Cascaded Datapath DSP nodes on the Datapath DSP diagram must be arranged in adjacent positions in the same column:

[0062] The cascade relationship between multiple data path DSP nodes is decomposed into the cascade relationship between multiple DSP pairs. The cascade relationship between each DSP pair is a cascade pair. Let C be the set of all cascade pairs. Each cascade pair in set C is denoted as (c p ,c s ), c p ,c s Represent the predecessor and successor nodes respectively, the following constraints must be met: p and c s They must be adjacent in layout to ensure continuity and efficiency of signal transmission, which means they need to meet the following constraints:

[0063]

[0064] x cp,j Represents the predecessor node c p The binary decision variable at the jth DSP position, xcs,j+1 Represents the successor node c s The binary decision variable at the j+1th DSP position;

[0065] S203. In the constraints of this part, we also used the previously converted data path DSP diagram. In FPGA, the data path is composed of DSPs, which are directly involved in the calculation and determine the key timing path of the design. Therefore, in the DSP layout, not only the cascade constraints need to be met, but also the characteristics of the data path need to be further optimized to improve the compactness and timing performance of the layout. The cascade constraints ensure that the interconnected DSP nodes in the data path are closely arranged in the physical layout, reducing the delay of signal transmission. The overall layout of the data path DSP aims to reduce the wiring length of the DSP nodes by minimizing the path, and maintain the directionality of the path, reduce detours in signal transmission, and avoid unnecessary path bending. Here we incorporate the cascade constraints and data path constraints into the overall optimization goals, convert them into soft constraints, and unify the modeling. The overall mathematical model combines the directionality of the data path and the compactness of the layout, and is expressed as follows:

[0066] (1)

[0067] in, Represents the edge set between any datapath DSP node and its connected other component nodes in the directed graph after netlist conversion. e for The edge of represents the edge set in the datapath DSP graph, eD for The edge of

[0068] e p and e s Represent the predecessor node and successor node of edge e respectively, and Represent the assignment variables of the predecessor node and the successor node respectively, both of which are vectors of length S; where S represents the number of layout positions of all components on the FPGA; if e p is assigned to the pth position, then The pth element of is 1, and the rest of the elements are 0; if e s is assigned to the qth position, then The pth element of is 1, and the rest of the elements are 0; where p = 1, 2, ..., S; q = 1, 2, ..., S; and p is not equal to q;

[0069] p xA vector consisting of the x-coordinates of the placement positions of all components on the FPGA; p y is a vector of the y coordinates of the placement positions of all components on the FPGA; since the number of placement positions is S, p x 、 p y The length is S; the superscript T indicates transposition, A distance matrix is ​​formed;

[0070] - is a vector With vector The difference vector has one element of 1, one element of -1, and the other elements of 0. The positions of elements 1 and -1 represent the positions of the predecessor and successor nodes.

[0071] Represents an edge e D Predecessor node e Dp The angle between the line connecting to the ARM core in the FPGA and the horizontal line; Represents an edge e D The successor node e Ds The angle between the line connecting to the ARM core in the FPGA and the horizontal line; the ARM core is located in the lower left corner of the FPGA by default; λ is a hyperparameter that controls the trade-off between distance and data path, and η is the penalty factor;

[0072] The hard constraints directly limit the layout variable x i,j The legality of the DSP node to ensure that the resource allocation on the FPGA meets the basic rules:

[0073] Single location constraint: Each datapath DSP node i must and can only be assigned to one location j, that is:

[0074]

[0075] This constraint ensures that each datapath DSP node does not occupy multiple locations repeatedly;

[0076] Resource limitation constraint: Each position j can only accommodate at most one datapath DSP node, expressed as:

[0077]

[0078] This constraint ensures that the limited resources on the FPGA are used reasonably and avoids resource conflicts;

[0079] Variable validity constraints: x i,j is a binary variable indicating whether data path DSP node i is assigned to position j, that is:

[0080]

[0081] S204. The three penalty terms in the objective function together constitute soft constraints, which enable flexible layout adjustments through the weight parameters of the optimization process:

[0082] Distance constraints between datapath DSP nodes and other connected components:

[0083]

[0084] This term represents the sum of the squares of the distances between the DSP node and other component nodes connected to it in the path. The optimization goal is to minimize the data path length to reduce routing delay and routing complexity.

[0085] Data path directionality constraints:

[0086]

[0087] This item is measured by the difference in direction cosine values. The optimization goal is to maintain the directionality of the data path and avoid path detours. The weight parameter λ is used to adjust the intensity of directional optimization.

[0088] Cascading constraints:

[0089]

[0090] Where C is the set of all cascade pairs, x cp,j Represents the predecessor node c p The binary decision variable at the jth DSP position, x cs,j+1 Represents the successor node c s The binary decision variable at the j+1th DSP position;

[0091] This term ensures that the cascaded DSP nodes are closely arranged in physical locations, and the weight parameter η controls the priority of the cascade constraints in the overall optimization;

[0092] S205. Solve the constructed mathematical model of the optimization problem.

[0093] The step S205 includes:

[0094] B1. After completing the linearization, further improve the optimization problem into the minimum cost flow (Min-

[0095] The MCF model is a classic network optimization problem that is well-suited for resource allocation. Its core idea is as follows:

[0096] 1. Network Modeling: The datapath DSP nodes and FPGA resources are modeled as nodes in a directed graph, with the DSP nodes considered "sources" and the available slots in the FPGA considered "sinks." The weight of each edge represents the "cost" of assigning a DSP node to a particular slot. This cost includes path length, directionality deviation, and the degree of cascade violation.

[0097] 2. Capacity Constraints: Modeling hard constraints is achieved through edge capacity constraints. For example, setting the capacity of an edge to 1 means that a location can accommodate at most one datapath DSP node.

[0098] 3. Solution: Leveraging the monotonicity and integer solution properties of the MCF model, the analytical algorithm described above is used to minimize the total allocation cost while satisfying the constraints. Compared to directly solving integer programming problems, the MCF model leverages the characteristics of network optimization algorithms to quickly obtain integer solutions, significantly improving solution efficiency.

[0099] B2. After the data path DSP layout optimization step is completed, because the cascade constraints were placed in the objective function to reduce the complexity of the algorithm, there will be a few conflicts that violate the DSP cascade constraints or exceed resource limits. Figure 2 As shown, a legalization step is added as a post-processing step: ensuring that the final result meets all design rules.

[0100] Perform inter-column legalization and adjust the horizontal position of the data path DSP to ensure that the path DSP nodes in the cascade relationship are located in the same column while minimizing horizontal displacement. This goal is achieved through the hard constraints of integer linear programming. The optimization problem is formalized as the following formula:

[0101]

[0102] t i,k is a binary variable indicating whether path DSP node i is assigned to column k: if yes, it takes 1, if not, it takes 0; D col (i, k) is the horizontal displacement of path DSP node i from the current column to column k, Ncol represents the number of path DSP node columns on the FPGA, M k represents the number of available DSP positions for column k; C represents the set of all cascade pairs, c p and c s It is the front and back cascade nodes, that is, the predecessor node and the successor node; and A binary variable representing the predecessor node and the successor node;

[0103] Objective function (2) minimizes the weighted sum of the horizontal displacements of all path DSP nodes. Constraint (2a) ensures that each path DSP node is assigned to and only to one column; constraint (2b) ensures that the number of path DSP nodes in each column does not exceed the number of available positions; constraint (2c) ensures that path DSP nodes in a cascade relationship must be assigned to the same column. By solving this optimization problem, the horizontal position of the data path DSP can be effectively adjusted.

[0104] In-column legalization:

[0105] After obtaining the solution for inter-column legalization, intra-column legalization further optimizes the vertical position of DSP nodes in each column, ensuring that cascaded DSP nodes are closely arranged while allowing gaps between non-cascaded nodes. The optimization problem is formalized as follows:

[0106]

[0107] r i represents the vertical position of the path DSP node i; R col (i) is the vertical position of path DSP node i in the initial layout, M k is the total number of DSP positions in column k, N k is the number of DSP nodes in column k, C k represents the set of all cascade relationships in column k;

[0108] Objective function (3) minimizes the deviation between the current vertical position of a DSP node and its initial position; constraint (3a) ensures that the DSP nodes in the cascade relationship are tightly arranged; constraint (3b) allows gaps between non-cascaded nodes to avoid overlap. By solving this optimization problem, the impact of vertical displacement on layout compactness is minimized while maintaining the cascade constraints.

[0109] like Figure 3 As shown, in order to further improve the compactness of the layout and satisfy the cascade constraints, the present invention adopts an incremental iterative optimization method: first, the initial positions of all logic components in the design netlist are given (which can be generated using existing FPGA layout tools, such as Vivado or AMF2.0); after completing the data path DSP identification and data path DSP graph construction according to step S1, the positions of all components except the path DSP nodes are fixed, and according to step S2, only the data path DSP layout is optimized to generate preliminary results;

[0110] Then, fix the position of the datapath DSP node. Then, use commercial (or open source) layout tools (such as Vivado or AMF2.0, which use a rule to minimize the distance between connected components to shorten the wire length and speed up data communication) to adjust the positions of all components except the datapath DSP node.

[0111] In subsequent iterations, the positions of fixed datapath DSP nodes and other components are alternated to iteratively optimize the overall layout;

[0112] Convergence judgment: When the change of the objective function of formula (1) is lower than the preset threshold, the iteration is stopped and the final result is output.

[0113] The DSP layout optimization framework of the present invention was evaluated in multiple experimental environments. The experimental settings included target devices, benchmark designs, GCN training and layout configurations, and the optimization effect of the framework was verified through comparative analysis. This experiment selected Xilinx Zynq UltraScale+ MPSoC ZCU104 FPGA as the target device. The device has column-distributed heterogeneous resources (such as LUTs, DSPs, and RAM, etc.), which can fully verify the effectiveness of the present invention in complex designs. The benchmark design selected multiple test cases from the DAC system design competition, including iSmart DNN, SkyNet, and the expanded SkrSkr series design. The number of DSPs ranged from 197 to 1431, covering CNN accelerators of different sizes. The experimental environment runs on a Linux workstation equipped with a 10-core CPU, and is combined with tools such as Vivado 2020.2 and RapidWright for synthesis and layout routing.

[0114] During the data path DSP extraction phase, this paper utilizes a DSP node classification method based on a graph neural network (GCN). By extracting features such as closeness centrality, betweenness centrality, in-degree, out-degree, and average shortest path length from the design netlist, it comprehensively reflects the global and local properties of DSP nodes. The GCN model employs a two-layer convolutional structure, combined with three fully connected layers and a softmax classifier for classification, and uses a weighted loss function to address class imbalance. Cross-validation results demonstrate that the GCN achieves an average classification accuracy of 96%, significantly outperforming the 81% achieved by traditional support vector machines (SVMs), fully demonstrating the accuracy of data path information extraction.

[0115] During the placement optimization phase, an initial layout was generated using Vivado, with the clock frequency gradually increased to test the tool's performance limits. Subsequently, the proposed DSP placement optimization framework was applied, using a minimum cost flow (MCF) model for optimization. The maximum number of iterations was set to 50, and the weight parameter λ for datapath constraints was set to 100. After placement, the layout was further adjusted through inter-column and intra-column legalization to strictly satisfy cascade constraints and minimize displacement deviation. Optimization results demonstrate that the proposed framework demonstrates excellent performance on multiple benchmark designs. Compared to Vivado, the proposed framework improves the worst negative slack (WNS) by an average of 32%; compared to AMF-Placer, the improvement reaches 65%. Furthermore, while the compact placement may slightly increase routing length, the impact is minimal, and the placement remains efficient and controllable. DSPlacer reduces runtime by approximately 50% compared to AMF-Placer, demonstrating exceptional efficiency.

[0116] Analysis of the layout results reveals two key strengths of this method. First, the compact datapath-driven layout effectively reduces routing detours. Second, strict adherence to cascade constraints improves the timing consistency of the layout. Layout visualization further validates these advantages. Compared to Vivado and AMF-Placer, the layout generated by DSPlacer is more regular and compact in the datapath, significantly reducing the number of cross-column connections.

[0117] In summary, the DSP layout optimization framework of the present invention outperforms existing tools in terms of timing performance, layout compactness, and operational efficiency by integrating data path information and efficient optimization algorithms, providing a general and efficient solution for the design of CNN accelerators on FPGAs.

[0118] The foregoing description is a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Instead, the present invention can be used in other combinations, modifications, and environments and can be modified within the scope of the concept described herein through the above teachings or techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention are intended to be protected by the appended claims.

Claims

1. A DSP layout optimization method for CNN accelerators for FPGA, characterized by: The following steps are involved: S1. DSP path information extraction: Convert the design netlist into a graph representation, identify datapath DSP nodes, and construct the datapath DSP graph. S2. Datapath-driven DSP placement: Based on the extracted datapath DSP graph, datapath DSP nodes are assigned to specific locations on the FPGA. The step S2 comprises: S201. Convert the data path DSP node layout problem into a constrained optimization problem and perform data path DSP layout problem modeling: Assume that the data path DSP graph contains N data path DSP nodes that need to be laid out. Each data path DSP node Needs to be mapped to one of the M DSP locations in the FPGA; The coordinates of the jth DSP position are ; Introducing a binary decision variable x i，j Indicates the distribution relationship: The basic hard constraints that must be met for a given datapath DSP node layout are: Single location constraint: Each datapath DSP node must be assigned to exactly one location: Position uniqueness constraint: Each position can be assigned at most one datapath DSP node: S202. To maintain the compactness and directionality of the data link, the data path DSP nodes can be arranged in a cascaded manner in the layout. The following data path DSP node cascade constraints are introduced: Cascaded Datapath DSP nodes on the Datapath DSP diagram must be arranged in adjacent positions in the same column: The cascade relationship between multiple data path DSP nodes is decomposed into the cascade relationship between multiple DSP pairs. The cascade relationship between each DSP pair is a cascade pair. Let C be the set of all cascade pairs. Each cascade pair in set C is denoted as (c p ,c s ), c p ,c s Represent the predecessor and successor nodes respectively, the following constraints must be met: p and c s They must be adjacent in layout to ensure continuity and efficiency of signal transmission, which means they need to meet the following constraints: x cp,j Represents the predecessor node c p The binary decision variable at the jth DSP position, x cs,j+1 Represents the successor node c s The binary decision variable at the j+1th DSP position; S203. In DSP layout, not only must cascade constraints be met, but data path characteristics must also be further optimized to improve layout compactness and timing performance. Cascade constraints and data path constraints are incorporated into the overall optimization objectives and converted into soft constraints, leading to unified modeling. The overall mathematical model combines the directionality of the data path and the compactness of the layout, and is expressed as follows: (1) in, Represents the edge set between any datapath DSP node and its connected other component nodes in the directed graph after netlist conversion. e for The edge of represents the edge set in the datapath DSP graph, e D for The edge of e p and e s Represent the predecessor node and successor node of edge e respectively, and Represent the assignment variables of the predecessor node and the successor node respectively, both of which are vectors of length S; where S represents the number of layout positions of all components on the FPGA; if e p is assigned to the pth position, then The pth element of is 1, and the rest of the elements are 0; if e s is assigned to the qth position, then The pth element of is 1, and the rest of the elements are 0; where p = 1, 2, ..., S; q = 1, 2, ..., S; and p is not equal to q; p x A vector consisting of the x-coordinates of the placement positions of all components on the FPGA; p y is a vector of the y coordinates of the placement positions of all components on the FPGA; since the number of placement positions is S, p x 、 p y The length is S; the superscript T indicates transposition, A distance matrix is ​​formed; - is a vector With vector The difference vector has one element of 1, one element of -1, and the other elements of 0. The positions of elements 1 and -1 represent the positions of the predecessor and successor nodes. Represents an edge e D Predecessor node e Dp The angle between the line connecting to the ARM core in the FPGA and the horizontal line; Represents an edge e D The successor node e Ds The angle between the line connecting to the ARM core in the FPGA and the horizontal line; the ARM core is located in the lower left corner of the FPGA by default; λ is a hyperparameter, and η is the penalty factor; The hard constraints directly limit the layout variable x i,j The legality of the DSP node to ensure that the resource allocation on the FPGA meets the basic rules: Single location constraint: Each datapath DSP node i must and can only be assigned to one location j, that is: This constraint ensures that each datapath DSP node does not occupy multiple locations repeatedly; Resource limitation constraint: Each position j can only accommodate at most one datapath DSP node, expressed as: This constraint ensures that the limited resources on the FPGA are used reasonably and avoids resource conflicts; Variable validity constraints: x i,j is a binary variable indicating whether data path DSP node i is assigned to position j, that is: S204. The three penalty terms in the objective function together constitute soft constraints, which enable flexible layout adjustments through the weight parameters of the optimization process: Distance constraints between datapath DSP nodes and other connected components: This term represents the sum of the squares of the distances between the DSP node and other component nodes connected to it in the path. The optimization goal is to minimize the data path length to reduce routing delay and routing complexity. Data path directionality constraints: This term is measured by the difference in direction cosine values. The optimization goal is to maintain the directionality of the data path and avoid path detours. λ is used to adjust the intensity of directional optimization. Cascading constraints: Where C is the set of all cascade pairs, x cp,j Represents the predecessor node c p The binary decision variable at the jth DSP position, x cs,j+1 Represents the successor node c s The binary decision variable at the j+1th DSP position; This term ensures that the cascaded DSP nodes are closely arranged in physical locations, and the weight parameter η controls the priority of the cascade constraints in the overall optimization; S205. Solve the constructed mathematical model of the optimization problem.

2. The DSP layout optimization method for a CNN accelerator for FPGA according to claim 1, characterized in that: The step S1 comprises: S101. Graph Representation and Datapath DSP Node Identification: Convert the design netlist into a directed graph and classify the datapath DSP nodes and control path DSP nodes to identify the datapath DSP nodes. S102. After completing the DSP classification, only the identified data path DSP nodes are retained, and a data path DSP graph is constructed based on them.

3. The DSP layout optimization method for a CNN accelerator for FPGA according to claim 2, characterized in that: The step S101 includes: A1. Given a design netlist, the design netlist includes logic components and connection relationships between the logic components; A2. Convert the design netlist into a directed graph G = (V, E). The nodes in the node set V in the directed graph represent the logical components in the design netlist, and the edges in the edge set E represent the connection relationships between the logical components. A3. Extracting features of each node from the design netlist, including local features and global features; the local features include in-degree and out-degree; the global features include closeness centrality, betweenness centrality, eccentricity, and the number of feedback loops; For any node v in the node set V, the definitions of in-degree, out-degree, betweenness centrality, closeness centrality, eccentricity and number of feedback loops are as follows: The in-degree of a node v is: the number of nodes connected to the input of node v; The out-degree of a node v is: the number of nodes connected to the output of node v; Betweenness centrality measures the importance of a node as an intermediary for the shortest path between other nodes. The betweenness centrality of a node v is defined as follows: in, Representation node u i and u l The number of shortest paths between Representation node u i and u l The number of paths passing through node v in the shortest path between represents the betweenness centrality of node v; Closeness centrality is used to measure the inverse of the average distance between a node and other nodes. The closeness centrality of node v is defined as follows: Among them, d(v,u) represents the shortest path distance between node v and node u, represents the closeness centrality of node v; The eccentricity measures the distance between a node and the farthest node. The eccentricity of a node v is defined as follows: Among them, d(v,u) represents the shortest path distance between node v and node u, represents the eccentricity of node v; The number of feedback loops of node v refers to the number of paths starting from node v and returning to node v; A4. Model the DSP identification problem as a binary classification problem and label some DSP nodes in the node set V: data path DSP nodes are labeled as one category, while control path DSP nodes are labeled as another category. A5. Construct a graph neural network model, which includes: a graph convolutional neural network (GCN), a ReLU activation function, a dropout layer, a graph convolutional neural network (GCN), a ReLU activation function, a dropout layer, a fully connected layer, and a Softmax classifier connected in sequence; A6. Use the marked node features to train the graph neural network model, and classify the unlabeled DSP nodes in the node set V through the trained mature graph neural network model, thereby realizing data path DSP node identification.

4. The DSP layout optimization method for a CNN accelerator for FPGA according to claim 3, characterized in that: The logic components include DSP, LUT and RAM.

5. The DSP layout optimization method for a CNN accelerator for FPGA according to claim 3, characterized in that: The step S102 includes: Shortest path calculation: Using an iteratively deepened depth-first search algorithm, starting from each data path DSP node, calculate the shortest path to other data path DSP nodes; Based on the shortest path information, a subgraph containing only data path DSP nodes and their connection relationships is constructed. This subgraph describes the topological structure between data path DSPs and is recorded as a data path DSP graph. The connection relationships between data path DSP nodes are the edges of the data path DSP graph.

6. The DSP layout optimization method for a CNN accelerator for FPGA according to claim 1, characterized in that: The step S205 includes: B1. The objective function and constraints of the optimization problem contain nonlinear terms. The nonlinear problem is transformed into a linear problem through linearization technology. After the linearization is completed, the optimization problem is further improved into a minimum cost flow model for solution: Network modeling: Datapath DSP nodes and FPGA resources are modeled as nodes in a directed graph, with FPGA resources represented as available locations in the FPGA. Path DSP nodes are considered sources, and available locations in the FPGA are considered sinks. The weight of each edge represents the cost required to assign the path DSP node to the corresponding location. This cost includes path length, directionality deviation, and the degree of cascade violation. Capacity Constraints: Modeling hard constraints through edge capacity constraints, minimizing the total allocation cost while satisfying the constraints; The layout location of the data path DSP nodes is obtained by solving the minimum cost flow model; B2. After the datapath DSP layout optimization step is completed, because the cascade constraints were previously placed in the objective function and calculated as soft constraints to reduce algorithm complexity, there will be a few violations of the DSP cascade constraints or exceeding resource limits because the constraints have been converted to soft constraints. A legalization step is added as post-processing: Perform inter-column legalization and adjust the horizontal position of the data path DSP to ensure that the path DSP nodes in the cascade relationship are located in the same column while reducing horizontal displacement. This goal is achieved through the hard constraints of integer linear programming. The optimization problem is formalized as the following formula: t i,k is a binary variable indicating whether path DSP node i is assigned to column k: if yes, it takes 1, if not, it takes 0; D col (i, k) is the horizontal displacement of path DSP node i from the current column to column k, Ncol represents the number of path DSP node columns on the FPGA, M k represents the number of available DSP positions for column k; C represents the set of all cascade pairs, c p and c s It is the front and back cascade nodes, that is, the predecessor node and the successor node; and A binary variable representing the predecessor node and the successor node; Objective function (2) minimizes the weighted sum of the horizontal displacements of all path DSP nodes. Constraint (2a) ensures that each path DSP node is assigned to and only to one column; constraint (2b) ensures that the number of path DSP nodes in each column does not exceed the number of available positions; constraint (2c) ensures that path DSP nodes in a cascade relationship must be assigned to the same column. By solving this optimization problem, the horizontal position of the data path DSP can be effectively adjusted. In-column legalization: After obtaining the solution for inter-column legalization, intra-column legalization further optimizes the vertical position of DSP nodes in each column, ensuring that cascaded DSP nodes are closely arranged while allowing gaps between non-cascaded nodes. The optimization problem is formalized as follows: r i represents the vertical position of path DSP node i; Rcol ( i ) is the vertical position of path DSP node i in the initial layout, M k is the total number of DSP positions in column k, N k is the number of DSP nodes in column k, C k represents the set of all cascade relationships in column k; Objective function (3) minimizes the deviation between the current vertical position of a DSP node and its initial position; constraint (3a) ensures that the DSP nodes in the cascade relationship are tightly arranged; constraint (3b) allows gaps between non-cascaded nodes to avoid overlap. By solving this optimization problem, the impact of vertical displacement on layout compactness is minimized while maintaining the cascade constraints.

7. The DSP layout optimization method for a CNN accelerator for FPGA according to claim 1, characterized in that: The method further comprises: First, the initial positions of all logic components in the design netlist are given; after completing the datapath DSP identification and datapath DSP graph construction according to step S1, the positions of all components except the path DSP nodes are fixed, and according to step S2, only the datapath DSP layout is optimized to generate preliminary results; Then, the position of the datapath DSP node is fixed, and the positions of other components except the path DSP node are adjusted using a commercial layout tool; In subsequent iterations, the positions of the fixed datapath DSP nodes and other components are alternated to iteratively optimize the overall layout; Convergence judgment: When the change of the objective function of formula (1) is lower than the preset threshold, the iteration is stopped and the final result is output.

Images