Fpga layout method based on reinforcement learning

Abstract

The application belongs to the field of integrated circuits, and particularly relates to a FPGA layout method based on reinforcement learning. First, according to an input netlist file, logical units contained in a FPGA design circuit are extracted, and then initialization layout operation of the logical units is completed; in view of the slow convergence problem of a traditional simulated annealing method layout, multiple search region construction methods are proposed, which can effectively improve the search efficiency of the layout solution space; on this basis, an optimal search region selection method based on reinforcement learning is proposed, which can adaptively select the optimal search region to perform the exchange operation of the logical units. The layout method can greatly reduce the time required for FPGA layout while maintaining the required wire length and critical path delay.

Description

Technical Field

[0001] This invention belongs to the field of integrated circuits, and in particular relates to an FPGA placement method based on reinforcement learning. Background Technology

[0002] With the increasing complexity and design scale of large-scale integrated circuit devices, Field Programmable Gate Arrays (FPGAs), as a semi-custom circuit, have gained widespread attention and application in fields such as the Internet of Things, high-performance computing, cloud computing, and artificial intelligence due to their advantages such as low design cost, high speed, and abundant heterogeneous logic resources. To enable the design of more complex circuit systems, the integration density of FPGAs is constantly increasing, but this also places a heavier burden on FPGA electronic design automation (EDA) tools. Circuits designed using hardware description languages ​​are first compiled into binary code stream files by FPGA EDA tools. These binary code stream files then configure the programmable logic and interconnects within the FPGA chip, ultimately realizing the required circuit functions on the FPGA. However, as the scale of FPGAs continues to increase, the time required for FPGA EDA tools to compile hardware description languages ​​into binary code stream files is also increasing, severely restricting the healthy and sustainable development of FPGA chips. Placement, as a core and time-consuming step in FPGA EDA tools, makes the optimization of the placement process crucial. Summary of the Invention

[0003] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide an FPGA placement method based on reinforcement learning, which can select the optimal search region in each placement iteration and finally complete the FPGA placement process, effectively improving the exploration efficiency of the placement solution space.

[0004] The solution of this invention is: an FPGA placement method based on reinforcement learning, which includes the following steps:

[0005] S1. Perform layout initialization operations on the logic units used in the FPGA design;

[0006] S2. Determine whether the current FPGA layout state meets the exit condition. If the exit condition is met, exit the layout process; otherwise, initialize the iteration count and proceed to step S3.

[0007] S3. Employ reinforcement learning to select search regions for layout optimization within the FPGA;

[0008] S4. Within the selected search area, select a logical unit as the logical unit to be disturbed, and estimate the change in the layout cost function caused by the position swapping operation of the logical unit to be disturbed.

[0009] S5. Determine whether the change ΔC of the layout cost function before and after the exchange of the logic unit to be disturbed is less than 0. If it is less than 0, then perform the position exchange operation of the logic unit to be disturbed; otherwise, perform the position exchange operation of the logic unit to be disturbed with a certain probability, and then proceed to step S6.

[0010] S6. Determine if the number of iterations is greater than the preset threshold. If it is, return to step S2 and re-execute steps S2 to S6; otherwise, re-execute steps S3 to S6.

[0011] Preferably, step S1 specifically comprises:

[0012] Step S101: Initialize the FPGA netlist file and extract the set of FPGA logic units contained in the FPGA design based on the netlist file;

[0013] Step S102: For each logic unit in the FPGA logic unit set, randomly select a physical location inside the FPGA chip that is allowed by the logic unit type, and place the logic unit there.

[0014] Preferably, the layout exit condition in step S2 is:

[0015] T t <0.005×Cost / N nets

[0016] Where Cost is the value of the cost function at the current moment, and N nets This indicates the number of nets in the FPGA netlist file.

[0017] Preferably, the reinforcement learning process in step S3 is as follows:

[0018] S301. Using the FPGA layout as an intelligent agent, determine the current layout state based on the temperature value of the FPGA layout at the current moment. The layout state includes a first layout state s1 and a second layout state s2.

[0019] The switching condition between the first layout state s1 and the second layout state s2 is:

[0020]

[0021] Among them, T t This represents the temperature value at time t, which can be calculated based on the current layout. T t-1The temperature value is the previous time step, and β∈(0,1) is the scaling factor.

[0022] S302. Based on the current layout state, determine the action set of the agent and calculate the probability of selecting each action in the action set;

[0023] When the agent is in state s1, the action set contains three actions: median search region construction method, zero-forcing search region construction method, and random search region construction method.

[0024] When the agent is in state s2, the action set contains 6 actions, namely the median search region construction method, the zero-forcing search region construction method, the weighted median search region construction method, the weighted zero-forcing search region construction method, the critical time-series search region construction method, and the random search region construction method.

[0025] S303. Based on the calculated probabilities of each action, select the corresponding action-determined region as the search region for layout optimization.

[0026] Preferably, the probability of choosing action a in state u at current time t is... With action value function Q(a t,u They are positively correlated, and the specific formula is:

[0027]

[0028] Where g represents the number of actions in the action set in the current state, and q is the action number in state u at the current time t.

[0029] Preferably, the update method for the action value function is as follows:

[0030] Q(a t+1,u )=Q(a t,u )+θ(R t+1 -Q(a t,u ))

[0031] Where θ represents the scaling factor, which is a constant greater than 0, R t+1 This represents the reward function.

[0032] Preferably, the value of the reward function is defined as follows:

[0033]

[0034] Preferably, the median search region is constructed as follows:

[0035] S1-1, Obtain all logic units lb to be disturbed. i Net is a collection of interconnected wires. iWhere 1≤i≤τ, and τ represents the number of logic units contained in the FPGA netlist file;

[0036] S1-2, For the net set Net i The inner net n j (j∈[1,|Net i |]), obtain the net n j bounding box ((x) j,min ,y j,min ),(x j,max ,y j,max )), where |Net i | represents the set of nets. i The number of internal networks, (x j,min ,y j,min (x) represents the coordinates of the bottom left vertex of the netframe in the FPGA chip layout coordinate system. j,max ,y j,max ) represents the coordinates of the upper right corner vertex of the wireframe in the FPGA chip layout coordinate system; the origin of the FPGA chip layout coordinate system is the lower left corner of the FPGA chip, and the X-axis and Y-axis are the two mutually perpendicular sides of the FPGA chip.

[0037] S1-3, Define the horizontal quantity x vec Used to store the variable of the net bounding box in the x-direction of the FPGA chip layout coordinate system, its initial value is empty; define the vertical quantity y. vec Used to store variables in the y-direction of the wireframe boundary, with an initial value of empty;

[0038] S1-4, For the net set i All the wires n j (j∈[1,|Net i |]), will y j,min, y j,max Insert into horizontal quantity x vec In the middle, y j,min ,y j,max Insert into vertical quantity y vec In the middle, and the horizontal quantity x vec With longitudinal quantity y vec The elements in the text are arranged in ascending order;

[0039] S1-5, Take the horizontal quantity x vec The first in The value of each element is denoted as the first horizontal coordinate x1, and the horizontal dimension x is taken as... vec The first in The value of each element is denoted as the second horizontal coordinate x2, and the vector y is taken as... vec The first in The value of each element is denoted as the first vertical coordinate y1, and the vector y is taken as... vec The first in The value of each element is denoted as the second vertical coordinate y2, where Represents |x vec | / 2 rounded down Represents |y vec | / 2 is rounded down.

[0040] S1-6. Define the region corresponding to x1≤x≤x2, y1≤y≤y2 as the median search region.

[0041] Preferably, the weighted median search region is constructed as follows:

[0042] After step S1-3 and before step S1-4, perform the following steps: For the net set Net i Internal wire network The coordinates of the bottom left vertex of the netbox in the x-direction of the FPGA chip layout coordinate system are x... j,min The coordinates of the bottom left vertex of the netbox in the y-direction of the FPGA chip layout coordinate system are y=0. j,min The coordinates of the top right corner vertex of the bounding box in the x-direction of the FPGA chip layout coordinate system are x. j,max The coordinates of the top right corner vertex of the netframe in the y-direction of the FPGA chip layout coordinate system are y=0. j,max Multiply by the corresponding weight coefficients to update the wire mesh bounding box vector ((x j,min ,y j,min ),(x j,max ,y j,max The weight coefficients are integer multiples of the minimum criticality of the nets connected to the nodes on the left and bottom borders of the net boundary, and integer multiples of the maximum criticality of the nets connected to the nodes on the right and top borders.

[0043] Preferably, the zero-forcing search region is constructed as follows:

[0044] S2-1. Calculate the zero-forcing position:

[0045]

[0046]

[0047] in,( xz_p y z_p ) represents the coordinates of the zero-forcing position in the FPGA chip layout coordinate system, output(lb) i ) represents all output logic units lb i The set of wire meshes, (x s(l) y s(l)The input(lb) represents the coordinates of the source node of net l in the FPGA chip layout coordinate system. i ) indicates that all source nodes are the logical unit lb. i The set of nets, dest(r) represents the set of all leaky nodes in net r, (x m , y m ) represents the coordinates of the drain node m in the FPGA chip layout coordinate system, and Count() represents the counting operation.

[0048] S2-2. Based on the zero-forcing position, the zero-forcing search region is a square region with a side length of dis, centered on the zero-forcing search position and extending outwards. Dis must satisfy the condition 0 < dis < dis max , among which, dis max =min(X) FPGA ,Y FPGA ), where X FPGA With Y FPGA These represent the length values ​​of the FPGA chip in the X and Y directions of the FPGA chip layout coordinate system, respectively.

[0049] Preferably, the weighted zero-forcing search region is constructed as follows:

[0050] S3-1. Calculate the weighted zero-forcing position:

[0051]

[0052]

[0053] Where, x wz_p Let x and y be the weighted zero-forcing position in the FPGA chip layout coordinate system. wz_p The weighted zero-forcing position is the AND coordinate in the FPGA chip layout coordinate system, where k is the criticality of the net connection; output(lb) i ) represents all output logic units lb i The set of wire meshes, (x s(l) ,y s(l) ) represents the coordinates of the source node of the net l in the FPGA chip layout coordinate system, input(lb) i ) indicates that all source nodes are the logical unit lb. i The set of nets, dest(r) represents the set of all leaky nodes in net r, (x m , y m ) represents the coordinates of the drain node m in the FPGA chip layout coordinate system, and Count() represents the counting operation;

[0054] S3-2. Based on the weighted zero-forcing position, the zero-forcing search region is a square region with a side length of dis, centered on the weighted zero-forcing position and extending outwards. Dis must satisfy the condition 0 < dis < dis max , among which, dis max =min(X) FPGA ,Y FPGA ), where X FPGA With Y FPGA These represent the length values ​​of the FPGA chip in the X and Y directions of the FPGA chip layout coordinate system, respectively.

[0055] Preferably, the key time-series search region is constructed as follows:

[0056] S4-1. Define the threshold α for the criticality of wire network connection, and α satisfies the condition α∈[0,1];

[0057] S4-2. Select the set of all net connections with a criticality greater than α, and then select a logic unit lb to be disturbed from the selected net connection set. i ;

[0058] S4-3, For the selected logic unit lb to be disturbed i Identify the logic unit lb to be disturbed. i Coordinates (x(lb) in the FPGA chip layout coordinate system) i ),y(lb i ));

[0059] S4-4, will be combined with the logic unit to be disturbed lb i The criticality of the connected wires is sorted to identify the logic unit lb to be disturbed. i The κ input nodes with high criticality that are connected together form the set of critical input nodes {h1, h2, ..., hk}. κ}; Obtain the logic unit lb to be disturbed i The coordinates (x(v), y(v)) of the output node v with the highest criticality in the FPGA chip layout coordinate system;

[0060] S4-4. Based on the key time series search principle, determine the key time series search area ((xl(lb)). i ),yb(lb i )),(xr(lb i ),yt(lb i ))), where (xl(lb) i ),yb(lb i (xr(lb) represents the coordinates of the lower left corner vertex of the critical timing search region in the FPGA chip layout coordinate system. i),yt(lb i )) represents the coordinates of the top right vertex of the critical timing search region in the FPGA chip layout coordinate system;

[0061] The key time series search principles are as follows:

[0062] lb i Key input node h g The minimum and maximum values ​​of the x and y coordinates of (g∈[1,κ]) are denoted as min_x(lb) and min_y(lb) respectively. i ), max_x(lb) i ), min_y(lb i ) and max_y(lb i );

[0063] 1) When x(v) <min_x(lb i When xl(lb) i )=min(x(lb i ),x(v)),xr(lb) i ) = max(min_x(lb) i ),x(lb i ));

[0064] 2) When x(v)∈[min_x(lb) i ),max_xx(lb i )],xl(lb i )=min(x(lb i ),x(v)),xr(lb) i )=max(x(lb i ),x(v));

[0065] 3) When x(v) > max_x(lb) i When xl(lb) i )=min(x(lb i ),max_x(lb i )), xr(lb i )=max(x(lb i ),x(v)).

[0066] Preferably, the change in the layout cost function caused by the operation of swapping the positions of the logic units to be disturbed is calculated using the following formula:

[0067]

[0068] in,

[0069] ΔTiming_Cost = Timing_Cost - Previous_Timing_Cost represents the change in the delay cost function, ΔWriting_Cost = Writing_Cost - Previous_Writing_Cost represents the change in the line length cost function, λ∈[0,1] is a weight constant; Previous_Timing_Cost and Previous_Wiring_Cost are the delay cost function value and line length cost function value before each logic unit exchange, respectively, and Timing_Cost and Wiring_Cost are the delay cost function value and line length cost function value after each logic unit exchange.

[0070] Preferably, in step S5, there is a certain probability that... Where ΔC is the change in the layout cost function before and after the logic unit to be disturbed is swapped, and T t This represents the annealing temperature value at time t.

[0071] Preferably, the preset threshold value in step S6 is as follows:

[0072] move_lim=inner_num×τ 4 / 3

[0073] Here, inner_num is a constant with a value between 1 and 10.

[0074] The advantages of this invention compared to the prior art are:

[0075] (1) This invention proposes a variety of search region construction methods, which can effectively search the layout solution space, avoid many unnecessary exploration processes, and can very efficiently reduce the time required in the FPGA layout process.

[0076] (2) This invention proposes a search region selection strategy based on reinforcement learning, which can automatically select the optimal search region to complete the layout process without human intervention, thereby improving the intelligence level of the layout method.

[0077] (3) Compared with the traditional simulated annealing layout method, the layout method proposed in this invention can significantly shorten the layout time while maintaining the line length and critical path delay. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is an execution flowchart of an FPGA placement method based on reinforcement learning in this invention;

[0080] Figure 2 This is a schematic diagram of the weighted median search region construction method in this invention;

[0081] Figure 3 This is a schematic diagram of the key temporal search region construction method in this invention;

[0082] Figure 4 This is a schematic diagram of the curve showing how the line length changes with the system running time in this invention;

[0083] Figure 5 This is a schematic diagram of the curve showing the change of critical path delay as a function of system runtime in this invention. Detailed Implementation

[0084] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] The concept of this invention is as follows: Figure 1 As shown:

[0086] S1. Perform layout initialization operations on the logic units used in the FPGA design;

[0087] S2. Determine whether the current FPGA layout state meets the exit condition. If the exit condition is met, exit the layout process; otherwise, initialize the iteration count and proceed to step S3.

[0088] S3. Employ reinforcement learning to select search regions for layout optimization within the FPGA;

[0089] S4. Within the selected search area, select a logical unit as the logical unit to be disturbed, and estimate the change in the layout cost function caused by the position swapping operation of the logical unit to be disturbed.

[0090] S5. Determine whether the change ΔC of the layout cost function before and after the exchange of the logic unit to be disturbed is less than 0. If it is less than 0, then perform the position exchange operation of the logic unit to be disturbed; otherwise, perform the position exchange operation of the logic unit to be disturbed with a certain probability, and then proceed to step S6.

[0091] S6. Determine if the number of iterations is greater than the preset threshold. If it is, return to step S2 and re-execute steps S2 to S6; otherwise, re-execute steps S3 to S6.

[0092] Step S1 is performed as follows:

[0093] Step S101: Initialize the FPGA netlist file and extract the set of FPGA logic units contained in the FPGA design based on the netlist file;

[0094] Step S102: For each logic unit in the FPGA logic unit set, randomly select a physical location inside the FPGA chip that allows the placement of the logic unit, and place the logic unit there.

[0095] Furthermore, the layout exit condition in step S2 is:

[0096] T t <0.005×Cost / N nets

[0097] Where Cost is the value of the cost function at the current moment, and N nets This indicates the number of nets in the FPGA netlist file.

[0098] Furthermore, in step S3, the search area includes five types: median search area, weighted median search area, zero-forcing search area, weighted zero-forcing search area, and key time-series search area. The median search area and zero-forcing search area mainly focus on line length indicators in the layout optimization process, while the weighted median search area, weighted zero-forcing search area, and key time-series search area focus on both line length and time-series indicators in the layout optimization process.

[0099] Furthermore, the median search region is constructed as follows:

[0100] S1-1, Obtain all logic units lb to be disturbed. i Net is a collection of interconnected wires. i Where 1≤i≤τ, and τ represents the number of logic units contained in the FPGA netlist file;

[0101] S1-2, For the net set Net i The inner net n j(j∈[1,|Net i |]), obtain the net n j bounding box ((x) j,min ,y j,min ),(x j,max ,y j,max )), where |Net i | represents the set of nets. i The number of internal networks, (x j,min ,y j,min (x) represents the coordinates of the bottom left vertex of the netframe in the FPGA chip layout coordinate system. j,max ,y j,max ) represents the coordinates of the upper right corner vertex of the wireframe in the FPGA chip layout coordinate system; the origin of the FPGA chip layout coordinate system is the lower left corner of the FPGA chip, and the X-axis and Y-axis are the two mutually perpendicular sides of the FPGA chip.

[0102] S1-3, Define the horizontal quantity x vec Used to store the variable of the net bounding box in the x-direction of the FPGA chip layout coordinate system, its initial value is empty; define the vertical quantity y. vec Used to store variables in the y-direction of the wireframe boundary, with an initial value of empty;

[0103] S1-4, For the net set i All the wires n j (j∈[1,|Net i |]), will y j,min, y j,max Insert into horizontal quantity x vec In the middle, y j,min ,y j,max Insert into vertical quantity y vec In the middle, and the horizontal quantity x vec With longitudinal quantity y vec The elements in the text are arranged in ascending order;

[0104] S1-5, Take the horizontal quantity x vec The first in The value of each element is denoted as the first horizontal coordinate x1, and the horizontal dimension x is taken as... vec The first in The value of each element is denoted as the second horizontal coordinate x2, and the vector y is taken as... vec The first in The value of each element is denoted as the first vertical coordinate y1, and the vector y is taken as... vec The first in The value of each element is denoted as the second vertical coordinate y2, where Represents |x vec | / 2 rounded down Represents |y vec | / 2 is rounded down.

[0105] S1-6. Define the region corresponding to x1≤x≤x2, y1≤y≤y2 as the median search region.

[0106] Furthermore, the construction steps of the weighted median search region are largely the same as those of the median search region, except that the median search region is constructed by inserting the wireframe bounding box into the vector x. vec With y vec In the middle of the hour, x j,min ,x j,max ,y j,min With y j,max The weight values ​​are all 1, but when constructing the weighted median search region, when x j,min ,x j,max ,y j,min With y j,max Insert into vector x vec With y vec Before proceeding, a corresponding weighting coefficient needs to be added, which is related to the criticality of the wire connection.

[0107] The weighted median search region is constructed as follows:

[0108] After step S1-3 and before step S1-4, perform the following steps: For the net set Net i The inner net n j (j∈[1,|Net i |]), the coordinates of the lower left corner vertex of the net bounding box in the x-direction of the FPGA chip layout coordinate system are x j,min The coordinates of the bottom left vertex of the netbox in the y-direction of the FPGA chip layout coordinate system are y=0. j,min The coordinates of the top right corner vertex of the bounding box in the x-direction of the FPGA chip layout coordinate system are x. j,max The coordinates of the top right corner vertex of the netframe in the y-direction of the FPGA chip layout coordinate system are y=0. j,max Multiply by the corresponding weight coefficients to update the wire mesh bounding box vector ((x j,min ,y j,min ),(x j,max ,y j,max The weighting coefficients are integer multiples of the minimum criticality of the nets connected to the nodes on the left and bottom borders of the net boundary, and integer multiples of the maximum criticality of the nets connected to the nodes on the right and top borders of the net boundary.

[0109] Specifically, such as Figure 2 As shown, Figure 2The numbers next to the dashed lines represent the criticality information of the net connections, which are then inserted into vector x. vec The elements in can be represented as (0.15γx) min1 ,0.7γx max1 ,0.2γx min2 ,0.3γx max2 ), where γ > 10 represents the weight scaling factor, and the vector y vec Perform the same operation. For the net set... i All the wires n j (j∈[1,|Net i |]), will (x j,min ,x j,max Multiply by the weighting coefficients respectively, then update x j,min1 The weighting coefficient is the minimum criticality of the net connected to the nodes on the left border of the net bounding box multiplied by γ; j,max1 The weighting coefficient is the maximum criticality of the net connected to the nodes on the right bounding box of the net multiplied by γ; j,min2 The weighting coefficient is the minimum criticality of the net connected to the nodes on the lower border of the net bounding box multiplied by γ; j,max2 The weighting coefficient is the maximum criticality of the net connected to the nodes on the upper border of the net boundary frame multiplied by γ.

[0110] Furthermore, the zero-forcing search region is constructed as follows:

[0111] S2-1. Calculate the zero-forcing position:

[0112]

[0113]

[0114] Among them, (x z_p y z_p ) represents the coordinates of the zero-forcing position in the FPGA chip layout coordinate system, output(lb) i ) represents all output logic units lb i The set of wire meshes, (x s(l) y s(l) The input(lb) represents the coordinates of the source node of net l in the FPGA chip layout coordinate system. i ) indicates that all source nodes are the logical unit lb. i The set of nets, dest(r) represents the set of all leaky nodes in net r, (x m， y m ) represents the coordinates of the drain node m in the FPGA chip layout coordinate system, and Count() represents the counting operation.

[0115] S2-2. Based on the zero-forcing position, the zero-forcing search region is a square region with a side length of dis, centered on the zero-forcing search position and extending outwards. Dis must satisfy the condition 0 < dis < dis max , among which, dis max =min(X) FPGA ,Y FPGA ), where X FPGA With Y FPGA These represent the length values ​​of the FPGA chip in the X and Y directions of the FPGA chip layout coordinate system, respectively.

[0116] Furthermore, the weighted zero-forcing search region is constructed as follows:

[0117] S3-1. Calculate the weighted zero-forcing position:

[0118]

[0119]

[0120] Where, x wz_p Let x and y be the weighted zero-forcing position in the FPGA chip layout coordinate system. wz_p The weighted zero-forcing position is the AND coordinate in the FPGA chip layout coordinate system, where k is the criticality of the net connection; output(lb) i ) represents all output logic units lb i The set of wire meshes, (x s(l) ,y s(l) ) represents the coordinates of the source node of the net l in the FPGA chip layout coordinate system, input(lb) i ) indicates that all source nodes are the logical unit lb. i The set of nets, dest(r) represents the set of all leaky nodes in net r, (x m， y m ) represents the coordinates of the drain node m in the FPGA chip layout coordinate system, and Count() represents the counting operation;

[0121] S3-2. Based on the weighted zero-forcing position, the zero-forcing search region is a square region with a side length of dis, centered on the weighted zero-forcing position and extending outwards. Dis must satisfy the condition 0 < dis < dis max , among which, dis max =min(X) FPGA ,Y FPGA ), where X FPGA With Y FPGA These represent the length values ​​of the FPGA chip in the X and Y directions of the FPGA chip layout coordinate system, respectively.

[0122] Furthermore, the key temporal search region is constructed as follows:

[0123] S4-1. Define the threshold α for the criticality of wire network connection, and α satisfies the condition α∈[0,1];

[0124] S4-2. Select the set of all net connections with a criticality greater than α, and then select a logic unit lb to be disturbed from the selected net connection set. i ;

[0125] S4-3, For the selected logic unit lb to be disturbed i Identify the logic unit lb to be disturbed. i Coordinates (x(lb) in the FPGA chip layout coordinate system) i ),y(lb i ));

[0126] S4-4, will be combined with the logic unit to be disturbed lb i The criticality of the connected wires is sorted to identify the logic unit lb to be disturbed. i The κ input nodes with high criticality that are connected together form the set of critical input nodes {h1, h2, ..., hk}. κ}; Obtain the logic unit lb to be disturbed i The coordinates (x(v), y(v)) of the output node v with the highest criticality in the FPGA chip layout coordinate system;

[0127] S4-4. Based on the key time series search principle, determine the key time series search area ((xl(lb)). i ),yb(lb i )),(xr(lb i ),yt(lb i ))), where (xl(lb) i ),yb(lb i (xr(lb) represents the coordinates of the lower left corner vertex of the critical timing search region in the FPGA chip layout coordinate system. i ),yt(lb i )) represents the coordinates of the top right vertex of the critical timing search region in the FPGA chip layout coordinate system;

[0128] The key time series search principles are as follows:

[0129] lb i Key input node h g The minimum and maximum values ​​of the x and y coordinates of (g∈[1,κ]) are denoted as min_x(lb) and min_y(lb) respectively. i ), max_x(lb) i), min_y(lb i ) and max_y(lb i );

[0130] 1) When x(v) <min_x(lb i When xl(lb) i )=min(x(lb i ),x(v)),xr(lb) i ) = max(min_x(lb) i ),x(lb i ));

[0131] 2) When x(v)∈[min_x(lb) i ),max_xx(lb i )],xl(lb i )=min(x(lb i ),x(v)),xr(lb) i )=max(x(lb i ),x(v));

[0132] 3) When x(v) > max_x(lb) i When xl(lb) i )=min(x(lb i ),max_x(lb i )), xr(lb i )=max(x(lb i ),x(v)).

[0133] A schematic diagram of the key temporal search region construction method is shown below. Figure 3 As shown.

[0134] Furthermore, the reinforcement learning process in step S3 is as follows:

[0135] S301. Treat the FPGA layout as an intelligent agent and determine the current layout state based on the temperature value at the current moment under the FPGA layout.

[0136] S302. Based on the current layout state, determine the action set of the agent and calculate the probability of selecting each action in the action set;

[0137] S304. Based on the calculated probabilities of each action, select the corresponding action-defined region as the search region for layout optimization.

[0138] Using reinforcement learning methods to select the most suitable search region, the definitions of the agent, action set, policy, state, and reward function in reinforcement learning methods are as follows:

[0139] Intelligent agent: An FPGA layout based on reinforcement learning described in this invention;

[0140] State: The above layout states include the first layout state s1 and the second layout state s2;

[0141] The switching condition between the first layout state s1 and the second layout state s2 is:

[0142]

[0143] Among them, T t This represents the temperature value at time t, which can be calculated based on the current layout. T t-1 The temperature value from the previous iteration is β, where β∈(0,1) represents the scaling factor.

[0144] Action Set: When the agent is in state s1, the FPGA placement method based on reinforcement learning described in this invention mainly optimizes line length. The action set includes three search regions to choose from: median search region construction method, zero-forcing search region construction method, and random search region construction method. When the agent is in state s2, the FPGA placement method based on reinforcement learning described in this invention optimizes both line length and timing. The action set includes six controllable search regions to choose from: median search region construction method, zero-forcing search region construction method, weighted median search region construction method, weighted zero-forcing search region construction method, critical timing search region construction method, and random search region construction method.

[0145] Strategy: The FPGA placement method based on reinforcement learning described in this invention determines the probability of choosing action a in state u at time t. With action value function Q(a t,u They are positively correlated, and the specific formula is:

[0146]

[0147] Where g represents the number of actions in the action set in the current state, and q is the action number in state u at the current time t.

[0148] The action value function is updated in the following way during each iteration:

[0149] Q(a t+1,u )=Q(a t,u )+θ(R t+1 -Q(a t,u ))

[0150] Where θ represents the scaling factor, which is a constant greater than 0, Rt+1 This represents the reward function.

[0151] Reward Function: In the reinforcement learning-based FPGA placement method described in this invention, the reward function is defined as follows:

[0152]

[0153] Where ΔC is the change in the layout cost function of the logic unit to be disturbed after performing a swap operation within the selected search region.

[0154] The change in the layout cost function caused by performing the position swapping operation of the logic unit to be disturbed is calculated by the following formula:

[0155]

[0156] in,

[0157] ΔTiming_Cost = Timing_Cost - Previous_Timing_Cost represents the change in the delay cost function, ΔWriting_Cost = Writing_Cost - Previous_Writing_Cost represents the change in the line length cost function, λ∈[0,1] is a weight constant; Previous_Timing_Cost and Previous_Wiring_Cost are the delay cost function value and line length cost function value before each logic unit exchange, respectively, and Timing_Cost and Wiring_Cost are the delay cost function value and line length cost function value after each logic unit exchange.

[0158] Furthermore, step S5 specifically includes:

[0159] When the change in the layout cost function ΔC before and after the logic unit swap is less than 0, the position swap operation of the logic unit to be disturbed is executed.

[0160] When the change in the layout cost function ΔC before and after the logic unit swap is ≥0, The probability of performing a position swap operation on the logic unit to be disturbed is given by ΔC, where ΔC is the change in the layout cost function before and after the swap, and T is the position swap operation. t This represents the annealing temperature value at time t.

[0161] In a specific embodiment of the present invention If it is 50%, then it can be achieved in the following way: The probability of performing a position swap operation on the logic unit to be disturbed is:

[0162] Generate a random number between 0 and 1. If the random number is greater than 0.5, swap the numbers; otherwise, do not swap them.

[0163] In a specific embodiment of the present invention, the value of the preset threshold in step S6 is as follows:

[0164] move_lim=inner_num×τ 4 / 3

[0165] Here, inner_num is a constant with a value between 1 and 10.

[0166] Figure 4 and Figure 5 The graphs showing the changes in line length and critical path delay as a function of system runtime are presented. As can be seen from the graphs, the curves of the placement method proposed in this invention are consistently below those of the traditional simulated annealing method, indicating that the performance of the proposed placement method is superior to that of the traditional simulated annealing method. For a given set of parameters such as line length and critical path delay, the runtime of the placement method proposed in this invention is reduced by 59% and 46% respectively compared to the traditional simulated annealing method.

[0167] In summary, this invention proposes multiple search region construction mechanisms that can effectively improve the exploration efficiency of the layout solution space. Based on this, a reinforcement learning-based search region selection strategy is proposed, which can select the optimal search region in each layout iteration, ultimately completing the FPGA layout process.

[0168] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0169] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

Claims

1. A reinforcement learning based FPGA placement method, characterized in that The method comprises the following steps: S1, performing a layout initialization operation on a logic unit used in FPGA design; S2, judging whether the current FPGA layout state reaches an exit condition, if yes, exiting the layout process; Otherwise, initializing the iteration number and entering step S3; S3, selecting a search area for layout optimization in the FPGA by using a reinforcement learning method; S4, selecting a logic unit as a to-be-disturbed logic unit in the selected search area, and estimating a layout cost function variation caused by a position exchange operation of the to-be-disturbed logic unit; S5, judging the variation of the layout cost function before and after the exchange of the to-be-disturbed logic unit whether less than 0, if less than 0, performing the to-be-disturbed logic unit position exchange operation; otherwise, performing the to-be-disturbed logic unit position exchange operation with a certain probability, and then entering step S6; S6, judging whether the iteration number is greater than a preset threshold, if yes, returning to step S2 to re-execute steps S2-S6; otherwise, re-executing steps S3-S6; The step S3 reinforcement learning process is as follows: S301, regarding the FPGA layout as an agent, determining a current layout state according to a temperature value of the FPGA layout at the current time, wherein the layout state comprises a first layout state s1 and a second layout state s2; first layout state switching condition between the first layout state and the second layout state wherein, T t represents the temperature value at the current time t, which can be calculated according to the current layout state, is the temperature value at the previous time, is a scaling factor; S302, determining an action set of the agent according to the current layout state, and calculating probabilities of actions in the action set; When the agent is in the s1 state, the action set comprises three actions, which are a median search area construction method, a zero-forcing search area construction method and a random search area construction method; When the agent is in the s2 state, the action set comprises six actions, which are the median search area construction method, the zero-forcing search area construction method, a weighted median search area construction method, a weighted zero-forcing search area construction method, a critical timing search area construction method and the random search area construction method; S303, selecting a region determined by a corresponding action as a search area for layout optimization according to the calculated probabilities of the actions.

2. The FPGA placement method based on reinforcement learning of claim 1, wherein The layout exit condition in the step S2 is: wherein, is the value of the cost function at the current time instant, represents the number of lines in the FPGA netlist file.

3. The FPGA placement method based on reinforcement learning of claim 1, wherein, Current time t In state Select action Probability Positive correlation with action value function The specific formula is: wherein g represents the number of actions in the action set at the current state, is the action number at the current time t in the state .

4. The FPGA placement method based on reinforcement learning of claim 3, wherein The update mode of the action value function is: wherein, represents a scaling factor, is a constant greater than 0, represents a reward function.

5. The FPGA placement method based on reinforcement learning of claim 4, wherein The value of the reward function is defined as follows: 。 6. The FPGA placement method based on reinforcement learning of claim 1, wherein The construction mode of the median search area is as follows: S1-1, acquire all logic units to be disturbed lb i a set of connected wire nets Net i wherein , represents the number of logic units contained in the FPGA netlist file; S1-2, for the line net set Net i The line net in , acquire the line net n j The bounding box of , wherein Indicates the line net set Net i The number of line nets in, Is the coordinate of the left lower corner vertex of the line net bounding box in the FPGA chip layout coordinate system, Is the coordinate of the right upper corner vertex of the line net bounding box in the FPGA chip layout coordinate system; the origin of the FPGA chip layout coordinate system is the left lower corner of the FPGA chip, and the X axis and the Y axis are two edges of the FPGA chip perpendicular to each other; S1-3, define the horizontal vector x vec A variable used to store the horizontal vector of the line net bounding box in the x direction of the FPGA chip layout coordinate system, whose initial value is empty; define the vertical vector y vec A variable used to store the y direction of the line net bounding box, whose initial value is empty; S1-4, for the line net set Net i all the line nets , the insert into the transverse vector , the insert into the longitudinal vector , and arrange the elements in the transverse vector and the longitudinal vector in ascending order; S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector x S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector x S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector y S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector y S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector S1-5, the value of the first element of the transverse vector S1-6, to The corresponding region is defined as the median search region.

7. The FPGA placement method based on reinforcement learning of claim 6, wherein The construction mode of the weighted median search area is as follows: After step S1-3, before step S1-4, the following steps are performed: for the line net set Net i in the line net , the coordinate of the left bottom corner vertex of the line net bounding box in the x direction of the FPGA chip layout coordinate system , the coordinate of the left bottom corner vertex of the line net bounding box in the y direction of the FPGA chip layout coordinate system , the coordinate of the right top corner vertex of the line net bounding box in the x direction of the FPGA chip layout coordinate system , the coordinate of the right top corner vertex of the line net bounding box in the y direction of the FPGA chip layout coordinate system are respectively multiplied by the corresponding weight coefficients, and the line net bounding box vector is updated The weight coefficients are respectively an integer multiple of the minimum value of the criticality of the line net connected to the nodes on the left frame and the lower frame of the line net bounding box, and an integer multiple of the maximum value of the criticality of the line net connected to the nodes on the right frame and the upper frame of the line net bounding box.

8. The FPGA placement method based on reinforcement learning of claim 1, wherein The construction mode of the zero-forcing search area is as follows: S2-1, calculating a zero-forcing position: in,( , ( ) represents the coordinates of the zero-forcing position in the FPGA chip layout coordinate system. Represents all output logic units The set of wire networks, ( , ) indicates wire mesh l The coordinates of the source node in the FPGA chip layout coordinate system. This indicates that all source nodes are the logical units. A collection of wire meshes, dest(r) Indicates wire mesh r The set of all leaky nodes, ( x m， , y m ) indicates the drain node m Coordinates in the FPGA chip layout coordinate system Indicates counting operations; S2-2, on the basis of the zero-forcing position, the zero-forcing search region is a square region with the zero-forcing search position as the center point, and the side length dis of the square region is dis satisfying the condition wherein, wherein and are the length values of the X direction and the Y direction of the FPGA chip in the FPGA chip layout coordinate system, respectively.

9. The FPGA placement method based on reinforcement learning of claim 8, wherein The construction mode of the weighted zero-forcing search area is as follows: S3-1, calculating a weighted zero-forcing position: in, The weighted zero-forcing position is the x-coordinate in the FPGA chip layout coordinate system. The weighted zero-forcing position is the y-coordinate in the FPGA chip layout coordinate system. and These represent the criticality of the connection between net l and node m, respectively. Represents all output logic units A collection of wire meshes, For wire mesh l The coordinates of the source node in the FPGA chip layout coordinate system This indicates that all source nodes are the logical units. A collection of wire meshes, dest(r) Indicates wire mesh r The set of all leaky nodes, ( x m， , y m ) indicates the drain node m Coordinates in the FPGA chip layout coordinate system Indicates counting operations; S3-2, on the basis of the weighted zero-forcing position, the zero-forcing search region is a square region with the weighted zero-forcing position as the center point, and the square region is extended outward to obtain a side length dis of the square region, dis The condition must be met, wherein wherein and are length values of the X direction and the Y direction of the FPGA chip in the FPGA chip layout coordinate system, respectively.

10. The FPGA placement method based on reinforcement learning of claim 1, wherein The construction mode of the critical timing search area is as follows: S4-1, defining a threshold of line network connection criticality , and satisfies the condition ; S4-2, select a set of all wire-net connections whose criticality is greater than S4-3, select a to-be-disturbed logic cell from the set of selected wire-net connections ; S4-3, for the selected logic unit to be disturbed , find out the logic unit to be disturbed In the coordinate system of the FPGA chip layout (x, y) x ( lb i ), y ( lb i )) S4-4, the logic unit to be disturbed The size of the criticality of the connected wire net is sorted to find the logic unit to be disturbed The logic unit connected with the greater criticality The input nodes are grouped to form a set of critical input nodes ; acquiring a logic cell to be disturbed the most critical output node connected to the logic cell v coordinates in a coordinate system of the layout of the fpga chip x ( v ), y ( v） ); S4-4, determining a critical timing search region according to the critical timing search principle wherein is a coordinate of a left lower corner vertex of the critical timing search region in a coordinate system of an FPGA chip layout, is a coordinate of a right upper corner vertex of the critical timing search region in the coordinate system of the FPGA chip layout. The critical timing search principle is as follows: key input node the minimum and maximum of the x and y coordinates of , , and ; 1) when x ( v ) < min_ x ( lb i ), xl ( lb i ) = min( x ( lb i ), x ( v )), xr ( lb i ) = max(min_ x ( lb i ), x ( lb i )) ; 2) when , xl ( lb i ) = min( x ( lb i ), x ( v )), xr ( lb i ) = max( x ( lb i ), x ( v )); 3) when x ( v ) > max_ x ( lb i ), xl ( lb i ) = min( x ( lb i ), max_x ( lb i ) ), xr ( lb i ) = max( x ( lb i ), x ( v ) ).

11. The FPGA placement method based on reinforcement learning of claim 1, wherein The layout cost function variation caused by the position exchange operation of the to-be-disturbed logic unit is calculated by the following formula: wherein, denotes a change in the latency cost function, denotes a change in the wire length cost function; is a weight constant; , are the latency cost function value and the wire length cost function value, respectively, before the exchange logic unit each time, , denote the latency cost function value and the wire length cost function value after the exchange logic unit each time.

Images