Chip backend global routing method based on state-aware reinforcement learning

Abstract

The application discloses a chip back-end global routing method based on state-aware reinforcement learning, comprising the following steps: designing an input file based on information of a single or multiple benchmark design objects, establishing a standardized global routing running environment, and constructing a neutral global baseline result; performing effect screening and region capability verification to obtain a safety factor value set of each benchmark design object, a training action space, and a region capability mapping; performing probe routing to obtain a probe routing result and a structured probe index thereof; constructing a state-aware feature vector; performing reinforcement learning decision and deriving regional resource adjustment parameters to obtain formal global routing parameter decision results and regional resource adjustment parameters; calculating a reward value and performing update gating, equivalent deduplication, and candidate confirmation for learning iteration training; and outputting a structured result archive to obtain traceable and reproducible global routing optimization results. The application improves the stability of training convergence and the consistency of final optimization results.

Description

Technical Field

[0001] This invention belongs to the field of Electronic Design Automation (EDA) technology, specifically relating to a chip back-end global routing method based on state-aware reinforcement learning. Background Technology

[0002] Global routing in chip back-end design is used to allocate global routing paths to interconnect networks under design rule constraints. The results directly affect detailed routing feasibility, as well as trace length, vias, congestion patterns, and runtime. Existing global routers typically read process and design input files and output a global routing guide and log. Common evaluation metrics include total overflow, wirelength, total number of vias, runtime, hotspot ratio, and coefficient of variation (CV).

[0003] In existing technologies, common approaches mainly include fixed-parameter operation, manual multi-round parameter testing, and direct search methods without safety constraints. While these approaches can yield usable results on specific computational examples, their parameter selection process typically lacks the combined utilization of the current congestion status of the cabling design, layer resource distribution, hotspot locations, and historical feedback. Consequently, it is difficult to stably obtain an optimal solution with interpretability and reproducibility under multi-objective conflict conditions.

[0004] In existing technical solutions, one approach involves using a fixed combination of Tcl parameters to call global_route for global routing. Then, metrics such as total overflow, wirelength, and total number of vias are extracted from the global routing boot file and logs, allowing designers to manually compare the advantages and disadvantages of different parameter combinations. While this approach is relatively straightforward, parameter selection relies heavily on manual experience and lacks an adaptive decision-making mechanism tailored to the current design state.

[0005] Another type of existing technical solution employs multi-round script search or automated parameter optimization processes to directly score and compare multiple sets of formal global cabling results. While this type of solution improves automation, it typically evaluates the formal results directly and still lacks mechanisms for state characterization based on probe cabling, neutral global baseline Via barriers, equivalent signature deduplication, supplementary confirmation reruns, and verification of regional resource adjustment capabilities.

[0006] Furthermore, existing solutions often simply link "probe cabling," "formal global cabling," "results comparison," and "strategy update" together, lacking systematic constraints on high-risk parameter values, equivalent actions, regional resource adjustments, and confirmation of reruns. This results in at least the following problems:

[0007] (1) There are a large number of invalid, high-risk or equivalent parameter values ​​in the parameter space. Existing solutions usually lack pre-screening based on neutral baseline, Via security constraints and result signature deduplication mechanism, resulting in a lot of repeated execution, training samples are easily contaminated, parameter search cost is high and stability is insufficient.

[0008] (2) Existing multi-objective optimization processes usually score the formal global routing results directly, lacking the state characterization of the probe routing conditions and the measurement of the improvement relative to the probe results and baseline results. Therefore, it is difficult to effectively couple parameter decisions with the current congestion state, hotspot layers and historical feedback, which can easily lead to local indicators improving while overall quality deteriorates, especially the problem of Via deteriorating significantly.

[0009] (3) Regional resource adjustments and result outputs are usually implemented by direct script calls and log post-processing. There is a lack of regional capability verification, failure gating, structured native indicators, confirmation rerun and archiving mechanisms, which makes it difficult to determine whether regional adjustments are actually effective, and the traceability, reproducibility and reliability of the optimization process are insufficient. Summary of the Invention

[0010] The purpose of this invention is to provide a chip back-end global routing method based on state-aware reinforcement learning, which overcomes the shortcomings of existing global routing optimization processes such as lack of safe screening of parameter space, insufficient state coupling, unstable regional adjustment, lack of global constraints in Via, and incomplete result archiving.

[0011] The method of this invention is achieved through the following technical solution:

[0012] A chip back-end global routing method based on state-aware reinforcement learning, comprising:

[0013] Step S1: Based on the process / library information or design layout information of one or more baseline design objects, design input files, establish a standardized global routing operation environment, construct neutral global baseline results, and obtain the standardized global routing operation environment, the list of baseline design objects, and the neutral global baseline index corresponding to each baseline design object.

[0014] Step S2: Based on the candidate factor set, regional coordinate pattern, and neutral global baseline index obtained in step S1, perform effect screening and regional capability verification to obtain the safety factor value set, training action space, and regional capability mapping of each benchmark design object.

[0015] Step S3: Based on the standardized global cabling operating environment and configured fixed detection parameters obtained in step S1, perform detection cabling to obtain the detection cabling results and their structured detection indicators.

[0016] Step S4: Based on the structured detection indicators, historical rewards and historical improvement amounts, and baseline design object identification information obtained in step S3, construct the state-aware feature vector to obtain the state-aware feature vector.

[0017] Step S5: Based on the state-aware feature vector obtained in step S4, the set of security factor values ​​obtained in step S2, the training action space, and the regional capability mapping, perform reinforcement learning decision-making and derive regional resource adjustment parameters to obtain the formal global wiring parameter decision results and regional resource adjustment parameters.

[0018] Step S6: Based on the formal global cabling parameter decision results and regional resource adjustment parameters obtained in step S5, perform formal global cabling to obtain formal global cabling results, structured evaluation indicators and result signatures.

[0019] Step S7: Based on the probe routing results and structured probe metrics obtained in Step S3, the formal global routing metrics obtained in Step S6, the neutral global baseline metrics obtained in Step S1, and historical training records, calculate the reward value and perform update gating, equivalent deduplication, and candidate confirmation to obtain the reward value, policy update result, signature deduplication result, confirmation result, and specification optimal result. At the same time, determine whether the set iteration termination condition is met. If it is met, proceed to Step S8. If it is not met, repeat Steps S4-S7 to continue reinforcement learning training.

[0020] Step S8: Based on the data from the entire process from steps S1 to S7, output structured results for archiving, and obtain traceable and reproducible global routing optimization results.

[0021] Furthermore, the neutral global baseline metrics include at least total overflow, bus length, total number of vias, runtime, hotspot ratio, and congestion dispersion.

[0022] Further, step S2 specifically includes:

[0023] Step S2-1: The candidate factors are designed to include layer grouping, layer resource adjustment range, global capacity adjustment and layer window. Each candidate factor has a finite number of predefined candidate values. The effect screening is performed on each benchmark design object in a single factorial test manner to obtain the set of safety factor values ​​and the training action space.

[0024] Step S2-2: Set a retention upper limit for each factor in the candidate factor set. While satisfying the retention upper limit constraint, retain the default value of each factor so that the training action space always contains a stable backoff point.

[0025] Step S2-3: Perform baseline design object-level capability verification for regional resource adjustments: Apply regional resource adjustment parameters to each baseline design object individually and perform test routing to determine whether the global router can normally accept and execute regional resource adjustment commands; Baseline design objects that pass verification are recorded as having available regional capabilities, and those that fail verification are recorded as having unavailable regional capabilities, thus forming a regional capability mapping; Regional resource adjustment parameters are only allowed to be derived during the training phase when the regional capability verification of the corresponding baseline design object passes.

[0026] Further, step S2-1 specifically includes:

[0027] Step S2-1-1: Based on the total via count ratio of the neutral global baseline, high-risk values ​​are screened out, and the total via count ratio is calculated. for:

[0028]

[0029] in, This represents the total number of vias in the selected sample. The total number of vias for the neutral global baseline corresponding to the reference design object. To prevent the total number of vias from becoming zero constant;

[0030] like If the value exceeds the preset safety threshold, the candidate factor is determined to be a high-risk value and is filtered out.

[0031] Step S2-1-2: Sign the calculation results of the screened candidate factors. for:

[0032]

[0033] in, For hash mapping functions, , , , , These are the total overflow of formal global routing, bus length, total number of vias, hotspot ratio, and congestion dispersion, respectively.

[0034] Deduplication is performed on equivalent values ​​that result in the same signature within the same baseline design object, and the remaining values ​​are recorded as filtered equivalence mappings to obtain a set of safety factor values.

[0035] The safety factor values ​​are sequentially processed to form a Cartesian product to create the training action space.

[0036] Furthermore, the detection metrics in step S3 include total overflow, bus length, total number of vias, running time, hotspot ratio, congestion dispersion, layer distribution, number of candidate networks for hotspot layer groups, hotspot bounding box area ratio, and database statistics. The hotspot ratio is determined based on the 95th percentile value of the density grid, and the congestion dispersion is used to measure the degree of dispersion of congestion in each layer.

[0037] In step S4, the state-aware feature vector is composed of structured detection indicators, historical memory items, and baseline design object hotspot identifiers concatenated in a predetermined field order. The historical memory items include the previous round reward value and the improvement amount of the previous round hotspot ratio and congestion dispersion. The baseline design object hotspot identifier is used to distinguish the congestion characteristics of different design objects.

[0038] Furthermore, the reinforcement learning decision in step S5 is specifically as follows:

[0039] Let the candidate complete actions in the training action pool be... For each candidate action, calculate the action score using the following formula:

[0040]

[0041] in, For action The corresponding linear parameters, For action The corresponding covariance matrix, Representation matrix The inverse matrix, For the first Exploration coefficient, This is a state-aware feature vector. express transpose;

[0042] Let the total number of training movements be The training budget is Determine the number of pre-coverage rounds for:

[0043]

[0044] in, To minimize the number of actions covered, The proportion of pre-coverage rounds relative to the training budget, To train the total number of movements, For training budget;

[0045] If the current iteration number Forcefully cover unsampled actions according to the pre-coverage sequence; if And the probability of a random number is less than the probability of random exploration. If the result is positive, then random exploration will be performed; otherwise, a search based on the given conditions will be performed. The highest-scoring candidate move; the probability of the random exploration is:

[0046]

[0047] Before formally executing the selected action, the equivalent factor value is first folded into the standard value. The equivalent factor is: if a factor value of the current benchmark design object produces the same result signature as a factor value obtained in the effect screening stage of step S2, then the factor is considered to be an equivalent factor. Then, context combination equivalence filtering is performed. That is, if the current action is only equivalent to the existing signature on an independent factor, it is directly replaced with the known standard action.

[0048] The specific parameters for adjusting derived region resources in step S5 include:

[0049] For the selected action involving layer grouping and layer resource adjustment, and combining the regional capability mapping obtained in step 2 and the detection index information obtained in step 3, regional resource adjustment parameters are derived. If the regional capability verification of the corresponding benchmark design object fails, or if the current layer group does not have a hotspot bounding box, regional resource adjustment is not enabled; otherwise, the regional resource adjustment magnitude is calculated as follows:

[0050]

[0051] in, Adjust the layer resource range for the selected action. The `clamp` function adjusts the scaling factor for a region, fixing a specified value to a specified minimum and maximum range.

[0052] Furthermore, the calculation of the reward value in step S7 specifically includes:

[0053] S7-1, Calculate the improvement in hotspot ratio, congestion dispersion, wire length, and runtime relative to probe routing in formal global routing. for:

[0054]

[0055] in, For the first Wheel detection wiring stage index values, For the first Round of formal global cabling index values, As an indicator The zero-prevention constant;

[0056] For the total number of vias index, calculate its relative improvement. for:

[0057]

[0058] in, The total number of vias is a reference value, which is taken as the total number of vias at the neutral global baseline of the corresponding reference design object. For the first Total number of vias in the formal global routing round. This is a zero-prevention constant for the total number of vias;

[0059] S7-2, Calculate the... The ratio of the total number of vias in the formal global routing result relative to the neutral global baseline ,when At the set soft penalty starting threshold With hard guardrail threshold In between, calculate the soft penalty term for the total number of through holes. ;

[0060]

[0061] in, It is a stable constant in the denominator of the ratio;

[0062] S7-3, based on the improvement amount, the total number of vias ratio, and the soft penalty term, calculate the first... Round reward value :

[0063]

[0064] in, For the first Round-route integrity flag: 1 for complete, 0 for incomplete; For the first Global routing overflow in round 1; This is the total overflow threshold; , , , , These represent the improvements in hotspot ratio, congestion dispersion, line length, total number of vias, and runtime, respectively. These are the corresponding weighting coefficients.

[0065] Further, in step S7, the equivalent deduplication and candidate confirmation are as follows:

[0066] First, candidate actions with valid samples are selected from the summary results of actions obtained during the training phase, and sorted according to the priority order of hotspot ratio, congestion dispersion, line length, number of vias, and running time. For duplicate actions with the same result signature set on each benchmark design object, only one representative action is retained. Then, a preset number of candidates are selected from the representative actions, and each candidate is repeatedly executed multiple times on each benchmark design object to obtain candidate confirmation results. Subsequently, the candidate actions with valid samples from the training phase are combined and statistically analyzed with the candidate confirmation results to calculate the strategy evaluation mean, valid sample ratio, and total number of samples for each candidate action. The candidate actions are sorted in descending order according to the strategy evaluation mean. When the strategy evaluation mean is the same, the action is ranked and decided in turn according to the hotspot ratio, congestion dispersion, line length, number of vias, running time, valid sample ratio, and total number of samples to determine the optimal result. The strategy evaluation mean is the statistical mean of the learning signal corresponding to the same candidate action.

[0067] Furthermore, in step S7, the gating is updated to use a learned signal relative to a neutral global baseline. Update action parameters, the learning signal being:

[0068]

[0069] in, These represent the improvements of each metric in the formal global routing results relative to the neutral global baseline. These are the weighting coefficients for the corresponding learning signal.

[0070] Furthermore, the specific steps in step S7 for determining whether the set iteration termination condition is met are as follows:

[0071] Let the window that appears most frequently in the most recent W rounds be the first one. The result signature set is Calculate stability coverage:

[0072]

[0073] Training is terminated when the number of training rounds reaches the set minimum number of training rounds, there is no substantial improvement for several consecutive rounds, and the stability coverage reaches a preset threshold. The method for judging that there is no substantial improvement for several consecutive rounds is as follows: after the most recent best effective result is updated, within a certain number of consecutive training rounds, the effective formal global routing result of the current round is compared with the best effective result in the priority order of hotspot ratio, congestion dispersion, line length, number of vias, and running time. If the difference between the two results does not exceed the preset minimum improvement ratio threshold, then there is no substantial improvement for several consecutive rounds.

[0074] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0075] (1) This invention first constructs a neutral global baseline result, and then performs effect screening, total via ratio safety screening, and result signature deduplication on the candidate factor values. Only the values ​​that are effective for the corresponding benchmark design object and whose risks are controlled are retained as the training action space. Since invalid values, high-risk values, and other valuable results have been eliminated before formal training, it can reduce repeated formal wiring and invalid strategy updates, reduce parameter search costs, and improve the stability of training convergence and the consistency of the final optimization results.

[0076] (2) Before each round of formal decision-making, this invention first performs probe routing to extract diagnostic information such as total overflow, hotspot ratio, congestion dispersion, layer distribution, and the number of candidate networks for hotspot layer groups. This information is then combined with historical feedback to form a state-aware feature vector, and action scoring and selection are performed in the training action space. At the same time, the improvement of the formal results relative to probe routing and the total via number guardrail relative to the neutral global baseline are both included in the reward calculation and gating update. Since the parameter decision is based on the current routing conditions and explicit constraints are set for the deterioration of the total via number, it can suppress the abnormal increase of the total via number while improving the hotspot and congestion patterns, thus achieving more stable collaborative optimization among multiple objectives.

[0077] (3) This invention first performs baseline design object-level capability verification for regional resource adjustments, and only derives regional resource adjustment parameters when the verification is passed and the corresponding layer group hotspots do exist during the detection phase; after formal execution, it also outputs native structured indicators, result signatures, additional confirmation results, and archived files. Since regional adjustments have an execution chain of "verification first, then activation, gating, and confirmation", and the result recording no longer relies solely on log text, it can improve the effectiveness judgment capability of regional resource adjustments and enhance the traceability, reproducibility, and credibility of the optimization process and the final result. Attached Figure Description

[0078] Figure 1 This is a flowchart illustrating the overall process of a chip back-end global routing method based on state-aware reinforcement learning according to the present invention.

[0079] Figure 2 This is a block diagram showing the interaction between the software module and the global router of the present invention.

[0080] Figure 3 This is a schematic diagram illustrating the neutral global baseline, effect screening, and training action space generation of this invention.

[0081] Figure 4 This is a schematic diagram illustrating the derivation of state-aware feature aggregation and regional resource adjustment in this invention.

[0082] Figure 5This diagram illustrates the signature deduplication, additional confirmation, and optimal result selection for standardization in this invention.

[0083] Figure 6 This is a timing diagram of the persistent execution task based on the Python program interface of this invention. Detailed Implementation

[0084] Combination Figure 1 and Figure 3 This invention provides a chip back-end global routing method based on state-aware reinforcement learning (RL), comprising:

[0085] Step S1: Based on the LEF / DEF design input files of one or more benchmark design objects, establish a standardized global routing runtime environment and construct a neutral global baseline result to obtain the standardized global routing runtime environment, the list of benchmark design objects, and the neutral global baseline index corresponding to each benchmark design object.

[0086] The neutral global baseline refers to the reference result obtained by performing a complete formal global routing with predetermined default parameters without applying additional regional resource adjustments, global capacity adjustments, or layer window biases. Specifically, the LEF / DEF design input is parsed to establish a standardized global routing runtime environment; the output directory, cache directory, metric record structure, and parameter injection interface are initialized; a formal global routing with fixed neutral parameter configuration is performed once for each baseline design object to obtain the neutral global baseline metrics for the corresponding baseline design object. The neutral global baseline metrics include at least total overflow, wirelength, total number of vias, runtime, hotspot ratio, and congestion dispersion.

[0087] Step S2: Based on the neutral global baseline index, candidate factor set, and regional coordinate pattern obtained in step S1, perform effect screening and regional capability verification to obtain the safety factor value set, training action space description, and regional capability mapping of each benchmark design object.

[0088] The candidate factor set includes at least the following four types of factors: layer group, layer resource adjustment range (adj), global capacity adjustment (cap_adj), and layer window (layer_window), with a finite number of predefined candidate values ​​for each factor.

[0089] The effect screening adopted a single-factor experiment approach, that is, only the value of one tested factor was changed each time, while the values ​​of the other factors remained unchanged. Formal global routing was performed one by one for each baseline design object to obtain the formal global routing results and their evaluation indicators corresponding to the values ​​of each candidate factor. For each screened sample, based on the neutral global baseline Via obtained in step S1, the Via ratio of the sample relative to the neutral global baseline was calculated according to the following formula. :

[0090]

[0091] in, For this screening sample , Neutral global baseline for the corresponding benchmark design object , for Zero-prevention constant. If If the value exceeds the preset safety threshold, it is determined to be a high-risk value and is filtered out.

[0092] Then, the signature of each selected sample is calculated using the following formula. :

[0093]

[0094] in, For hash mapping functions, , , , , These are the Overflow, Wirelength, Via, Hotspot Ratio, and Congestion Dispersion for formal global routing. The result signature is used to identify equivalent results; for equivalent values ​​within the same baseline design object that result in the same result signature, deduplication is performed, retaining only the standard values, and recording the remaining values ​​as filtered equivalent mappings.

[0095] Set retention limits for each factor. While satisfying the retention limit constraints, retain the default values ​​of each factor to ensure that the training action space always contains a stable backoff point.

[0096] For regional resource adjustments, baseline design object-level capability verification is performed: Before training, test routing is executed by applying regional resource adjustment parameters to each baseline design object individually to determine whether the global router can normally accept and execute regional resource adjustment commands. Baseline design objects that pass verification are recorded as having usable regional capabilities, while those that fail verification are recorded as having unusable regional capabilities, thus forming a regional capability mapping. Regional resource adjustment parameters are only allowed to be derived in subsequent training phases when the regional capability verification of the corresponding baseline design object passes.

[0097] Step S3: Based on the operating environment and fixed probe parameter configuration obtained in step S1, perform probe routing to obtain the probe routing results and their structured diagnostic indicators.

[0098] Specifically, at the start of each training round, a probe routing is performed once with fixed probe parameters. This probe routing uses fewer congestion iterations than the formal global routing to obtain congestion diagnostic information for the current design condition in a shorter time, rather than obtaining the complete routing result. Structured probe metrics are extracted from the probe routing results. These metrics include at least total overflow, wirelength, total number of vias, runtime, hotspot ratio, congestion dispersion, layer distribution, number of candidate networks for hotspot layer groups, hotspot bounding box area ratio, and design database statistics. The hotspot ratio is determined based on the 95th percentile of the density grid, and the congestion dispersion measures the degree of congestion dispersion across layers.

[0099] Step S4: Based on the detection indicators, historical rewards and historical improvement amounts, and baseline design object identification information obtained in Step S3, construct the state-aware feature vector to obtain the state-aware feature vector. and its field definitions.

[0100] Specifically, the detection indicators, historical memory items, and the unique thermal identifier of the benchmark design object are concatenated according to a predetermined field order to form a state-aware feature vector. The historical memory entries include the previous round's reward value and the improvements in the previous round's hotspot ratio and congestion dispersion; the baseline design object's unique hotspot identifier is used to distinguish the congestion characteristics of different design objects.

[0101] Step S5, based on the state-aware feature vector obtained in step S4 The training action space and region capability mapping obtained in step S2 are used to perform reinforcement learning decisions and derive region resource adjustment parameters, resulting in formal global wiring parameter decision results and region resource adjustment parameters.

[0102] Specifically, an explicit training action pool is constructed from the set of safe values ​​for each factor obtained in step S2: the safe values ​​of each factor are combined using a Cartesian product to generate all candidate complete actions; if the layer window range of an action does not overlap with the layer grouping, the incompatible action is directly removed during the construction phase. A separate training action pool is maintained for each benchmark design object, and the action pool switches as the benchmark design object changes.

[0103] Let the candidate complete actions in the training action pool be... For each candidate action, calculate the action score using the following formula:

[0104]

[0105] in, For action The corresponding linear parameters, For action The corresponding covariance matrix, For the first Exploration coefficient.

[0106] In the initial stage of training, a dynamic pre-coverage mechanism is used to forcibly cover unsampled actions. Let the total number of training actions be... The training budget is Then the number of pre-coverage rounds Determine using the following formula:

[0107]

[0108] in, Minimum number of actions covered. The ratio of the number of pre-covered rounds to the training budget.

[0109] After the pre-coverage phase, actions are selected based on the exploration decay strategy: using random exploration probability. Perform random exploration; otherwise, select the candidate action with the highest score among the aforementioned actions. The probability of random exploration decreases with each round according to the following formula:

[0110]

[0111] Before formally executing the selected action, two types of deduplication filtering are performed: the first is the equivalence mapping filtering, which folds the equivalence factor value into the normalized value. The equivalence factor is: if a factor value of the current baseline design object produces the same result signature as a factor value obtained in the effect filtering stage of step S2, then the factor is considered to be an equivalence factor; the second is the context combination equivalence filtering, which means that if the current action is only equivalent to an existing signature on a single independent factor, then it is directly replaced with a known normalized action.

[0112] For the selected action involving layer grouping and layer resource adjustment, the region resource adjustment parameters are derived by combining the region capability mapping obtained in step S2 and the probe hotspot information obtained in step S3. Region resource adjustment is not enabled when the region capability verification of the corresponding baseline design object fails, or when there is no hotspot bounding box in the current layer group. Otherwise, the region resource adjustment magnitude is calculated using the following formula:

[0113]

[0114] in, Adjust the layer resource range for the selected action. Adjust the scaling factor for the region.

[0115] Step S6: Based on the formal global cabling parameter decision results and regional resource adjustment parameters obtained in step S5, perform formal global cabling to obtain formal global cabling results, structured evaluation indicators, and result signatures.

[0116] Specifically, formal global routing parameters and area resource adjustment parameters are injected into the global router to uniformly execute a complete formal global routing. Evaluation metrics, including at least total overflow, wirelength, total number of vias, runtime, hotspot ratio, congestion dispersion, and route integrity flags, are collected from the formal global routing results. Simultaneously, a result signature is generated for the formal global routing results according to the result signature calculation method defined in step S2. This is used for subsequent equivalent deduplication.

[0117] Step S7: Based on the detection metrics obtained in Step S3, the formal global wiring metrics obtained in Step S6, the neutral global baseline metrics obtained in Step S1, and historical training records, calculate the reward value and perform update gating, equivalent deduplication, and candidate confirmation to obtain the reward value, policy update result, signature deduplication result, confirmation result, and optimal specification result. The signature deduplication method is the same as that in Step S2.

[0118] First, calculate the improvement of each metric in formal global routing compared to probe routing. This includes metrics such as hotspot ratio, congestion dispersion, cable length, and runtime. Its relative improvement Calculate using the following formula:

[0119]

[0120] in, For the first Wheel detection wiring stage index values, For the first Round of formal global cabling index values, As an indicator The zero-constant value for Via. The relative improvement of Via is not referenced to the Via of the probe wiring, but is calculated separately with priority given to the Via of the neutral global baseline, so that the benchmark for measuring the Via improvement remains consistent throughout the process.

[0121] Then, calculate the first... Round of formal global routing results relative to the neutral global baseline ratio :

[0122]

[0123] in, For the first Round of formal global cabling , Via is the neutral global baseline corresponding to the benchmark design object. For Via, use a zero constant.

[0124] when At the soft penalty initiation threshold With hard guardrail threshold During this period, calculate the Via soft penalty term using the following formula. :

[0125]

[0126] in, The denominator of the ratio is a stable constant. The Via soft penalty term is used to apply a continuously increasing penalty when the Via ratio approaches the hard fence threshold, in order to avoid actions that rely excessively on Via to improve hotspot performance.

[0127] Based on the aforementioned improvements, Via ratio, and soft penalty term, the following formula is used to calculate the... Round reward value :

[0128]

[0129] in, For the first Round-route integrity flag: 1 for complete, 0 for incomplete; For the first Global routing overflow in round 1; The overflow threshold; , , , , These represent the relative improvements in hotspot ratio, congestion dispersion, line length, Via, and runtime, respectively. For the corresponding weighting coefficients, the above formula judges from top to bottom according to priority: route integrity failure is directly penalized with a fixed penalty; when Overflow exceeds the threshold, a penalty is imposed according to Overflow; when the Via ratio exceeds the hard fence, a penalty is imposed according to the Via ratio; in other cases, the reward is calculated based on the multi-objective weighted improvement amount.

[0130] The reinforcement learning policy is updated only for rounds that meet the valid sample conditions. The valid sample conditions include at least: successful formal global routing, valid routing integrity flag, and no abnormal collapse of the formal result relative to the probe result. For rounds that do not meet the valid sample conditions, or actions involving Overflow penalty, Via hard fence penalty, invalid metrics, or formal global routing returning a non-zero exit code, these are uniformly recorded in the invalid action record table. The invalid action record table records at least the action identifier, baseline design object identifier, invalid reason, and subsequent processing flag; wherein, the subsequent processing flag is used to distinguish whether the entry is merely recorded as an invalid sample or whether filtering and avoidance are performed in subsequent candidate selections. Simultaneously, a signature equivalence table and equivalent action mapping relationships under context conditions are maintained to avoid duplicate samples contaminating the training.

[0131] When updating the reinforcement learning policy, a learning signal relative to the neutral global baseline is used. The learning signal is calculated by updating the action parameters using the following formula:

[0132]

[0133] in, These represent the improvements of each metric in the formal global routing results relative to the neutral global baseline. These are the weight coefficients for the corresponding learning signal. The learning signal is only applied when the valid sample condition is met, the reward source does not belong to routing integrity failure, or... Hard fence penalties are only used to update reinforcement learning policy parameters when the current sample does not match an existing duplicate signature.

[0134] In addition, a dynamic early stopping mechanism is introduced during training: the mechanism is defined as the one that appears most frequently within the most recent W-round window. The result signature set is The stability coverage is then calculated using the following formula:

[0135]

[0136] When three conditions are met simultaneously—the number of training rounds reaches the minimum number of training rounds, there is no substantial improvement for multiple consecutive rounds, and the stability coverage reaches a preset threshold—the subsequent training is terminated and the additional confirmation phase begins. The result set obtained after the additional confirmation run is the confirmation result set.

[0137] After training, select the top representative parameter combination from the set of unique result signatures. Each candidate is subjected to an additional confirmation rerun to obtain the optimal canonical result. The optimal canonical result refers to the result obtained from the confirmation set formed by merging the training result and the additional confirmation result, first according to the learning signal... The calculated strategy evaluation mean is sorted in descending order, and then the final optimal result is obtained by judging the hotspot ratio, congestion dispersion, line length, Via, running time, effective sample ratio, and total sample size in turn. This result is not equivalent to the result with the largest reward value.

[0138] Step S8: Based on the data from the entire process from steps S1 to S7, output structured results for archiving, and obtain traceable and reproducible global routing optimization results.

[0139] Specifically, the output includes configuration files, baseline metrics, effect screening results, training results, confirmation results, summary of the best action, summary of the best formal global routing results, heatmap, and native metric description files, which are then structured and archived according to the baseline design object and round.

[0140] Accordingly, through a closed-loop process of “neutral global baseline construction—effect screening and regional capability verification—probe cabling diagnosis—state construction—training action decision-making—formal global cabling—reward gating update—confirmation and result archiving output”, the state adaptive optimization of global cabling parameters is achieved.

[0141] Example

[0142] This embodiment provides a specific implementation process for a chip back-end global routing method based on state-aware reinforcement learning, including:

[0143] Step 1: Set implementation conditions and input / output interfaces

[0144] Combination Figure 2 This embodiment adopts an implementation structure of "Python program interface scheduling module + persistent global routing execution unit + global routing boot file statistics module". The Python program interface scheduling module is responsible for baseline establishment, effect filtering, probe cache management, state construction, action selection, reward calculation, append confirmation, and result archiving and writing. The persistent global routing execution unit is responsible for loading the design, applying routing parameters, calling the global router, and returning formal metrics and native database statistics. The global routing boot file statistics module is responsible for parsing the global routing boot file, calculating density grids, calculating hotspot ratios and congestion dispersion statistics, and rendering heatmaps. This includes:

[0145] The input object in this embodiment is one or more benchmark design objects. Each benchmark design object contains at least: INPUT_LEF (path to process / library information file), INPUT_DEF (path to design layout information file), and a design identifier, benchmark_id. In scenarios with multiple benchmark design objects, the input can be a benchmark design object manifest file, where each line contains three fields: design identifier, path to process / library information file, and path to design layout information file.

[0146] The Python program interface scheduling module also maintains a probe cache directory; when the probe routing for the same benchmark design object, the same probe parameter configuration, and the same execution unit fingerprint has been completed, subsequent rounds directly reuse the cached probe results without repeating the probe routing, thereby reducing the total time overhead of multiple rounds of training.

[0147] The output directory in this embodiment is organized according to one experiment. The core results include configuration files, baseline index files, effect screening result files, training result files, confirmation result files, training summary files, standard optimal action configuration files, and optimal formal global routing result directory.

[0148] The parameter injection interface uses environment variables or equivalent structured task loads, and the mapping relationship is shown in Table 1.

[0149] Table 1, Interface Parameter Mapping Relationship Table

[0150]

[0151] One implementation example of GRT_LAYER_ADJ_SPEC is: Metal1:0.20, Metal2:0.20, Metal3:0.20; another implementation example of GRT_REGION_SPEC (regional resource adjustment parameters) is: bbox (boundary box coordinates) = 1200, 3400, 1400, 3600; layers (routing layers) = M1-3; adj (resource adjustment range) = 0.20; the formal global routing, additional confirmation, and specification optimal result generation in this implementation all adopt complete global routing, and do not adopt incremental rerouting paths that are only performed on a part of the network.

[0152] Step 2: Neutral global baseline, effect screening, and regional capability verification

[0153] Before entering reinforcement learning training, a neutral global baseline result is established for each benchmark design object. Then, the action space is compressed by using single-factor effects, and the regional resource adjustment capability is pre-validated.

[0154] The neutral global baseline parameters are set as follows: layer group = M1-3, where group represents the target layer group to which layer resource adjustments are applied; layer resource adjustment magnitude adj = 0.00, where adj represents the magnitude of resource adjustment applied to the target layer group; global capacity adjustment cap_adj = 0.00, where cap_adj represents the magnitude of uniform adjustment applied to global cabling capacity; layer window layer_window = (0,0), where layer_window represents the range of signal cabling layers allowed for formal global cabling, and (0,0) indicates no additional limitation on the layer range; congestion iterations congestion_iterations = 60, where congestion_iterations represents the number of congestion iteration rounds for formal global cabling; critical nets_percentage = 0, where critical nets_percentage represents the percentage of networks prioritized based on criticality; region resource adjustment switch region_enable = 0, where region_enable indicates whether region resource adjustment is enabled. The candidate factor set is defined as shown in Table 2.

[0155] Table 2, Candidate Factor Set

[0156]

[0157] A simplified filter budget file was used. The actual test values ​​during the filter were: group={M1-3,M4-6,M7-9}; adj={0.00,0.10,0.20}; cap_adj={0.00,0.10,0.15}; layer_window={(0,0),(1,6)}.

[0158] Single-factor screening follows the rule of "changing only the tested factor, while keeping the other screening default values". The default screening values ​​are: group=M1-3; layer_adj=0.15; cap_adj=0.00; layer_window=(0,0); congestion_iterations=60; critical_nets_percentage=0; region_enable=0.

[0159] When the selection factor is cap_adj, layer_adj=0.00 is fixed to eliminate layer adjustment interference.

[0160] For each screened sample, calculate the Via ratio to the neutral global baseline. .like If the selected sample is not included in the training action candidate set, then the selected sample will not be included in the training action candidate set.

[0161] For the same factor value of the same benchmark design object, if different values ​​produce the same signature result. If so, only the normalized values ​​are retained, and the remaining values ​​are recorded as the filtering equivalent mapping.

[0162] The retention limits for each factor are as follows: group retains 2 values; adj retains 3 values; cap_adj retains 2 values; and layer_window retains 2 values. While satisfying the upper limit constraints, the default values ​​of each factor are retained to ensure that the training action space always contains a stable backoff point.

[0163] Regional resource adjustments are performed separately for regional capability verification (referred to as regionsmoketest in this embodiment) before training. Specifically, a test routing task containing the regional resource adjustment parameter GRT_REGION_SPEC is constructed separately for each baseline design object, a complete global routing is performed, and it is checked whether the global router successfully accepts and executes the regional resource adjustment command. If the global router returns a non-zero exit code, or a warning flag indicating that the regional resource adjustment cannot be recognized appears in the log, the regional capability verification of the baseline design object is determined to have failed. Regional capability verification also supports automatic coordinate adaptation: in this embodiment, the regional coordinates are first attempted to be converted from DBU (DesignBlockUnit) to micrometer units for submission; if the verification still fails after conversion, the original DBU coordinates are reverted and submitted again. In the main scheme, a regional capability mapping is formed according to the baseline design object, and GRT_REGION_SPEC is only allowed to be derived in the subsequent training phase when the regional capability verification of the corresponding baseline design object passes.

[0164] In this embodiment, a stricter global gating strategy can be adopted; when the capability verification of any benchmark design object area fails, the training phase that enables regional resource adjustment will not be entered.

[0165] Step 3: Extraction of detection indicators and construction of state-aware feature vectors

[0166] At the start of each training round, a probe routing is performed once. The probe parameters are fixed as follows: layer_group=M1-3; layer_adj=0.00; cap_adj=0.00; layer_window=(0,0); congestion_iterations=25; critical_nets_percentage=0; region_enable=0.

[0167] The rules for extracting detection indicators are as follows:

[0168] 1. The priority of metrics is as follows: native metric files take precedence, followed by the statistical results of the global routing guide file, and finally the parsing results of log files and report files. In this embodiment, the native metric file can be named pyapi_native_metrics.json, and the global routing guide file can be named route.guide.

[0169] 2. Essential basic metrics include: final_overflow, wirelength, total number of vias, runtime_s, and number of routed nets.

[0170] 3. The density statistics of the global routing bootstrap file adopt a unified "rectangle + layer" approach. A density grid is constructed for each layer, and the hotspot ratio (hot_ratio) and congestion dispersion (cv) are calculated. The hotspot criterion is that the grid count is not less than the 95th percentile of that layer. In this embodiment, the hotspot binning step size is 200 DBU, and the density statistics are selected from the first four layers. A unified preprocessing result with a layer-by-layer index can be established for the global routing bootstrap file first, and then an event-scanning aggregation method can be used to calculate the density grid to reduce the statistical overhead in large-scale designs.

[0171] 4. Extract the following from the global routing guidance file statistics: maximum hotspot ratio of each layer (hot_ratio_max), maximum congestion dispersion of each layer (cv_max), mean hotspot ratio of the top several layers (hot_topk_mean), standard deviation of hotspot ratio of the top several layers (hot_topk_std), mean congestion dispersion of the top several layers (cv_topk_mean), standard deviation of congestion dispersion of the top several layers (cv_topk_std), percentage of highest density grid (top1_bin_ratio), percentage of top three density grids (top3_bin_ratio), and density distribution (bin_entropy).

[0172] 5. Extract the following metrics from the Python API's native metrics: total number of pins (db_pin_count), total number of nets (db_net_count), number of signal nets (db_signal_net_count), and mean net degree (net_degree_mean).

[0173] The 95th percentile of network degree (net_degree_p95), the maximum network degree (net_degree_max), and hotspot group information (hotspot_groups).

[0174] 6. For the three layer groups M1-3, M4-6, and M7-9, respectively, calculate the number of hotspot candidate networks and the ratio of hotspot bounding box area to chip area. The number of hotspot candidate networks is normalized by dividing by 100; the hotspot bounding box area ratio is directly obtained from bbox_area / die_area, and the maximum value among the three layer groups is taken as the maximum hotspot bounding box area ratio (hotspot_bbox_area_ratio_max).

[0175] In this embodiment, the hotspot group information is retained during the probe routing phase for use in state construction and regional resource adjustment derivation; the formal global routing phase only retains the hotspot ratio, congestion dispersion and related statistics, thereby avoiding repeated calculation of hotspot candidate information that does not participate in the current decision.

[0176] Let the total number of reference design objects be Then the dimension of the state-aware feature vector is Defined as:

[0177] =[1,wl_norm,via_norm,overflow0,hot0,cv0,rt0_norm,top1_bin_ratio,top3_bin_ratio,bin_entropy,hot_topk_mean,hot_topk _std,cv_topk_mean,cv_topk_std,db_pin_norm,net_degree_mean_norm,net_degree_p95_norm,net_degree_max_norm,signal_net _ratio,hotspot_net_m13_norm,hotspot_net_m46_norm,hotspot_net_m79_norm,hotspot_bbox_area_ratio_max,prev_reward,prev_delta_hot,prev_delta_cv,benchmark_one_hot], where: the first element is fixed at 1 (bias term) to allow the linear model to learn the intercept, the wire length normalization value wl_norm = wirelength_probe / 1e6, and the via normalization value `via_norm=vias_probe / 1e6`, runtime normalization `rt0_norm=runtime_probe / 100`, pin count normalization `db_pin_norm=db_pin_count / 1e6`, mean network degree normalization `net_degree_mean_norm=net_degree_mean / 100`, 95th percentile network degree normalization `net_degree_p95_norm=net_degree_p95 / 100`, maximum network degree normalization `net_degree_max_norm=ne` t_degree_max / 100, signal_net_ratio=db_signal_net_count / db_net_count, is used to characterize the proportion of signal networks in the current design. The normalized values ​​of hotspot candidate networks for the three layer groups, hotspot_net_m13_norm, hotspot_net_m46_norm, and hotspot_net_m79_norm, are the number of hotspot candidate networks for the three layer groups divided by 100. benchmark_one_hot is the one-hot encoding vector of the benchmark design object, and missing values ​​are uniformly set to 0.

[0178] The above fields constitute the state-aware feature vector of this embodiment; without changing the state construction concept of "detection index + historical feedback + benchmark design object identifier", the number of fields and the specific normalization method can be increased or decreased.

[0179] Step 4: Construction of the training action pool, reinforcement learning decision-making, and derivation of regional resource adjustments.

[0180] Combination Figure 4 The set of safety factor values ​​obtained through effect screening is used to construct a complete action pool, and action decisions are made using a combination of LinUCB and controlled random exploration. During training initialization, for each candidate complete action... Initialize the inverse covariance matrix and return vector ,in for Regularization coefficient (taken in this embodiment) ), for The identity matrix. Subsequent policy updates will use the Sherman-Morrison formula for online incremental updates. :

[0181]

[0182] And update the reward vector simultaneously. ,in This is the learning signal for this round. The action parameters are... The calculation yielded the result. This incremental update method avoids recalculating the matrix inverse in each round, ensuring the computational efficiency of online learning.

[0183] This implementation learns only four factors: group, adj, cap_adj, and layer_window. The number of congestion iterations (congestion_iterations) for formal global routing is fixed at 60, and the critical nets percentage (critical_nets_percentage) is fixed at 0, no longer considered as independent learning factors. The training action pool is composed of the Cartesian product of the four factors; if the layer_window of an action does not overlap with the group, the action is directly discarded during the construction phase. A separate training action pool is maintained for each baseline design object; in multi-baseline design object training scenarios, the action pool switches with the switching of baseline design objects.

[0184] In this embodiment, the number of pre-coverage rounds is calculated. Time to take , When calculating the action score, let .

[0185] Before formal execution, two types of deduplication filtering are performed: equivalence mapping filtering: known equivalent factor values ​​are folded into canonical values; context combination equivalence filtering: if the current action is only equivalent to an existing signature on a single independent factor, it is directly replaced with a known canonical action.

[0186] Regional resource adjustment: If the regional capability verification fails, then region_enable=0; if adj=0.00, then region_enable=0; prioritize reading the current layer group bounding box from the hotspot group information in the native index file; if the native bounding box is missing, then infer from the global routing guide file route.guide generated by probe routing according to the current layer group; if there is no hotspot bounding box in the current layer group, then region_enable=0; otherwise, generate the regional adjustment magnitude region_adj=clamp(adj×region_adj_scale,0,0.95), in this embodiment, region_adj_scale=1.0.

[0187] During training, if a benchmark design object first appears as a valid sample in a specific group and layer_window subspace and the corresponding layer group has no hotspot bounding box, then the subspace is marked as a collapsed subspace. Subsequently, candidate actions in the same subspace will have their adj directly collapsed to 0.00 before execution to avoid repeatedly applying layer resource adjustments to the non-hotspot layer group.

[0188] In this implementation, the formal training, additional confirmation, and generation of the optimal standard result all use the complete global route. The region resource adjustment parameter region_spec is only used for local resource adjustment and does not change the scope of the complete route solution.

[0189] Step 5: Perform formal global wiring, multi-target rewards, and update the gating and invalid action log.

[0190] After the formal global wiring is completed, the reward is calculated based on the detection results, the formal results, and the neutral global baseline, and the model update is controlled.

[0191] In this embodiment, if the total number of signal networks is available, the number of deployed networks (routed_net_count) must be no less than 0.85 times the total number of signal networks. In this embodiment, if the ratio of the line length or via ratio of the formal global routing result to the detection result is less than 0.50, it is determined to be an abnormal collapse sample, and no policy update is performed.

[0192] Therefore, in this embodiment, a valid sample must at least meet the following conditions: the formal global cabling process is successfully executed, the route integrity flag is valid, and the formal result does not show any abnormal collapse relative to the probe result; for samples that do not meet the above conditions, they are uniformly recorded in the invalid action record table, and no policy update is performed.

[0193] Let the final global routing result be `final_t` and the probe result be `probe_t`, then the relative change is defined as follows, where the Via term is defined separately according to the Via reference value:

[0194] ΔWL_t=(WL(final_t)-WL(probe_t)) / max(|WL(probe_t)|,1)

[0195] ΔVia_t=(Via(final_t)-Via_ref) / max(|Via_ref|,1)

[0196] ΔH_t=(H(final_t)-H(probe_t)) / max(|H(probe_t)|,1e-6)

[0197] ΔC_t=(C(final_t)-C(probe_t)) / max(|C(probe_t)|,1e-6)

[0198] ΔT_t=(T(final_t)-T(probe_t)) / max(|T(probe_t)|,1e-3)

[0199] Via_ref is preferentially taken from the Via of the neutral global baseline. If the baseline Via is missing, it will fall back to take the Via of the probe wiring.

[0200] In this embodiment, the reward calculation is performed according to the following priority:

[0201] If route integrity fails, the reward value reward_t = -1.0e4, denoted as invalid_metrics; if final_overflow is missing, reward_t = -1.0e4, denoted as missing_overflow; if final_overflow > 0, reward_t = -1.0e4 × final_overflow, denoted as overflow_penalty; if ρ_t > 1.5, reward_t = -1.0e4 × ρ_t, denoted as via_guardrail; in other cases, the relative improvement reward relative_reward is calculated according to the reward calculation formula.

[0202] In this embodiment, the multi-objective loss weight is taken as: Overflow penalty coefficient. , , , , , Via soft penalty initial ratio Via hard guardrail ratio Via soft penalty weight .

[0203] like Then, extra_via_loss is calculated according to the Via soft penalty term formula and added to the total loss to avoid actions relying too much on Via to improve hotspots.

[0204] In this embodiment, reinforcement learning updates use a learned signal relative to a neutral global baseline. Defined as:

[0205]

[0206] In this embodiment, the method is used only if the following conditions are met simultaneously. Update LinUCB (Linear Upper Confidence Bound Policy): Valid sample flag `valid_sample=1`, reward source `reward_source` does not belong to invalid metrics or `via_guardrail`, and the current sample does not match an existing duplicate signature. Here, `valid_sample=1` already includes conditions such as successful formal global cabling execution, valid route integrity, and no abnormal collapse. In this embodiment, if `final_rc!=0`, the sample will typically not satisfy `valid_sample=1`.

[0207] For rounds that do not meet the above update conditions, they are uniformly recorded in the invalid action record table, and the subsequent processing flag is set to "no update". Furthermore, if reward_source is overflow_penalty, via_guardrail, invalid_metrics, or the global router returns a non-zero exit code (final_rc != 0), the corresponding action is recorded in the invalid action record table, and the subsequent processing flag is further set to "filtering and avoidance" for subsequent candidate filtering and avoidance. overflow_penalty is used to identify actions that incur overflow penalties; such actions are recorded in the invalid action record table, but can still participate in the current round if they still satisfy valid_sample=1 and do not hit duplicate result signatures. renew.

[0208] It should be noted that this implementation involves two evaluation values ​​with different uses: one is the reward value, used for early stopping judgment and training process recording within the scheduling module; the other is the policy reward value, which is based on the learning signal. The resulting single-sample policy evaluation values ​​are used for the actual updating of LinUCB policy parameters and the statistics of candidate actions. The selection criteria for additional confirmation and standardization of the optimal results are primarily based on the policy evaluation mean (policy_reward_mean), rather than the statistical mean of the reward.

[0209] Step 6: Deduplication of result signatures, append confirmation, and dynamic early stop.

[0210] To avoid training contaminated by duplicate samples with "different parameters and the same result" and to ensure the reproducibility of the final conclusion, this implementation introduces signature deduplication, additional confirmation, and dynamic early stopping mechanisms in the training closed loop.

[0211] Combination Figure 5 Deduplication of signatures: Each official result generates a signature based on the signature formula in step 2. The signature fields are fixed as: final_overflow, wirelength, vias, hot_ratio_max, and cv_max; if the signature of a certain official result... If a result signature has already appeared, the current sample is recorded as a duplicate sample (i.e., a sample whose signature has already appeared) and is not counted again in the effective learning update. If there is a historical record of "changing only one factor but obtaining the same signature" within the same baseline design object, then a context combination equivalence rule is established, and folding is performed directly before subsequent selection actions.

[0212] Additional Confirmation: Let the number of additional confirmation candidates be K_conf. After training, from all the specification representative actions corresponding to the unique result signature sets, first sort them in ascending order of physical quality priority (hotspot ratio → congestion dispersion → line length → number of vias → runtime), then calculate the action signature fingerprint of the sorted results (i.e., group and summarize all valid result signatures corresponding to the action by the baseline design object), remove duplicate actions with the same fingerprint, and select the top K_conf candidates for additional confirmation; in this embodiment, K_conf=3. Each candidate performs a complete formal global routing twice for each baseline design object.

[0213] After the additional confirmation is completed, a summary confirmation result is generated. The additional confirmation result is then combined with the results of the same action from the training phase, and the optimal specification result is determined according to the following sorting criteria: First, sorted in descending order by the policy reward mean (policy_reward_mean); second, sorted in ascending order by the hotspot ratio (hot_ratio_mean); then, sorted in ascending order by the congestion dispersion mean, bus length mean, via count mean, and runtime mean; finally, ranked in ties by the effective sample rate and total sample count. Therefore, the primary criterion for determining the optimal specification result is the policy reward mean (policy_reward_mean), followed by physical quality indicators for ranking.

[0214] Let the nearest The most frequently appearing in the wheel window The signature set is Then the stability coverage is defined as:

[0215]

[0216] The dynamic early stopping conditions are: a minimum of 20 training rounds; and no substantial improvement exceeding 0.01 for 12 consecutive rounds. The method for determining substantial improvement is as follows: Take the valid result row of the current round and compare it with the historical best valid result row in order of priority: hotspot ratio → congestion dispersion → line length → number of vias → running time. Only when the value of the current row at a certain level is lower than the corresponding value of the historical best row by more than min_gain (0.01 in this embodiment) is it considered a substantial improvement; the coverage rate of the first two signatures within the last 8 rounds... When all of the above conditions are met, the subsequent training is terminated and the process proceeds to the stage of additional confirmation and standardization of optimal result generation.

[0217] Step 7: Implement based on Python program interface

[0218] In this embodiment, a persistent execution structure based on a Python program interface is adopted to reduce the time overhead caused by repeatedly starting the global router and improve the consistency of native metric extraction. The task execution sequence can be found in [reference needed]. Figure 6 At the start of the experiment, the scheduling module performs a self-check to confirm that the Python program interface execution unit, OpenROAD commands, and input baseline design objects are accessible. After the self-check passes, the persistent Python session process is started. Once started, the persistent execution process waits for subsequent tasks within the same process. One or more session pools are created for each baseline design object; tasks without regional resource adjustments enter the plain session pool, while tasks with regional resource adjustments enter the corresponding regional session pool.

[0219] In this embodiment, when a baseline design object enters a session pool for the first time, the design object can be cold-loaded by inputting a design file, and a design snapshot corresponding to the baseline design object can be written. If a new session pool with the same baseline design object is needed later, the design object can be restored from the design snapshot.

[0220] For an established session pool, subsequent tasks will no longer repeatedly re-read the design snapshot. Instead, they will call the global router's internal cleanup interface to clear the previous round of routing state, restore the default layer resource state, and reapply the area resource adjustment parameters as needed, and then execute a new complete global route.

[0221] In this embodiment, the global router internal cleanup interface can be implemented by the global router cleanup command GlobalRouter.clear() exposed by the Python program interface; when this interface is unavailable, this hot reuse implementation method is not enabled.

[0222] The scheduling module and the execution process communicate using a standard input / output JSON message protocol. The scheduling module sends a structured task payload (type=job) to the execution process: a baseline design object identifier, input file path, output directory, a set of GRT_* environment variables, and task stage identifiers (baseline / probe / screening / region_smoke / final / confirm / best_full). Upon receiving the task, the execution process returns start and end markers, task exit codes, and runtime. When the scheduling module detects an unexpected interruption of the communication channel, it checks whether the result file in the output directory has been completely written; if it has, it accepts the results of that round and restarts the session to continue subsequent tasks.

[0223] The execution process completes design loading, layer table reading, native database statistics, and result writing at the Python program interface layer; when global routing is required, it executes global_route via the Tcl interface. The execution process outputs two types of key documentation files for each task: a native metrics file, recording native database metrics, hotspot group information, and metric source descriptions; and an execution manifest file, recording the design loading mode, session reuse information, actual application parameters, and output files.

[0224] In this embodiment, the native metrics file and execution manifest file may also record the design loading mode, reset strategy, session pool type, number of hot session reuses, and route reset commands, which are used to indicate whether the current round of tasks is executed by creating a new session pool or by hot reuse.

[0225] In implementation scenarios where retaining round-level state files is not required, it is not necessary to write a state file for each round; in implementation scenarios where retaining debug logs is not required, debug files are not necessary, thus forming a concise result set suitable for engineering deployment and experimental reproduction.

[0226] In this embodiment, the scheduling module maintains a probe cache for each baseline design object. The cache key is calculated jointly by the fingerprint of the baseline design object's input file (file path, file size, and modification timestamp), probe parameter configuration, and execution unit code fingerprint. If the cache key of the current training round matches the cached probe result, the cached probe metric and the original metric file are directly reused without re-executing probe routing. This mechanism can significantly reduce the number of repeated probe routing calls in multiple training rounds, reduce the total training time, and ensure that cache hits only take effect when the input file remains unchanged and the execution environment is consistent.

[0227] Step 8: Output the organization and implementation results

[0228] The following provides the implementation results of this invention on the benchmark design objects ispd18_test1, ispd18_test4, and ispd18_test7 to illustrate that this implementation can form a complete process from screening, training, validation to generating the optimal specification result. The output of the effect screening includes an effect screening result file, an effect screening summary file, and a recommendation factor file. In this embodiment, the final safety factor values ​​retained by the effect screening are: group={M4-6,M1-3}, adj={0.20,0.00,0.10}, cap_adj={0.00,0.15}, layer_window={(0,0),(1,6)}.

[0229] During the training phase, after completing 30 rounds with these parameter configurations, the system enters a stable convergence region and is terminated by dynamic early stopping. The main training record is written to the training result record file, and a training summary file is output synchronously at the end of the experiment. In the supplementary confirmation phase, the representative actions corresponding to the first K_conf=3 unique result signature sets are executed twice, generating supplementary confirmation result files and supplementary confirmation summary files. In this embodiment, the optimal specification result is: layer=M1-3@0.20|iters=60|crit=0|cap=0.10|window=0-0|region=1. Representative results are shown in Table 3.

[0230] Table 3. Representative results after 30 rounds

[0231]

[0232] After completing 44 rounds of training, the system entered a stable convergence region and was terminated by dynamic early stopping (the reason for stopping was the exhaustion of consecutive patient rounds and the coverage rate of the first two signatures in the last 8 rounds reached 0.875). Its optimal result is: layer=M1-3@0.20|iters=60|crit=0|cap=0.15|window=0-0|region=1, and its representative results are shown in Table 4.

[0233] Table 4. Representative results after 44 rounds

[0234]

[0235] After 48 rounds of training, the system entered a stable convergence region and was terminated by dynamic early stopping. The optimal result was: layer=M1-3@0.20|iters=60|crit=0|cap=0.10|window=1-6|region=1. The representative results are shown in Table 5.

[0236] Table 5. Representative results after 48 rounds

[0237]

[0238] In the three embodiments with 30, 44, and 48 iterations respectively, the action space size is approximately 24 candidate complete actions, and the pre-coverage rounds are 9, 9, and 7 rounds respectively. The effective sample rate is 1.0 in all three embodiments, and no actions were recorded in the invalid action log during training. The task specification document of the persistent execution unit indicates that when the baseline design object first enters a session pool, it can be restored through a design snapshot, while most subsequent tasks use hot_reset to reuse the session; this reduces the cost of repeated startups and reloading while ensuring the consistency of the complete global routing path.

[0239] Compared to the single-test results of their respective neutral global baselines, the additional confirmation mean values ​​corresponding to the above representative embodiments can be further explained as follows: In the embodiment with 30 iterations, the optimal canonical result, while maintaining Overflow=0 and an effective sample rate of 1.0, reduces the hotspot ratio relative to the baseline by approximately 47.67% and the congestion dispersion relative to the baseline by approximately 39.13%. This result indicates that the present invention can simultaneously improve both the hotspot ratio and congestion dispersion on a large-scale benchmark design object. In the embodiment with 48 iterations, the optimal canonical result, also under the conditions of Overflow=0 and an effective sample rate of 1.0, reduces the hotspot ratio relative to the baseline by approximately 22.38% and the congestion dispersion relative to the baseline by approximately 19.50%, indicating that the present invention can achieve simultaneous improvement of both the hotspot ratio and congestion dispersion on a large-scale benchmark design object.

[0240] The final output includes the standard optimal formal global routing guide file, heatmap, standard optimal result summary and necessary explanatory documents, forming a structured result archive suitable for engineering deployment, experimental reproduction and solution auditing.

[0241] This invention takes a design object with a determined layout as input and only performs adaptive parameter optimization in the backend global routing stage, without involving the layout solution process. The reinforcement learning mentioned in this invention includes online decision-making methods based on state-reward feedback; in this embodiment, LinUCB is used as the state-aware action selection strategy.

[0242] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and incorporate common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A chip back-end global routing method based on state-aware reinforcement learning, characterized in that, include: Step S1: Based on the process / library information or design layout information of one or more baseline design objects, design input files, establish a standardized global routing operation environment, construct neutral global baseline results, and obtain the standardized global routing operation environment, the list of baseline design objects, and the neutral global baseline index corresponding to each baseline design object. Step S2: Based on the candidate factor set, regional coordinate pattern, and neutral global baseline index obtained in step S1, perform effect screening and regional capability verification to obtain the safety factor value set, training action space, and regional capability mapping of each benchmark design object. Step S3: Based on the standardized global cabling operating environment and configured fixed detection parameters obtained in step S1, perform detection cabling to obtain the detection cabling results and their structured detection indicators. Step S4: Based on the structured detection indicators, historical rewards and historical improvement amounts, and baseline design object identification information obtained in step S3, construct the state-aware feature vector to obtain the state-aware feature vector. Step S5: Based on the state-aware feature vector obtained in step S4, the set of security factor values ​​obtained in step S2, the training action space, and the regional capability mapping, perform reinforcement learning decision-making and derive regional resource adjustment parameters to obtain the formal global wiring parameter decision results and regional resource adjustment parameters. Step S6: Based on the formal global cabling parameter decision results and regional resource adjustment parameters obtained in step S5, perform formal global cabling to obtain formal global cabling results, structured evaluation indicators and result signatures. Step S7: Based on the probe routing results and structured probe metrics obtained in Step S3, the formal global routing metrics obtained in Step S6, the neutral global baseline metrics obtained in Step S1, and historical training records, calculate the reward value and perform update gating, equivalent deduplication, and candidate confirmation to obtain the reward value, policy update result, signature deduplication result, confirmation result, and specification optimal result. At the same time, determine whether the set iteration termination condition is met. If it is met, proceed to Step S8. If it is not met, repeat Steps S4-S7 to continue reinforcement learning training. Step S8: Based on the data from the entire process from steps S1 to S7, output structured results for archiving, and obtain traceable and reproducible global routing optimization results.

2. The chip back-end global routing method based on state-aware reinforcement learning according to claim 1, characterized in that, The neutral global baseline metrics include total overflow, bus length, total number of vias, runtime, hotspot ratio, and congestion dispersion.

3. The chip back-end global routing method based on state-aware reinforcement learning according to claim 1, characterized in that, Step S2 specifically includes: Step S2-1: The candidate factors are designed to include layer grouping, layer resource adjustment range, global capacity adjustment and layer window. Each candidate factor has a finite number of predefined candidate values. The effect screening is performed on each benchmark design object in a single factorial test manner to obtain the set of safety factor values ​​and the training action space. Step S2-2: Set retention upper limits for each factor. While satisfying the retention upper limit constraints, retain the default values ​​of each factor so that the training action space always contains a stable backoff point. Step S2-3: Perform baseline design object-level capability verification for regional resource adjustment: Apply regional resource adjustment parameters to each baseline design object individually and perform test routing to determine whether the global router can normally accept and execute regional resource adjustment commands; Baseline design objects that pass verification are recorded as having available regional capabilities, and those that fail verification are recorded as having unavailable regional capabilities, thus forming a regional capability mapping.

4. The chip back-end global routing method based on state-aware reinforcement learning according to claim 3, characterized in that, Step S2-1 specifically includes: Step S2-1-1: Based on the total via count ratio of the neutral global baseline, high-risk values ​​are screened out, and the total via count ratio is calculated. for: in, This represents the total number of vias in the selected sample. The total number of vias for the neutral global baseline corresponding to the reference design object. To prevent the total number of vias from becoming zero constant; like If the value exceeds the preset safety threshold, the candidate factor is determined to be a high-risk value and is filtered out. Step S2-1-2: Sign the calculation results of the screened candidate factors. for: in, For hash mapping functions, , , , , These are the total overflow of formal global routing, bus length, total number of vias, hotspot ratio, and congestion dispersion, respectively. Deduplication is performed on equivalent values ​​that result in the same signature within the same baseline design object, and the remaining values ​​are recorded as filtered equivalence mappings to obtain a set of safety factor values. The safety factor values ​​are sequentially processed to form a Cartesian product to create the training action space.

5. The chip back-end global routing method based on state-aware reinforcement learning according to claim 1, characterized in that, The detection metrics in step S3 include total overflow, bus length, total number of vias, running time, hotspot ratio, congestion dispersion, layer distribution, number of candidate networks for hotspot layer groups, hotspot bounding box area ratio, and database statistics. The hotspot ratio is determined based on the 95th percentile value of the density grid. In step S4, the state-aware feature vector is composed of structured detection indicators, historical memory items, and baseline design object hotspot identifiers concatenated in a predetermined field order. The historical memory items include the previous round reward value and the improvement amount of the previous round hotspot ratio and congestion dispersion. The baseline design object hotspot identifier is used to distinguish the congestion characteristics of different design objects.

6. The chip back-end global routing method based on state-aware reinforcement learning according to claim 1, characterized in that, The reinforcement learning decision in step S5 is as follows: Let the candidate complete actions in the training action pool be... For each candidate action, calculate the action score using the following formula: in, For action The corresponding linear parameters, For action The corresponding covariance matrix, Representation matrix The inverse matrix, For the first Exploration coefficient, This is a state-aware feature vector. express transpose; Let the total number of training movements be The training budget is Determine the number of pre-coverage rounds for: in, To minimize the number of actions covered, The proportion of pre-coverage rounds relative to the training budget, To train the total number of movements, For training budget; If the current iteration number Forcefully cover unsampled actions according to the pre-coverage sequence; if And the probability of a random number is less than the probability of random exploration. If the result is positive, then random exploration will be performed; otherwise, a search based on the given conditions will be performed. The highest-scoring candidate move; the probability of the random exploration is: Before formally executing the selected action, the equivalent factor value is first folded into the standard value. The equivalent factor is: if a factor value of the current benchmark design object produces the same result signature as a factor value obtained in the effect screening stage of step S2, then the factor is considered to be an equivalent factor. Then, context combination equivalence filtering is performed. That is, if the current action is only equivalent to the existing signature on an independent factor, it is directly replaced with the known standard action. The specific parameters for adjusting derived region resources in step S5 include: For the selected action involving layer grouping and layer resource adjustment, and combining the regional capability mapping obtained in step S2 and the detection index information obtained in step S3, regional resource adjustment parameters are derived. If the regional capability verification of the baseline design object fails, or if the current layer group does not have a hotspot bounding box, regional resource adjustment is not enabled; otherwise, the regional resource adjustment magnitude is calculated as follows: in, Adjust the layer resource range for the selected action. The `clamp` function adjusts the scaling factor for a region, fixing a specified value to a specified minimum and maximum range.

7. The chip back-end global routing method based on state-aware reinforcement learning according to claim 1, characterized in that, The calculation of the reward value in step S7 specifically includes: S7-1, Calculate the improvement in hotspot ratio, congestion dispersion, wire length, and runtime relative to probe routing in formal global routing. for: in, For the first Wheel detection wiring stage index values, For the first Round of formal global cabling index values, As an indicator The zero-prevention constant; For the total number of vias index, calculate its relative improvement. for: in, The total number of vias is a reference value, which is taken as the total number of vias at the neutral global baseline of the corresponding reference design object. For the first Total number of vias in the formal global routing round. This is a zero-prevention constant for the total number of vias; S7-2, Calculate the... The ratio of the total number of vias in the formal global routing result relative to the neutral global baseline ,when At the set soft penalty starting threshold With hard guardrail threshold In between, calculate the soft penalty term for the total number of through holes. ; in, It is a stable constant in the denominator of the ratio; S7-3, based on the improvement amount, the total number of vias ratio, and the soft penalty term, calculate the first... Round reward value : in, For the first Round-route integrity flag: 1 for complete, 0 for incomplete; For the first Global routing overflow in round 1; This is the total overflow threshold; , , , , These represent the improvements in hotspot ratio, congestion dispersion, line length, total number of vias, and runtime, respectively. These are the corresponding weighting coefficients.

8. The chip back-end global routing method based on state-aware reinforcement learning according to claim 1, characterized in that, The equivalent deduplication and candidate confirmation in step S7 are as follows: First, candidate actions with valid samples are selected from the action summary results obtained during the training phase, and then initially sorted according to the priority order of hot spot ratio, congestion dispersion, line length, number of vias and running time. For repeated actions with the same result signature set on each benchmark design object, only one standard representative action is retained; then a preset number of candidates are selected from the standard representative actions, and for each candidate, the complete formal global routing is repeatedly performed multiple times on each benchmark design object to obtain the candidate confirmation result. Subsequently, the candidate actions with valid samples from the training phase are combined and statistically analyzed with the candidate confirmation results. The mean policy evaluation, the proportion of valid samples, and the total number of samples for each candidate action are calculated. The candidate actions are sorted in descending order according to the mean policy evaluation. When the mean policy evaluations are the same, they are ranked and decided in turn according to the hotspot ratio, congestion dispersion, line length, number of vias, running time, proportion of valid samples, and total number of samples to determine the optimal result. The mean policy evaluation is the statistical mean of the learning signal corresponding to the same candidate action.

9. A chip back-end global routing method based on state-aware reinforcement learning according to claim 1, characterized in that, In step S7, the gating is updated using a learned signal relative to the neutral global baseline. Update action parameters, the learning signal being: in, These represent the improvements of each metric in the formal global routing results relative to the neutral global baseline. These are the weighting coefficients for the corresponding learning signal.

10. A chip back-end global routing method based on state-aware reinforcement learning according to claim 1, characterized in that, The specific steps in step S7 to determine whether the set iteration termination condition is met are as follows: Let the one that appears most frequently in the most recent W rounds of the window be the first one. The result signature set is Calculate stability coverage: Training is terminated when the number of training rounds reaches the set minimum number of training rounds, there is no substantial improvement for several consecutive rounds, and the stability coverage reaches a preset threshold. The method for judging that there is no substantial improvement for several consecutive rounds is as follows: after the most recent best effective result is updated, within a certain number of consecutive training rounds, the effective formal global routing result of the current round is compared with the best effective result in the priority order of hotspot ratio, congestion dispersion, line length, number of vias, and running time. If the difference between the two results does not exceed the preset minimum improvement ratio threshold, then there is no substantial improvement for several consecutive rounds.

Images