An evaluation system for integrated circuit physical design tools based on benchmark test sets

Abstract

This invention relates to an integrated circuit physical design tool evaluation system based on a benchmark test set, comprising: a test case management module, a tool under test (DUT) management module, and a test execution and reporting module. The test case management module creates new test cases based on user instructions using benchmark examples with standardized naming and data formats, and uploads the new test cases to a benchmark test set library. The DUT management module uploads the device under test (DUT) based on user instructions. The test execution and reporting module, based on user instructions, runs the test cases that pass the standardization check on the DUT to obtain the DUT's output values, and converts the DUT's output values ​​to the output values ​​of the benchmark tool running the test cases using the same standard through an evaluation algorithm, outputting a visual evaluation report of the DUT's performance evaluation values ​​for each indicator. This evaluation system's indicator scoring method is simple and practical, and can quickly evaluate the performance of the physical design tool based on the scores of different indicators.

Description

Technical Field

[0001] This invention belongs to the field of integrated circuit physical design technology, specifically relating to an integrated circuit physical design tool evaluation system based on a benchmark test set. Background Technology

[0002] Integrated circuit (IC) physical design is a crucial component of modern electronic product design and manufacturing, encompassing the entire process from logic design to final physical implementation. With the advancement of Moore's Law, the size of integrated circuits continues to shrink, and their integration density increases, allowing more transistors and other electronic components to be housed on a single chip. This necessitates physical design techniques capable of supporting high-density, high-performance circuit placement and routing. Physical design directly impacts chip performance, including speed, power consumption, and heat dissipation. By optimizing placement, routing, timing, and power distribution, physical design can significantly improve chip performance metrics while reducing power consumption and thermal effects. As integrated circuit technology continues to advance, chip design complexity and integration density are rapidly increasing, demanding higher performance and accuracy from physical design tools. The integrated circuit design flow comprises multiple stages, from logic design to final physical placement, each requiring efficient design tools. The physical design stage is particularly critical, directly determining chip performance, power consumption, and cost.

[0003] Currently, in the field of integrated circuit design, physical design tools are mainly used to convert logic circuit diagrams into physical layouts, including layout planning, routing, and optimization. The quality of these tools directly affects the yield and market competitiveness of the final product. Therefore, developing high-quality physical design tools has become one of the industry's goals. However, existing physical design tool performance evaluation methods have the following shortcomings: 1) Lack of standardization: There is a lack of unified standards in the industry to measure the performance of different physical design tools, making it difficult for users to objectively compare the advantages and disadvantages of different tools. 2) Incomplete evaluation: Existing evaluation methods often focus on a single indicator (such as layout area or routing quality) while ignoring other important factors. 3) Insufficient benchmark testing: Existing benchmarks may not fully reflect the various challenges in real-world application scenarios, and due to the lack of continuous maintenance and updating mechanisms, some benchmark sets become outdated and cannot accurately reflect the needs of modern designs. 4) Cumbersome evaluation systems: Most current evaluation systems rely on manual evaluation, which is complex, inefficient, and cannot quickly evaluate tools and provide evaluation results. Furthermore, the presentation of evaluation results varies widely. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, this invention provides an integrated circuit physical design tool evaluation system based on a benchmark test set. The technical problem to be solved by this invention is achieved through the following technical solution:

[0005] This invention provides an integrated circuit physical design tool evaluation system based on a benchmark test set, comprising: a test case management module, a test tool management module, and a test execution and reporting module, wherein...

[0006] The test case management module is used to create new test cases based on user instructions, using cleaned benchmark examples with unified naming and data format under different data categories of integrated circuits, upload the new test cases to the benchmark test set library, and edit, delete, classify, copy and query the test cases in the benchmark test set library to update the benchmark test set library.

[0007] The device under test (DUT) management module is used to upload the DUT according to user instructions;

[0008] The test execution and reporting module is used to select a set of test cases from the benchmark test set library according to user instructions and perform standardization checks. After running the test cases that have passed the standardization checks on the device under test, the output value of the device under test is obtained. The output value of the device under test and the output value of the benchmark tool running the test cases are converted to the same standard using the evaluation algorithm. The module then outputs the performance evaluation value of each indicator of the device under test and a visual evaluation report.

[0009] In one embodiment of the present invention, the method for cleaning the data content of a benchmark example includes: extracting the data features of the circuit in each benchmark example, modifying the circuit in the benchmark example according to the data features and the needs of the business scenario, and obtaining the cleaned benchmark example.

[0010] In one embodiment of the present invention, the naming convention for benchmark examples in the benchmark test set library is: Data Category_Service Segment_Circuit Size_DESIGN Name_Number; wherein,

[0011] The data classification includes any one of the following: process model, generalized analog compound, synthesis, physical design, wafer manufacturing, packaging, standards and substrates, and signing off.

[0012] The business segment is a process under the data category;

[0013] The circuit scale refers to the size of the unit components in the circuit design;

[0014] DESIGN is the name of the circuit design.

[0015] In one embodiment of the present invention, the data specification format of the benchmark examples in the benchmark test set library is implemented through a hierarchical directory specification format, wherein,

[0016] The first-level directory includes use case naming, and the second-level directory includes design documents, relevant scripts during the benchmarking tool's operation, the benchmarking tool's operation results, and explanatory documents. The design documents include input files and common databases involved in the operation process.

[0017] In one embodiment of the present invention, uploading newly created test cases to a benchmark test set library includes:

[0018] Submit the newly created use case;

[0019] The newly created use case is run using the benchmarking tool to verify its usability.

[0020] If the newly created test case is verified as unusable, the newly created test case is edited and modified and temporarily saved for resubmission; if the newly created test case is verified as usable, the newly created test case is reviewed for inclusion in the database based on whether the test case content is compliant and whether the test case is valuable.

[0021] If the newly created test case is deemed unsuitable for inclusion in the database, it is edited, modified, and temporarily saved for resubmission. If the newly created test case is deemed acceptable for inclusion in the database, it is uploaded to the benchmark test suite database.

[0022] In one embodiment of the present invention, the device under test includes any one of components, script code, and third-party tools.

[0023] In one embodiment of the present invention, the output value of the device under test and the output value of the benchmark tool running the evaluation test case are converted to the same standard using an evaluation algorithm, including:

[0024] For each index, the absolute difference between the output value of the test piece and the output value of the benchmark tool is calculated to obtain the absolute error of each index;

[0025] Based on the absolute error of each indicator, calculate the proportion of the number of test cases for each difference category to the total number of test cases for the current indicator, and use the proportion as the weight to calculate the weighted average of the difference category scores corresponding to each difference category, so as to obtain the weighted average of each indicator.

[0026] Substitute the weighted average of each indicator into the indicator comparison formula to calculate the performance evaluation value of each indicator in the tested component.

[0027] In one embodiment of the present invention, the difference classification and its corresponding difference classification score are as follows: the difference classification score is 100 when the absolute error is <0.00001, the difference classification score is 95 when the absolute error is <=0.0001, the difference classification score is 90 when the absolute error is <=0.001, the difference classification score is 85 when the absolute error is <=0.01, the difference classification score is 80 when the absolute error is <=0.1, the difference classification score is 70 when the absolute error is <=1, and the difference classification score is 60 when the absolute error is <=1.

[0028] In one embodiment of the present invention, when the indicator is a physical indicator, the indicator comparison formula is: (1+(golden-test) / golden)*100;

[0029] When the indicator is a time-series indicator, the indicator comparison formula is: (1+(test-golden) / period)*100;

[0030] When the indicator is a period indicator, the indicator comparison formula is: (1+(test-golden) / (period*golden_NVP))*100;

[0031] Wherein, golden is the output value of the benchmark tool, test is the weighted average of the metrics of the device under test, Period is the clock cycle, and golden_nvp is the number of violation paths.

[0032] In one embodiment of the present invention,

[0033] When the output value of the index comparison formula is greater than 200, the performance evaluation value of the current index of the tested component is 200.

[0034] When the output value of the index comparison formula is less than 100, the performance evaluation value of the current index of the tested component is -100.

[0035] When the output value of the benchmarking tool is 0 and when processing the WNS, TNS, DRC_VIO, and Overflow metrics, if the output value of the metric comparison formula is 0, then the performance evaluation value of the current metric of the device under test is 100. If the output of the metric comparison formula is not 0, then the performance evaluation value of the current metric of the device under test is 100-test.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] The evaluation system of this invention uses cleaned benchmark examples with unified naming and data format to create new test cases and upload them to the benchmark test set library, thus improving the benchmark of the benchmark test set library, enabling continuous updates and maintenance, and constantly expanding the benchmark library to make the test benchmarks increasingly complete and rich. The output values ​​of the test cases running on the device under test (DUT) and the output values ​​of the benchmark test cases running on the benchmark tool are converted to the same standard through the evaluation algorithm. For different DUTs, only appropriate test cases need to be selected from the benchmark test set library for evaluation, and then the evaluation results are compared with the output values ​​of the benchmark tool. The index scoring method is simple and practical, and can quickly, conveniently and accurately score various indicators of the DUT, helping users to evaluate and select tools, greatly improving evaluation efficiency and reducing labor costs. Therefore, the evaluation system of this invention can quickly evaluate the performance of physical design tools, thereby promoting the improvement of design tools, providing impetus for technological innovation and development in the field of integrated circuit design, and providing strong support for the development of high-performance, low-power integrated products. Attached Figure Description

[0038] Figure 1 A schematic diagram of the structure of an integrated circuit physical design tool evaluation system based on a benchmark test set provided in an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram illustrating the process of uploading newly created test cases to a benchmark test set library, provided as an embodiment of the present invention.

[0040] Figure 3 A framework diagram for use case management provided in embodiments of the present invention;

[0041] Figure 4 This is a flowchart illustrating the placement, cts, and routing of the test piece provided in an embodiment of the present invention. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0043] Example 1

[0044] This invention proposes an integrated circuit physical design tool evaluation system based on a benchmark test set. By running the same benchmark test set on the same platform with different physical design tools, the system comprehensively evaluates the results to obtain the performance evaluation results of the evaluated tools. This provides a comprehensive, professional system that can be continuously updated and quickly evaluated, which can help designers gain an intuitive understanding of the overall performance of the tools, thereby guiding them to improve or select the right tools.

[0045] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of an integrated circuit physical design tool evaluation system based on a benchmark test set, provided as an embodiment of the present invention. The evaluation system includes a test case management module, a test tool management module, and a test execution and reporting module.

[0046] The test case management module is used to create new test cases according to user instructions, using cleaned benchmark examples under different data categories of integrated circuits, and following a unified naming and data specification format. The newly created test cases are uploaded to the benchmark test set library, and the module can edit, delete, classify, copy, and query the test cases in the benchmark test set library to update the benchmark test set library.

[0047] Benchmark sets are crucial for evaluating physics design tools. Currently, publicly available benchmark sets are mainly scattered across open-source communities, examples provided by some domestic EDA competitions, and simple examples released by some enterprises. With the emergence of new technologies and the development of older technologies, existing benchmarks may become outdated or no longer adequately reflect the current technological state. Therefore, benchmarks need continuous updates and expansion to ensure they accurately evaluate the latest technologies and solutions. This embodiment addresses the problem of scattered and unupdable benchmark sets by establishing a unified, standardized benchmark set library.

[0048] Specifically, the first step is to collect publicly available benchmark examples. Benchmark examples related to each stage of physical design, such as Placement, CTS, and Routing, are collected from open-source communities, various domestic Electronic Design Automation (EDA) competitions, and examples from various companies. These benchmark examples are only used for competitions or certain stages and have not been further maintained or utilized. Then, these benchmark examples are collected, cleaned, categorized, and numbered to transform them into standardized and universally applicable examples. Finally, these benchmark examples are uploaded to a benchmark test set library for future use.

[0049] Furthermore, cleaning the benchmark examples includes cleaning the data content, naming conventions, and data format of the benchmark examples.

[0050] The method for cleaning the data content of benchmark examples includes: extracting data features of the circuits in each benchmark example, such as cells, nets, primary I / O, and die area; modifying the circuits in the benchmark examples according to the extracted data features and specific business scenario requirements to make the benchmark examples meet the requirements of the business use scenario, thus obtaining the cleaned benchmark examples. For example, based on the extracted data features and specific business scenario requirements, the shape of the circuits in the benchmark examples can be modified to clean them to meet the requirements of the business use scenario.

[0051] Naming conventions and data format cleaning for benchmark examples refer to standardizing the naming conventions and data formats of benchmark examples.

[0052] The naming convention for benchmark examples in the benchmark set library is: Data Category_Service Segment_Circuit Size_DESIGN Name_Number, for example, PD_PLACE_10K_DES_001; where, Data Category includes any one of Process Model (PM), generalized analog compound, synthesis, Physical Design (PD), wafer fabrication, packaging, standards and substrates, and approval; Service Segment is a process under Data Category, such as the service segment of Physical Design (PD) can be divided into PR / PLACE / CTS / ROUTE, etc.; Circuit Size is the size of the unit component in the circuit design; DESIGN Name is the DESIGN name of the circuit design.

[0053] The data specification format of benchmark examples in the benchmark test set library is implemented through a hierarchical directory specification format. The first-level directory includes test case naming, and the second-level directory includes design files, scripts related to the benchmark tool operation, results of the benchmark tool operation, and README documentation. The design files include input files and common database pdk involved in the operation process.

[0054] Taking PD_CTS_15K_AES_001 as an example, its data format specification is as follows:

[0055]

[0056]

[0057] Among them, design.v is the Verilog netlist file for the design, design.sdc is the timing constraint file for the design, placement.def records the physical information after the design layout planning, the lef directory stores the design process information, design rule information, via information and information of each element in the element library, the lib directory stores the timing logic information of the design, the rlc directory stores the parasitic parameter information extracted from the design, the script directory stores the relevant scripts during the operation, the result directory stores the operation results, and README is the documentation.

[0058] After data cleaning and normalization, benchmark examples are used to create new test cases under different data categories (e.g., process models, generalized simulation compounds, synthesis, physical design, wafer fabrication, packaging, standards and substrates, approval, etc.). These benchmark examples are then uploaded to the benchmark test suite library individually or in batches. The form for creating a new test case includes a test case directory, test case name, test case description, and test case file upload (selecting the benchmark test case data to be used, which can be local benchmark test case data or data from the benchmark test case library). The process for uploading benchmark examples individually or in batches to the benchmark test suite library is as follows: Figure 2 As shown, Figure 2 This is a schematic diagram of the process for uploading newly created test cases to a benchmark test set library, provided by an embodiment of the present invention. The process specifically includes the following steps:

[0059] S1. Submit the newly created test case;

[0060] S2. Run the newly created use case using the benchmarking tool to verify its usability.

[0061] Specifically, the newly created test cases are run using the benchmarking tool to determine if the benchmarking tool can run and output normal results. If it can, the newly created test cases are considered usable and the verification is successful; otherwise, the newly created test cases are considered unusable and the verification fails.

[0062] S3. When a newly created test case is found to be unusable, edit and modify the test case and save it for resubmission. When a newly created test case is found to be usable, review whether the test case can be added to the database based on whether the test case content is compliant and whether the test case is valuable. Compliance refers to whether the test case contains sensitive words or involves red line issues.

[0063] S4. If it is determined that the newly created test case cannot be added to the database, edit and modify the newly created test case and save it for resubmission; if it is determined that the newly created test case can be added to the database, upload the newly created test case to the benchmark test set database.

[0064] Furthermore, the benchmark test set library displays attributes such as test case name, test case status, test case size, test case type, uploader, upload time, scale, technology, update time, usage count, and download count for uploaded examples, allowing users to quickly find the test cases they need. Users can also delete, categorize, copy, query, and edit test cases (editing is only permitted for cases uploaded by the uploader), enabling test case management. Figure 3 As shown, Figure 3 This is a framework diagram for use case management provided in this embodiment of the invention. Uploaded use cases can be modified through editing. When updating a use case, if you want to preserve the old use case, you can upload a new example. New and old use cases are distinguished by number, and the use case description explains that this use case is an update based on a certain use case. To use a specific use case, you can download it using the download function. If you don't want to retain the original use case during an update, you can directly update the use case in the editing function by uploading a new use case package. Through this use case library, use cases can be continuously expanded, updated, and maintained.

[0065] This embodiment improves the benchmark test suite library through the test case management module, enabling it to be continuously updated and maintained, constantly expanding the benchmark library, and making the test benchmark increasingly complete and rich.

[0066] The Tool Under Test (DUT) management module is used to upload DUTs according to user instructions. DUTs include, but are not limited to, components, script code, and third-party tools.

[0067] Specifically, if the component under test is a complete component or script code, users can choose to upload the component to the evaluation platform. If it is a third-party tool, users can choose to install it on the platform and use it directly when needed. When uploading a tool to the platform, users need to fill in the tool type, directory where it is located, name of the component under test, version, description, etc., in the form, and then upload the component under test.

[0068] The test execution and reporting module is used to select a set of test cases from the benchmark test set library according to user instructions and perform standardization checks. After running the test cases that have passed the standardization checks on the device under test, the output value of the device under test is obtained. The output value of the device under test is then converted to the same standard by the benchmark tool's test cases using the same evaluation algorithm. The module outputs the performance evaluation value of each indicator of the device under test and a visual evaluation report.

[0069] Specifically, the test execution and reporting module primarily associates the device under test (DUT) with test cases for testing and outputs reports. First, a new test task is created. Creating a task includes selecting a directory, evaluation phase (choosing placement, CTS, routing, etc.), selecting the DUT (e.g., the tool under test), configuring the environment, and specifying the environment on which to run the test. Once the task is created, it will be displayed on the interface. The task operation bar allows for actions such as running, copying the test task, viewing task details, deleting the task, and viewing the report. Then, automated execution is performed. During automated execution, the required benchmark set is selected from the benchmark set library as the test cases, the execution task is created, the runtime environment information is configured, and the task begins execution. After the task completes, the execution result is displayed: success or failure. Upon success, clicking "View Report" shows the score of the tool under test under that test case or group of test cases, as well as the execution details. If it fails, a failure log is displayed for troubleshooting. After the task completes, users can also rerun the task, delete the task, and view the execution record.

[0070] Furthermore, during automated execution, the test cases are first checked for compliance. If a test case is non-standard, the evaluation system will report an error, indicating that the test case does not conform to the specifications or fails the specification check, and request that it be modified before use. Then, the test cases are run on the device under test (DUT) to perform a usability check: for different types of test cases, different benchmarking tools (generally considered authoritative tools in the industry) are used for pre-evaluation to obtain the output values ​​of the benchmarking tools; on the same platform and system, a set of test cases is selected for the DUT to be evaluated, and the set of test cases is run on the DUT to provide the output values; the output values ​​of the benchmarking tools and the output values ​​of the DUT are converted to the same standard using the same evaluation method to provide the performance evaluation value of each indicator of the DUT.

[0071] Specifically, the process of converting the output value of the device under test (DUT) and the output value of the benchmark tool's test cases to the same standard using the evaluation algorithm is as follows: For each indicator, calculate the absolute difference between the output value of the DUT and the output value of the benchmark tool to obtain the absolute error of each indicator; based on the absolute error of each indicator, calculate the proportion of the number of test cases for each difference category to the total number of test cases for the current indicator, and use the proportion as a weight to perform a weighted average of the difference category scores corresponding to each difference category to obtain the weighted average of each indicator; substitute the weighted average of each indicator into the indicator comparison formula to calculate the performance evaluation value of each indicator in the DUT.

[0072] Specifically, assuming the device under test is m, there are n metrics to focus on, and h test cases.

[0073] First, the device under test (DUT) runs a specific test case to obtain the output value of each metric of the DUT. Then, the benchmark tool runs the same test case to obtain the output value of each metric of the benchmark tool. For each metric, the absolute difference between the output value of the DUT and the output value of the benchmark tool is calculated to obtain the absolute error of each metric.

[0074] Then, combining the absolute error of each indicator, the proportion of the current indicator's sample size (number of test cases) in each of the 7 difference categories to the total current sample size (h) is calculated. The difference category scores of each category are then weighted to obtain the weighted average score of the indicator, with the weights being the proportions of each category. The difference categories and their corresponding difference category scores are as follows: 100 for absolute error < 0.00001, 95 for absolute error < 0.0001, 90 for absolute error < 0.001, 85 for absolute error < 0.01, 80 for absolute error < 0.1, 70 for absolute error < 1, and 60 for absolute error < 1.

[0075] Finally, the weighted average of each metric is substituted into the metric comparison formula to calculate the performance evaluation value of the device under test (DUT) for each metric. The comparison formulas differ for different metrics, and there are three types: When the metric is a physical metric, the comparison formula is: (1 + (golden - test) / golden) * 100; when the metric is a timing metric, the comparison formula is: (1 + (test - golden) / period) * 100; when the metric is a stage metric, the comparison formula is: (1 + (test - golden) / (period * golden_NVP)) * 100. Where golden is the output value of the benchmarking tool, test is the weighted average of the DUT's metrics, period is the clock cycle, and golden_NVP is the number of violation paths. Using the above metric comparison formulas, the score of the DUT for each metric is calculated.

[0076] Furthermore, when the output score of the indicator comparison formula is abnormal, it should be handled as follows:

[0077] When the output value of the index comparison formula is greater than 200, the performance evaluation value of the current index of the tested component is 200.

[0078] When the output value of the index comparison formula is less than 100, the performance evaluation value of the current index of the tested component is -100.

[0079] When the output value of the benchmarking tool is 0 and when processing the WNS, TNS, DRC_VIO, and Overflow metrics, if the output value of the metric comparison formula is 0, then the performance evaluation value of the current metric of the device under test is 100. If the output of the metric comparison formula is not 0, then the performance evaluation value of the current metric of the device under test is 100-test.

[0080] Taking the process mainly involving placement, CTS, and routing as an example, the evaluation process of the test execution and reporting module is as follows: First, a standardization check is performed. If a test case is in a non-standard form, the evaluation system will first report an error, indicating that the test case does not conform to the standard or fails the standardization check, and request that the standard be modified before use. Then, a usability check is performed. The test case input is read using the device under test, and the placement, CTS, and routing steps are performed respectively, such as... Figure 4 As shown, Figure 4 This is a flowchart illustrating the placement, cts, and routing of the test piece provided in an embodiment of the present invention.

[0081] The metrics for the placement phase include area_stdcell, H_overflow, V_overflow, placement_violations, total_leakage_power, total_switch_power, total_internal_power, reg2reg_setup_wns, reg2reg_setup_tns, all_setup_wns, all_setup_tns, and density.

[0082] The indicators of the Cts stage scenario include: area_stdcell, H_overflow, V_overflow, total_leakage_power, total_switch_power, total_internal_power, reg2reg_setup_wns, reg2r Indicators such as eg_setup_tns, reg2reg_hold_wns, reg2reg_hold_tns, clk_max_id, clk_skew, all_setup_wns, all_setup_tns, all_hold_wns, all_hold_tns, density, etc.

[0083] The indicators of the Routing stage scenario include area_stdcell, H_overflow, V_overflow, total_leakage_power, total_switch_power, total_internal_power, reg2reg_setup_wns, reg2reg_se Indicators such as tup_tns, reg2reg_hold_wns, reg2reg_hold_tns, total_drc, all_setup_wns, all_setup_tns, all_hold_wns, all_hold_tns, density, via_number, wire_length, etc.

[0084] Then, the report command of the device under test (DUT) is used to obtain the circuit output value. For the same metric, the absolute difference between the output value of the DUT and the output value of the benchmark tool is calculated to obtain the absolute error of the metric. Based on the calculated absolute error, the proportion of the sample size (number of test cases) of each of the seven difference categories to the total number of samples (h) is calculated. The difference category scores of each category are weighted and averaged to obtain the weighted average score of the metric, with the weights being the proportions of each category. Then, based on the weighted average score of each metric, the performance evaluation value of each metric is calculated using the metric comparison formula. Based on experience, the metrics in placement, cts, and routing are divided into physical metrics, timing metrics, and stage metrics. The comparison formulas for each metric are shown in Table 1.

[0085] Table 1

[0086]

[0087] Furthermore, when there are abnormal output scores in the indicator comparison formula, they are handled according to the abnormality handling method, and finally the performance evaluation value of each indicator is obtained. The performance evaluation value of each indicator is reflected in the automatically output visual evaluation report.

[0088] This embodiment proposes a simple and practical index scoring method, providing a framework for evaluating similar problems in the future; moreover, the evaluation system can run automatically, making the evaluation process more intelligent and convenient, greatly improving efficiency and reducing labor costs.

[0089] This embodiment establishes a standardized and systematic evaluation system for physical backend design tools, which can quickly, conveniently and accurately score the tools under test, helping users to evaluate and select tools.

[0090] The evaluation system of this invention uses cleaned benchmark examples with unified naming and data format to create new test cases and upload them to the benchmark test set library, thus improving the benchmark of the benchmark test set library, enabling continuous updates and maintenance, and constantly expanding the benchmark library to make the test benchmarks increasingly complete and rich. The output values ​​of the test cases running the device under test (DUT) and the output values ​​of the benchmark test cases running the benchmark tools are converted to the same standard through the evaluation algorithm. For different DUTs, only appropriate test cases need to be selected from the benchmark test set library for evaluation, and then the evaluation results are compared with the evaluation values ​​of the benchmark tools. The index scoring method is simple and practical, and can quickly, conveniently and accurately score various indicators of the DUT, helping users to evaluate and select tools, greatly improving evaluation efficiency and reducing labor costs. Therefore, the evaluation system of this invention can quickly evaluate the performance of physical design tools, thereby promoting the improvement of design tools, providing impetus for technological innovation and development in the field of integrated circuit design, and providing strong support for the development of high-performance, low-power integrated products.

[0091] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A benchmark test set-based evaluation system for integrated circuit physical design tools, characterized in that, include: The test case management module, the tool under test management module, and the test execution and reporting module are included. The test case management module is used to create new test cases based on user instructions, using cleaned benchmark examples with unified naming and data format under different data categories of integrated circuits, upload the new test cases to the benchmark test set library, and edit, delete, classify, copy and query the test cases in the benchmark test set library to update the benchmark test set library. The device under test (DUT) management module is used to upload the DUT according to user instructions; The test execution and reporting module is used to select a set of test cases from the benchmark test set library according to user instructions and perform standardization checks. After running the test cases that have passed the standardization checks on the device under test, the output value of the device under test is obtained. The output value of the device under test and the output value of the benchmark tool running the test cases are converted to the same standard using the evaluation algorithm. The module then outputs the performance evaluation value of each indicator of the device under test and a visual evaluation report.

2. The integrated circuit physical design tool evaluation system based on a benchmark test set according to claim 1, characterized in that, The method for cleaning the data content of the benchmark example includes: extracting the data features of the circuit in each benchmark example, modifying the circuit in the benchmark example according to the data features and the needs of the business scenario, and obtaining the cleaned benchmark example.

3. The integrated circuit physical design tool evaluation system based on a benchmark test set according to claim 1, characterized in that, The naming convention for benchmark examples in the benchmark test set library is: Data Category_Service Segment_Circuit Size_DESIGN Name_Number; where, The data classification includes any one of the following: process model, generalized analog compound, synthesis, physical design, wafer manufacturing, packaging, standards and substrates, and signing off. The business segment is a process under the data category; The circuit scale refers to the size of the unit components in the circuit design; DESIGN is the name of the circuit design.

4. The integrated circuit physical design tool evaluation system based on a benchmark test set according to claim 1, characterized in that, The data specification format for benchmark examples in the benchmark test set library is implemented through a hierarchical directory specification format, wherein... The first-level directory includes use case naming, and the second-level directory includes design documents, relevant scripts during the benchmarking tool's operation, the benchmarking tool's operation results, and explanatory documents. The design documents include input files and common databases involved in the operation process.

5. The integrated circuit physical design tool evaluation system based on a benchmark test set according to claim 1, characterized in that, Upload the newly created test cases to the benchmark test suite library, including: Submit the newly created use case; The newly created use case is run using the benchmarking tool to verify its usability. If the newly created test case is verified as unusable, the newly created test case is edited and modified and temporarily saved for resubmission; if the newly created test case is verified as usable, the newly created test case is reviewed for inclusion in the database based on whether the test case content is compliant and whether the test case is valuable. If the newly created test case is deemed unsuitable for inclusion in the database, it is edited, modified, and temporarily saved for resubmission. If the newly created test case is deemed acceptable for inclusion in the database, it is uploaded to the benchmark test suite database.

6. The integrated circuit physical design tool evaluation system based on a benchmark test set according to claim 1, characterized in that, The device under test includes any one of the following: components, script code, and third-party tools.

7. The integrated circuit physical design tool evaluation system based on a benchmark test set according to claim 1, characterized in that, The output value of the test piece and the output value of the benchmark tool running the test case are converted to the same standard using the evaluation algorithm, including: For each index, the absolute difference between the output value of the test piece and the output value of the benchmark tool is calculated to obtain the absolute error of each index; Based on the absolute error of each indicator, calculate the proportion of the number of test cases for each difference category to the total number of test cases for the current indicator, and use the proportion as the weight to calculate the weighted average of the difference category scores corresponding to each difference category, so as to obtain the weighted average of each indicator. Substitute the weighted average of each indicator into the indicator comparison formula to calculate the performance evaluation value of each indicator in the tested component.

8. The integrated circuit physical design tool evaluation system based on a benchmark test set according to claim 7, characterized in that, The difference classification and its corresponding difference classification score are as follows: when the absolute error is <0.00001, the difference classification score is 100; when the absolute error is <=0.00001, the difference classification score is 95; when the absolute error is <=0.001, the difference classification score is 90; when the absolute error is <=0.01, the difference classification score is 85; when the absolute error is <=0.1, the difference classification score is 80; when the absolute error is <=1, the difference classification score is 70; and when the absolute error is <=1, the difference classification score is 60.

9. The integrated circuit physical design tool evaluation system based on a benchmark test set according to claim 7, characterized in that, When the indicator is a physical indicator, the indicator comparison formula is: (1+(golden-test) / golden)*100; When the indicator is a time-series indicator, the indicator comparison formula is: (1+(test-golden) / period)*100; When the indicator is a period indicator, the indicator comparison formula is: (1+(test-golden) / (period*golden_NVP))*100; Wherein, golden is the output value of the benchmark tool, test is the weighted average of the metrics of the device under test, Period is the clock cycle, and golden_nvp is the number of violation paths.

10. The integrated circuit physical design tool evaluation system based on a benchmark test set according to claim 7, characterized in that, When the output value of the index comparison formula is greater than 200, the performance evaluation value of the current index of the tested component is 200. When the output value of the index comparison formula is less than 100, the performance evaluation value of the current index of the tested component is -100. When the output value of the benchmarking tool is 0 and when processing the WNS, TNS, DRC_VIO, and Overflow metrics, if the output value of the metric comparison formula is 0, then the performance evaluation value of the current metric of the device under test is 100. If the output of the metric comparison formula is not 0, then the performance evaluation value of the current metric of the device under test is 100-test.

Images