Integrated circuit physical design tool evaluation system based on benchmark test set

Abstract

The invention relates to an integrated circuit physical design tool evaluation system based on a benchmark test set. The integrated circuit physical design tool evaluation system comprises a use case management module, a tested tool management module and a test execution and report module, the use case management module is used for creating a use case by using a reference case with a unified naming specification format and a data specification format according to a user instruction, and uploading the newly created use case to the reference test set library; the tested tool management module is used for uploading a tested piece according to a user instruction; the test execution and report module is used for obtaining an output value of the tested piece after the tested piece runs the evaluation case passing the normativity check according to a user instruction, and performing same standard conversion on the output value of the tested piece and an output value of the evaluation case run by the benchmarking tool through an evaluation algorithm; and outputting a visual evaluation report of the performance evaluation value of each index of the tested piece. The index scoring method of the evaluation system is simple and practical, and the performance of the physical design tool can be quickly evaluated according to score values of different indexes.

Description

Technical Field

[0001] The present invention belongs to the technical field of integrated circuit physical design, and in particular relates to an integrated circuit physical design tool evaluation system based on a benchmark test set. Background Art

[0002] Integrated Circuit (IC) physical design is an important part of modern electronic product design and manufacturing. It involves the entire process from logic design to final physical implementation. With the advancement of Moore's Law, the size of integrated circuits continues to shrink and the integration continues to increase, so that more transistors and other electronic components can be accommodated on a single chip, which requires physical design technology to support high-density, high-performance circuit layout and routing. Physical design directly affects the performance of the chip, including speed, power consumption and heat dissipation. Physical design can significantly improve the performance indicators of the chip by optimizing layout and routing, timing and power distribution, while reducing power consumption and thermal effects. With the continuous advancement of integrated circuit technology, the complexity and integration of chip design are rapidly increasing, which requires physical design tools to have higher performance and accuracy. The integrated circuit design process includes multiple stages, from logic design to the final physical layout, and each step requires efficient design tools to support it. Among them, the physical design stage is particularly critical, which directly determines the performance, power consumption and cost of the chip.

[0003] At present, in the field of integrated circuit design, physical design tools are mainly used to realize the conversion from logic circuit diagrams to physical layouts, including layout planning, wiring, optimization and other steps. The quality of these tools directly affects the yield and market competitiveness of the final product. Therefore, developing high-quality physical design tools has become one of the goals pursued by the industry. However, the existing physical design tool performance evaluation methods have the following shortcomings: 1) Lack of standardization. There is a lack of unified standards in the industry to measure the performance of different physical design tools, which makes it difficult for users to objectively compare the advantages and disadvantages of different tools. 2) Incomplete evaluation. Existing evaluation methods often focus on a single indicator (such as layout area or wiring quality) and ignore other important factors. 3) Insufficient benchmarks. Existing benchmarks may not fully reflect the various challenges in actual application scenarios. Due to the lack of continuous maintenance and update mechanisms, some benchmark test sets cannot accurately reflect the needs of modern design once they are outdated. 4) The evaluation system is cumbersome. Most of the current evaluation systems are manually evaluated, with complex evaluation methods and low evaluation efficiency. It is impossible to quickly evaluate tools and give evaluation results, and the evaluation results are presented in a variety of ways. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides an integrated circuit physical design tool evaluation system based on a benchmark test set. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] The embodiment of the present invention provides an integrated circuit physical design tool evaluation system based on a benchmark test set, including: a use case management module, a tested tool management module and a test execution and reporting module, wherein:

[0006] The use case management module is used to create new use cases under different data classifications of integrated circuits using cleaned benchmark examples with unified naming and data standard formats according to user instructions, upload the newly created use cases to the benchmark test set library, and edit, delete, classify, copy and query the use cases in the benchmark test set library to update the benchmark test set library;

[0007] The tool under test management module is used to upload the tested device according to user instructions;

[0008] The test execution and reporting module is used to select a set of evaluation cases from the benchmark test set library and perform a compliance check according to user instructions, obtain the output value of the test piece after running the evaluation cases that have passed the compliance check using the test piece, and convert the output value of the test piece and the output value of the benchmark tool running the evaluation case to the same standard through an evaluation algorithm, and output the performance evaluation value of each indicator of the test piece and a visual evaluation report.

[0009] In one embodiment of the present invention, a method for cleaning the data content of a benchmark example includes: extracting data features of a circuit in each benchmark example, modifying the circuit in the benchmark example according to the data features in combination with business scenario requirements, and obtaining the cleaned benchmark example.

[0010] In one embodiment of the present invention, the naming standard format of the benchmark examples in the benchmark test set library is: data classification_business segment_circuit scale_DESIGN name_number; wherein,

[0011] The data classification includes any one of process model, pan-analog compound, synthesis, physical design, wafer manufacturing, packaging, standard and base, and sign-off;

[0012] The business segment is a process under the data classification;

[0013] The circuit scale is the size of the unit element in the circuit design;

[0014] The DESIGN name is the DESIGN name of the circuit design.

[0015] In one embodiment of the present invention, the data specification format of the benchmark examples in the benchmark test set library is implemented by a hierarchical directory specification format, wherein:

[0016] The first-level directory includes use case naming, and the second-level directory includes design files, relevant scripts during the operation of the benchmark tool, the operation results of the benchmark tool and description documents. The design files include input files and public databases involved in the operation process.

[0017] In one embodiment of the present invention, uploading a newly created use case to a benchmark test set library includes:

[0018] Submit the newly created use case;

[0019] Running the newly created use case through the benchmark tool to verify whether the newly created use case is available;

[0020] When the newly created use case is verified to be an unusable use case, the newly created use case is edited and modified and temporarily saved for resubmission; when the newly created use case is verified to be an available use case, whether the newly created use case can be included in the database is reviewed based on whether the use case content is compliant and whether the use case is valuable;

[0021] When the newly created use case is found to be unacceptable for storage, the newly created use case is edited and modified and temporarily stored for resubmission; when the newly created use case is found to be acceptable for storage, the newly created use case is uploaded to the benchmark test set library.

[0022] In one embodiment of the present invention, the device under test includes any one of a component, a script code, and a third-party tool.

[0023] In one embodiment of the present invention, the output value of the device under test and the output value of the benchmark tool running the test case are converted to the same standard through a test algorithm, including:

[0024] For each indicator, the difference between the output value of the measured object and the output value of the benchmark tool is calculated to obtain the absolute error of each indicator;

[0025] According to the absolute error of each indicator, calculate the ratio of the number of use cases of the current indicator in each difference classification to the current total number of use cases, and use the ratio as the weight to perform weighted average on the difference classification scores corresponding to each difference classification to obtain the weighted average of each indicator;

[0026] Substitute the weighted average value of each indicator into the indicator comparison formula to calculate the performance evaluation value of each indicator in the tested device.

[0027] In one embodiment of the present invention, the difference classification and its corresponding difference classification score are: when the absolute error is <0.00001, the difference classification score is 100, when 0.00001<=absolute error<0.0001, the difference classification score is 95, when 0.0001<=absolute error<0.001, the difference classification score is 90, when 0.001<=absolute error<0.01, the difference classification score is 85, when 0.01<=absolute error<0.1, the difference classification score is 80, when 0.1<=absolute error<1, the difference classification score is 70, and when 1<=absolute error, the difference classification score is 60.

[0028] In one embodiment of the present invention, when the indicator is a physical indicator, the indicator comparison formula is: (1+(golden-test) / golden)*100;

[0029] When the indicator is a time series indicator, the indicator comparison formula is: (1+(test-golden) / period)*100;

[0030] When the indicator is a stage indicator, the indicator comparison formula is: (1+(test-golden) / (period*golden_NVP))*100;

[0031] Among them, golden is the output value of the benchmark tool, test is the weighted average of the indicators of the device under test, Period is the clock period, and golden_nvp is the number of violation paths.

[0032] In one embodiment of the present invention,

[0033] When the output value of the index comparison formula is greater than 200, the performance evaluation value of the current index of the device under test is 200;

[0034] When the output value of the indicator comparison formula is less than 100, the performance evaluation value of the current indicator of the device under test is -100;

[0035] When the output value of the benchmark tool is 0 and when processing the wns, tns, drc_vio, and overflow indicators, if the output value of the indicator comparison formula is 0, the performance evaluation value of the current indicator of the device under test is 100; if the output of the indicator comparison formula is not 0, the performance evaluation value of the current indicator of the device under test is 100-test.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The evaluation system of the present invention uses the cleaned benchmark examples with unified naming and data formats to create new use cases and upload them to the benchmark test set library, thereby improving the benchmark of the benchmark test set library, enabling it to be continuously updated and maintained, and constantly expanding the benchmark library, making the test benchmark more and more complete and rich; the output value of the running evaluation case of the tested device and the output value of the running evaluation case of the benchmark tool are converted to the same standard through the evaluation algorithm. For different tested devices, it is only necessary to select appropriate evaluation cases in the benchmark test set library for evaluation, and then compare the evaluation results with the output values ​​of the benchmark tool. The indicator scoring method is simple and practical, and can quickly, conveniently and accurately score various indicators of the tested tool, helping users to evaluate and select tools, greatly improving evaluation efficiency and reducing labor costs; therefore, the evaluation system of the present invention can quickly evaluate the performance of physical design tools, thereby promoting the improvement of design tools, providing impetus for the innovation and development of technology in the field of integrated circuit design, and providing strong support for the development of high-performance, low-power integrated products. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the structure of an integrated circuit physical design tool evaluation system based on a benchmark test set provided by an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of a process for uploading a newly created test case to a benchmark test set library provided by an embodiment of the present invention;

[0040] Figure 3 A framework diagram of use case management provided by an embodiment of the present invention;

[0041] Figure 4 A flow chart of placement, CTS, and routing of a device under test provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0043] Embodiment 1

[0044] The embodiment of the present invention proposes an integrated circuit physical design tool evaluation system based on a benchmark test set. Different physical design tools run the same benchmark test set on the same platform, and then the running results are comprehensively evaluated to obtain performance evaluation results of the evaluated tools. A comprehensive, professional, continuously updated and rapidly evaluated system is provided, which can help designers have an intuitive understanding of the overall performance of the tools, thereby guiding designers to improve or select the tools.

[0045] See also Figure 1 , Figure 1 The present invention provides a schematic diagram of the structure of an integrated circuit physical design tool evaluation system based on a benchmark test set. The evaluation system includes a use case management module, a tested tool management module and a test execution and reporting module.

[0046] The use case management module is used to create new use cases in accordance with a unified naming specification format and data specification format using cleaned benchmark examples under different data classifications of integrated circuits according to user instructions, upload the newly created use cases to the benchmark test set library, and edit, delete, classify, copy and query the use cases in the benchmark test set library to update the benchmark test set library.

[0047] Benchmark test sets are crucial for evaluating physical design tools. Currently, the publicly available benchmark test sets are mainly scattered in open source communities, examples provided by some domestic EDA competitions, and simple examples released by some companies. With the emergence of new technologies and the development of old technologies, the original benchmarks may become outdated or no longer fully reflect the current state of the art. Therefore, benchmarks need to be continuously updated and expanded to ensure that they can accurately evaluate the latest technologies and solutions. Therefore, this embodiment solves the problem that benchmark test sets are scattered and cannot be updated by establishing a unified standardized benchmark test set library.

[0048] Specifically, we first collect publicly available benchmark examples. We collect benchmark examples related to various stages of physical design, such as Placement, CTS, Routing, etc., from open source communities, domestic electronic design automation (EDA) competitions, and public examples from various companies. These benchmark examples are only used in competitions or as part of a process, and are not further maintained or used. We then collect these benchmark examples, clean them, classify them, and number them to make them standardized and universal examples. Finally, we upload these benchmark examples to the benchmark test set library for future use.

[0049] Furthermore, cleaning the benchmark examples includes cleaning the data content, naming conventions, and data formats of the benchmark examples.

[0050] The method for cleaning the data content of the benchmark example includes: extracting data features of the circuit in each benchmark example, such as cells, nets, primary IO, die area, etc., and making relevant modifications to the circuit in the benchmark example according to the extracted data features and in combination with specific business scenario requirements, so that the benchmark example meets the business usage scenario requirements, and obtains the cleaned benchmark example. Exemplarily, according to the extracted data features, the shape of the circuit in the benchmark example is modified in combination with the specific business scenario requirements, and the benchmark example is cleaned to meet the business usage scenario requirements.

[0051] Cleaning the naming conventions and data formats of benchmark examples refers to unifying the naming conventions and data formats of benchmark examples.

[0052] The naming standard format of the benchmark examples in the benchmark test set library is: data classification_business segment_circuit scale_DESIGN name_number, for example, PD_PLACE_10K_DES_001; wherein, the data classification includes any one of process model (PM), pan-analog compound, synthesis, physical design (PD), wafer manufacturing, packaging, standards and base, and sign-off; the business segment is a process under the data classification, for example, the business segment of physical design (PD) can be divided into PR / PLACE / CTS / ROUTE, etc.; the circuit scale is the size of the unit component in the circuit design; the DESIGN name is the DESIGN name of the circuit design.

[0053] The data specification format of the benchmark examples in the benchmark test set library is implemented through a hierarchical directory specification format, where the first-level directory includes the use case name, the second-level directory includes the design file design, the relevant scripts script during the running process of the benchmark tool, the running results result of the benchmark tool and the description document README. The design file design includes the input file input and the public database pdk involved in the running process.

[0054] Taking PD_CTS_15K_AES_001 as an example, its data format specification is:

[0055]

[0056]

[0057] Among them, design.v is the Verilog netlist file of the design, design.sdc is the timing constraint file of the design, placement.def records the physical information of the design layout planning, the lef directory stores the design process information, design rule information, through-hole information and information of each unit in the unit library, the lib directory stores the timing logic information of the design, the rlc directory stores the parasitic parameter information extracted by the design, the script directory stores the relevant scripts during the operation process, the result directory stores the operation results, and README is the instruction document.

[0058] After the data content of the benchmark examples is cleaned and normalized, new use cases are created under different data categories (for example, process models, pan-analog compounds, synthesis, physical design, wafer manufacturing, packaging, standards and bases, sign-off, etc.), and the benchmark examples are uploaded to the benchmark test set library individually or in batches. When creating a new use case, the form for creating a new use case includes a use case directory, a use case name, a use case description, and a use case file upload (select the benchmark use case data to be used, which can be local benchmark use case data or data in the benchmark use case library). The process of uploading benchmark examples individually or in batches to the benchmark test set library is as follows: Figure 2 As shown, Figure 2 A schematic diagram of a process for uploading a newly created use case to a benchmark test set library provided by an embodiment of the present invention, the process specifically includes the following steps:

[0059] S1. Submit the newly created use case;

[0060] S2. Run the newly created use case through the benchmark tool to verify whether the newly created use case is available.

[0061] Specifically, the newly created use case is run using the benchmark tool to determine whether the benchmark tool can run and the result output is normal; if so, the newly created use case is an available use case and the verification is successful; if not, the newly created use case is an unavailable use case and the verification fails.

[0062] S3. When the newly created use case is verified to be an unusable use case, edit and modify the newly created use case and temporarily save it for resubmission. When the newly created use case is verified to be an available use case, review whether the newly created use case can be included in the database based on whether the use case content is compliant and whether the use case is valuable. Compliance refers to whether the use case contains sensitive words, whether it involves red line issues, etc.

[0063] S4. When it is determined that the newly created use case cannot be stored in the database, the newly created use case is edited and modified and temporarily stored for resubmission; when it is determined that the newly created use case can be stored in the database, the newly created use case is uploaded to the benchmark test set library.

[0064] Furthermore, in the categories of the benchmark test set library, the use case name, use case status, use case size, use case type, uploader, upload time, scale, process, update time, number of uses and downloads of the uploaded examples will be displayed, so that users can quickly find the use cases they need. At the same time, the use cases can also be deleted, classified, copied, queried and edited (the editing operation can only be done by the uploader himself who has the authority to modify the use cases uploaded by himself), so as to achieve the management of the use cases, such as Figure 3 As shown, Figure 3 The framework diagram of the use case management provided by the embodiment of the present invention. The uploaded use case can be modified through editing. When updating the use case, if you want to save the old use case, you can upload a new example, distinguish the new and old use cases by number, and explain in the use case description that the use case is an updated use case based on a certain use case; if you want to use a certain use case, you can download it through the download function; if you do not want to keep the original use case during the update, you can directly update the use case in the edit and upload a new use case package; through the above use case library, the use cases can be continuously expanded, updated and maintained.

[0065] This embodiment improves the benchmark test set library through the use case management module, so that it can be continuously updated and maintained, and the benchmark library is continuously expanded, making the test benchmark more and more complete and rich.

[0066] The tool management module is used to upload the tested device according to the user's instructions, wherein the tested device includes but is not limited to any one of components, script codes, and third-party tools.

[0067] Specifically, if the DUT is a complete component or script code, the user can select component upload and upload it to the evaluation platform. If it is a third-party tool, you can choose to install it on the platform and directly choose to use it when you need to use it. When uploading a tool to the platform, you need to fill in the tool type, directory, name, version, and description of the DUT in the form, and upload the DUT.

[0068] The test execution and reporting module is used to select a set of evaluation cases from the benchmark test set library and perform a compliance check according to user instructions. The output value of the test piece is obtained by running the evaluation cases that have passed the compliance check on the test piece. The output value of the test piece and the output value of the evaluation case run by the benchmark tool are converted to the same standard through the evaluation algorithm, and the performance evaluation value of each indicator of the test piece and a visual evaluation report are output.

[0069] Specifically, the test execution and reporting module mainly associates the DUT and the use case for testing and outputs a report. First, create a new test task. The new task includes the directory, evaluation stage (select evaluation stage placement, cts, routing, etc.), select the DUT (such as the tool under test), configure the environment, which environment to run on, etc. After the task is created, the interface will display the newly created task. In the task operation bar, you can operate the task, such as running, copying the test task, viewing task details, deleting the task, viewing the report, etc. Then perform automated execution. During automated execution, select the benchmark test set to be executed as the evaluation case in the benchmark test set library, create an execution task, configure the running environment information, and start running the task. After the task is completed, the execution result will be displayed: success or failure. After success, click View Report to see the score of the tool under test under this or this group of use cases and the details of the operation. If it fails, a failure log will be prompted for troubleshooting. After the task is completed, you can also re-run the task, delete the task, and view the execution record.

[0070] Furthermore, during the automated execution, the test cases are first checked for compliance. If a test case is in a non-standard form, the test system will first report an error, indicating that the test case does not meet the specifications or fails the specification check. Please modify the specifications before using it. Then, the test case is run with the test piece to perform a usability check: for different types of use cases, different benchmark tools (generally considered to be authoritative tools in the industry) are used in advance to perform evaluations and obtain the output values ​​of the benchmark tools; on the same platform and system, the test piece to be evaluated is used to select a set of evaluation cases to be used, and the output values ​​are given by running the set of evaluation cases through the test piece; the output values ​​of the benchmark tool and the test piece are converted to the same standard using the same evaluation method to give the performance evaluation value of each indicator of the test piece.

[0071] Specifically, the process of converting the output value of the device under test and the output value of the benchmark tool running the evaluation case to the same standard through the evaluation algorithm is as follows: for each indicator, calculate the absolute value difference between the output value of the device under test and the output value of the benchmark tool to obtain the absolute error of each indicator; based on the absolute error of each indicator, calculate the proportion of the number of cases of the current indicator in each difference classification to the current total number of cases, and use the proportion as the weight to perform weighted averaging on the difference classification scores corresponding to each difference classification to obtain the weighted average value of each indicator; substitute the weighted average value of each indicator into the indicator comparison formula to calculate the performance evaluation value of each indicator in the device under test.

[0072] Specifically, assume that there are m devices under test, n indicators to be concerned about, and h use cases.

[0073] First, the DUT runs a certain use case to obtain the output value of each indicator of the DUT, and the benchmark tool runs the use case to obtain the output value of each indicator of the benchmark tool. For each indicator, the absolute value difference between the output value of the DUT and the output value of the benchmark tool is calculated to obtain the absolute error of each indicator.

[0074] Then, combined with the absolute error of each indicator, the proportion of the number of samples (number of use cases) of each category in the 7 difference categories of the current indicator to the current total samples (h) is calculated, and the weighted average of the difference classification scores of each category is obtained to obtain the weighted average score of the indicator, and the weight is the proportion of each category. The difference classification and its corresponding difference classification score are: when the absolute error is <0.00001, the difference classification score is 100, when 0.00001<=absolute error<0.0001, the difference classification score is 95, when 0.0001<=absolute error<0.001, the difference classification score is 90, when 0.001<=absolute error<0.01, the difference classification score is 85, when 0.01<=absolute error<0.1, the difference classification score is 80, when 0.1<=absolute error<1, the difference classification score is 70, and when 1<=absolute error, the difference classification score is 60.

[0075] Finally, substitute the weighted average value of each indicator into the indicator comparison formula to calculate the performance evaluation value of the DUT for each indicator. The comparison formulas for different indicators are different, and there are three types: when the indicator is a physical indicator, the indicator comparison formula is: (1+(golden-test) / golden)*100; when the indicator is a timing indicator, the indicator comparison formula is: (1+(test-golden) / period)*100; when the indicator is a stage indicator, the indicator comparison formula is: (1+(test-golden) / (period*golden_NVP))*100. Among them, golden is the output value of the benchmark tool, test is the weighted average value of the indicator of the DUT, Period is the clock period, and golden_nvp is the number of violation paths. Through the above indicator comparison formula, the score of each indicator of the DUT is calculated.

[0076] Furthermore, when the output score of the indicator comparison formula is abnormal, it is handled as follows:

[0077] When the output value of the index comparison formula is greater than 200, the performance evaluation value of the current index of the device under test is 200;

[0078] When the output value of the indicator comparison formula is less than 100, the performance evaluation value of the current indicator of the device under test is -100;

[0079] When the output value of the benchmark tool is 0 and when processing the wns, tns, drc_vio, and overflow indicators, if the output value of the indicator comparison formula is 0, the performance evaluation value of the current indicator of the device under test is 100; if the output of the indicator comparison formula is not 0, the performance evaluation value of the current indicator of the device under test is 100-test.

[0080] Taking the process mainly involving placement, cts and routing as an example, the process of evaluation in the test execution and reporting module is as follows: first, a normative check is performed. If a use case is in a non-normative form, the evaluation system will first report an error, prompting that the use case does not meet the norms, or the normative check fails. Please modify the norms before using it. Then, a usability check is performed, and the use case input is read into the test piece, and the placement, cts and routing steps are performed respectively, such as Figure 4 As shown, Figure 4 A flow chart of placement, CTS, and routing of a device under test provided in an embodiment of the present invention.

[0081] Among them, the indicators of the placement stage scenario include area_stdcell, H_overflow, V_overflow, placement_violations, total_leakage_power, total_switch_power, total_internal_power, reg2reg_setup_wns, reg2reg_setup_tns, all_setup_wns, all_setup_tns, density and other indicators.

[0082] The indicators of the Cts stage scenario include: area_stdcell, H_overflow, V_overflow, total_leakage_power, total_switch_power, total_internal_power, reg2reg_setup_wns, reg2r Indicators such as eg_setup_tns, reg2reg_hold_wns, reg2reg_hold_tns, clk_max_id, clk_skew, all_setup_wns, all_setup_tns, all_hold_wns, all_hold_tns, density, etc.

[0083] The indicators of the Routing stage scenario include area_stdcell, H_overflow, V_overflow, total_leakage_power, total_switch_power, total_internal_power, reg2reg_setup_wns, reg2reg_se Indicators such as tup_tns, reg2reg_hold_wns, reg2reg_hold_tns, total_drc, all_setup_wns, all_setup_tns, all_hold_wns, all_hold_tns, density, via_number, wire_length, etc.

[0084] Then use the report command of the device under test to obtain the circuit output value. For the same indicator, calculate the absolute value difference between the output value of the device under test and the output value of the benchmark tool to obtain the absolute error of the indicator. Combined with the calculated absolute error, calculate the proportion of the number of samples (number of use cases) of each category in the 7 difference categories of the current indicator to the current total samples (h), and perform weighted average of the difference classification scores of each category to obtain the weighted average score of the indicator, with the weight being the proportion of each category. Then, combined with the weighted average score of each indicator, use the indicator comparison formula to calculate the performance evaluation value of each indicator. According to experience, the indicators in placement, cts, and routing are divided into physical indicators, timing indicators, and stage indicators. The comparison formulas corresponding to each indicator are shown in Table 1:

[0085] Table 1

[0086]

[0087] Furthermore, when there is an abnormality in the output score of the indicator comparison formula, it is processed according to the exception handling method, and finally the performance evaluation value of each indicator is obtained, and the performance evaluation value of each indicator is reflected in the automatically output visual evaluation report.

[0088] This embodiment proposes a simple and practical indicator scoring method, which provides a way of thinking for the subsequent evaluation of similar problems; and the evaluation system can run automatically, making the evaluation process more intelligent and convenient, greatly improving efficiency and reducing labor costs.

[0089] This embodiment establishes a standardized and systematic physical back-end design tool evaluation system, which can quickly, conveniently and accurately score the tools under test, and help users evaluate and select the tools.

[0090] The evaluation system of the present invention creates new use cases with cleaned benchmark examples with unified naming and data formats and uploads them to the benchmark test set library, thereby improving the benchmark of the benchmark test set library, enabling it to be continuously updated and maintained, and continuously expanding the benchmark library, making the test benchmark more and more complete and rich; the output value of the running evaluation case of the tested device and the output value of the running evaluation case of the benchmark tool are converted to the same standard through an evaluation algorithm. For different tested devices, it is only necessary to select appropriate evaluation cases in the benchmark test set library for evaluation, and then compare the evaluation results with the evaluation values ​​of the benchmark tool. The indicator scoring method is simple and practical, and can quickly, conveniently and accurately score various indicators of the tested tool, helping users to evaluate and select tools, greatly improving evaluation efficiency and reducing labor costs; therefore, the evaluation system of the present invention can quickly evaluate the performance of physical design tools, thereby promoting the improvement of design tools, providing impetus for the innovation and development of technology in the field of integrated circuit design, and providing strong support for the development of high-performance, low-power integrated products.

[0091] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. An integrated circuit physical design tool evaluation system based on a benchmark test set, characterized in that: include: Use case management module, tested tool management module and test execution and reporting module, among which, The use case management module is used to create new use cases under different data classifications of integrated circuits using cleaned benchmark examples with unified naming and data standard formats according to user instructions, upload the newly created use cases to the benchmark test set library, and edit, delete, classify, copy and query the use cases in the benchmark test set library to update the benchmark test set library; The tool under test management module is used to upload the tested device according to user instructions; The test execution and reporting module is used to select a set of evaluation cases from the benchmark test set library and perform a compliance check according to user instructions, obtain the output value of the test piece after running the evaluation cases that have passed the compliance check using the test piece, and convert the output value of the test piece and the output value of the benchmark tool running the evaluation case to the same standard through an evaluation algorithm, and output the performance evaluation value of each indicator of the test piece and a visual evaluation report.

2. The integrated circuit physical design tool evaluation system based on the benchmark test set according to claim 1, characterized in that: The method for cleaning the data content of the benchmark example includes: extracting data features of the circuit in each benchmark example, modifying the circuit in the benchmark example according to the data features and business scenario requirements, and obtaining the cleaned benchmark example.

3. The integrated circuit physical design tool evaluation system based on the benchmark test set according to claim 1, characterized in that: The naming standard format of the benchmark examples in the benchmark test set library is: data classification_business segment_circuit scale_DESIGN name_number; where, The data classification includes any one of process model, pan-analog compound, synthesis, physical design, wafer manufacturing, packaging, standard and base, and sign-off; The business segment is a process under the data classification; The circuit scale is the size of the unit element in the circuit design; DESIGN name is the DESIGN name of the circuit design.

4. The integrated circuit physical design tool evaluation system based on the benchmark test set according to claim 1, characterized in that: The data specification format of the benchmark examples in the benchmark test set library is implemented by a hierarchical directory specification format, where: The first-level directory includes use case naming, and the second-level directory includes design files, relevant scripts during the operation of the benchmark tool, the operation results of the benchmark tool and description documents. The design files include input files and public databases involved in the operation process.

5. The integrated circuit physical design tool evaluation system based on the benchmark test set according to claim 1, characterized in that: Upload the newly created test cases to the benchmark library, including: Submit the newly created use case; Running the newly created use case through the benchmark tool to verify whether the newly created use case is available; When the newly created use case is verified to be an unusable use case, the newly created use case is edited and modified and temporarily saved for resubmission; when the newly created use case is verified to be an available use case, whether the newly created use case can be included in the database is reviewed based on whether the use case content is compliant and whether the use case is valuable; When the newly created use case is found to be unacceptable for storage, the newly created use case is edited and modified and temporarily stored for resubmission; when the newly created use case is found to be acceptable for storage, the newly created use case is uploaded to the benchmark test set library.

6. The IC physical design tool evaluation system based on a benchmark test set according to claim 1, characterized in that: The device under test includes any one of a component, a script code, and a third-party tool.

7. The integrated circuit physical design tool evaluation system based on a benchmark test set according to claim 1, characterized in that: The output value of the device under test and the output value of the benchmark tool running the test case are converted to the same standard through a test algorithm, including: For each indicator, the difference between the output value of the measured object and the output value of the benchmark tool is calculated to obtain the absolute error of each indicator; According to the absolute error of each indicator, calculate the ratio of the number of use cases of the current indicator in each difference classification to the current total number of use cases, and use the ratio as the weight to perform weighted average on the difference classification scores corresponding to each difference classification to obtain the weighted average of each indicator; Substitute the weighted average value of each indicator into the indicator comparison formula to calculate the performance evaluation value of each indicator in the tested device.

8. The benchmark test set-based integrated circuit physical design tool evaluation system according to claim 7, characterized in that: The difference classification and its corresponding difference classification score are: when the absolute error is <0.00001, the difference classification score is 100, when 0.00001<=absolute error<0.0001, the difference classification score is 95, when 0.0001<=absolute error<0.001, the difference classification score is 90, when 0.001<=absolute error<0.01, the difference classification score is 85, when 0.01<=absolute error<0.1, the difference classification score is 80, when 0.1<=absolute error<1, the difference classification score is 70, and when 1<=absolute error, the difference classification score is 60.

9. The benchmark test set-based integrated circuit physical design tool evaluation system according to claim 7, characterized in that: When the indicator is a physical indicator, the indicator comparison formula is: (1+(golden-test) / golden)*100; When the indicator is a time series indicator, the indicator comparison formula is: (1+(test-golden) / period)*100; When the indicator is a stage indicator, the indicator comparison formula is: (1+(test-golden) / (period*golden_NVP))*100; Among them, golden is the output value of the benchmark tool, test is the weighted average of the indicators of the device under test, Period is the clock period, and golden_nvp is the number of violation paths.

10. The benchmark test set-based integrated circuit physical design tool evaluation system according to claim 7, characterized in that: When the output value of the index comparison formula is greater than 200, the performance evaluation value of the current index of the device under test is 200; When the output value of the indicator comparison formula is less than 100, the performance evaluation value of the current indicator of the device under test is -100; When the output value of the benchmark tool is 0 and when processing the wns, tns, drc_vio, and overflow indicators, if the output value of the indicator comparison formula is 0, the performance evaluation value of the current indicator of the device under test is 100; if the output of the indicator comparison formula is not 0, the performance evaluation value of the current indicator of the device under test is 100-test.

Images