Testing methods and systems for EDA tools, and storage media
By managing metadata and filtering EDA function fields, the problems of complex dependencies and semantic-level filtering in EDA tool testing are solved, enabling efficient and reliable test environment reproduction and result analysis, thus improving testing efficiency and accuracy.
Patent Information
- Application Number
- CN202611619103.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-20
- Publication Date
- 2026-11-17
AI Technical Summary
Existing EDA tools and testing methods cannot effectively describe the complex dependencies between test cases, make it difficult to accurately filter multi-dimensional attributes, and fail to meet complex semantic-level filtering requirements, resulting in insufficient testing efficiency and accuracy.
Metadata is used to manage test cases, including EDA function fields and version fields. Target test cases are determined through EDA function fields and filtering conditions, and EDA tools are driven to execute tests in the test environment to generate and parse raw results, and specific EDA-related metrics are extracted for analysis.
It achieves reproducibility and traceability of tests, improves testing efficiency and accuracy, can accurately reproduce historical test environments, supports multi-dimensional filtering and intelligent analysis, and improves the reliability and relevance of test results.
Smart Images

Figure CN123114986A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of testing technology for EDA (Electronic Design Automation) tools, specifically to a testing method and system for EDA tools, and a storage medium. Background Technology
[0002] EDA tools, also known as EDA software, are core tools in integrated circuit design and manufacturing. Their quality and stability directly determine the success or failure of chip design. As process nodes evolve to 5nm, 3nm, and even more advanced levels, the complexity of EDA tools has increased dramatically, with a growing number of functional modules. Comprehensive, efficient, and in-depth testing of these tools has become a major challenge for the industry. Currently, industry and academia have conducted extensive research in the field of EDA tool testing, resulting in several typical solutions. However, these solutions still have significant shortcomings in terms of systematicity, intelligence, and adaptation to the characteristics of the EDA domain.
[0003] The two most typical problems are: First, EDA test cases typically contain various heterogeneous data such as layouts, configuration files, model files, netlists, simulation stimuli, and reference results. Existing technologies usually manage these data using simple file directories or basic databases, resulting in rudimentary test case management. This makes it difficult to effectively describe the complex dependencies between test cases, achieve precise filtering based on multi-dimensional attributes, and store only file paths, completely losing version control capabilities and failing to guarantee test reproducibility and traceability. Second, existing technologies typically filter based only on simple fields such as test case ID and name, which cannot meet complex, semantic-based testing requirements such as "verifying the rule implementation of a physical effect (e.g., antenna effect) across multiple process nodes." This fails to match the verification intent of testers, easily leading to omissions in filtering and insufficient targeted verification, thus affecting testing efficiency and accuracy. Summary of the Invention
[0004] In view of this, embodiments of this application provide a testing method and system for EDA tools, as well as a storage medium, which can at least improve the problems of poor reproducibility and traceability of tests due to the extensive management of test cases, and the difficulty in matching the verification intentions of testers to achieve semantic-level screening and targeted verification of test cases.
[0005] This application provides a testing method for EDA tools, including: Maintain metadata in the first storage area and entity files of test cases in the second storage area. The metadata includes at least EDA function fields, version fields pointing to the version of the entity file, and EDA tool-related information. Each test case is associated with one or more EDA function fields in the metadata. Receive test requests and parse them to obtain filter conditions that include at least EDA function fields; Based on the EDA function fields and metadata, at least one target test case is determined, and the version field and EDA tool-related information corresponding to the target test case are obtained from the metadata. Based on the version field, retrieve the corresponding version entity file from the second storage area, and configure the test environment for running the EDA tool in conjunction with the relevant EDA tool information; Drive EDA tools in the test environment to execute tests on the target test cases and generate raw results.
[0006] Optionally, the method further includes: Extract structured key information from the original results; The key information is divided into multiple levels according to preset rules; The key information from the multiple levels is presented in a structured form.
[0007] Optionally, the preset rules include stratifying the key information from general to detailed and / or from results to recommendations.
[0008] Optionally, the method further includes: Extract EDA domain-specific metrics from the original results. These EDA domain-specific metrics are information used to characterize the performance and quality of the EDA tool, extracted from the original results using parsing rules specific to the output format of the EDA tool. The EDA-specific metrics are associated with timestamps, EDA tool versions, test case IDs, and identifiers for each test case execution, and at least one of trend analysis, statistical analysis, and regression testing is performed. The analysis results are fed back to the user and / or written back as new tags to the metadata in the first storage area.
[0009] Optionally, the filtering criteria may also include other dimensions; determining at least one target test case based on the EDA functional fields and metadata includes: Select test cases associated with the EDA function fields from the metadata, and select test cases corresponding to the other dimensions from the metadata as target test cases.
[0010] Optionally, the other dimensions include at least one of the following: tags, priority, EDA tool type, EDA tool version, process node, creator, creation time, input / output characteristics of test cases, and impact analysis of EDA tool code changes.
[0011] Optionally, driving the EDA tool to perform tests on the target test cases in the test environment includes: determining a preset strategy, the preset strategy including scheduling according to the priority of the target test cases from high to low, and / or scheduling according to the resource dependency from low to high; The target test cases are scheduled to be executed according to the preset strategy.
[0012] Optionally, the version field includes a repository URL and a hash value, wherein the repository URL is the repository address of the corresponding version of the entity file in the second storage area, and the hash value is a unique identifier of the corresponding version of the entity file.
[0013] This application provides a testing system for EDA tools, comprising: The intelligent management module is used to maintain the metadata of the first storage area and the entity files of test cases in the second storage area. The metadata includes at least EDA function fields, version fields that point to the version of the entity file, and EDA tool-related information. Each test case is associated with one or more EDA function fields in the metadata. The dynamic selection module is used to receive test requests and parse the filtering conditions, which include at least EDA function fields; and to determine at least one target test case based on the EDA function fields and metadata, and to obtain the version field and EDA tool related information corresponding to the target test case from the metadata. The test execution module is used to obtain the entity file of the corresponding version from the second storage area according to the version field, and configure the test environment for running the EDA tool in combination with the relevant information of the EDA tool; and drive the EDA tool to perform tests on the target test cases in the test environment to generate raw results.
[0014] Optionally, the testing system for EDA tools further includes an intelligent feedback module, which is used to extract structured key information from the raw results, divide the key information into multiple levels according to preset rules, and present the key information of the multiple levels in a structured form.
[0015] Optionally, the testing system for EDA tools further includes a data analysis module, used to extract EDA domain-specific metrics from the raw results. These EDA domain-specific metrics are information used to characterize the performance and quality of the EDA tool, extracted from the raw results using parsing rules specific to the output format of the EDA tool. The module associates the EDA domain-specific metrics with timestamps, EDA tool versions, test case IDs, and identifiers for each execution of the test cases, and performs at least one of trend analysis, statistical analysis, and regression testing. The module also feeds back the analysis results to the user and / or writes them back to the metadata in the first storage area as new tags.
[0016] This application provides a storage medium storing a computer program, which, when executed by a processor, implements the testing method for EDA tools as described in any of the preceding claims.
[0017] As described above, this application stores lightweight, queryable metadata in the first storage area, supporting complex and efficient filtering; it stores large-volume entity files of various versions in a second storage area, such as Git, enjoying the mature branching, merging, and historical comparison management functions of the second storage area, achieving a balance between efficient querying and fine-grained version management; furthermore, it establishes the association between entity files of various versions and test cases through version fields. Every time a test case changes, whether it is metadata or entity files, a new record will be generated and stored in the corresponding storage area, thereby accurately reproducing the complete test environment of any historical test, achieving test reproducibility and traceability; in addition, testers can directly filter test cases based on verification intent through EDA function fields, without needing to care about the specific file structure and storage path of the test cases, thereby achieving semantic-level filtering, which not only avoids filtering omissions, but also significantly improves testing efficiency while ensuring targeted verification.
[0018] Furthermore, this application can combine other dimensions with EDA functional fields as filtering conditions to form complex filtering logic, thereby meeting the needs of different scenarios such as regression testing, smoke testing, and special testing through multi-dimensional filtering.
[0019] This application can also extract key information from test results (i.e., raw results), hierarchically divide them, and present them in a structured form. Through intelligent parsing and layered presentation, testers can quickly grasp the overall picture, identify key points, and delve into details, thereby greatly shortening the time for problem localization and report analysis. Furthermore, this application can extract and analyze EDA-specific indicators from test results, shifting testing activities from passive "verification pass" to proactive "quality insight" and "risk warning," providing data-driven decision-making support for EDA tool version releases, performance optimization, and problem localization. Attached Figure Description
[0020] Figure 1 This is a flowchart illustrating a testing method for EDA tools according to an embodiment of this application. Figure 2 This is a flowchart illustrating another testing method for EDA tools according to an embodiment of this application; Figure 3 This is a schematic diagram of an intelligent feedback interface for test results provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a test system for EDA tools according to an embodiment of this application. Detailed Implementation
[0021] To address the aforementioned technical problems, this application provides a testing method and system for EDA tools, as well as a storage medium. These protection subjects are based on the same concept, and the principles for solving the problems are basically the same or similar. The implementation methods of each protection subject can be referred to mutually, and repeated details will not be elaborated.
[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described below are only a part of the embodiments of this application, and not all of them. Unless otherwise specified, the following embodiments and their technical features can be combined with each other, and also belong to the technical solutions of this application.
[0023] Figure 1 This is a flowchart illustrating a testing method for EDA tools according to an embodiment of this application. The testing method for EDA tools may also be referred to as a "method" or "testing method," and the executing entity for each step may be an appropriate testing system (e.g., a computer with EDA tool testing capabilities), or a storage medium, processor, controller, etc., with EDA tool testing capabilities.
[0024] See Figure 1 As shown, the method includes at least the following steps S1 to S5.
[0025] S1. Maintain the metadata of the first storage area and the entity files of the test cases in the second storage area. The metadata includes at least EDA function fields, version fields that point to the version of the entity file, and EDA tool-related information. Each test case is associated with one or more EDA function fields in the metadata.
[0026] The first and second storage areas are two independent storage areas used to store metadata and test case entity files separately. In a real-world scenario, the first storage area can be regarded as the metadata database subsystem, and the second storage area can be regarded as the entity file version control subsystem.
[0027] The entity file version control subsystem can be constructed using a version control system such as Git. For large files, a Git LFS extension can be used. Alternatively, a version management system optimized for large binary files, such as a commercial artifact repository like Perforce Helix Core or JFrog Artifactory, can be used. Entity files are collections of test case files, including but not limited to at least one of the following: integrated circuit layout files in GDSII or OASIS format, design rule files in DRC or LVS Rule File format, process model files in SPICE Model or .lib format, simulation stimulus files, reference result files, and execution scripts.
[0028] The metadata database subsystem can be constructed using relational databases such as PostgreSQL, or document databases that support complex queries such as MongoDB. It stores the heterogeneous attributes of test cases through a flexible document model. The metadata describes the test cases and includes at least EDA function fields, version fields, and EDA tool-related information. The version field points to a specific entity file version or an entity file with specific version information. The EDA function fields, as key fields in the metadata, describe specific functionalities performed by the EDA tool, such as Clock Tree Synthesis (CTS) optimization, Signal Integrity (SI) analysis, and Antenna Effect checks. Each test case is associated with one or more EDA function fields in the metadata. It should be understood that the metadata may also include other information from existing metadata, including but not limited to structured information such as the test case's unique identifier (ID), name, description, tags, creator, creation time, and status.
[0029] The version field acts as a versioned pointer or index, telling the testing system which version of the entity file for a given test case it belongs to, and in which repository of the entity file version control subsystem. During test execution, the testing system uses this pointer or index to retrieve the complete entity file of that version from the entity file version control subsystem, thus accurately reproducing the test environment at that time. In one example, the version field may include a repository URL (Uniform Resource Locator) and a hash value. These can be concatenated into a single string or stored separately. The repository URL is the repository address of the corresponding version of the entity file in the second storage area, and the hash value is a unique identifier for the corresponding version of the entity file, which can be calculated based on the entity file content.
[0030] S2. Receive the test request and parse it to obtain the filter conditions that include at least the EDA function fields.
[0031] S3. Based on the EDA function fields and metadata, determine at least one target test case, and obtain the version field and EDA tool related information corresponding to the target test case from the metadata.
[0032] In real-world scenarios, testing systems can provide graphical interfaces or application programming interfaces (APIs) to receive filtering criteria input by testers. These criteria can be expressed in natural or structured language, then converted into structured query statements (SQL) targeting the metadata database subsystem. For example, when a tester selects "antenna effect check" as the function to be tested, the constructed structured query statement contains the value of that function. The testing system executes this structured query statement, searches for all test cases in the metadata whose EDA function field contains that value, and can return a list containing a unique identifier for each test case and its corresponding version field.
[0033] In one example, the filtering criteria may also include other dimensions, including but not limited to at least one of the following: tags, priority, EDA tool type, EDA tool version, process node, creator, creation time, input / output characteristics of test cases, and impact analysis of EDA tool code changes.
[0034] The input / output features of test cases are not simply filenames or paths, but rather features extracted from the input files and historical outputs of the test cases, which can be used for comparison and matching. Input features include, but are not limited to, at least one of the following: layout file size, number of layers, metal layers, density, and spacing range; process node, such as 7nm, 5nm; rule file type, such as DRC, LVS, PEX; netlist size and structural features; simulation stimulus type and configuration parameters; whether it contains a special structure, such as antenna structure, clock tree, SRAM, etc. Output features include, but are not limited to, at least one of the following: test pass / fail status; number of violations and violation type; timing indicators, such as WNS, TNS; simulation convergence status, accuracy, and power consumption; runtime and peak memory usage; error mode and warning type, etc. These input and / or output features can form a feature profile of the test cases. For example, the screening criteria include input features such as "metal layer M1, minimum spacing 0.05um, high density," and the test system can use this to filter out test cases with similar input features that have previously been found to have "metal spacing violations."
[0035] EDA tool code change impact analysis refers to analyzing which functional modules, code paths, or tool behaviors are affected when the source code, rule files, scripts, or configuration of an EDA tool are changed, and then mapping these impacts to relevant test cases. For example, if the code responsible for "antenna effect checking" in an EDA tool is modified, the testing system can determine the impact of this modification on the "antenna effect checking" function through code dependency graphs, call relationships, historical coverage, defect correlations, etc., and then automatically recommend all test cases related to "antenna effect checking" or those that have historically covered this code.
[0036] In step S3, test cases associated with EDA function fields are selected from the metadata, and test cases corresponding to other dimensions are selected from these as target test cases. In other words, the embodiments of this application can adopt a "multi-dimensional intelligent filtering based on EDA functions" mechanism to solve the technical problems in the prior art where the filtering conditions are too simple and cannot match the verification intentions of testers.
[0037] S4. Retrieve the corresponding version entity file from the second storage area based on the version field, and configure the test environment for running the EDA tool in conjunction with the relevant EDA tool information.
[0038] S5. Drive the EDA tool in the test environment to perform tests on the target test cases and generate raw results.
[0039] The testing system allocates and manages the computing resources required to execute tests, including but not limited to server nodes, EDA tool licenses, and storage space. For each scheduled test case, it performs the following operations: 1. Based on the version field in the test case's metadata, it retrieves the complete entity file of the corresponding version from the entity file version control subsystem (i.e., the second storage area) and places it in the local working directory; 2. Based on the EDA tool, EDA tool version, and environment variables specified in the test case's metadata, it configures the corresponding test environment; 3. It calls a predefined startup script in the local working directory, such as run.sh, to drive the EDA tool to execute tests in the test environment.
[0040] In one example, embodiments of this application may first determine a preset strategy, and then schedule target test cases to execute tests according to the preset strategy. The preset strategy may be to schedule the target test cases according to their priority from high to low, and / or according to their resource dependency from low to high.
[0041] Based on the aforementioned steps S1 to S5, the embodiments of this application can achieve the following effects: First, lightweight, queryable metadata is stored in the first storage area, supporting complex and efficient filtering. Larger entity files of various versions are stored in the second storage area, enjoying its mature management functions such as branching, merging, and historical comparison, achieving a balance between efficient querying and fine-grained version management. Furthermore, a version field establishes a relationship between entity files of different versions and test cases. Each time a test case changes, whether it's metadata or entity files, a new record is generated and stored in the corresponding storage area. This allows for precise reproduction of the test environment for any historical test, ensuring test reproducibility and traceability. This fundamentally eliminates the problem of unreliable test results caused by inconsistent entity file versions and environments. For example, when reproducing a problem found in version 1.2 of the EDA tool, the testing system can ensure complete reproduction of the original test environment, including test case files, EDA tool version, configuration parameters, etc., greatly improving the efficiency of fault location and problem resolution. Simulation verification shows that using this method can reduce unreproducible problems caused by environmental differences by more than 90%.
[0042] Second, testers can directly filter test cases based on verification intent using EDA function fields, without needing to concern themselves with the specific file structure and storage path of the test cases. This semantic-level filtering not only avoids omissions but also significantly improves testing efficiency while ensuring targeted verification. Furthermore, this application can combine other dimensions with EDA function fields as filtering conditions to form complex filtering logic, meeting the needs of different scenarios such as regression testing, smoke testing, and specialized testing through multi-dimensional filtering. Testers no longer need to perform fuzzy searches by name or path among thousands of test cases, greatly improving the efficiency of test case filtering and combination, and realizing intelligent test-driven testing from "file-level" to "function intent-level". For example, to verify the "correctness of antenna effect rules under 7nm process", only the corresponding EDA function field, process node, and EDA tool version need to be selected once, and the testing system can automatically filter out all relevant test cases. Practical tests show that compared with traditional filtering methods based on folder traversal and script matching, this application can reduce the filtering time from several hours to several minutes, improving efficiency by more than 10 times, while avoiding subjective omissions in manual filtering and ensuring the completeness of test coverage.
[0043] Figure 2 This is a flowchart illustrating another testing method for EDA tools according to an embodiment of this application. Figure 2 As shown, the test method in this embodiment includes the following steps S21 to S26.
[0044] S21. Maintain the metadata of the first storage area and the entity files of the test cases in the second storage area. The metadata shall contain at least the EDA function field, the version field and the EDA tool related information. Each test case shall be associated with one or more EDA function fields in the metadata.
[0045] S22. Receive the test request and parse it to obtain the filtering conditions that include at least the EDA function fields.
[0046] S23. Determine at least one target test case based on EDA function fields and metadata, and obtain the version field and EDA tool related information corresponding to the target test case from the metadata.
[0047] S24. Obtain the corresponding version entity file from the second storage area according to the version field, and configure the test environment for running the EDA tool in combination with the relevant information of the EDA tool.
[0048] S25. Drive the EDA tool in the test environment to perform tests on the target test cases and generate raw results.
[0049] S26. Extract structured key information from the original results, divide the key information into multiple levels according to preset rules, and present the key information of multiple levels in a structured form.
[0050] This embodiment is the same as the aforementioned Figure 1 For the same technical features of the embodiments, please refer to the foregoing description. For example, steps S21 to S25 are implemented in the same way as steps S1 to S5.
[0051] In the actual scenario of this embodiment, after receiving the raw result output in step S25, the testing system can extract key information from the raw result using regular expressions, keyword matching, dedicated parsing plugins, etc., such as test status (pass / fail / error), runtime, peak memory usage, number and type of violations found, error message summary, etc., and then divide it into multiple levels according to preset rules. The preset rules include stratifying the key information from general to detailed and / or from result to suggestion. Figure 3 This is a schematic diagram of a smart feedback interface for test results provided in an embodiment of this application, combined with... Figure 3 As shown, the resulting multiple levels can be represented as the following five levels.
[0052] Execution summary layer: Displays the most essential information such as test status, pass rate, and total time.
[0053] Key metrics layer: Displays performance metrics (e.g., runtime, memory usage) and quality metrics (e.g., number of violations, number of errors) using structured data (e.g., lists, charts).
[0054] Issue Refinement Layer: Categorizes (errors, warnings, messages) and summarizes the issues found in the tests, and provides links to view detailed information.
[0055] Deep information link layer: Provides access points for deep information such as "Download complete logs", "Download detailed reports", "View performance charts", and "View comparative analysis".
[0056] Intelligent suggestion layer: Based on test results and analysis, it provides actionable suggestions for failed test cases, such as "re-execute," "re-execute with a different version," "re-execute after updating the baseline," and "submit a defect report." The baseline is a comparison benchmark or reference value. For example, if the current runtime is 120 seconds, the baseline might be a value from a previous test execution or a historical average.
[0057] The testing system can present the extracted key information to users through a web interface, API response, or report document. The presentation effect can be specifically manifested as follows: Figure 3 The interface shown.
[0058] Continue reading Figure 2 As shown, the method in this embodiment may further include the following step S27.
[0059] S27. Extract EDA domain-specific metrics from the raw results, associate the EDA domain-specific metrics with timestamps, EDA tool versions, test case IDs, and identifiers for each execution of test cases, and perform at least one of trend analysis, statistical analysis, and regression testing; and feed the analysis results back to the user and / or write them back to the metadata of the first storage area as new tags.
[0060] It should be noted that although the embodiments of this application use step codes such as S21 and S22, their purpose is to describe the corresponding content more clearly and concisely, and they do not constitute a substantial limitation on the order. When implementing the solution of this application, those skilled in the art may execute step S27 first and then step S26, etc., and these should all be within the protection scope of this application.
[0061] EDA-specific metrics are information extracted from the raw results using parsing rules tailored to the output format of EDA tools, used to characterize the performance and quality of EDA tools.
[0062] The testing system incorporates built-in rules for parsing the raw results output by different EDA tools. For example, for Calibre's DRC report, it can extract "Total errors found"; for Prime Time's time-series report, it can extract "Worst Negative Slack (WNS)" and "Total Negative Slack (TNS)"; and for HSPICE simulation logs, it can extract simulation time, convergence status, etc. These EDA-specific metrics are key to evaluating the performance and quality of EDA tools. Then, through time-series extensions such as InfluxDB or PostgreSQL, the EDA-specific metrics extracted for each test are persistently stored and associated with information such as timestamps, EDA tool versions, test case IDs, and identifiers for each test case execution. Next, analysis is performed; for example, it can generate performance-over-time curves, success rate trend graphs, and automatically detect regressions, such as a significant increase in average runtime or a deterioration in the WNS value when running a specific test case set for a particular version of the EDA tool.
[0063] Compared to the raw feedback method of log dumping, this application embodiment, through intelligent parsing and layered presentation, enables testers to quickly grasp the overall picture, pinpoint key issues, and delve into details, thereby significantly shortening the time for problem localization and report analysis. Furthermore, this application embodiment extracts and analyzes EDA-specific indicators from the test results, shifting testing activities from passive "verification pass" to proactive "quality insight" and "risk warning," providing data-driven decision-making support for EDA tool version releases, performance optimization, and problem localization. In practical scenarios, see [reference needed]. Figure 3 As shown, testers can quickly move from the execution summary layer (e.g., "pass rate 95%)" to the key performance indicator layer (e.g., "average runtime 120s, a 5% decrease from the baseline"), and then focus on the issue refinement layer (e.g., "5 new DRC short-circuit violations found, located in module X"). This changes the traditional model of manually sifting through thousands of lines of log files to troubleshoot issues. Based on experimental comparative testing, for regression testing containing hundreds of test cases, this embodiment can reduce the average time for result analysis and root cause localization from the traditional 4-6 person-hours to less than 0.5 person-hours. Simultaneously, automated trend analysis can provide early warnings of performance degradation. For example, immediately after a version of an EDA tool is launched, it can be found that the average time for running specific simulation test cases has increased by 20%, triggering an in-depth investigation.
[0064] Furthermore, the management, filtering, execution, analysis, and feedback of test-related data in this application embodiment are not isolated, but are tightly coupled through the data flow and state flow described in the preceding steps, forming a complete closed-loop feedback system. Insights generated from the analysis (such as "Version A's false alarm rate increases in DRC checks") can be directly transformed into action recommendations ("It is recommended to focus on verifying the DRC rules of Version A in the next regression test"), and corresponding verification suites can be quickly built. This makes test data no longer a dormant archive, but rather "fuel" driving continuous improvement in product quality. For example, a project analyzed historical data using this application and discovered that a certain type of layout and routing use case is highly prone to timing violations in specific scenarios. Based on this, the testing strategy was optimized, reducing the occurrence rate of similar problems in subsequent versions by 70%.
[0065] Figure 4 This is a schematic diagram of the structure of a testing system for EDA tools according to an embodiment of this application. Figure 4 As shown, the test system for EDA tools (hereinafter referred to as "System") 40 includes: The intelligent management module 41 is used to maintain the metadata of the first storage area and the entity files of test cases in the second storage area. The metadata includes at least EDA function fields, version fields that point to the version of the entity file, and EDA tool-related information. Each test case is associated with one or more EDA function fields in the metadata. The dynamic selection module 42 is used to receive test requests and parse the filtering conditions, which include at least EDA function fields; and to determine at least one target test case based on the EDA function fields and metadata, and to obtain the version field and EDA tool related information corresponding to the target test case from the metadata. Test execution module 43 is used to retrieve the corresponding version entity file from the second storage area according to the version field, and configure the test environment for running the EDA tool in combination with EDA tool related information; and drive the EDA tool to execute tests on the target test cases in the test environment to generate raw results.
[0066] Through the cooperation of the above modules, the testing of EDA tools can be completed.
[0067] It should be understood that the various modules of the test system 40 for EDA tools can be represented as physical devices or virtual modules (i.e., logical modules) in actual scenarios. A module can be implemented by a single physical device or by two or more physical devices working together. Similarly, the function performed by a module can be implemented by a single physical device or by two or more physical devices working together.
[0068] Furthermore, the functions corresponding to each of the above modules can be implemented by the corresponding steps of the testing method for EDA tools in any of the foregoing embodiments, thus the system has the same beneficial effects. For example, see [link to relevant documentation]. Figure 4 As shown, the testing system 40 for EDA tools may further include an intelligent feedback module 44 and a data analysis module 45. The intelligent feedback module 44 is used to extract structured key information from the raw results, divide the key information into multiple levels according to preset rules, and present the key information at multiple levels in a structured form; the preset rules may include hierarchical arrangement of key information from general to detailed and / or from results to recommendations. The data analysis module 45 is used to extract EDA domain-specific metrics from the raw results. These EDA domain-specific metrics are information used to characterize the performance and quality of the EDA tool, extracted from the raw results using parsing rules specific to the EDA tool's output format. Then, the EDA domain-specific metrics are associated with timestamps, EDA tool versions, test case IDs, and identifiers for each test case execution, and at least one of trend analysis, statistical analysis, and regression testing is performed. The analysis results are then fed back to the user and / or written back as new tags to metadata in the first storage area.
[0069] This application embodiment also provides a storage medium storing a test program for EDA tools, which may be simply referred to as a "program" or "test program". The test program for EDA tools is essentially a computer program, and when the test program for EDA tools is executed by a processor, it implements the steps of the test method for EDA tools in any of the foregoing examples.
[0070] The storage medium includes, but is not limited to, any one of read-only memory (ROM), random access memory (RAM), magnetic disk, and optical disk.
[0071] Since the test program for EDA tools stored in the storage medium can execute the steps in the test method for EDA tools of any embodiment provided in this application, the beneficial effects that the test method for EDA tools of any of the foregoing embodiments can achieve can be realized. For details, please refer to the foregoing embodiments, which will not be repeated here.
[0072] This application also provides a test device (hereinafter referred to as "test device" or "device") or chip for EDA tools, including a memory and a processor. The memory stores a test program for EDA tools. When the test program for EDA tools is executed by the processor, it implements the steps of the test method for EDA tools of any of the foregoing embodiments. And / or, the test device or chip for EDA tools is provided with a storage medium as shown in the above example. The processor loads the storage medium to execute the steps of the test method for EDA tools of the corresponding embodiment, thereby achieving the beneficial effects that the test method for EDA tools of the corresponding embodiment can achieve.
[0073] The above are only some embodiments of this application and do not limit the patent scope of this application. For those skilled in the art, any equivalent structural transformations made using the content of this specification and drawings are similarly included within the patent protection scope of this application.
[0074] Although this document uses terms such as "first," "second," etc., to describe various types of information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. Furthermore, the singular forms "a," "an," and "the" are intended to also include the plural forms. The terms "or" and "and / or" are interpreted as inclusive, or meaning either one or any combination thereof. Exceptions to this definition only arise when combinations of elements, functions, steps, or operations are inherently mutually exclusive in some way.
Claims
1. A testing method for EDA tools, characterized in that, include: Maintain metadata in the first storage area and entity files of test cases in the second storage area. The metadata includes at least EDA function fields, version fields pointing to the version of the entity file, and EDA tool-related information. Each test case is associated with one or more EDA function fields in the metadata. Receive test requests and parse them to obtain filter conditions that include at least EDA function fields; Based on the EDA function fields and metadata, at least one target test case is determined, and the version field and EDA tool-related information corresponding to the target test case are obtained from the metadata. Based on the version field, retrieve the corresponding version entity file from the second storage area, and configure the test environment for running the EDA tool in conjunction with the relevant EDA tool information; Drive EDA tools in the test environment to execute tests on the target test cases and generate raw results.
2. The method according to claim 1, characterized in that, The method further includes: Extract structured key information from the original results; The key information is divided into multiple levels according to preset rules; The key information from the multiple levels is presented in a structured form.
3. The method according to claim 2, characterized in that, The preset rules include stratifying the key information from general to detailed and / or from results to recommendations.
4. The method according to claim 2, characterized in that, The method further includes: Extract EDA domain-specific metrics from the original results. These EDA domain-specific metrics are information used to characterize the performance and quality of the EDA tool, extracted from the original results using parsing rules specific to the output format of the EDA tool. The EDA-specific metrics are associated with timestamps, EDA tool versions, test case IDs, and identifiers for each test case execution, and at least one of trend analysis, statistical analysis, and regression testing is performed. The analysis results are fed back to the user and / or written back as new tags to the metadata in the first storage area.
5. The method according to claim 1, characterized in that, The filtering criteria also include other dimensions; The determination of at least one target test case based on the EDA function fields and metadata includes: Select test cases associated with the EDA function fields from the metadata, and select test cases corresponding to the other dimensions from the metadata as target test cases.
6. The method according to claim 5, characterized in that, The other dimensions include at least one of the following: tags, priority, EDA tool type, EDA tool version, process node, creator, creation time, input / output characteristics of test cases, and impact analysis of EDA tool code changes.
7. The method according to claim 6, characterized in that, The process of driving EDA tools to perform tests on target test cases in the test environment includes: A preset strategy is determined, which includes scheduling according to the priority of the target test cases from high to low, and / or scheduling according to the resource dependence from low to high. The target test cases are scheduled to be executed according to the preset strategy.
8. The method according to claim 1, characterized in that, The version field includes a repository URL and a hash value. The repository URL is the repository address of the corresponding version of the entity file in the second storage area, and the hash value is a unique identifier of the corresponding version of the entity file.
9. A testing system for EDA tools, characterized in that, include: The intelligent management module is used to maintain the metadata of the first storage area and the entity files of test cases in the second storage area. The metadata includes at least EDA function fields, version fields that point to the version of the entity file, and EDA tool-related information. Each test case is associated with one or more EDA function fields in the metadata. The dynamic selection module is used to receive test requests and parse them to obtain filtering conditions that include at least EDA function fields; Furthermore, at least one target test case is determined based on the EDA function fields and metadata, and the version field and EDA tool-related information corresponding to the target test case are obtained from the metadata; The test execution module is used to obtain the entity file of the corresponding version from the second storage area according to the version field, and configure the test environment for running the EDA tool in combination with the relevant information of the EDA tool; and drive the EDA tool to perform tests on the target test cases in the test environment to generate raw results.
10. A storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the testing method for EDA tools as described in any one of claims 1 to 8.
