Simulation software testing method, computer device, storage medium and program product

Abstract

The application relates to a simulation software testing method, a computer device, a storage medium and a program product. The method comprises the following steps: obtaining a test data set of simulation software; the test data set comprises at least one test data; the test data comprises layout extraction data and simulation configuration data; based on the test data, a test scene corresponding to the test data is determined; based on the test scene and the test data, the simulation software is tested to obtain at least one preliminary test result; and based on the preliminary test result of each test data, a test result of the simulation software is determined. The method can reduce test time consumption and improve test flexibility.

Description

Technical Field

[0001] This application relates to the field of testing technology, and in particular to a simulation software testing method, computer equipment, storage medium, and program product. Background Technology

[0002] In the integrated circuit industry today, simulation software is often used for verification during circuit manufacturing. However, with the development of the integrated circuit industry, the complexity of the simulation parameters input in the early stage and the amount of simulation data output in the later stage have gradually increased. The amount of data required to verify the correctness of the simulation software results has also increased, and there are certain data differences in different test scenarios. Therefore, there are problems such as long test time and low test flexibility.

[0003] This shows that the current field of simulation software testing still suffers from problems such as long testing time and low testing flexibility. Summary of the Invention

[0004] Therefore, it is necessary to provide a simulation software testing method, computer equipment, storage medium, and program product that can reduce testing time and improve testing flexibility in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a simulation software testing method, the simulation software testing method comprising:

[0006] Obtain a test dataset for the simulation software; the test dataset includes at least one set of test data; the test data includes layout extraction data and simulation configuration data;

[0007] Based on the test data, determine the test scenario corresponding to the test data;

[0008] Based on the test scenario and the test data, the simulation software is tested to obtain at least one preliminary test result;

[0009] Based on the preliminary test results of each of the test data, the test results of the simulation software are determined.

[0010] In one embodiment, the test data further includes data attributes; the test scenario includes at least one of regression testing, process testing, and model testing; determining the test scenario corresponding to the test data based on the test data includes:

[0011] Based on the data attributes, determine the test scenario corresponding to the data attributes.

[0012] In one embodiment, the test scenario includes regression testing; the step of testing the simulation software based on the test scenario and the test data to obtain at least one preliminary test result includes:

[0013] Based on the simulation configuration data, the simulation software is configured;

[0014] The extracted layout data is input into the configured simulation software to obtain the output results;

[0015] Based on the output results, the preliminary test results of the simulation software are determined.

[0016] In one embodiment, the test scenario includes process testing; the simulation configuration data includes runtime configuration parameters, which include process tools and the number of processes; the step of testing the simulation software based on the test scenario and the test data to obtain at least one preliminary test result includes:

[0017] Obtain the process toolset and preset process number set corresponding to the simulation software;

[0018] The process management tools in the process toolset and the number of processes in the preset process number set are combined to obtain at least one process configuration combination;

[0019] The running configuration parameters are adjusted based on the at least one process configuration combination to obtain the adjusted test data for at least one process.

[0020] Based on at least one of the process test data, process testing is performed on the simulation software to obtain at least one preliminary test result.

[0021] In one embodiment, the process testing of the simulation software based on at least one of the process test data to obtain at least one preliminary test result includes:

[0022] The simulation software is configured based on the simulation configuration data in the process test data;

[0023] The extracted layout data is input into the configured simulation software to obtain the output results;

[0024] Based on the output results, the preliminary test results of the simulation software are determined.

[0025] In one embodiment, the test scenario includes model testing; the simulation configuration data further includes software configuration parameters; and the step of testing the simulation software based on the test scenario and the test data to obtain at least one preliminary test result includes:

[0026] Obtain at least one parameter template file and at least one parameter range file for the simulation software;

[0027] By combining at least one of the parameter template files and at least one of the parameter range files, at least one model configuration combination is obtained;

[0028] Based on the at least one model configuration combination, the software configuration parameters are adjusted respectively to obtain at least one model test data after adjustment;

[0029] Based on at least one of the adjusted model test data, the simulation software is tested to obtain at least one preliminary test result.

[0030] In one embodiment, the step of performing model testing on the simulation software based on at least one of the model test data to obtain at least one preliminary test result includes:

[0031] The simulation software is configured based on the simulation configuration data in the model test data;

[0032] The extracted layout data is input into the configured simulation software to obtain the output results;

[0033] Based on the output results, the preliminary test results of the simulation software are determined.

[0034] In one embodiment, determining the preliminary test results of the simulation software based on the output results includes:

[0035] Based on the simulation result comparison data items required for the current test scenario, obtain each data item in the output result;

[0036] Compare each data item in the output result with each data item in the preset output result;

[0037] If the differences of each data item all meet the preset allowable range, then the preliminary test result is passed;

[0038] If the difference between any data items does not meet the preset allowable range, the preliminary test result is a failure, and one or more of the average difference, maximum difference, and variance of the data items are output.

[0039] In one embodiment, determining the preliminary test results of the simulation software based on the output results includes:

[0040] The test data and the output results are fed into the simulation result verification module, which is a simulation model implemented based on at least part of the algorithm logic of the simulation software.

[0041] Based on the test data, the influencing parameters are obtained, including: grid density, simulation stage, parameter template file, and parameter range file;

[0042] The simulation result verification module performs simulation based on the influencing parameters to obtain simulation reference results;

[0043] The output results are compared with the simulation reference results to determine the preliminary test results of the simulation software.

[0044] In a second aspect, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect above.

[0045] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0046] Fourthly, this application provides a computer program product that, when executed by a processor, implements the method described in the first aspect above.

[0047] The aforementioned simulation software testing method, computer equipment, storage medium, and program product acquire a test dataset for the simulation software. The test dataset includes at least one set of test data, comprising layout extraction data and simulation configuration data. Based on the test data, a test scenario corresponding to the test data is determined. Based on the test scenario and the test data, the simulation software is tested to obtain at least one preliminary test result. Based on the preliminary test results for each set of test data, the test result of the simulation software is determined. Through standardized encapsulation of the test data, it can be reused in various simulation software programs and versions of the simulation software at different times. Furthermore, by selecting the corresponding test scenario based on the test data, the test data can be flexibly applied to different test scenarios to perform corresponding tests on the simulation data. This objectively eliminates the subjectivity and delays inherent in manual testing. Compared to the traditional method of manually inputting parameters one by one and manually triggering simulations scene by scene, it can significantly reduce parameter configuration time, scene switching delays, and repetitive simulation overhead, thereby achieving the technical effect of reducing testing time and improving testing flexibility. Attached Figure Description

[0048] Figure 1 This is a diagram illustrating the application environment of a simulation software testing method in one embodiment.

[0049] Figure 2 This is a flowchart illustrating a simulation software testing method in one embodiment;

[0050] Figure 3 This is a flowchart illustrating the process testing process in one embodiment;

[0051] Figure 4 This is a schematic diagram of the model testing process in one embodiment;

[0052] Figure 5 This is a flowchart illustrating the simulation software testing method in another embodiment;

[0053] Figure 6 This is a flowchart illustrating the test data verification process in one embodiment;

[0054] Figure 7 This is a flowchart illustrating the process of verifying simulation result files in one embodiment;

[0055] Figure 8 This is a log illustration of the comparison tool in one embodiment;

[0056] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] The simulation software testing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 is used to acquire test datasets for simulation software; the test datasets include at least one set of test data; the test data includes layout extraction data and simulation configuration data; based on the test data, a test scenario corresponding to the test data is determined; based on the test scenario and the test data, the simulation software is tested to obtain at least one preliminary test result; based on the preliminary test result of each set of test data, the test result of the simulation software is determined. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and wearable devices. Portable wearable devices can be head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0059] In one embodiment, such as Figure 2As shown, a simulation software testing method is provided. Taking the application of this method in a simulation software testing scenario as an example, the simulation software testing method includes:

[0060] Step S100: Obtain the test dataset for the simulation software.

[0061] The simulation software can be an EDA simulation tool for integrated circuit manufacturing processes such as CMP (Chemical Mechanical Polishing) simulation. It accepts layout extraction data and simulation configuration data as input, performs simulation calculations, and outputs response data.

[0062] A test dataset can be a structured collection containing at least one set of test data, used to input into simulation software to verify the software's performance. For example, the test dataset can be stored in a test data folder or obtained through batch import. For example, the test dataset can be generated by extracting and aggregating data from a historical simulation project library, layout design tool output files, or a configuration management platform. For example, the test dataset can adopt various dataset types such as single layout with multiple configurations, multiple layouts with single configurations, and multiple layouts with multiple configurations.

[0063] Test data can be a single unit used for simulation testing. In this embodiment, the test data includes at least layout extraction data and simulation configuration data. For example, the system can store corresponding expected simulation results for each piece of test data. The test data can be matched to the test scenario, corresponding to the operating conditions and expected output range of the simulation software.

[0064] Layout extraction data can be geometric and electrical parameter information extracted from the physical layout of an integrated circuit, which can characterize the actual structure of the circuit. For example, layout extraction data can include physical data such as grid density, spacing, linewidth, and perimeter of the layout to ensure consistency between simulation and actual physical implementation. For example, layout extraction data can be extracted from layout files using layout parsing tools.

[0065] Simulation configuration data can be configuration information that sets the simulation running conditions, and can control the accuracy, speed, coverage, etc., of the simulation software's behavior. For example, simulation configuration data may include simulation parameters, running parameters, etc. In an exemplary embodiment, simulation configuration data can be loaded from the simulation tool's configuration file, script template, or design specifications.

[0066] For example, obtaining the test dataset for the simulation software could be done by reading a pre-built structured test dataset file from a storage system. Furthermore, obtaining the test dataset could involve obtaining a fixed test dataset, such as pulling a snapshot of the test dataset that matches the current version of the simulation software, or obtaining a dynamically combined test dataset, for example, by calling the API interface of a configuration management platform to dynamically assemble the test dataset according to the tags of the simulation software.

[0067] Step S200: Based on the test data, determine the test scenario corresponding to the test data.

[0068] The test scenario can be a scenario used to verify the performance of simulation software in a specific dimension. Different test scenarios can use different test algorithms to verify whether the test data can obtain the expected results in the test scenario.

[0069] Based on the test data, the corresponding test scenarios are determined. For example, the test data may be pre-labeled with the test scenarios to be performed, and the test scenarios to be performed can be determined by extracting the metadata in the test data. Alternatively, data may be extracted from the test data layout and simulation configuration data, and clustering algorithms may be used to determine the matching test scenarios based on the data features. Another option is to use a machine learning-based classifier to classify the data extracted from the layout and simulation configuration data to determine the corresponding test scenarios.

[0070] Step S300: Based on the test scenario and test data, test the simulation software to obtain at least one preliminary test result.

[0071] The preliminary test results can be a set of simulation output data generated after the simulation software executes in a single test scenario, or a preliminary evaluation obtained from basic test result analysis of the simulation software's raw response data. The preliminary test results are output by the simulation software after performing calculations based on the input test data and the test scenario. It can be understood that by comparing the preliminary test results with the expected results, the simulation performance of the simulation software in that test scenario, for that specific test data, can be determined.

[0072] Step S400: Based on the preliminary test results of each test data, determine the test results of the simulation software.

[0073] The test results of the simulation software can be verification conclusions generated by comprehensive analysis of multiple preliminary test results, which are used to provide a unified evaluation of the overall performance of the simulation software.

[0074] For example, the test results of simulation software can be obtained by first analyzing each preliminary test result separately to obtain a single analysis result, and then by comprehensively analyzing multiple analysis results to obtain the final test result of the simulation software. Furthermore, each preliminary test result can be analyzed to determine one or more analytical results such as key indicators, anomalies, and pass rates for each preliminary output result, and finally, the overall test result can be obtained.

[0075] In one exemplary embodiment, the test results of the simulation software can be determined by assessing the degree of consistency between the preliminary test results and the expected results in all or part of the results, assigning a score to each preliminary test result, and then weighting the scores of multiple preliminary test results to obtain the test results of the simulation software. Alternatively, the performance test results of the simulation software can be determined by comprehensively analyzing whether there is abnormal output data in each preliminary test result. Another method is to verify the preliminary test results in other ways to obtain the test results, which is not limited in this embodiment.

[0076] Furthermore, based on the preliminary test results of each test data, the test results of the simulation software can be determined by analyzing the preliminary test results according to the test result evaluation criteria corresponding to the test scenario, and obtaining a single analysis result for each preliminary test result.

[0077] This embodiment provides a simulation software testing method. It involves acquiring a test dataset for the simulation software, including at least one set of test data, comprising layout extraction data and simulation configuration data. Based on the test data, a test scenario corresponding to the test data is determined. The simulation software is then tested based on the test scenario and the test data to obtain at least one preliminary test result. Based on the preliminary test results for each set of test data, the test result of the simulation software is determined. Through standardized encapsulation of the test data, it can be reused in various simulation software programs and versions of the simulation software at different times. Furthermore, by selecting the corresponding test scenario based on the test data, the test data can be flexibly applied to different test scenarios to perform corresponding tests on the simulation data. This objectively eliminates the subjectivity and delays inherent in manual testing. Compared to the traditional method of manually inputting parameters one by one and manually triggering simulations scene by scene, this method effectively reduces parameter configuration time, scene switching delays, and repetitive simulation overhead, thereby achieving the technical effect of reducing testing time and improving testing flexibility.

[0078] In one embodiment, the test data further includes data attributes; based on the test data, determining the test scenario corresponding to the test data includes:

[0079] Based on the data attributes, determine the test scenarios corresponding to the data attributes.

[0080] Data attributes can be metadata tags attached to test data, serving as a basis for determining test scenarios, matching test types with data characteristics, and improving the accuracy of test scheduling. For example, data attributes can be injected during the test data construction phase by design analysis tools or manual annotation systems, and can be embedded in the test data metadata through field enumeration, feature vectors, etc. For example, data attributes can have an input-mapping relationship with test scenarios, enabling automated determination.

[0081] In this embodiment, the test scenario includes at least one of regression testing, process testing, and model testing.

[0082] Regression testing can be a test scenario designed to verify the consistency of simulation software output after design changes or parameter adjustments. It can detect simulation result drift caused by minor modifications during design iterations, ensuring functional stability. In an exemplary embodiment, regression testing can be determined as a test scenario when data attributes include labels indicating the impact of changes. Preliminary test results are obtained by performing simulation tasks on the test data, and the historical benchmark results corresponding to the test data are compared for difference.

[0083] Process testing can be a type of test that verifies the stability of simulation software execution under different operating environments or configurations. It can detect the reliability and consistency of the simulation engine under one or more conditions such as parallel computing, resource scheduling, and changes in environmental variables. For example, process testing can be determined as a test scenario when data attributes include tags related to execution stability. Furthermore, process testing can be conducted by repeatedly executing the same test data and monitoring output fluctuations to obtain preliminary test results for evaluation.

[0084] Model testing can be a type of test that verifies the accuracy and physical modeling integrity of the algorithm model used by simulation software, thereby confirming whether the simulation model's modeling accuracy for complex physical phenomena meets expectations. Furthermore, the verified algorithm model can include physical modules, machine learning models, or mathematical models from different process stages. Model testing can include testing of the algorithm model itself, as well as verification of model parameter configurations. For example, model testing can be triggered when data attributes include tags indicating high modeling complexity. This involves loading the corresponding model file and parameter constraint file at a specific process stage, executing the simulation, obtaining preliminary test results, and then comparing and verifying them.

[0085] Determining the test scenario based on data attributes can be achieved by reading the data attribute tags embedded in the test data, matching the attributes with predefined tests through preset mapping rules or lookup tables, thereby realizing the automated selection of test scenarios, eliminating the delay and subjective bias of manual classification, and ensuring that the test type is accurately aligned with the data features.

[0086] This embodiment provides a simulation software testing method that determines test scenarios corresponding to data attributes based on data attributes. It introduces a structured test type that automatically maps data attributes to regression testing, process testing, and model testing, forming a complementary test dimension system. This system verifies the impact of design changes, the stability of the simulation process, and the accuracy of the algorithm model, thereby achieving intelligent triggering of test scenarios and precise resource allocation. This significantly improves the configurability and response speed of test scenarios, achieving the technical effect of reducing test time and increasing test flexibility without increasing hardware resources.

[0087] In one embodiment, the test scenario includes regression testing; based on the test scenario and test data, the simulation software is tested to obtain at least one preliminary test result, including:

[0088] Configure the simulation software based on the simulation configuration data;

[0089] The extracted layout data is input into the configured simulation software to obtain the output results;

[0090] Based on the output results, the preliminary test results of the simulation software are determined.

[0091] In this embodiment, simulation configuration data is used to dynamically set the operating parameters of the simulation software. By adjusting the parameters of the simulation software, it can be adapted to different test objectives. For example, configuring the simulation software based on the simulation configuration data can involve reading the simulation configuration data and dynamically modifying the internal parameters of the simulation engine to bring it into a running state adapted to the current test scenario. For example, the simulation configuration data can be a pre-compiled configuration snapshot file, which is used to replace the simulation software's configuration file, thereby enabling the configuration of the simulation software.

[0092] The layout extraction data is input into the configured simulation software. Alternatively, the layout extraction data can be used as input to a simulation software instance with completed parameter configuration, triggering simulation calculations and outputting the original response to obtain the output result.

[0093] The output can be the raw response data generated by the simulation software after receiving the configuration and performing simulation calculations on the layout data. Based on the output, the preliminary test results of the simulation software are determined. For example, this can be achieved by applying predefined difference detection rules to the output, extracting key node indicators, and generating binary or hierarchical judgment results, thereby automating the generation of preliminary test results, shortening the interpretation delay, and improving consistency.

[0094] This embodiment provides a simulation software testing method that configures the simulation software based on simulation configuration data, inputs layout extraction data into the configured simulation software to obtain output results, determines the preliminary test results of the simulation software based on the output results, and achieves the technical effects of reducing testing time and improving testing flexibility by reconstructing simulation environment parameters and simulating layout extraction data based on the configured simulation software.

[0095] In one embodiment, the simulation software is tested based on the test scenario and test data to obtain at least one preliminary test result, including:

[0096] Obtain the process toolset and preset process count set corresponding to the simulation software;

[0097] Combine the process management tools in the process toolset with the number of processes in the preset process number set to obtain at least one process configuration combination;

[0098] The runtime configuration parameters are adjusted based on at least one process configuration combination to obtain the adjusted test data for at least one process.

[0099] Based on at least one process test data, process testing is performed on the simulation software to obtain at least one preliminary test result.

[0100] In this embodiment, the test scenario includes process testing; the simulation configuration data includes runtime configuration parameters, which include process tools and the number of processes.

[0101] A process toolset can be an enumerated set of various parallel computing process management tools supported by the simulation software. As a variable dimension for resource scheduling strategies in process testing, it can support a systematic evaluation of simulation performance under different parallel architectures and execution modes. For example, a process toolset may include, but is not limited to, one or more of the following: distributed message passing process tools, shared memory thread process tools, and heterogeneous computing acceleration process tools.

[0102] A preset set of process counts represents the number of parallel execution units in simulation software. It is used to set the scale of computing resources, thereby supporting comparative analysis of simulation efficiency, resource contention intensity, and task scheduling latency under different concurrency levels. For example, the preset set of process counts can be predefined by the system administrator or performance benchmark, and can generate non-overlapping enumeration values ​​based on factors such as the number of CPU cores, memory capacity, cache structure, or number of cluster nodes on the hardware platform.

[0103] At least one process configuration combination is obtained by combining the process management tools in the process toolset with the number of processes in the preset process number set. The process management tool can be a single callable parallel computing execution framework in the process toolset, used to control thread or process allocation, communication mechanisms, and synchronization mechanisms for simulation tasks. In this embodiment, the process management tool can be extracted from the process toolset and invoked through command-line parameters of the simulation tool, configuration file injection, API calls, etc.

[0104] By combining process management tools from the process toolset with process numbers from a preset process number set, or by pairing process generation tools with process numbers, a process configuration combination can be obtained. This process configuration combination can be used to compare the parallelization performance of simulations under different resource scheduling strategies, parallel frameworks, and execution modes.

[0105] Adjusting the runtime configuration parameters based on at least one process configuration combination can be done by replacing each runtime configuration parameter with the process management tool and the number of processes in the current process configuration combination, based on the original test data.

[0106] In this embodiment, the preliminary test results can be performance indicators collected after running the simulation software under different process configuration combinations based on the adjusted process test data, such as running time, peak memory usage, and convergence steps, to reflect the differences in simulation performance under different parallel configurations.

[0107] This embodiment provides a simulation software testing method. It involves acquiring a process toolset and a preset set of process numbers corresponding to the simulation software; combining the process management tools in the process toolset and the process numbers in the preset set to obtain at least one process configuration combination; adjusting the running configuration parameters based on each of the at least one process configuration combination to obtain at least one adjusted process test data; and performing process testing on the simulation software based on the at least one process test data to obtain at least one preliminary test result. By combining different process management tools and different process numbers, diverse process configuration combinations can be generated. This allows for the parallel coverage of multiple resource scheduling strategies in a single test process, transforming the performance tuning tests that originally required multiple independent runs into a single automated experiment executed serially or in parallel. This enables the simulation software to run under different parallel architectures, simultaneously collecting multi-dimensional performance indicators such as throughput, memory usage, and convergence speed. This allows for the rapid identification of the optimal resource configuration combination and dynamic adaptation to the running requirements of different hardware platforms without modifying the original test data. This significantly improves the compatibility and response speed of testing in heterogeneous environments, achieving the technical effects of reducing testing time and increasing testing flexibility.

[0108] In one embodiment, process testing is performed on the simulation software based on at least one process test data to obtain at least one preliminary test result, including:

[0109] The simulation software is configured based on the simulation configuration data in the process test data;

[0110] The extracted layout data is input into the configured simulation software to obtain the output results;

[0111] Based on the output results, the preliminary test results of the simulation software are determined.

[0112] In this embodiment, simulation configuration data is used to dynamically set the running parameters of the simulation software, including the process tools and the number of processes used. For example, configuring the simulation software based on the simulation configuration data can involve reading the simulation configuration data and dynamically modifying the internal parameters of the simulation engine to adapt it to the current test scenario. For example, the simulation configuration data can be a pre-compiled configuration snapshot file, which is used to replace the simulation software's configuration file, thereby configuring the simulation software.

[0113] The layout extraction data is input into the configured simulation software. Alternatively, the layout extraction data can be used as input to a simulation software instance with completed parameter configuration, triggering simulation calculations and outputting the original response to obtain the output result.

[0114] The output can be the raw response data generated by the simulation software after receiving the configuration and performing simulation calculations on the layout data. Based on the output, the preliminary test results of the simulation software are determined. For example, this can be achieved by applying predefined difference detection rules to the output, extracting key node indicators, and generating binary or hierarchical judgment results, thereby automating the generation of preliminary test results, shortening the interpretation delay, and improving consistency.

[0115] This embodiment provides a simulation software testing method that configures the simulation software based on simulation configuration data from process test data; inputs layout extraction data into the configured simulation software to obtain output results; determines the preliminary test results of the simulation software based on the output results; configures the simulation software using reconstructed process test data; and performs simulation on the layout extraction data based on the configured simulation software. This method can achieve simulation software testing for different process configuration combinations, thereby reducing testing time and improving testing flexibility.

[0116] In one embodiment, the test scenario includes model testing; the simulation configuration data also includes software configuration parameters; based on the test scenario and test data, the simulation software is tested to obtain at least one preliminary test result, including:

[0117] Obtain at least one parameter template file and at least one parameter range file from the simulation software;

[0118] Combine at least one parameter template file and at least one parameter range file to obtain at least one model configuration combination;

[0119] The software configuration parameters are adjusted based on at least one model configuration combination to obtain test data for at least one adjusted model.

[0120] Based on at least one adjusted model test data, the simulation software is tested to obtain at least one preliminary test result.

[0121] Parameter template files can be standardized configuration skeletons that define the parameter structure and field relationships of simulation software, providing a reusable parameter configuration framework for model testing. In this embodiment, the parameter template file can include parameter names, data types, hierarchical nesting, dependencies, and field constraint rules to ensure structural consistency in parameter configuration. For example, parameter template files can be extracted from the configuration template library of the simulation tool, design specification documents, etc., and serialized into structured format files such as JSON and YAML.

[0122] A parameter range file can be a constraint file that defines the range of possible values ​​for simulation parameters, and can be used to limit the exploration boundaries of the parameter space. For example, a parameter range file can set minimum values, maximum values, step sizes, sampling strategies, etc., for parameters. For example, a parameter range file can be determined based on process characteristics, historical simulation results, or other prior knowledge such as design specifications.

[0123] A model configuration combination can be a complete set of simulation parameters generated by combining parameter template files and parameter range files through automated rules. It can be used to systematically enumerate the simulation parameter space. In this embodiment, a model configuration combination can represent an independent parameter configuration instance used to drive model test execution. For example, the model configuration combination can be obtained by combining the field structure of the template file and the value set of the range file using a combination algorithm. This enables systematic enumeration of the simulation parameter space, avoiding the inefficient method of manually constructing test cases line by line, and improving the coverage and efficiency of configuration generation.

[0124] The software configuration parameters are adjusted based on at least one model configuration combination. This can be achieved by replacing the corresponding fields in the simulation configuration data of the original test data with parameter values ​​from each model configuration combination, generating new test data instances. Based on at least one adjusted model test data set, the simulation software is tested to obtain at least one preliminary test result. This can be achieved by using each adjusted model test data set as an independent input to drive the simulation software to execute the model test and output the corresponding original simulation response data as the preliminary test result.

[0125] In one specific embodiment, the software configuration parameters are adjusted based on at least one model configuration combination. This can be achieved by randomly generating one or more random numbers according to the parameter range determined in the parameter range file and filling them into the parameter template file to obtain the software configuration parameters.

[0126] This embodiment provides a simulation software testing method that involves acquiring at least one parameter template file and at least one parameter range file for the simulation software; combining the at least one parameter template file and at least one parameter range file to obtain at least one model configuration combination; adjusting the software configuration parameters based on the at least one model configuration combination to obtain at least one adjusted model test data; and performing model testing on the simulation software based on the at least one adjusted model test data to obtain at least one preliminary test result. By introducing parameter template files and parameter range files to construct reusable and combinable parameter configuration units, the flexibility of configuration can be effectively improved. The adjusted model test data can make the test input have parameter space coverage and diversity, transforming the original serial test mode that required multiple independent runs and manual parameter switching into a parallel test flow that can be configured once, generated in batches, and executed automatically. This significantly reduces the time overhead of repeatedly loading the simulation environment and manual intervention, thereby achieving the technical effect of reducing test time and improving test flexibility without increasing hardware resources.

[0127] In one embodiment, the simulation software is tested based on at least one model test data to obtain at least one preliminary test result, including:

[0128] The simulation software is configured based on the simulation configuration data in the model test data;

[0129] The extracted layout data is input into the configured simulation software to obtain the output results;

[0130] Based on the output results, the preliminary test results of the simulation software are determined.

[0131] In this embodiment, simulation configuration data is used to dynamically set the operating parameters of the simulation software, including software configuration parameters constructed from parameter template files and parameter range data. For example, configuring the simulation software based on the simulation configuration data can involve reading the simulation configuration data and dynamically modifying the internal parameters of the simulation engine to bring it into a running state adapted to the current test scenario.

[0132] The layout extraction data is input into the configured simulation software. Alternatively, the layout extraction data can be used as input to a simulation software instance with completed parameter configuration, triggering simulation calculations and outputting the original response to obtain the output result.

[0133] The output can be the raw response data generated by the simulation software after receiving the configuration and performing simulation calculations on the layout data. Based on the output, the preliminary test results of the simulation software are determined. For example, this can be achieved by applying predefined difference detection rules to the output, extracting key node indicators, and generating binary or hierarchical judgment results, thereby automating the generation of preliminary test results, shortening the interpretation delay, and improving consistency.

[0134] This embodiment provides a simulation software testing method that configures the simulation software based on simulation configuration data from model test data; inputs layout extraction data into the configured simulation software to obtain output results; determines preliminary test results of the simulation software based on the output results; configures the simulation software again using reconstructed software configuration parameters; and performs simulation on layout extraction data based on the configured simulation software. This method enables simulation software testing under different software configuration parameters, thereby achieving the technical effects of reducing testing time and improving testing flexibility.

[0135] In one embodiment, the preliminary test results of the simulation software, based on the output, include:

[0136] Based on the simulation results comparison data items required for the current test scenario, obtain each data item in the output result;

[0137] Compare each data item in the output result with each data item in the preset output result;

[0138] If the differences of each data item all meet the preset allowable range, the preliminary test result is passed;

[0139] If the difference between any data items does not meet the preset allowable range, the preliminary test result is a failure, and one or more of the average difference, maximum difference, and variance of the data items will be output.

[0140] Among them, the simulation result comparison data items required for the current test scenario can be a set of key data points with relevance in the simulation output selected according to the verification target of the current test scenario. This can be used to limit the focus of the test evaluation, avoid redundant calculations in the comparison of full output data, and improve the comparison efficiency and diagnostic accuracy.

[0141] To obtain each data item in the output result, the corresponding field values ​​or waveform sampling points can be extracted from the simulation output result based on the comparison data item list to form a structured comparison dataset. This allows only data items that are strongly related to the current verification target to be processed, greatly reducing the overhead of reading and processing invalid data.

[0142] The preset output results can be simulation results that have been verified in the past, or reference output datasets defined by design specifications. For example, they can include expected values, waveform characteristics, etc., that correspond one-to-one with the current comparison data items. For example, the preset output results can be a set of baseline snapshot parameters that match the current design version loaded from the version control system, or a set of theoretical parameters determined according to the design specifications.

[0143] The difference can be the numerical deviation between each data item in the output result and the corresponding item in the preset output result, and can be expressed in the form of absolute value, relative error, etc. Understandably, the difference can be used as a quantitative basis for judging whether a test passes or fails, reflecting the degree of deviation between the simulation output and the expected behavior.

[0144] Comparing each data item in the output result with each data item in the preset output result can be done by calculating the difference between each extracted output data item and the corresponding item in the preset output result to generate a difference set.

[0145] The preset allowable range can be an acceptable deviation interval pre-defined for each comparison data item, which can be determined based on design specifications, process libraries, historical verification data statistics, etc. If the difference of each data item meets the preset allowable range, the preliminary test result is passed. This can be achieved by iterating through all differences; if all are less than or equal to the corresponding preset allowable range, the preliminary test result is marked as passed. Furthermore, it can also include short-circuit logic judgment; if any difference is found to be greater than the preset allowable range, the iteration is terminated, and a failure is output prematurely. Further, it can also be achieved through weighted judgment, setting higher priority for critical items; if a critical item passes, or the weighted result including the critical item meets the preset allowable range, the result is considered passed.

[0146] The average difference can be the arithmetic mean of the differences of all data items, or the arithmetic mean of the differences of data items that do not meet the preset allowable range. It can reflect the overall trend of systematic deviation and determine whether there is a global offset in the simulation output, such as systematic gain drift or bias error.

[0147] The maximum difference can be the largest absolute value among all differences, used to locate the most severe anomaly. In this embodiment, the maximum difference can be used to identify local anomalies with extreme deviations in the simulation output, assisting in locating the physical location of design defects or modeling errors, and achieving rapid location of the root cause of the fault.

[0148] Variance can be the statistical variance of all differences, which can characterize the consistency and stability of the simulation output across multiple comparison points. In this embodiment, variance can be used to identify whether the simulation results have local instability or noise sensitivity issues, such as random jitter, convergence divergence, etc.

[0149] The output data can include one or more of the following: average difference, maximum difference, and variance. For example, when an out-of-limit difference exists, the initial test result can be marked as a failure, and the average difference, maximum difference, and variance can be calculated as diagnostic indicators. Furthermore, only the maximum difference and its corresponding data item name can be output for quick anomaly location, or all three indicators can be output and a visual report generated to support subsequent root cause analysis. This provides multi-dimensional quantitative diagnostic clues, transforming failed tests from black-box judgments into analyzable fault signals, thereby improving debugging efficiency and test closed-loop capabilities.

[0150] This embodiment provides a simulation software testing method that obtains each data item in the output result by comparing the simulation results required for the current test scenario. Each data item in the output result is compared with each data item in the preset output result. If the difference between each data item meets the preset allowable range, the preliminary test result is passed. If the difference between any data item does not meet the preset allowable range, the preliminary test result is failed, and one or more of the average difference, maximum difference, and variance of the data item are output. By comparing key output nodes strongly correlated with the current test objective, redundant calculations caused by full data comparison are avoided. When the test fails, statistical indicators such as average difference, maximum difference, and variance are automatically output, providing multi-dimensional quantitative clues for root cause analysis of the fault. This can significantly improve debugging efficiency, thereby achieving the technical effects of reducing test time and improving test flexibility.

[0151] In one embodiment, the preliminary test results of the simulation software, based on the output, include:

[0152] The test data and output results are fed into the simulation result verification module, which is a simulation model based on at least part of the algorithm logic of the simulation software.

[0153] Based on the test data, obtain the influencing parameters, including: grid density, simulation stage, parameter template file, and parameter range file;

[0154] The simulation result verification module performs simulations based on the influencing parameters and obtains simulation reference results.

[0155] The output results are compared with the simulation reference results to determine the preliminary test results of the simulation software.

[0156] The simulation result verification module can be a verification module independent of the simulation software. It acts as a fast reference generator for the simulation software, replacing the full simulation to generate reference results for comparison with the output results. For example, the simulation result verification module may include at least part of the algorithm logic of the simulation software to achieve functional replication. Its computational overhead is lower than that of the simulation software, thereby reducing the computational load and time cost of the verification process.

[0157] For example, the simulation result verification module can be obtained by extracting the core solver, convergence mechanism, and physical modeling module of the simulation software, and then packaging the unnecessary components into independent, callable modules. Furthermore, the simulation result verification module can receive test data and influencing parameters as input, generate simulation reference results, and thus establish a comparison relationship with the main simulation output results.

[0158] Influencing parameters can be key input variables that determine the behavior of the simulation result verification module. In this embodiment, influencing parameters include grid density, simulation stage, parameter template file and parameter range file, which are used to control the accuracy, range and modeling strategy of the verification module, thereby enabling parameter alignment between the verification module and the main simulation environment.

[0159] Grid density refers to the precision of the mesh used to discretize the circuit space during simulation. It reflects the density of computational nodes and the resolution of physical modeling, and can control the spatial modeling granularity of the verification module, thereby affecting the accuracy and computation speed of the simulation reference results. For example, grid density can be obtained by parsing mesh information from the layout data of the test data, such as parsing cell size, number of nodes, and local densification region identifiers.

[0160] The simulation phase can represent the process phase to be simulated. It is understood that different process phases use different physical or mathematical models, and different process phases have different modeling logic and key parameters. Therefore, it is necessary to ensure that the verification module executes the corresponding algorithm only in specific phases to avoid invalid calculations and improve verification efficiency and relevance.

[0161] The parameter template file and parameter range file can refer to the parameter template file and parameter range file described in any of the above embodiments, and will not be repeated here.

[0162] In an exemplary embodiment, influencing parameters can be obtained by parsing and extracting from test data. For example, grid density can be obtained by analyzing the grid division information in the layout extraction data, the simulation stage can be obtained from the stage identifier in the simulation configuration data, and the parameter template file and parameter range file can be obtained from the reference paths of the parameter template file and parameter range file of the current simulation software. Alternatively, the configuration management service can be called to dynamically load the corresponding combination of influencing parameters according to the test scenario label, thereby realizing the automated generation of the verification module configuration, ensuring that its behavior is completely aligned with the main simulation, and eliminating manual configuration errors.

[0163] The simulation reference result can be the expected response data generated by the simulation result verification module after performing simulation based on the influencing parameters. This data is used to compare the parameters with the output of the simulation software, serving as a reference benchmark for the simulation software's output. In this embodiment, the simulation reference result can be obtained by the simulation result verification module after dynamic configuration based on the influencing parameters. Its output format is consistent with the output of the simulation software, thereby achieving lightweight simulation and significantly reducing the computational load.

[0164] Comparing the output results with the simulation reference results can involve point-by-point and dimension-by-dimensional comparisons. Furthermore, a pass / fail determination can be made by calculating a difference index and applying preset rules. For example, preset rules could include using a difference threshold comparison method to detect point-by-point deviations between the output results and the simulation reference results, or using feature vector similarity analysis to calculate the cosine similarity of higher-order features such as the spectral energy distribution and rising edge slope of the output results. This upgrades the process from black-box difference judgment to algorithm-level behavioral verification, improving test accuracy and interpretability, and supporting the automated generation of pass / fail markers.

[0165] This embodiment provides a simulation software testing method. Test data and output results are input into a simulation result verification module, which is a simulation model implemented based on at least part of the algorithmic logic of the simulation software. Based on the test data, influencing parameters are obtained, including: grid density, simulation stage, parameter template file, and parameter range file. The simulation result verification module performs simulation based on the influencing parameters to obtain simulation reference results. The output results are compared with the simulation reference results to determine the preliminary test results of the simulation software. By constructing an independent verification module separate from the simulation software, and implementing lightweight replication based on part of the simulation software's algorithmic logic, and dynamically configuring and generating reference results, the computational load of the verification process is significantly reduced. Furthermore, the comparison process is transformed from black-box difference judgment to algorithm-level behavioral verification, improving test accuracy and interpretability. This achieves the technical effects of reducing testing time and increasing testing flexibility.

[0166] To more clearly illustrate the technical solution of this application, a detailed embodiment is also provided.

[0167] In the chemical mechanical polishing (CMP) simulation software used in the integrated circuit industry, the complexity of the initial input simulation parameters and the large volume of output simulation data present several challenges in verifying the correctness of the simulation results: First, the large data volume is time-consuming and labor-intensive; second, there are numerous test scenarios with reasonable data discrepancies, such as different process management tools, process counts, different simulation parameter configurations, and simulation model optimization regression testing. Therefore, testing with multiple scenarios and large volumes of simulation data requires writing processing scripts to design different test data execution and verification methods for different scenarios to ensure the correctness of the simulation results and the stability of the model.

[0168] In one embodiment, a simulation software testing method is provided to verify the correctness of simulation results and the stability of the simulation model under different scenarios and parameter configurations. This simulation software testing method mainly includes a scenario testing process and a test data verification process, wherein:

[0169] I. Scenario Testing Process.

[0170] Step S101: Store the input data containing layout extraction data (i.e., layout extraction data) and simulation configuration data (including simulation parameters and running parameters) and the expected output simulation results as a single test data set in the test data folder as a test dataset.

[0171] Step S102: Traverse the test data folder from step S101 to obtain the test data, including the expected results.

[0172] Step S103: Determine whether regression testing is required for the test scenario. If yes, proceed to step S104; otherwise, proceed to step S107.

[0173] Determining whether a test scenario requires regression testing can be done by extracting data from the layout, simulation parameters, or runtime parameters, or by identifying whether the test data corresponds to a regression test scenario. Alternatively, it can be determined by identifying tags within the test data indicating whether regression testing is required, i.e., data attributes. Subsequent determinations regarding the need for process and model testing can be conducted in the same way as those for regression testing, and will not be elaborated upon in this embodiment.

[0174] Step S104: Input the test data, including layout extraction data, simulation parameters (including simulation model files and parameters), and running parameters (including the number of processes and simulation layers), into the simulation software as is, and run the simulation.

[0175] Step S105: By calling the preset simulation result comparison tool, the obtained simulation results are compared with the expected results, and the average difference, maximum difference and variance of each column or item will be output.

[0176] Furthermore, if the test scenario is a regression test, the simulation data is marked as passing when every value of every data item in the expected result is exactly the same as the simulation data; otherwise, it is marked as failing; and then proceed to step S106.

[0177] Step S106: Determine if the current test data is the last data in the test set. If it is, end the process; otherwise, return to step S102.

[0178] In the loop process of this automated test, a test data can be subjected to regression testing, process testing and model testing in parallel. Step S106 is the loop test end statement. Accordingly, for the loop method, it can be that all regression testing, process testing and / or model testing are performed on a test data first, and then other test data are tested. Alternatively, regression testing can be performed on all test data first, and then process testing / model testing can be performed on all test data. Different types of tests can also be performed, which is not limited in this embodiment.

[0179] Step S107: Determine whether the test scenario needs to be tested. If yes, proceed to step S108; otherwise, proceed to step S111.

[0180] Step S108: Based on the pre-set combination of process tools and process sets, set the running parameters of the test data, input the modified test data into the simulation, and run the simulation until all process combinations are executed.

[0181] This can involve retrieving test data from a test dataset, with input data including layout extraction data, simulation parameters, and runtime parameters. The original runtime parameters of the test data can be replaced by combinations of processes to obtain modified test data.

[0182] Step S109: Combine the elements of each process tool set [tool_A, tool_B] and process number set [1, 3, 4, ..., 16] to obtain process configuration combinations such as [(tool_A, 1), (tool_A, 3), ..., (tool_B, 16)]. Each combination is passed in as a running parameter to run the simulation.

[0183] In this embodiment, the process tools can be existing ones. It is understood that MPI (Message Passing Interface) is a standard interface for parallel computing, widely used in the field of high-performance computing. MPI process tools are used to manage, debug, and analyze MPI programs. These tools help developers optimize the performance of parallel programs and resolve debugging issues.

[0184] like Figure 3 As shown, process testing may include the following steps:

[0185] Obtain the set of process management tools and the set of process numbers. Iterate through each set of process management tools and process numbers, including selecting the next process management tool and the next process number in the current state. Process the test data according to the selected process management tool and process number, set the process management tool and process number, and then run the simulation to obtain the simulation results.

[0186] Determine if the range of differences in the simulation results is reasonable. If it is, mark it as passed and determine if the current process number is the last element of the set. If not, iterate through the next process number and perform further testing. If it is, determine if the current process management tool is the last element of the set. If not, iterate through the next process management tool and perform further testing. If not, the process test ends.

[0187] Step S110: By calling a preset simulation result comparison tool, the simulation results are compared with the expected results to determine whether the differences in each column of the simulation data are within a reasonable range. The reasonable range can be determined based on prior knowledge. If it is not within a reasonable range, the variance of the comprehensive evaluation index is referenced. If it is within a reasonable range, it is marked as passing; otherwise, it is marked as failing, and the process proceeds to step S106.

[0188] The simulation result verification in the process test involves comparing the simulation results of all process combinations with the standard file. The number of processes in a process combination corresponds to the number of simulation result data sets; that is, each process combination yields a set of simulation result data after running once.

[0189] Step S111: Determine whether the test scenario needs to be tested with the model. If yes, proceed to step S112; otherwise, proceed to step S106.

[0190] Step S112: Based on the simulation parameter template file and parameter range file, randomly generate N sets of parameter combinations, save them as the simulation parameter file for testing, i.e., simulation configuration parameters, input the modified test data and run the simulation until all N sets of parameter combinations have been executed;

[0191] The test data is extracted from the test dataset. The input data includes layout extraction data, simulation parameters, and running parameters. Each simulation configuration parameter is used to replace the original simulation parameters of the test data to obtain the modified test data.

[0192] like Figure 4 As shown, the model testing scenario includes the following process: Obtain the parameter template file and parameter range data; initialize the counter i=0; determine if i is less than N, where N is the preset number of parameter combinations. If not, the process ends; if so, i=i+1, and iterate through each parameter, generating a random number based on the suggested parameter range. Use this random number as the parameter's value, fill all the randomly generated values ​​into the parameter template file, generate the test parameter file, and run the simulation by passing the test parameter file. After the simulation ends, the counter is checked again.

[0193] Step S113: Verify each set of simulation result data. Use the preset simulation file comparison tool to output the maximum, minimum and average values ​​of each column. Determine whether the average value is within the expected range. If it is, mark it as passed; otherwise, mark it as failed and proceed to step S106.

[0194] like Figure 5 As shown, in one embodiment, the overall testing process for the test dataset is as follows:

[0195] Get the test dataset and read the next test data.

[0196] Determine if the test data requires regression testing. If so, input the test data as is and run the simulation. Compare the output simulation results with the expected results. If they match, mark it as passed; otherwise, mark it as failed. Determine if this is the last test data. If yes, end the process; otherwise, read the next test data.

[0197] If regression testing is not required, determine whether process testing is needed. If so, process the test data, set the number of processes in the process management tool, and then run the simulation. Determine if the range of differences in the simulation results is reasonable. If yes, mark it as passed; otherwise, mark it as failed. Check if this is the last test data. If yes, end the process; otherwise, read the next test data.

[0198] If process testing is not required, determine whether model testing is necessary. If so, process the test data, run the simulation according to the determined simulation parameters, and verify whether the simulation results are reasonable. If so, mark it as passed; otherwise, mark it as failed.

[0199] If the simulation test ends, or if model testing is not required, determine if this is the last test data. If yes, end the test; otherwise, read the next test data.

[0200] It is understood that the accompanying drawings in the above specification are intended to provide an example of an automated test loop. The specific loop method may be to first perform all regression tests, process tests and / or model tests on a test data set before testing other test data, or to first perform regression tests on all test data sets and then perform process tests / model tests on all test data sets, or to perform tests of different test scenarios. This embodiment does not limit the specific test scenarios.

[0201] II. Test data verification.

[0202] The preset simulation result comparison tool used in steps S105 (comparison of simulation results for regression testing), S110 (comparison of simulation results for process testing), and S113 (comparison of simulation results for model testing) can first receive one or more simulation result files as input, input them into the comparison tool, and then run the comparison. The comparison tool determines whether the input file is a single simulation result file. If it is a single simulation result file, it outputs the maximum, minimum, and average values ​​of each column of parameters in the file. If two simulation result files are input, it compares the corresponding parameters at the same grid point position point by point, outputs the maximum difference, average difference, and comprehensive variance of each column, and generates a detailed comparison file containing the difference of each data point. If the number of input files is neither 1 nor 2, it is determined to be an illegal input, and an error is reported and the tool exits.

[0203] During the comparison process, the choice of data for comparison is crucial. In the first part, the comparison is based on expected results, expected ranges, and reasonable ranges, meaning a standard document needs to be obtained beforehand. However, the verification of regression and process test results mainly relies on line-by-line comparison with the standard document, determining whether the results pass by judging whether the difference rate meets the deadline. While this "gold standard" approach is intuitive, it has significant limitations: Firstly, when simulation algorithms, physical models, or numerical solvers are updated, the original standard document often becomes inapplicable, requiring manual regeneration and review, resulting in high maintenance costs; secondly, the standard document itself may contain historical errors or not cover boundary scenarios, leading to insufficient test sensitivity or misjudgments.

[0204] Existing technologies lack verification of the consistency of the file content itself, and when the model algorithm formula is updated, the standard file needs to be updated frequently, which consumes manpower and time. This embodiment designs a set of verifications for the result file itself based on the different relationships between fields at different stages of simulation results, thereby adding a verification dimension and improving the robustness of interface automation. That is, based on the simulation result verification requirements of model testing, this embodiment involves a method for verifying test data for simulation software. It uses simulation results (the test data itself) for verification, no longer relying on externally preset standard files, but dynamically constructing verification rules based on the content of the simulation results themselves, combined with process stages, model configurations, and input layout features. This method achieves self-consistency verification of the result data by mining the physical constraint relationships between parameters at different simulation stages, thereby adding a verification dimension independent of traditional comparisons, significantly improving the robustness and adaptability of automated testing. This mechanism can be applied not only to model testing, but also to result verification in regression testing and process testing.

[0205] In this embodiment, data for comparison and verification is obtained from the result file itself, which can be obtained through information such as the simulation run stage, simulation layer, model file, and parameters. Due to the special nature of simulation software, different results and corresponding relationships exist for different simulation stages. In this embodiment, special verification logic is added for specific grid points. For example, when the simulation is specified to run to the ECD / CMP1 stage, the output results include a specific set of parameters, and the parameter columns of the generated simulation result file are the same. However, when running to the CMP2 / CMP3 stage, although the output parameters are internally consistent and the parameters of the generated simulation result file are the same, the parameter columns are different from those of the simulation result file generated in the ECD / CMP1 stage. This is because the process modules of ECD electrochemical deposition and CMP chemical mechanical polishing are completely different. ECD focuses more on metal filling thickness and uniformity, while CMP focuses more on the high residual after surface planarization. Therefore, different calculation models and output fields need to be activated. When the grid point Density value is in (0, 1), and the value = 0 or the value = 1, different logic verifications are performed for different parameter values. Figure 6 As shown, the data verification of the simulation results of the simulation software in this embodiment is as follows:

[0206] Read in the input data and obtain the simulation run phase and model file, including obtaining the simulation run phase, model file and parameters, and grid coordinates.

[0207] Determine if the simulation layer number is 1. If not, determine if the stack-up reference parameters are obtained. Then determine if the simulation stage is ECD / CMP1. If so, directly determine if the simulation stage is ECD / CMP1.

[0208] If the simulation stage is ECD / CMP1, then the simulation stage is CMP2 / CMP3, and it is determined whether Density ∈ (0, 1). If it is not ECD / CMP1, then it is directly determined whether Density ∈ (0, 1).

[0209] If Density does not belong to (0, 1), then determine whether to set Density=0 or Density=1, thereby enabling the validation rules for blank areas and fully filled areas respectively.

[0210] To determine whether data validation has passed, for example, one could check whether the parameters of all grid points meet the physical constraints. If any grid point does not meet the constraints, an error message is stored and the process ends; otherwise, the process ends directly.

[0211] The specific process for data verification of the simulation results from the simulation software is as follows:

[0212] Simulation Acquisition: The input data for the test data includes a Recipe file, which corresponds to the steps of each simulation stage. When preparing the dataset, specify the steps to be run, such as M1:7, meaning the simulation of layer M1 up to step S107. For multi-layer simulations, the data must be calculated based on the simulation data of the first layer; that is, the stack-up baseline parameter values ​​refer to the simulation data of the first layer. Here, "stack-up" refers to a multi-layer metal interconnect structure. The simulation of the upper layer depends on the morphology results of the lower layer as initial conditions; therefore, it is necessary to accurately transmit the lower-level data to ensure overall consistency.

[0213] The process involves obtaining data columns and their corresponding relationships: Both the layout extraction data from the input data (i.e., the "layout extraction data" in the test data input) and the output data (i.e., the data in the simulation result file) contain grid point coordinates. Each grid point corresponds to a set of coordinate values, such as (x, y). The layout extraction data is matched and verified against the simulation result file using these coordinates. Specifically, the grid density value of the grid point in the layout extraction data is obtained from the grid point coordinates in the output data. This density value is then used to perform the simulation result verification process. This matching mechanism ensures that the verification logic can accurately locate the same area in physical space, avoiding misjudgments caused by grid offset or index misalignment.

[0214] Obtain the model file and parameters. The simulation parameters in the input data of the test data include the simulation model file and parameters. The model file and parameters corresponding to each test data are different, so it is necessary to obtain the corresponding model and parameters for data calculation and verification. For example, the simulation run stage can be determined based on the model file. It is understandable that the model file not only defines the physical equations and boundary conditions, but also implicitly contains the semantics and valid range of the output fields; therefore, incorporating it into the verification context is key to achieving self-consistent verification.

[0215] By determining whether the simulation stage is ECD / CMP1, CMP2 / CMP3, whether the Density value is ∈ (0,1), whether Density=0, and whether Density=1, comparative data for the simulation results is obtained or generated, thus completing the step of "whether the data verification passes". It should be noted that different comparative data will be obtained or generated based on the results of each of the above steps. For example, when Density=0, it indicates that the region has no metal pattern; theoretically, Cu_Thickness should approach zero or the basic deposition layer thickness. When Density=1 (full fill), the surface height should exhibit a specific convex shape. This allows for the formation of built-in verification rules that can be executed without external standards, based on inherent constraints derived from physical laws.

[0216] For each simulation execution, such as when inputting data, multiple result files can be pre-set or specified to be acquired. For example, before regression testing, multiple result files can be acquired, and the simulation result output data file and the simulation result data file of a certain intermediate process can be compared with the expected results. The multi-stage result capture mechanism of this embodiment not only supports final output verification, but also monitors the intermediate states in the simulation process, which helps to quickly locate which process sub-step the problem occurs in, improving debugging efficiency and test coverage.

[0217] like Figure 7 As shown, the comparison between the simulation results and the expected results is as follows:

[0218] Obtain the simulation result files and run the comparison tool. Check if the number of simulation result files is one; if not, check if it is two. If the number of simulation result files is one, calculate the value of each column of data at each grid point coordinate, and then obtain the maximum, minimum, and average values ​​of each column of parameters, and end the process. If the number of simulation result files is two, calculate the maximum and average differences in each column of data, calculate the variance, and calculate the difference file for each data point. If the number of simulation result files is not two, report an error and end the process.

[0219] Each column of data above represents a parameter value corresponding to all grid points on the layout. The data point refers to a specific parameter value for a particular grid point on the layout. For example, Density is a parameter column, and the first row, Density=0.018567, represents the data point.

[0220] In one specific embodiment, taking regression testing as an example, the input layout extraction data is shown in Table 1. The input simulation parameters are shown in Table 2. The runtime configuration is as follows: the process management tool is OpenMPI, and the number of running processes is 3. The expected simulation results are shown in Table 3.

[0221] Table 1. Map Extraction Data

[0222]

[0223] Table 2 Simulation Parameters

[0224]

[0225] Table 3 Expected Simulation Results

[0226]

[0227] First, obtain the above test data, including parameter configuration, runtime configuration, and the path to the expected simulation results. Store some parameters in a configuration file, and then store the configuration file path as a string in a script variable.

[0228] The runtime configuration path is set to " / path / to / runset", the process management tool name is set to "openmpi", the number of processes is set to 3, and the expected result file path is set to " / path / to / expect_simures_file".

[0229] Input the above test data as is into the simulation running function run_simulation. The running command is "run_simulation(runset_dir, mpi_tool, process_num)". After the run is complete, output the simulation results.

[0230] The simulation result file path is stored as a string in the script variable `simures_dir`. The expected result file path `expec_simures_dir` and the output result path `simures_dir` are passed to the simulation result file comparison function. The comparison tool is then called to obtain the comparison results, and the log results are as follows: Figure 8 As shown, the maximum difference of each data item in the comparison results is obtained, namely the MAX value of surface height Surface_Height and the MAX value of copper thickness Cu_Thickness. It is then determined whether the difference is 0. If both are 0, the test data passes the test.

[0231] Typical regression testing procedures mostly involve inputting test data as is, running the test, and then comparing the expected results with the simulation results, which covers a limited range of scenarios. This method can be applied to various testing scenarios, and verification schemes can be designed for each scenario, thus better ensuring the reliability of the software and the stability of the model.

[0232] This embodiment provides a simulation software testing method that, through standardized encapsulation of test data, makes it reusable to various simulation software and different versions of simulation software at different times. Furthermore, by selecting appropriate test scenarios based on the test data, the method allows for flexible application of test data to different test scenarios to perform corresponding tests on the simulation data. This objectively eliminates the subjectivity and delays inherent in manual testing. Compared to the traditional method of manually inputting parameters one by one and manually triggering simulations scene by scene, this method effectively reduces parameter configuration time, scene switching delays, and repetitive simulation overhead, thereby achieving the technical effects of reducing testing time and improving testing flexibility.

[0233] By introducing data attributes that are automatically mapped to structured test types such as regression testing, process testing, and model testing, a complementary test dimension system is formed. This system verifies the impact of design changes, the stability of simulation processes, and the accuracy of algorithm models, enabling intelligent triggering of test scenarios and precise resource allocation. This significantly improves the configurability and response speed of test scenarios, thereby achieving the technical effect of reducing test time and increasing test flexibility without increasing hardware resources.

[0234] By reconstructing the simulation environment parameters and simulating the layout data based on the configured simulation software, the technical effects of reducing testing time and improving testing flexibility can be achieved.

[0235] By combining different process management tools and varying numbers of processes, diverse process configuration combinations can be generated. This allows for the parallel coverage of multiple resource scheduling strategies within a single test process. Performance tuning tests that originally required multiple independent runs are transformed into automated experiments executed serially or in parallel in a single batch. This enables the simulation software to run under different parallel architectures and simultaneously collect multi-dimensional performance metrics such as throughput, memory usage, and convergence speed. Consequently, it can quickly identify the optimal resource configuration combination and dynamically adapt to the operational requirements of different hardware platforms without modifying the original test data. This significantly improves the compatibility and response speed of testing in heterogeneous environments, achieving the technical effects of reducing test time and increasing test flexibility.

[0236] By introducing parameter template files and parameter range files to construct reusable and composable parameter configuration units, the flexibility of configuration can be effectively improved. The adjusted model test data can make the test input have the coverage and diversity of parameter space. The original serial test mode, which required multiple independent runs and manual parameter switching, is transformed into a parallel test flow that can be configured once, generated in batches, and executed automatically. This greatly reduces the time overhead of repeatedly loading the simulation environment and manual intervention, thereby achieving the technical effect of reducing test time and improving test flexibility without increasing hardware resources.

[0237] By comparing key output nodes that are strongly correlated with the current test objective, redundant calculations caused by full data comparison can be avoided. When the test fails, statistical indicators such as average difference, maximum difference, and variance are automatically output, providing multi-dimensional quantitative clues for root cause analysis of the failure. This can significantly improve debugging efficiency, thereby achieving the technical effects of reducing test time and improving test flexibility.

[0238] By constructing an independent verification module separate from the simulation software, a lightweight replication is achieved based on some of the simulation software's algorithm logic, and reference results are dynamically configured. This not only significantly reduces the computational load of the verification process, but also transforms the comparison process from black-box difference judgment to algorithm-level behavioral verification, improving test accuracy and interpretability. As a result, the technical effects of reducing test time and increasing test flexibility can be achieved.

[0239] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0240] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a simulation software testing method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0241] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0242] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the simulation software testing method of any of the above embodiments:

[0243] Obtain a test dataset for the simulation software; the test dataset includes at least one set of test data; the test data includes layout extraction data and simulation configuration data;

[0244] Based on the test data, determine the test scenario corresponding to the test data;

[0245] Based on the test scenario and the test data, the simulation software is tested to obtain at least one preliminary test result;

[0246] Based on the preliminary test results of each of the test data, the test results of the simulation software are determined.

[0247] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the simulation software testing method of any of the above embodiments.

[0248] Obtain a test dataset for the simulation software; the test dataset includes at least one set of test data; the test data includes layout extraction data and simulation configuration data;

[0249] Based on the test data, determine the test scenario corresponding to the test data;

[0250] Based on the test scenario and the test data, the simulation software is tested to obtain at least one preliminary test result;

[0251] Based on the preliminary test results of each of the test data, the test results of the simulation software are determined.

[0252] In one embodiment, a computer program product is provided, which, when executed by a processor, implements the simulation software testing method of any of the above embodiments:

[0253] Obtain a test dataset for the simulation software; the test dataset includes at least one set of test data; the test data includes layout extraction data and simulation configuration data;

[0254] Based on the test data, determine the test scenario corresponding to the test data;

[0255] Based on the test scenario and the test data, the simulation software is tested to obtain at least one preliminary test result;

[0256] Based on the preliminary test results of each of the test data, the test results of the simulation software are determined.

[0257] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0258] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0259] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0260] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A simulation software testing method, characterized in that, The simulation software testing method includes: Obtain a test dataset for the simulation software; the test dataset includes at least one set of test data; the test data includes layout extraction data and simulation configuration data; Based on the test data, determine the test scenario corresponding to the test data; Based on the test scenario and the test data, the simulation software is tested to obtain at least one preliminary test result; Based on the preliminary test results of each of the test data, the test results of the simulation software are determined.

2. The simulation software testing method according to claim 1, characterized in that, The test data also includes data attributes; the test scenario includes at least one of regression testing, process testing, and model testing; determining the test scenario corresponding to the test data based on the test data includes: Based on the data attributes, determine the test scenario corresponding to the data attributes.

3. The simulation software testing method according to claim 2, characterized in that, The test scenarios include regression testing; The step of testing the simulation software based on the test scenario and the test data to obtain at least one preliminary test result includes: Based on the simulation configuration data, the simulation software is configured; The extracted layout data is input into the configured simulation software to obtain the output results; Based on the output results, the preliminary test results of the simulation software are determined.

4. The simulation software testing method according to claim 2, characterized in that, The test scenario includes process testing; the simulation configuration data includes runtime configuration parameters, which include process tools and the number of processes. The step of testing the simulation software based on the test scenario and the test data to obtain at least one preliminary test result includes: Obtain the process toolset and preset process number set corresponding to the simulation software; The process management tools in the process toolset and the number of processes in the preset process number set are combined to obtain at least one process configuration combination; The running configuration parameters are adjusted based on the at least one process configuration combination to obtain the adjusted test data for at least one process. Based on at least one of the process test data, process testing is performed on the simulation software to obtain at least one preliminary test result.

5. The simulation software testing method according to claim 4, characterized in that, The process testing of the simulation software based on at least one of the process test data, to obtain at least one preliminary test result, includes: The simulation software is configured based on the simulation configuration data in the process test data; The extracted layout data is input into the configured simulation software to obtain the output results; Based on the output results, the preliminary test results of the simulation software are determined.

6. The simulation software testing method according to claim 2, characterized in that, The test scenario includes model testing; the simulation configuration data also includes software configuration parameters; the testing of the simulation software based on the test scenario and the test data to obtain at least one preliminary test result includes: Obtain at least one parameter template file and at least one parameter range file for the simulation software; By combining at least one of the parameter template files and at least one of the parameter range files, at least one model configuration combination is obtained; Based on the at least one model configuration combination, the software configuration parameters are adjusted respectively to obtain at least one model test data after adjustment; Based on at least one of the adjusted model test data, the simulation software is tested to obtain at least one preliminary test result.

7. The simulation software testing method according to claim 6, characterized in that, The step of performing model testing on the simulation software based on at least one of the model test data to obtain at least one preliminary test result includes: The simulation software is configured based on the simulation configuration data in the model test data; The extracted layout data is input into the configured simulation software to obtain the output results; Based on the output results, the preliminary test results of the simulation software are determined.

8. The simulation software testing method according to claim 3, claim 5, or claim 7, characterized in that, The preliminary test results of the simulation software, determined based on the output, include: Based on the simulation result comparison data items required for the current test scenario, obtain each data item in the output result; Compare each data item in the output result with each data item in the preset output result; If the differences of each data item all meet the preset allowable range, then the preliminary test result is passed; If the difference between any data items does not meet the preset allowable range, the preliminary test result is a failure, and one or more of the average difference, maximum difference, and variance of the data items are output.

9. The simulation software testing method according to claim 3, claim 5, or claim 7, characterized in that, The preliminary test results of the simulation software, determined based on the output, include: The test data and the output results are fed into the simulation result verification module, which is a simulation model implemented based on at least part of the algorithm logic of the simulation software. Based on the test data, the influencing parameters are obtained, including: grid density, simulation stage, parameter template file, and parameter range file; The simulation result verification module performs simulation based on the influencing parameters to obtain simulation reference results; The output results are compared with the simulation reference results to determine the preliminary test results of the simulation software.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 9.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 9.

12. A computer program product, characterized in that, When the computer program product is executed by a processor, it implements the method of any one of claims 1 to 9.

Images