Design verification method, device and storage medium of multi-simulation platform

Abstract

The present application relates to the field of integrated circuit simulation verification, and discloses a multi-simulation platform design verification method, equipment and storage medium, comprising: obtaining at least one test case for a design under test; dividing the at least one test case into a first scale level or a second scale level according to an execution scale; executing all test cases on a first simulation platform and exporting intermediate state data of all test cases; injecting the test cases of the first scale level and the corresponding intermediate state data into a second simulation platform to verify the design under test, and after verification passes in the second simulation platform, injecting into a third simulation platform for verification; injecting the test cases of the second scale level and the corresponding intermediate state data into the third simulation platform to verify the design under test. The problem that a single simulation platform cannot balance simulation accuracy, execution speed and debugging convenience is solved.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit simulation and verification, and in particular to design verification methods, equipment, and storage media using multiple simulation platforms. Background Technology

[0002] With the rapid development of fields such as artificial intelligence and high-performance computing, the design scale of system-on-a-chip continues to grow, and the verification work of integrated circuits accounts for an increasingly larger proportion of the entire chip development process. In order to ensure the functional correctness and performance indicators of the design under test before tape-out, it is necessary to run a large number of test cases of different sizes in the simulation environment to fully cover and verify the design under test.

[0003] Currently, the field of integrated circuit simulation and verification faces multiple challenges. On the one hand, a single simulation platform struggles to simultaneously meet the multi-dimensional requirements of simulation accuracy, execution speed, and debugging convenience, leading to a conflict between verification cycle and verification quality. On the other hand, although various simulation platforms with different characteristics exist in the industry, there is still a lack of mature methods for systematically coordinating and scheduling multiple simulation platforms based on the characteristics of the verification phase and test cases in actual projects. This results in low efficiency in data transfer and task allocation between multiple platforms, and there is still significant room for optimization in verification costs and cycle time. Summary of the Invention

[0004] The purpose of this invention is to provide a design verification method using multiple simulation platforms, in order to solve the problems in related technologies where a single simulation platform is difficult to balance simulation accuracy, execution speed and debugging convenience, and where the lack of a systematic collaborative scheduling strategy among multiple platforms leads to low verification efficiency and high verification cost.

[0005] To address the aforementioned technical problems, this invention provides a design verification method using multiple simulation platforms, comprising: acquiring at least one test case for the design under test; dividing the at least one test case into a first scale level or a second scale level based on its execution scale; executing all the test cases on a first simulation platform and exporting intermediate state data for all the test cases; injecting the test cases of the first scale level and the corresponding intermediate state data into a second simulation platform to verify the design under test, and after successful verification on the second simulation platform, injecting them into a third simulation platform for verification; injecting the test cases of the second scale level and the corresponding intermediate state data into the third simulation platform to verify the design under test; wherein the execution scale of the second scale level is greater than that of the first scale level; the execution speed of the third simulation platform is higher than that of the second simulation platform; and the simulation accuracy of the second simulation platform is higher than that of the third simulation platform.

[0006] The present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the design verification method of the multi-simulation platform as described above.

[0007] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the design verification method of the multi-simulation platform as described above.

[0008] In this embodiment of the invention, by dividing all test cases into different levels according to their execution scale and setting differentiated platform scheduling orders for different scale levels, small-scale test cases are prioritized to complete the basic path verification of the design under test on the second simulation platform with high simulation accuracy. After the path verification is passed, they are then deployed to the third simulation platform with faster execution speed for further verification. This avoids running directly on the third simulation platform with limited debugging means before the path has been verified, which would lead to low problem-solving efficiency. At the same time, for large-scale test cases, since their execution time on the second simulation platform is extremely long, scheduling them directly to the third simulation platform with faster execution speed can quickly expose potential defects in the design under test and significantly shorten the verification cycle in large-scale testing scenarios. In addition, by pre-executing all test cases and exporting intermediate state data on the first simulation platform, cross-platform state injection from the first simulation platform to the second and third simulation platforms is realized. This allows each simulation platform to carry out verification work based on a consistent initial state, reducing the data alignment cost between multiple platforms. Thus, without relying on expensive dedicated hardware accelerators, the systematic collaboration of multiple easily accessible platforms achieves a reduction in verification cost and an improvement in verification efficiency. Attached Figure Description

[0009] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0010] Figure 1 This is a flowchart of a design verification method for a multi-simulation platform according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the test case compilation process and execution steps on each platform according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the characteristic relationships between three simulation platforms according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the relationship between test case size levels and project progress time according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the overall architecture of multi-simulation platform collaborative scheduling according to an embodiment of the present invention; Figure 6 This is a flowchart illustrating the problem exposure of the second test case scale in the overall architecture of multi-simulation platform collaborative scheduling according to an embodiment of the present invention; Figure 7 This is a schematic diagram of a multi-platform collaborative operation process centered on a benchmark testing scenario according to an embodiment of the present invention; Figure 8 This is a schematic diagram illustrating the detailed execution flow and replay mechanism within each simulation platform according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of an electronic device according to another embodiment of the present invention. Detailed Implementation

[0011] As mentioned in the background technology, the field of integrated circuit simulation and verification faces the challenge that a single simulation platform cannot simultaneously achieve simulation accuracy, execution speed, and debugging convenience. Furthermore, the lack of a systematic collaborative scheduling strategy among multiple platforms results in low verification efficiency and high verification costs.

[0012] The applicant discovered through research that the root cause of the above problems lies in the fact that existing simulation platforms are limited by their respective physical implementation mechanisms, and their performance indicators exhibit an inherent inverse relationship.

[0013] 1. The first simulation platform performs functional abstract modeling of the design under test at the software level, bypassing the timing details of register transfer level. It has fast execution speed and flexible debugging methods, but the simulation accuracy is insufficient and it cannot reflect the clock cycle level behavior of the design under test at the hardware level.

[0014] 2. The second simulation platform uses an event-driven simulation engine to perform clock-cycle simulation of the register transfer level code of the design under test. The simulation accuracy is high and it supports comprehensive observation of any signal node. It is the core means to locate the root cause of design defects. However, the execution time increases non-linearly with the increase of design scale and test complexity. For large-scale test cases, the execution time can be more than a thousand times that of the behavioral simulation platform.

[0015] 3. The third simulation platform maps the design under test to a programmable logic device after logic synthesis, achieving a significantly higher execution speed than software simulation by leveraging the parallel execution capabilities of hardware. However, its debugging interface resources and signal probing depth are limited, making defect localization significantly more difficult than with the second simulation platform. Furthermore, while dedicated hardware accelerators can balance execution speed and debugging capabilities to some extent, their procurement costs are in the millions or even tens of millions of dollars, and their initial utilization rate is low, making them unaffordable for most chip design companies.

[0016] Against this backdrop, existing multi-platform usage methods typically involve verification engineers manually switching between different platforms based on their personal experience, lacking a systematic scheduling mechanism based on the scale characteristics of test cases. This approach has two key drawbacks: First, it fails to differentiate the varying platform requirements of test cases of different sizes. The core requirement for small-scale test cases is high-precision path verification and efficient defect localization, while the core requirement for large-scale test cases is rapid execution to expose boundary defects. The requirements for platform scheduling order are drastically different for these two types of test cases, but existing methods do not differentiate between them. Second, there is a lack of a unified state injection mechanism between platforms. Test cases need to independently complete environment initialization on different platforms, resulting in high cross-platform data alignment costs and a high risk of false positives due to environmental differences.

[0017] Therefore, how to systematically coordinate and schedule multiple simulation platforms with complementary characteristics to meet testing needs of different scales, so that each platform can do its best, has become an urgent technical problem to be solved.

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the embodiments of the present invention to facilitate a better understanding of the invention. However, the technical solutions claimed in the present invention can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the following embodiments is for ease of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0019] The multi-simulation platform design verification method involved in this invention can run on an electronic device with data processing capabilities. This electronic device includes, but is not limited to, workstations with simulation development environments, server clusters, cloud computing platforms, or integrated verification management systems configured with multi-platform management and scheduling modules. Optionally, the electronic device can be one or more servers configured with high-performance processors and large-capacity storage. The servers respectively deploy the runtime environment of a first simulation platform (e.g., gem5 simulator), the runtime environment of a second simulation platform (e.g., VCS, Verilog Compiler Simulator, or Xrun, Xcelium Logic Simulator simulation tool), and communicate with a third simulation platform (e.g., FPGA, Field Programmable Gate Array development board) carrying the design under test via a network interface. Alternatively, the three simulation platforms can also be deployed on the same server with heterogeneous computing capabilities, or distributed across multiple devices interconnected by a local area network, with a unified scheduling and management module handling task orchestration and data flow.

[0020] This invention provides a design verification method using multiple simulation platforms. The method acquires test cases for the design under test (DUT), categorizes all test cases into different scale levels based on their execution size, pre-executes all test cases on a first simulation platform to export intermediate state data, and then injects the test cases and intermediate state data into a second and third simulation platform respectively for verification according to a differentiated scheduling order based on the scale level of the test cases. Through this approach, on the one hand, for small-scale test cases, the second simulation platform, with its high simulation accuracy and strong debugging capabilities, is prioritized to complete the basic path verification of the DUT. After the path verification is successful, it is then deployed to the faster third simulation platform, avoiding direct execution on the third simulation platform with limited debugging capabilities before the basic path of the DUT has been verified, thereby reducing the time cost of defect localization. On the other hand, for large-scale test cases, the execution speed advantage of the third simulation platform is used to quickly expose potential defects in the DUT, avoiding the consumption of a large amount of time on the extremely slow second simulation platform, shortening the large-scale test verification cycle, which might originally take months or even years, to the order of weeks. Meanwhile, by pre-executing and exporting intermediate state data through the first simulation platform, cross-platform state injection and data alignment are achieved, eliminating the environmental initialization differences caused by independent startup of each platform and reducing data flow costs in multi-platform collaboration. This solution, without relying on expensive dedicated hardware accelerators, achieves a significant reduction in verification costs and an effective improvement in verification efficiency through systematic collaborative scheduling of multiple readily available platforms based on test scale.

[0021] The following is a detailed description of the implementation details of the design verification method for the multi-simulation platform according to an embodiment of the present invention. The following content is only for the convenience of understanding and is not necessary for implementing this solution.

[0022] It should be noted that the Design Under Test (DUT) is a chip design module described in register-transfer-level (RTD) code form. This DUT is loaded and executed in different ways on different simulation platforms: on the first simulation platform, the DUT runs as a behavioral model; on the second simulation platform, the DUT's RTD register-transfer-level code is directly compiled and simulated; on the third simulation platform, the DUT's RTD register-transfer-level code is synthesized and mapped to a programmable logic device for execution. The second and third simulation platforms share the same set of DUT register-transfer-level code, which is the basis for data alignment and cross-validation of defects between them.

[0023] Example 1 like Figure 1 As shown, the design verification method for a multi-simulation platform provided in this embodiment of the invention includes the following steps 101 to 105.

[0024] Step 101: Obtain at least one test case for the design under test.

[0025] In embodiments of the present invention, the design under test (DUT) is a system-on-a-chip (SoC) design module for a specific application scenario. This DUT needs to undergo thorough simulation verification before tape-out to ensure functional correctness and performance compliance. Test cases are sets of stimulus data used to drive the DUT to execute specific functional scenarios; each test case corresponds to one or more functional paths or performance metrics to be verified.

[0026] Step 102: Divide at least one test case into a first scale level or a second scale level according to the execution scale.

[0027] In embodiments of the present invention, since test cases of different sizes have fundamentally different requirements for the characteristics of the simulation platform, it is necessary to classify at least one test case into different sizes before execution verification, so that differentiated platform scheduling strategies can be adopted for different size levels. The first size level corresponds to test cases with smaller execution scales, and the second size level corresponds to test cases with larger execution scales, with the execution scale of the second size level being greater than that of the first size level.

[0028] Step 103: Execute all test cases on the first simulation platform and export the intermediate state data of all test cases.

[0029] In step 103, regardless of whether the test case belongs to the first or second scale level, it must be pre-executed on the first simulation platform. The first simulation platform plays a crucial role in the multi-platform collaborative architecture of this invention: on the one hand, leveraging its fast execution speed and convenient debugging, it performs preliminary functional verification on all test cases, ensuring the logical correctness of the test cases themselves—that is, the test cases can correctly drive the design under test to complete the expected operation at the functional abstraction level, eliminating defects in the test case writing itself; on the other hand, the first simulation platform exports intermediate state data during execution, which includes at least one of the following: initialization memory data, static random access memory dump data after execution, and data packet data. This intermediate state data will be injected into the second and third simulation platforms as the initial state for the second and third simulation platforms to start verification, thereby achieving cross-platform state alignment.

[0030] like Figure 2 As shown, after the test cases are executed on the first simulation platform, they go through stages such as memory initialization, running, outputting, and ending, exporting intermediate state data including memory initialization data, memory dump data, and data packet data. This intermediate state data is simultaneously output to the input interfaces of the second and third simulation platforms.

[0031] After obtaining intermediate state data in step 103, differentiated platform scheduling is performed according to the scale level of the test cases: test cases belonging to the first scale level are executed in step 104, and test cases belonging to the second scale level are executed in step 105.

[0032] Step 104: Inject the test cases of the first scale level and the corresponding intermediate state data into the second simulation platform to verify the design under test. After the verification is passed on the second simulation platform, inject the data into the third simulation platform for verification.

[0033] In an embodiment of the present invention, for test cases of the first scale level, the scheduling order is first the second simulation platform and then the third simulation platform, that is, the verification order is the first simulation platform, the second simulation platform, and the third simulation platform.

[0034] The technical considerations for adopting this scheduling order are as follows: the execution scale of test cases at the first scale level is relatively small, and the execution time on the second simulation platform is within an acceptable range of the project cycle. Since the second and third simulation platforms share the same design under test (DUT), the basic path verification of the DUT is completed first on the second simulation platform using its high simulation accuracy and strong testing capabilities. This ensures that the basic functional links of the DUT at the register transfer level are error-free. Then, the DUT with verified paths is deployed to the third simulation platform for further verification. This order avoids directly deploying the DUT to the third simulation platform, where debugging methods are limited, if there are still defects in the basic path. If there are defects in the basic path, these defects will manifest as a large number of difficult-to-locate abnormal behaviors on the third simulation platform. Due to the limited signal observation depth and debugging interface resources of the third simulation platform, checking these basic defects one by one will consume far more debugging time and human resources than expected.

[0035] Specifically, the process of injecting the first-scale test cases and corresponding intermediate state data into the second simulation platform includes: loading the initialization memory data from the intermediate state data into the storage space of the design under test (DUT) in the second simulation platform; configuring the run-through memory dump data and data packet data into the test environment of the second simulation platform; then starting the second simulation platform to execute the test cases; performing clock-cycle simulation on the DUT; and observing whether the signal changes of the DUT within each clock cycle conform to the expected behavior. After the second simulation platform verifies the design under test, i.e., after the basic path of the DUT is verified to be correct, the test cases are deployed to the third simulation platform for execution using the same intermediate state data injection method. This utilizes the hardware parallel execution characteristics of the third simulation platform for faster verification and examines the behavioral consistency of the DUT in an execution environment closer to real hardware.

[0036] In an optional embodiment, the first scale level can be further subdivided into two sub-levels: very small scale and small scale. For very small scale test cases (e.g., test cases that execute for approximately 1 to 2 seconds on the first simulation platform), since they are mainly used to verify a single functional path of the design under test, their execution time on the second simulation platform is extremely short (e.g., on the order of tens of minutes), thus enabling rapid path verification and deployment to the third simulation platform. For small scale test cases (e.g., test cases that execute for approximately 1 to 2 minutes on the first simulation platform), the execution time on the second simulation platform may reach several hours to tens of hours, but it is still within an acceptable range, making it suitable for path verification via the second simulation platform.

[0037] Step 105: Inject the second-scale test cases and corresponding intermediate state data into the third simulation platform to verify the design under test.

[0038] In embodiments of the present invention, for test cases of the second scale level, a scheduling order of first the first simulation platform and then the third simulation platform is adopted, i.e., the verification order is the first simulation platform followed by the third simulation platform. The technical consideration for adopting this scheduling order is that the execution scale of test cases of the second scale level is large. If they are directly deployed to the second simulation platform for execution, the execution time of large-scale test cases would reach several months or even years, far exceeding the tolerance of the project cycle, because the event-driven simulation engine of the second simulation platform needs to calculate the state changes of all active signals clockwise. The third simulation platform, utilizing the hardware parallel execution characteristics, can execute the same design under test at a much higher speed than the second simulation platform, compressing the execution time of large-scale test cases to an acceptable range.

[0039] The core purpose of executing large-scale test cases on a third-party simulation platform is to quickly expose potential defects in the design under test under boundary conditions and large data volume scenarios. Unlike small-scale test cases, which mainly verify single functional paths, large-scale test cases, through the continuous input of large amounts of data and the cross-coverage of complex functional paths, can uncover potential defects in the design boundary of the design under test. These boundary defects are often not triggered under the simple data scenarios of small-scale test cases, and will only be exposed under the continuous pressure of large data volumes and complex scenarios.

[0040] It should be noted that for test cases of the second scale level, pre-execution on the first simulation platform is also required in step 103 before verification on the third simulation platform. The first simulation platform plays a crucial role in the pre-execution of large-scale test cases: ensuring the logical correctness of the test cases themselves and exporting intermediate state data for state injection on the third simulation platform. Because the first simulation platform uses behavioral-level software simulation, its execution speed is the fastest among the three platforms, and it can complete pre-execution within a reasonable time even for large-scale test cases.

[0041] In a specific example, the second scale level includes two sub-levels: large-scale (execution time approximately 0.5 to 1 hour on the first simulation platform) and ultra-large-scale (execution time approximately 6 to 12 hours on the first simulation platform). For large-scale test cases, the execution time on the third simulation platform can be reduced to a fraction of a percent to a thousandth of that on the second simulation platform; for ultra-large-scale test cases, execution time on the second simulation platform may exceed one year, while on the third simulation platform it can be reduced to a few days to a few weeks.

[0042] like Figure 2 As shown, for large-scale test cases, intermediate state data is exported from the first simulation platform and injected into the third simulation platform. The third simulation platform performs rapid verification of the design under test to expose potential defects of the design under test in large data volume scenarios.

[0043] like Figure 5 The diagram illustrates the multi-platform verification process. For the first-scale level (small scale and below) test cases, the test cases are first pre-executed on the first simulation platform. The first simulation platform sequentially executes steps such as initialization and dumping memory and data packets, running, ending, and dumping memory and data packets again, and generates a result report. The exported intermediate state data is then injected into the test environment of the second simulation platform for path verification of the design under test. After successful path verification, the same intermediate state data is injected into the main control environment of the third simulation platform for further verification of the design under test.

[0044] like Figure 6 As shown, for the second-scale level of test cases, the test cases are first dumped using the first simulation platform. When the third simulation platform detects a defect during the execution of the test cases, the defect is reproduced on the third simulation platform. The test cases are then reduced and migrated to the second simulation platform for reproduction and debugging. After the defect in the design under test is fixed on the second simulation platform, it is re-verified on both the second and third simulation platforms. Once confirmed to be successful, the defect is closed.

[0045] like Figure 7 As shown, in a specific example, after completing the benchmark test scenario, the first simulation platform exports test statistics, dump data packets, and memory data. This exported data is then passed to the second simulation platform. The second platform performs performance analysis on the instruction statistics and performance monitoring unit data in the exported data, updates the toolchain and microarchitecture parameters based on the analysis results, and re-runs the benchmark test scenario on the first simulation platform, forming an iterative optimization loop centered on the benchmark test scenario. Simultaneously, the output of the second simulation platform is also passed to the third simulation platform for accelerated execution of large-scale test cases. Through this iterative process, multi-platform collaboration not only serves defect verification but also supports continuous iteration of performance optimization, further improving the coverage depth of the verification work on the overall quality of the design under test.

[0046] Example 2 In a specific example, the method also includes: when a defect is detected on the third simulation platform, reducing the size of the test cases that caused the defect to obtain reduced test cases; and migrating the reduced test cases and the corresponding intermediate state data to the second simulation platform for defect localization.

[0047] The aforementioned scale reduction refers to reducing the execution scale of test cases to the first scale level by reducing the amount of data in the test cases, shortening the execution instruction sequence, or reducing the complexity of the test scenarios, while keeping the core incentive conditions for triggering defects unchanged. This allows the test cases to complete simulation verification on the second simulation platform with an acceptable execution time.

[0048] Specifically, when the third simulation platform detects abnormal behavior in the design under test during the execution of large-scale test cases, the limited debugging interface resources and signal detection depth of the third simulation platform make root cause localization of the defect extremely difficult and time-consuming. Therefore, it is necessary to reduce the scale of the test cases that trigger the defect. That is, while keeping the core stimulus conditions that trigger the defect unchanged, the data size and execution scope of the test cases should be reduced, so that while retaining the ability to trigger the defect, the simulation can be completed on the second simulation platform with an acceptable execution time. The reduced test cases, along with the corresponding intermediate state data, are migrated to the second simulation platform. Utilizing the second simulation platform's comprehensive observation capabilities of any signal node and flexible debugging methods, the root cause of the defect can be accurately located. Thus, through the collaborative division of labor—rapidly exposing defects on the third simulation platform and accurately locating them on the second simulation platform—a balance between rapid exposure and accurate debugging is achieved. This avoids the inefficient approach of waiting months to execute large-scale test cases directly on the second simulation platform, and also avoids the predicament of being unable to locate defects for a long time due to limitations in debugging methods on the third simulation platform.

[0049] In an optional example, before migrating to the second simulation platform, the reduced test cases can be pre-validated on the first simulation platform to confirm that the reduced test cases can still trigger the same abnormal behavior as the original defect in the behavioral model. This avoids the defect being unreproducible on the second simulation platform due to improper reduction. If the reduced test cases can also reproduce the defect on the first simulation platform, preliminary defect analysis can be performed directly on the first simulation platform to further narrow down the investigation scope before migrating to the second simulation platform for precise localization.

[0050] Example 3 In a specific example, after migrating the reduced test cases and corresponding intermediate state data to the second simulation platform for defect localization, the process also includes: repairing defects in the design under test; and re-performing the verification of the reduced test cases and corresponding intermediate state data on the second and third simulation platforms, respectively.

[0051] Specifically, after root cause localization is completed on the second simulation platform, corresponding defect repairs are performed on the design under test (DUT). Since the second and third simulation platforms share the same DUT, the repairs will be reflected simultaneously in the verification environments of both platforms. After defect repair, the reduced test cases need to be re-executed on both the second and third simulation platforms to confirm that the defect has been correctly repaired through dual-platform cross-validation. Re-verification on the second simulation platform utilizes its high simulation accuracy to confirm that the repaired DUT behaves as expected at the register-transfer level; re-verification on the third simulation platform confirms that the repaired DUT also performs correctly in the hardware execution environment, eliminating the risk of inconsistent repair results due to platform differences. Therefore, through dual-platform cross-validation, the effectiveness of defect repair can be confirmed from different accuracy dimensions, improving the credibility of the repair verification.

[0052] Example 4 In a specific example, after the verification is successfully re-executed on the second and third simulation platforms, the process also includes: restoring the scaled-down test cases to their state before the scale reduction; and re-executing the verification on the third simulation platform with the restored test cases.

[0053] Specifically, while the scaled-down test cases can trigger defects and verify the effectiveness of the fixes, their data scale and execution scope have been compressed, making it impossible to fully cover all functional paths and boundary conditions encompassed by the original large-scale test cases. Therefore, after confirming the effectiveness of the defect fix through the scaled-down test cases, it is necessary to restore the test cases to their complete state before the scale reduction and re-execute the full large-scale verification on a third simulation platform. The purpose of this step is to confirm that the defect fix is ​​effective not only for the scaled-down scenario but also will not introduce new defects or cause regression failures in other functional paths under a complete large-scale data scenario. Due to the large execution scale of the restored test cases, execution is still carried out on a faster third simulation platform to complete the full regression verification within a reasonable timeframe. Figure 6 As shown on the right, after the reduced test cases have been verified on both the second and third simulation platforms, the test cases are restored to their original state before the reduction, and a full verification is re-executed on the third simulation platform until the defect is closed.

[0054] Thus, by adopting a progressive verification strategy from reduced verification to full regression, we can ensure that defect fixes are fully verified and avoid the waste of time caused by directly performing time-consuming full tests when the fixes have not been confirmed to be effective.

[0055] Example 5 In a specific example, intermediate state data includes at least one of the following: initialization memory data, post-run static random access memory dump data, and data packet data.

[0056] Specifically, the initialization memory data is a set of data generated by initializing and assigning values ​​to each memory unit in the behavioral model of the design under test (DUT) before the first simulation platform starts executing test cases. This data defines the initial contents of each memory space when the DUT begins executing test cases, including test program instructions in the instruction memory and initial operation data in the data memory. Injecting the initialization memory data into the second and third simulation platforms enables the DUT on both platforms to start execution from the same initial memory state as the first simulation platform, eliminating behavioral deviations caused by differences in memory initialization methods.

[0057] Static Random Access Memory (SRAM) dump data is a snapshot of the memory contents derived from the behavioral model of the design under test (DUT) by the first simulation platform during or after the execution of test cases. This data reflects the internal storage state of the DUT at a specific execution node, including the current values ​​of various levels of caches, register files, on-chip static random access memory (SRAM), and other memory units. By injecting this dump data into the second and third simulation platforms, both can resume execution from a specific execution section of the first simulation platform without having to start the simulation from scratch, thus saving the time overhead of repeated execution.

[0058] The data packet data is a record of data interaction between the design under test (DUT) and the external environment, recorded by the first simulation platform during the execution of test cases. It includes input stimulus data packets and output response data packets. Injecting the data packet data into the second and third simulation platforms allows for the reconstruction of data interaction timing consistent with the first simulation platform within their verification environments, ensuring consistency of stimulus conditions for cross-platform verification.

[0059] In an optional embodiment, the intermediate state data may further include performance statistics during the execution of the first simulation platform, such as instruction execution count, cache hit rate, and bus bandwidth utilization. Although these performance statistics do not directly participate in the state injection of the second and third simulation platforms, they can serve as a reference for subsequent performance analysis and architecture optimization.

[0060] Example 6 In a specific example, the process of classifying at least one test case into a first scale level or a second scale level based on its execution scale includes: obtaining the estimated execution time of at least one test case on a first simulation platform; classifying the test case into the first scale level if the estimated execution time is less than a preset time threshold; and classifying the test case into the second scale level if the estimated execution time is not less than the preset time threshold.

[0061] Specifically, the estimated execution time can be obtained by pre-executing test cases on the first simulation platform or by estimating based on the code size and complexity of the test cases. The preset time threshold can be adjusted according to the actual needs of the project. For example, using the execution time on the first simulation platform as a reference, test cases with an execution time of 1 to 2 minutes can be classified into the first scale level, and test cases with an execution time exceeding this threshold can be classified into the second scale level. Thus, by classifying test cases by scale based on execution time, a quantitative correlation can be established between the actual computational load of the test cases and the platform scheduling strategy, avoiding scheduling biases caused by subjective judgment based on experience.

[0062] In an optional embodiment, the scale level can be further refined into multiple levels. For example, using the execution time on the first simulation platform as a reference, test cases can be divided into four levels: extremely small scale (e.g., execution time of approximately 1 to 2 seconds), small scale (e.g., execution time of approximately 1 to 2 minutes), large scale (e.g., execution time of approximately 0.5 to 1 hour), and ultra-large scale (e.g., execution time of approximately 6 to 12 hours). Extremely small and small scale correspond to the first scale level, while large and ultra-large scale correspond to the second scale level. It should be noted that the above execution time values ​​are reference values ​​on the first simulation platform. If converted to the second simulation platform, the execution time needs to be multiplied by more than a thousand times. This difference in magnitude is the fundamental motivation for the differentiated platform scheduling strategy adopted in this invention. For test cases that only take a few minutes to execute on the first simulation platform, the execution time on the second simulation platform may reach several hours to tens of hours, which is still within an acceptable range. However, for test cases that take several hours to execute on the first simulation platform, the execution time on the second simulation platform will reach several months or even years, far exceeding the tolerance of the project cycle.

[0063] like Figure 4 As shown, Figure 4 This shows test case sets of different sizes and levels. Figure 4 The rectangular area below, from left to right, corresponds to extremely small scale, small scale, large scale, and very large scale, respectively, and the design scale. Figure 4 The rectangular area above, from left to right, corresponds to clusters, dies, chips, and multi-chips, respectively, as the project progresses ( Figure 4 The distribution relationship (horizontal axis). Among them, extremely small scale and small scale correspond to the first scale level in this invention, while large scale and ultra-large scale correspond to the second scale level in this invention. As the project progresses, the size of the test cases gradually increases, and the design scale also expands accordingly. Therefore, the scale level classification of test cases needs to be dynamically adjusted in conjunction with the project phase.

[0064] Furthermore, the classification of scale levels is not limited to execution time; it can also be based on a comprehensive evaluation of dimensions such as the number of lines of code in the test cases, data throughput, or the number of functional modules to be tested.

[0065] The aforementioned preset duration thresholds are not fixed values ​​and can be dynamically adjusted based on the project verification stage and available computing resources. For example, when verification resources are sufficient in the early stages of the project, a lower threshold can be set to allow more test cases to pass through the precise verification path of the second simulation platform; when the verification progress needs to be accelerated in the later stages of the project, the threshold can be appropriately increased to reduce the number of test cases executed on the second simulation platform.

[0066] Specifically, the estimated execution time can be obtained in several ways: First, the test cases are pre-executed on the first simulation platform, and the execution time is recorded as the estimated value. Second, a regression model is established based on the source code size of the test cases (e.g., number of instructions, number of lines of code) and historical execution data, and the execution time is predicted through the model. Third, empirical estimation is made based on the complexity of the functional modules involved in the test cases and the data throughput. Therefore, by using the execution time of the first simulation platform as a unified metric for scale classification, the actual computational load of the test cases can be transformed into a quantifiable scheduling basis, making the formulation of the platform scheduling strategy objective and repeatable.

[0067] Example 7 In a specific example, the first simulation platform is a behavioral-level software simulation platform, the second simulation platform is a register-transfer-level simulation platform, and the third simulation platform is a programmable logic device prototype verification platform.

[0068] Specifically, the first simulation platform, namely the behavioral software simulation platform, constructs a behavioral model equivalent to the function of the design under test using a high-level programming language and executes the simulation in software on a general-purpose processor. This platform has the fastest execution speed and the most flexible debugging methods among the three platforms, but its simulation accuracy is the lowest, making it suitable for functional verification and initial debugging of test cases. For example, the first simulation platform can be, but is not limited to, the gem5 simulator.

[0069] The second simulation platform, the register-transfer level simulation platform, uses an event-driven simulation engine to perform clock-cycle simulation of the design under test (DUT). It boasts the highest simulation accuracy among the three platforms, supporting comprehensive observation and waveform tracing of any signal node within the DUT, making it a core means of defect root cause localization. However, its execution speed is the slowest of the three platforms, making it suitable for precise verification and defect localization of small-scale test cases. For example, the simulation tool for the second platform can be, but is not limited to, VCS or Xrun.

[0070] The third simulation platform, or programmable logic device (FPGA) prototyping platform, maps the design under test (DUT) onto a programmable logic device after logic synthesis and place-and-route, utilizing the parallel execution capabilities of hardware for high-speed simulation. This platform's simulation accuracy falls between the first and second simulation platforms, its execution speed is significantly higher than the second platform but lower than the first, and its debugging capabilities are the weakest. This platform is suitable for the rapid execution of large-scale test cases and defect exposure. For example, the third simulation platform can be, but is not limited to, FPGA prototyping boards.

[0071] like Figure 3 As shown, the characteristic relationships among the three simulation platforms include execution speed, simulation accuracy, and debugging capability (i.e., adjustability). The execution speed ranking is: First simulation platform > Third simulation platform > Second simulation platform; the simulation accuracy ranking is: Second simulation platform > Third simulation platform > First simulation platform; the debugging capability ranking is: Second simulation platform > First simulation platform > Third simulation platform. Specifically, the Second simulation platform supports comprehensive observation and waveform tracing of any signal node within the design under test, and its debugging capability is superior to the First simulation platform. Although the First simulation platform offers flexible debugging methods and convenient operation, it cannot observe the signal details of the register transfer stage, and its debugging depth is less than that of the Second simulation platform.

[0072] like Figure 8 As shown, Figure 8 The detailed operation flow of each of the three simulation platforms is shown. The flow of the first simulation platform includes, in sequence: building a behavioral simulator, performing smoke tests, performing regression tests, and exporting replay statistics, and outputting a test report containing test statistics and performance monitoring unit data.

[0073] The second simulation platform follows a sequential process: First, it is built based on the design under test (DUT) and test baseline A. Then, smoke tests and regression tests are executed sequentially. After the regression tests are completed, a new round of building is performed based on the DUT and test baseline B. Smoke tests and regression tests are then executed sequentially again, and corresponding test reports are output. Test baseline A corresponds to a large set of test cases, used to verify the basic pathway on the second simulation platform. Test baseline B is introduced after the regression of test baseline A passes; it corresponds to a very small set of test cases and is used for more granular and precise verification of the DUT.

[0074] In addition, the second simulation platform also includes a replay subprocess: after loading the memory dump data, test cases are run synchronously on the first and second simulation platforms, and the output results on both sides are compared to cross-verify the behavioral consistency of the design under test.

[0075] The process of the third simulation platform includes, in sequence: synthesis and placement and routing, downloading bitstream files to programmable logic devices, performing playback smoke tests and performing playback regression tests, and outputting corresponding test reports.

[0076] Example 8 In a specific example, the process of obtaining test cases for the design under test includes: generating benchmark test scenarios based on market demands for the design under test; constructing source code for at least one test case based on the benchmark test scenarios; and compiling the source code to generate an executable file for at least one test case.

[0077] Specifically, benchmark test scenarios are standardized sets of test scenarios extracted from the target application market of the design under test (DUT). Each benchmark test scenario defines the sequence of functional operations and performance metrics that the DUT needs to perform under specific application conditions. Based on the benchmark test scenarios of the DUT, source code for at least one test case is constructed. This source code describes the instruction sequence and data configuration required to drive the DUT to execute the benchmark test scenarios. After cross-compilation, the source code generates executable files for the test cases that can be executed on the DUT's instruction set architecture. The executable files are further converted into binary formats that can be loaded by various simulation platforms.

[0078] In a specific example, benchmark test scenarios are generated based on market demands for the design under test (DUT); source code for at least one test case is constructed based on the benchmark test scenarios; and the source code is compiled to generate executable files for at least one test case. Specifically, a benchmark test scenario is a standardized test scenario description extracted from market demands based on the target application scenario of the DUT, used to define the functional operations and performance indicators that the DUT needs to perform under actual working conditions. The test case generation process includes: first, writing the source code for test cases based on the benchmark test scenarios; then, cross-compiling the source code to generate executable files that can be executed on the instruction set architecture of the DUT; and finally, converting the executable files into binary format data that can be loaded by various simulation platforms. Figure 2 As shown, a schematic diagram illustrates the compilation and conversion process of test cases and the execution stages of each simulation platform according to an embodiment of the present invention. It shows that the test cases are compiled from source code to generate executable files, and then converted into binary format data, which can be directly loaded and executed by subsequent simulation platforms. Furthermore, Figure 2 It also demonstrated the complete process of generating a results report after initializing the memory, running, outputting, and ending the process on various platforms.

[0079] Therefore, by generating benchmark test scenarios based on market demand, we can ensure that test cases can cover the core functional and performance requirements that the design under test needs to meet in actual applications, so that the verification work directly serves the product's market competitiveness goals.

[0080] In an optional embodiment, test cases can also be generated using an automated test generation tool, which automatically generates a set of test cases covering different functional paths and boundary conditions based on the functional specifications of the design under test, thereby improving test coverage and reducing the workload of manually writing test cases.

[0081] In another alternative embodiment, test cases can be continuously accumulated as the project progresses and verification work deepens. Newly generated test cases, together with existing test cases, constitute a test case library. Subsequent regression verification can select test cases from the library for execution to verify that modifications to the design under test have not introduced new defects. Test cases in the library are categorized and managed according to their size, facilitating the rapid selection of suitable test case sets at different verification stages.

[0082] In this embodiment of the invention, by dividing all test cases into different levels according to their execution scale and setting differentiated platform scheduling orders for different scale levels, small-scale test cases are prioritized to complete the basic path verification of the design under test on the second simulation platform with high simulation accuracy. After the path verification is passed, they are then deployed to the third simulation platform with faster execution speed for further verification. This avoids running directly on the third simulation platform with limited debugging means before the path has been verified, which would lead to low problem-solving efficiency. At the same time, for large-scale test cases, since their execution time on the second simulation platform is extremely long, scheduling them directly to the third simulation platform with faster execution speed can quickly expose potential defects in the design under test and significantly shorten the verification cycle in large-scale testing scenarios. In addition, by pre-executing all test cases and exporting intermediate state data on the first simulation platform, cross-platform state injection from the first simulation platform to the second and third simulation platforms is realized. This allows each simulation platform to carry out verification work based on a consistent initial state, reducing the data alignment cost between multiple platforms. Thus, without relying on expensive dedicated hardware accelerators, the systematic collaboration of multiple easily accessible platforms achieves a reduction in verification cost and an improvement in verification efficiency.

[0083] The steps described above are for clarity only. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this invention. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, without changing the core design of the algorithm and process, are also within the protection scope of this invention.

[0084] Furthermore, the examples mentioned in the above embodiments can be freely combined, and any combination can be understood as an embodiment. The terms "embodiment" or "example" appearing in various locations in the specification do not necessarily refer to the same embodiment, nor are they independent or alternative embodiments mutually exclusive with other embodiments. Those skilled in the art will understand that the embodiments described herein can be combined with other embodiments.

[0085] In the description of the embodiments of this invention, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this invention, "multiple" means two or more, unless otherwise explicitly defined.

[0086] Another embodiment of the present invention relates to an electronic device, such as Figure 9 As shown, it includes at least one processor 201; and a memory 202 communicatively connected to at least one processor 201; wherein the memory 202 stores instructions executable by at least one processor 201, the instructions being executed by at least one processor 201 to enable at least one processor 201 to execute the design verification method of the multi-simulation platform as described above.

[0087] The memory 202 and processor 201 are connected via a bus, which may include any number of interconnecting buses and bridges, connecting various circuits of one or more processors 201 and memory 202 together. The bus may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 201 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 201.

[0088] Processor 201 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 202 can be used for data accessed by processor 201 during operation.

[0089] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the method embodiments described above.

[0090] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0091] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.

Claims

1. A design verification method using multiple simulation platforms, characterized in that, include: Obtain at least one test case for the design under test; At least one of the test cases is divided into a first scale level or a second scale level according to the execution scale; Execute all the test cases on the first simulation platform and export the intermediate state data of all the test cases; The test cases and corresponding intermediate state data of the first scale level are injected into the second simulation platform to verify the design under test. After the design is verified on the second simulation platform, it is injected into the third simulation platform for verification. The test cases of the second scale level and the corresponding intermediate state data are injected into the third simulation platform to verify the design under test; The execution scale of the second scale level is larger than that of the first scale level; the execution speed of the third simulation platform is higher than that of the second simulation platform; and the simulation accuracy of the second simulation platform is higher than that of the third simulation platform.

2. The design verification method for a multi-simulation platform according to claim 1, characterized in that, The method further includes: If the third simulation platform detects a defect, the test cases that cause the defect are reduced in size to obtain the reduced test cases. The reduced test cases and corresponding intermediate state data are migrated to the second simulation platform for defect localization.

3. The design and verification method for a multi-simulation platform according to claim 2, characterized in that, After migrating the reduced test cases and corresponding intermediate state data to the second simulation platform for defect localization, the method further includes: Defect repair is performed on the design under test; The reduced test cases and the corresponding intermediate state data are re-verified on the second simulation platform and the third simulation platform, respectively.

4. The design and verification method for a multi-simulation platform according to claim 3, characterized in that, After successful re-execution of verification on the second and third simulation platforms, the process further includes: The reduced test cases are restored to their state before the size reduction; The restored test cases are then re-executed for verification on the third simulation platform.

5. The design verification method for a multi-simulation platform according to claim 1, characterized in that, The intermediate state data includes: At least one of the following: initialization memory data, post-run static random access memory dump data, and data packet data.

6. The design verification method for a multi-simulation platform according to claim 1, characterized in that, The step of classifying at least one of the test cases into a first scale level or a second scale level based on the execution scale includes: Obtain the estimated execution time of at least one of the test cases on the first simulation platform; If the estimated execution time is less than a preset time threshold, the test cases will be classified into the first scale level. If the estimated execution time is not less than the preset time threshold, the test cases are classified into the second scale level.

7. The design verification method for a multi-simulation platform according to claim 1, characterized in that, The first simulation platform is a behavioral-level software simulation platform; the second simulation platform is a register-transfer-level simulation platform; and the third simulation platform is a programmable logic device prototype verification platform.

8. The design verification method for a multi-simulation platform according to any one of claims 1 to 7, characterized in that, The step of obtaining at least one test case for the design under test includes: Generate benchmark test scenarios based on the market demand for the design under test; Build the source code for at least one of the test cases based on the benchmark test scenario; Compile the source code to generate an executable file for at least one of the test cases.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the design verification method of the multi-simulation platform as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the design verification method for the multi-simulation platform as described in any one of claims 1 to 8.

Images