Method for generating test case, electronic equipment and storage medium
By generating test cases corresponding to system-level testing operations, using the correlation between silicon front and back verification, and comparing the results to locate chip design errors, the problem of failure to effectively locate silicon back verification errors in the existing technology is solved, and testing efficiency and accuracy are improved sex.
Patent Information
- Application Number
- CN202310582469.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-12-03
AI Technical Summary
Existing system-level tests fail to effectively locate the root cause of errors in chip designs in post-silicon verification, resulting in abnormal chips being usually invalidated.
By generating test cases corresponding to system-level testing operations, using the correlation between pre-silicon verification and post-silicon verification, the results of the two are compared to locate the cause of the error. The method includes receiving system-level test operations, generating chip operations, determining test scenarios, acquiring configuration files, and generating test cases based on the scenarios and files.
It improves the ability to test the root cause of errors at the system level, provides more information to assist in the positioning of errors, reduces the time for eliminating chip design errors, and improves verification efficiency.
Smart Images

Figure CN119067062A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of logic system design technology, and in particular, to a method, electronic device, and storage medium for generating corresponding test cases for chip design in pre-silicon and post-silicon stages. Background Art
[0002] In the field of integrated circuit verification, in order to test the function of the chip design, it is usually necessary to verify the chip design. During the verification process, test cases are needed to test whether the chip functions correctly.
[0003] Chip verification can be divided into pre-silicon verification and post-silicon verification. Pre-silicon verification is usually for the verification of the chip's own functions, while post-silicon verification is usually system-level testing (SLT), which is used to verify whether the actual chip's system-level functions are correct in a real electronic system environment.
[0004] Currently, because system-level testing itself cannot locate the root cause of errors through complex electronic systems, the usual treatment for chips that have abnormalities during post-silicon verification is to simply discard them. Summary of the invention
[0005] In view of this, the present application proposes a method for generating a test case, an electronic device and a storage medium.
[0006] In a first aspect of the present application, a method for generating a test case is provided. The method comprises: receiving a system-level test operation of a chip design; generating multiple chip operations according to the system-level test operation, wherein the multiple chip operations include a first chip operation; determining a first test scenario according to the first chip operation; obtaining a first configuration file corresponding to the first test scenario, wherein the configuration file is associated with the system-level test operation; and generating multiple test cases corresponding to the first chip operation according to the first test scenario and the first configuration file.
[0007] In a second aspect of the present application, an electronic device is provided, comprising: a memory for storing a set of instructions; and at least one processor configured to execute the set of instructions to perform the method described in the first aspect.
[0008] In a third aspect of the present application, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores a set of instructions for an electronic device, wherein the set of instructions is used to enable the electronic device to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only for the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0010] Figure 1 Schematic diagram of a host according to an embodiment of the present application.
[0011] Figure 2 A schematic diagram of a simulation test system according to an embodiment of the present application is shown.
[0012] Figure 3 A schematic diagram of unifying pre-silicon verification and post-silicon verification using a PSS tool according to an embodiment of the present application is shown.
[0013] Figure 4 A flow chart of a method for generating a test case according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0014] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.
[0015] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in this application should be understood by people with ordinary skills in the field to which this application belongs. The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.
[0016] As mentioned above, system-level testing itself cannot locate the root cause of errors through complex electronic systems. How to enhance the ability of system-level testing to locate the root cause of errors is a technical problem that needs to be solved in the art.
[0017] The inventor of the present application realizes that pre-silicon verification and post-silicon verification have certain correlations, but currently no technology can utilize this correlation. The embodiments of the present application are based on this discovery and utilize the correlations between pre-silicon verification and post-silicon verification to enhance the ability of system-level testing to locate the root cause of errors.
[0018] Figure 1FIG. 1 is a schematic diagram of a host 100 according to an embodiment of the present application. The host 100 may be an electronic device running a simulation system. Figure 1 As shown, the host 100 may include: a processor 102, a memory 104, a network interface 106, a peripheral interface 108 and a bus 110. The processor 102, the memory 104, the network interface 106 and the peripheral interface 108 are connected to each other in communication within the host through the bus 110.
[0019] Processor 102 may be a central processing unit (CPU), an image processor, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or one or more integrated circuits. Processor 102 may be used to perform functions related to the technology described in this application. In some embodiments, processor 102 may also include multiple processors integrated into a single logical component. Figure 1 As shown, the processor 102 may include a plurality of processors 102a, 102b, and 102c.
[0020] The memory 104 may be configured to store data (eg, instruction sets, computer codes, intermediate data, etc.). In some embodiments, the simulation test system for simulation test design may be a computer program stored in the memory 104. Figure 1 As shown, the data stored in the memory may include program instructions (e.g., program instructions for implementing the simulation method of the present application) and data to be processed (e.g., the memory may store temporary codes generated during the compilation process). The processor 102 may also access the program instructions and data stored in the memory, and execute the program instructions to operate on the data to be processed. The memory 104 may include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 104 may include a random access memory (RAM), a read-only memory (ROM), an optical disk, a magnetic disk, a hard disk, a solid-state drive (SSD), a flash memory, a memory stick, etc.
[0021] The network interface 106 can be configured to provide communication with other external devices to the host 100 via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC), etc.), a cellular network, the Internet, or a combination thereof. It is understood that the type of network is not limited to the above specific examples. In some embodiments, the network interface 106 can include any number of network interface controllers (NICs), radio frequency modules, transceivers, modems, routers, gateways, adapters, cellular network chips, etc., in any combination.
[0022] The peripheral interface 108 can be configured to connect the host 100 to one or more peripheral devices to achieve information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touch pad, a touch screen, a microphone, and various sensors, and output devices such as a display, a speaker, a vibrator, and an indicator light.
[0023] The bus 110 may be configured to transmit information between various components of the host 100 (e.g., the processor 102, the memory 104, the network interface 106, and the peripheral interface 108), such as an internal bus (e.g., a processor-memory bus), an external bus (USB port, PCI-E bus), etc.
[0024] It should be noted that, although the above host architecture only shows the processor 102, the memory 104, the network interface 106, the peripheral interface 108 and the bus 110, in the specific implementation process, the host architecture may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above host architecture may also only include the components necessary for implementing the embodiment of the present application, and does not necessarily include all the components shown in the figure.
[0025] In the field of logic system design (such as chip design), a simulation system can be used to simulate the design. The simulation system can be running on Figure 1 A computer program on a host computer 100 is shown.
[0026] Figure 2 FIG. 2 is a schematic diagram of a simulation test system 200 according to an embodiment of the present application. Figure 2 As shown, the simulation test system 200 may include a PSS (Portable Stimulus Standard) tool 210, a test environment 220 and a device under test 230. The simulation test system 200 may generate multiple test cases according to user input 202, for testing multiple functional units of the logic system design during the simulation process of the logic system design.
[0027] The PSS tool 210 can generate multiple test cases across platforms based on user input 202. In the simulation test scenario, the efficiency of the test needs to be taken into account. For example, the same test case can be tested on different platforms or at different levels. The PSS tool 210 is proposed to shorten the test time and to achieve continuity in vertical reuse and cross-platform reuse of verification cases and verification plans. The PSS tool 210 generally uses DSL (Domain Specific Language) to process internal logic.
[0028] The test environment 220 is used to obtain multiple test cases from the PSS tool 210 and actually convert them into multiple test cases that can be executed. The test environment 220 may include a verification environment written in the SystemVerilog language, such as a Universal Verification Methodology (UVM) environment. In some embodiments, the test environment 220 may include a hardware simulation platform composed of a hardware simulator (emulator) and a host, etc., to verify the device under test 230. In some other embodiments, the test environment 220 may include a software simulator (simulator), a hardware simulator (emulator), a system-level test board, etc., to simulate and test the device under test 230.
[0029] Software simulators and hardware emulators are usually used in the design phase, i.e., pre-silicon verification. System-level test boards are usually used for post-silicon verification.
[0030] System-level tests for post-silicon verification usually include functional tests, performance tests, and reliability tests. The above tests are usually performed by the test environment sending real operations (for example, taking pictures) to the operating system, driver software, etc. Pre-silicon verification, on the other hand, directly applies specific signals to the chip design to stimulate the chip design. In addition, compared to the pre-silicon verification, where the object is the chip design, the test object of post-silicon verification is the real chip. It is precisely because of these differences that current verification tools cannot unify pre-silicon verification and post-silicon verification. However, the inventors of the present application realize that the system-level operations through post-silicon verification are essentially a series of incentives for chip design. If the system-level operations of post-silicon verification can be matched with the incentives for chip design, the errors generated in the post-silicon verification can be compared in the pre-silicon verification test, thereby providing help in locating the root cause of the errors in the post-silicon verification.
[0031] Figure 3 A schematic diagram of unifying pre-silicon verification and post-silicon verification using a PSS tool according to an embodiment of the present application is shown.
[0032] Based on the user's definition of the chip design, the system-level test operation of the chip design can be determined. For example, for a mobile phone SoC (system on chip) chip, a typical system-level test operation may be to take a photo with a camera; for an MCU chip used for industrial control, a typical system-level test operation may be to complete a specified mechanical operation instruction.
[0033] The following description uses a mobile phone SoC chip as an example of chip design and uses a camera to take photos as an example of system-level test operations. It is understandable that other chip designs and other system-level test operations may also be applicable to the method of the present application.
[0034] The operation of taking a photo with a camera may include receiving an instruction to call the camera, reading the image data of the CMOS sensor, and storing it in the memory via the bus. These operations can be further converted. For example, receiving an instruction to call the camera can be converted to sending an excitation signal to a specified address; reading the image data of the CMOS sensor can be converted to reading data from a specified address; storing it in the memory via the bus can be converted to writing data to a specified address. These operations of sending excitation signals, reading data, and writing data can be directly applied to the chip design.
[0035] Therefore, if Figure 3 As shown, the PSS tool 210 can receive a system-level test operation 300 from a user (for example, taking a picture with a camera), and convert the operation 300 into multiple chip operations 3021-3023 (receiving instructions to call the camera, reading image data from the CMOS sensor, and storing it in the memory via the bus). In an actual system, these conversions are performed by an operating system or a driver. However, in an embodiment of the present application, these operations are performed by the PSS tool 210. The PSS tool 210 can use a preset library to complete the conversion of a system-level test operation to multiple chip operations, and can also allow users to personalize the configuration to support more system-level test operations.
[0036] These chip operations can be further converted into test scenarios 3031-3033 by the PSS tool 210. For example, reading image data from a CMOS sensor can be converted into a test scenario of reading data from a specified address. Figure 3 As shown, multiple test scenarios 3031-3033 corresponding to chip operations can be generated.
[0037] Reading data from a specified address is a classic test scenario for the PSS tool 210. For example, the PSS tool 210 can generate multiple test cases by calling a scenario model and a user configuration file. The user configuration file can, for example, configure the address range for reading data and the method for reading data. It is understandable that there can be a corresponding configuration file for each test scenario.
[0038] Therefore, the PSS tool 210 can generate test cases 3051-3053 respectively according to the test scenarios 3031-3033 and the corresponding configuration files 304. It can be understood that each of the test cases 3051-3053 includes multiple test cases.
[0039] In this way, system-level test operation 300 can be used for system-level testing (i.e., post-silicon verification), while test cases 3051-3053 can be used for simulation verification of chip design (i.e., pre-silicon verification). The results of post-silicon verification and pre-silicon verification can be compared with each other to produce a comparison result. For example, when a chip fails the test of taking pictures with a camera in post-silicon verification, but all tests in pre-silicon verification are passed, then the tester can largely rule out the cause of chip design errors, and it is more likely to be a driver error. For another example, when a chip fails the test of taking pictures with a camera in post-silicon verification and there is an error in the read and write operations of a certain address in pre-silicon verification, the tester can do further testing to determine whether there is an error in the chip design, and can try to fix the error by modifying the driver.
[0040] It can be seen that by generating test cases for pre-silicon verification corresponding to system-level tests, more information can be provided to testers to assist in locating errors.
[0041] It is understandable that this method can be used in both post-silicon verification and pre-silicon verification. In post-silicon verification, test cases can be generated to verify the chip design again to obtain data for cross-verification. In pre-silicon verification, the generated test cases can be used in combination with other test cases to complete the verification of the chip design, and the data can be retained for cross-verification with future post-silicon verification data.
[0042] Figure 4 A flowchart of a method 400 for generating a test case according to an embodiment of the present application is shown.
[0043] Method 400 may be performed by Figure 1 More specifically, the method 400 may be executed by the PSS tool 210 running on the host 100. The method 400 may include the following steps.
[0044] In step 401, the PSS tool 210 receives a system-level test operation for a chip design (eg, Figure 3 The system-level test operation 300 ). The system-level test operation may be described using a domain-specific language (DSL), for example.
[0045] In step 403, the PSS tool 210 generates a plurality of chip operations (e.g., chip operations 3021-3023) according to the system-level test operation. The plurality of chip operations include a first chip operation (e.g., chip operation 3021). The first chip operation may be to read image data of a CMOS sensor. In some embodiments, the PSS tool 210 may generate the plurality of chip operations based on a preset library file according to the type of the chip design and the system-level test operation. The preset library may provide a variety of correspondences between system-level test operations and chip operations. This enables the PSS tool 210 to quickly decompose the system-level test operation into a plurality of chip operations. It is understandable that the PSS tool 210 may also guide the user to manually configure and generate a plurality of chip operations, thereby adapting to more personalized system-level test operations.
[0046] In step 405, the PSS tool 210 determines a first test scenario (e.g., test scenario 3031) according to the first chip operation. As described above, according to the first chip operation of reading image data of the CMOS sensor, the PSS tool 210 may determine that the first test scenario is to read data from a specified address. The specified address may be determined by the user through a configuration file.
[0047] In step 407, the PSS tool 210 obtains a first configuration file (e.g., configuration file 304) corresponding to the first test scenario. The configuration file is associated with the system-level test operation. The first configuration file may, for example, include parameters and constraints related to the first test scenario. The parameters may, for example, be an address range, a read / write mode (e.g., random read / write or sequential read / write), and a data volume.
[0048] In step 409, the PSS tool 210 generates a plurality of test cases corresponding to the first chip operation according to the first test scenario and the first configuration file. The PSS tool 210 may use the first test scenario as a scenario model, and generate a plurality of test cases in combination with the first configuration file. The first configuration file may also include the type of verification platform for running the plurality of test cases. Accordingly, a plurality of test cases may correspond to a verification platform. The types of verification platforms include software simulators, hardware simulators, or prototype verification systems. For example, for software simulators, the PSS tool 210 may generate test cases in SystemVerilog language or UVM code; for prototype verification systems, the PSS tool 210 may generate test cases in C / C++ code to run directly on prototyped chip designs.
[0049] After obtaining multiple test cases, the PSS tool 210 can use the multiple test cases to test the chip design on the verification platform to obtain pre-silicon verification results; and run the system-level test operation on the system-level test board to test the chip design to obtain post-silicon verification results.
[0050] By comparing pre-silicon verification results with post-silicon verification results, users can locate the root causes of chip design errors more quickly.
[0051] The present application also provides an electronic device. The electronic device may be Figure 1 The electronic device may include a memory for storing a set of instructions; and at least one processor configured to execute the set of instructions so that the electronic device performs the method 400.
[0052] It should be noted that the method of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the present application, and the multiple devices will interact with each other to complete the described method.
[0053] An embodiment of the present application also provides a storage medium, which stores at least one set of instructions, and when the instructions are executed, the method provided in the embodiment of the present application is performed.
[0054] The present application embodiment also provides a computer-readable storage medium storing instructions. The instruction is used to perform the above method when executed by an electronic device. The computer-readable storage medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be a computer-readable instruction, a data structure, a module of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassette, magnetic tape magnetic disk storage or other magnetic storage device or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.
[0055] Some embodiments of the present application are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0056] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the present application as described above, which are not provided in detail for the sake of simplicity.
[0057] In addition, to simplify the description and discussion, and in order not to make the present application difficult to understand, the well-known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device may be shown in the form of a block diagram so as to avoid making the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the present application will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the present application can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0058] Although the present application has been described in conjunction with specific embodiments of the present application, many alternatives, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the discussed embodiments.
[0059] This application is intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.
Claims
1. A method for generating a test case, comprising: Receive system-level test operations for chip designs; Generate a plurality of chip operations according to the system-level test operation, wherein the plurality of chip operations include a first chip operation; Determining a first test scenario according to the first chip operation; Acquire a first configuration file corresponding to the first test scenario, wherein the configuration file is associated with the system-level test operation; as well as A plurality of test cases corresponding to the first chip operation are generated according to the first test scenario and the first configuration file.
2. The method of claim 1, wherein: Generating a plurality of chip operations according to the system-level test operation further comprises: According to the type of the chip design and the system-level test operation, the plurality of chip operations are generated based on a preset library file.
3. The method of claim 1, wherein: The first configuration file includes a type of a verification platform for running the plurality of test cases or constraints corresponding to the first test scenario.
4. The method of claim 3, wherein: The types of the verification platform include a software simulator, a hardware simulator or a prototype verification system.
5. The method of claim 4, further comprising: Testing the chip design using the multiple test cases on the verification platform to obtain a pre-silicon verification result; The system-level test operation is run on a system-level test board to test the chip design to obtain a post-silicon verification result.
6. An electronic device comprising: a memory for storing a set of instructions; as well as At least one processor is configured to execute the set of instructions to perform the method according to any one of claims 1 to 5.
7. A non-transitory computer-readable storage medium storing a set of instructions for an electronic device, the set of instructions being used to cause the electronic device to execute the method of any one of claims 1 to 5.
