Generation method, test method, generator, electronic equipment and medium

Abstract

The embodiment of the invention relates to a test case generation method, a test method, a test case generator, electronic equipment and a medium. The test case generation method comprises the steps that a unified structure model of a tested design is obtained, and the unified structure model comprises at least one to-be-configured parameter; acquiring parameter configuration information of the at least one to-be-configured parameter; and on the basis of the parameter configuration information and the unified structure model, generating the test case so as to test the tested design by using the test case. According to the method, the test case generation efficiency can be improved, and the completeness and efficiency of chip verification are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to a test case generation method, a test method, a test case generator, an electronic device, and a medium. BACKGROUND

[0002] With the development of big data analysis and distributed computing systems, data processing tasks are becoming more and more complex and the performance requirements are becoming higher and higher, especially for large-scale parallel computing application systems and distributed training application systems. In order to ensure the correctness and reliability of the application system, the application system needs to be strictly and fully verified. SUMMARY

[0003] At least one embodiment of the present disclosure provides a test case generation method, comprising: obtaining a unified structure model of a design under test, wherein the unified structure model comprises at least one to-be-configured parameter; obtaining parameter configuration information of the at least one to-be-configured parameter; and generating a test case based on the parameter configuration information and the unified structure model, to test the design under test by using the test case.

[0004] For example, in the method provided by an embodiment of the present disclosure, generating the test case based on the parameter configuration information and the unified structure model comprises: inputting the parameter configuration information into the unified structure model to configure the at least one to-be-configured parameter to obtain a system form model; and generating the test case based on the system form model.

[0005] For example, in the method provided by an embodiment of the present disclosure, the design under test comprises a plurality of test nodes, and generating the test case based on the system form model comprises: determining task information of each of the plurality of test nodes based on the system form model; configuring the task information of each test node to a task register corresponding to the test node; and generating the test case based on the task register and a verification environment configured for the design under test.

[0006] For example, in the method provided by an embodiment of the present disclosure, configuring the task information of each test node to the task register corresponding to the test node comprises: creating a data object for each test node to obtain a plurality of data objects, wherein the value of a variable in each of the plurality of data objects is the task information of each test node; and assigning a value to the task register based on the task information.

[0007] For example, in the method provided by an embodiment of the present disclosure, configuring the task information of each test node to the task register corresponding to the test node further comprises: packaging the plurality of data objects into a wrapper; and extracting the task information of each test node from the wrapper.

[0008] For example, in the method provided by the embodiment of the present disclosure, inputting the parameter configuration information into the unified structure model to configure the at least one to-be-configured parameter to obtain the system mode model comprises: in response to the parameter configuration information indicating that the value of the at least one to-be-configured parameter is random, randomly setting the value of the at least one to-be-configured parameter; or in response to the parameter configuration information including the set value of at least part of the at least one to-be-configured parameter, configuring the set value to the unified structure model to obtain the system mode model.

[0009] For example, in the method provided by the embodiment of the present disclosure, the design under test includes a plurality of to-be-tested hardware, and the parameter configuration information includes the number or number of to-be-tested hardware participating in the current test among the plurality of to-be-tested hardware.

[0010] For example, in the method provided by the embodiment of the present disclosure, obtaining the unified structure model of the design under test comprises: obtaining attribute information of a plurality of test nodes included in the design under test; and generating the unified structure model based on the attribute information of each of the plurality of test nodes.

[0011] For example, in the method provided by the embodiment of the present disclosure, generating the unified structure model based on the attribute information of each of the plurality of test nodes comprises: performing digital modeling on the design under test based on the attribute information of each of the plurality of test nodes to obtain the unified structure model.

[0012] At least one embodiment of the present disclosure provides a test method, comprising: obtaining a test case, wherein the test case is generated according to the test case generation method provided by any one of the embodiments of the present disclosure; and executing the test case by using a verification environment.

[0013] At least one embodiment of the present disclosure provides a test case generator, comprising: a unified random model unit configured to obtain a unified structure model of a design under test, wherein the unified structure model includes at least one to-be-configured parameter; a parameter preprocessing unit configured to obtain parameter configuration information of the at least one to-be-configured parameter; and a test case generation unit configured to generate a test case based on the parameter configuration information and the unified structure model, so as to test the design under test by using the test case.

[0014] At least one embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory comprising one or more computer program instructions; wherein the one or more computer program instructions are executed by the processor to perform the method provided by any one of the embodiments of the present disclosure.

[0015] At least one embodiment of the present disclosure provides a computer readable storage medium, which non-transitorily stores computer readable instructions, wherein the computer readable instructions, when executed by a processor, implement the method provided by any one of the embodiments of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only relate to some of the embodiments of the present disclosure, but not limit the present disclosure.

[0017] Figure 1 A schematic diagram of a SHARP protocol architecture is shown;

[0018] Figure 2 A schematic diagram of a current verification method architecture is shown;

[0019] Figure 3 A flowchart of a test case generation method provided by at least one embodiment of the present disclosure is shown;

[0020] Figure 4 An architecture diagram of a system for generating test cases provided by at least one embodiment of the present disclosure is shown;

[0021] Figure 5 A flowchart of another test case generation method provided by at least one embodiment of the present disclosure is shown;

[0022] Figure 6 A schematic diagram of a test case generator provided by at least one embodiment of the present disclosure is shown;

[0023] Figure 7 A schematic block diagram of an electronic device provided by some embodiments of the present disclosure is shown;

[0024] Figure 8 A schematic block diagram of another electronic device provided by some embodiments of the present disclosure is shown; and

[0025] Figure 9 A schematic diagram of a computer readable storage medium of the embodiments of the present disclosure is shown schematically. DETAILED DESCRIPTION

[0026] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the protection scope of the present disclosure.

[0027] Unless otherwise defined, technical terms or scientific terms used in the present disclosure shall have the ordinary meaning of the terms to a person of ordinary skill in the art to which the present disclosure belongs. The terms “first”, “second” and similar terms used in the present disclosure do not denote any order, quantity or importance, but are used to distinguish different components. Similarly, the terms “one”, “a” or “the” and similar terms do not denote quantity limitation, but denote that there is at least one. The terms “include” or “contain” and similar terms mean that the components or objects before the terms encompass the components or objects listed after the terms and their equivalents, and do not exclude other components or objects. The terms “connect” or “connected” and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “up”, “down”, “left”, “right” and the like are only used to indicate relative positional relationships, and when the absolute positions of the described objects are changed, the relative positional relationships may also be changed accordingly.

[0028] In a large-scale parallel computing environment and a distributed training scenario, data aggregation and reduction operations are common requirements. For example, SHARP (Smart Hierarchical Aggregation and Reduction Protocol) is a protocol designed to optimize data aggregation and reduction operations in a large-scale parallel computing system.

[0029] Figure 1 An architectural schematic diagram of a SHARP protocol is shown.

[0030] As Figure 1As shown, the SHARP protocol includes a system software layer 101, switches 102, and compute nodes 103. The switches 102 are responsible for routing and forwarding data, containing a SHARP engine for performing aggregation and reduction operations at the network level. The system software layer 101 is responsible for configuring and managing the entire system, including setting the working mode, parameters, etc. of the SHARP engine. Under the SHARP architecture, each compute node 103 participating in a SHARP task generates raw computing data, which is sent to a specific switch 102 in the network. Next, the received data is aggregated or reduced by the switch according to the configuration, resulting in aggregated data, which can involve multiple stages and multiple layers of switch structure. Finally, the final result, i.e., the aggregated data, is returned to the compute node 103 according to the configuration.

[0031] As the complexity of the SHARP protocol application system increases, the computing stages and switch structures included in the design under test (e.g., a large-scale parallel computing system using the SHARP protocol) also gradually become large. In order to ensure the correctness and reliability of the SHARP technology application system, the design under test needs to be strictly and sufficiently verified. Currently, the verification method often relies on manually written test cases, which is very time-consuming and laborious to cover perfect and random test scenarios.

[0032] Figure 2 A schematic diagram of a current verification method architecture is shown.

[0033] As Figure 2 shown, the verification method architecture includes a design under test DUT and a verification environment TB. The design under test DUT represents the design system to be tested by the verification environment TB. The verification environment TB, for example, includes a testbench for verifying the environment of a digital design, which can include all necessary components to generate test stimuli and drive the test stimuli to the design under test.

[0034] When the design under test is a multi-die structure, including multiple different hardware to be tested (Design Under Test, DUT), in order to achieve comprehensive and sufficient testing of the design under test, the verification environment needs to have complete test stimuli and test scenarios.

[0035] In order to test the specific functions or characteristics of the design under test, ensure that certain critical paths or boundary conditions are covered, the verification needs to test the predefined behavior through directed test stimuli. As Figure 2As shown, when the number of directional test scenarios is large and does not intersect, it is usually necessary to construct a corresponding number of directional test stimuli. For example, n directional test parameter classes (directional test parameter class 1, …, directional test parameter class n) are used to construct n directional test stimuli (directional test stimulus a1, …, directional test stimulus an), and the n directional test stimuli and the n directional test parameter classes correspond one-to-one. Each directional test stimulus represents a directional test scenario.

[0036] In order to more comprehensively verify the design under test and find invisible design functional defects, in addition to directional test cases, random tests containing a large number of random test cases can effectively explore the boundary conditions of the design. As shown in Figure 2 As shown, under effective and reasonable constraints, the scale of random test scenarios is large, and a large number of random test cases are needed to correspond, which usually need to be manually crossed and randomly constructed, and there is a risk of omission. In random testing, various test stimuli are generated to try to cover various states and paths of the design. Here, "scenario" refers to different use cases or operation modes, and "cross" means the combination or interaction between these different scenarios. "Scenario cross random" means that when generating test cases randomly, not only a specific function or scenario is considered alone, but multiple functions or operation modes are combined together for testing. For example, n random test parameter classes (random test parameter class 1', …, random test parameter class n') are used to construct n random test stimuli (random test stimulus b1, …, random test stimulus bn), and the n random test stimuli and the n random test parameter classes correspond one-to-one. Each random test parameter class represents a random test scenario.

[0037] Figure 2 The direct case generation method shown needs to manually establish different test stimuli to meet the needs of different test scenarios; when the verification test scenario needs to cover different forms of the design under test, this method needs to increase more logic units and configuration modules, especially when a single die is expanded to multiple dies, new structures are needed to support complex scenarios.

[0038] Therefore, the current test case generation method needs to develop different test stimuli and add different test cases when it needs to cover different random test scenarios; when the verification test needs to cross different scenarios, this method is inefficient and not flexible; and when random test scenarios and directional test scenarios are compatible, code redundancy is easy to appear, resulting in low efficiency of test case development; when the design under test is adjusted or optimized, the modification cost is high, which is not conducive to the development of reusability and scalability; manual construction of random test scenarios has poor completeness and is easy to miss cross scenarios, which is not conducive to the effect of large-scale regression testing, resulting in increased verification cost.

[0039] Some embodiments of the present disclosure provide a test case generation method. The test case generation method comprises: obtaining a unified structure model of a design under test, the design under test comprising at least one to-be-configured parameter; obtaining parameter configuration information of the unified structure model; and generating a test case based on the parameter configuration information and the unified structure model, so as to test the design under test by using the test case. The test case generation method can generate corresponding test cases by adjusting the parameter configuration information when creating different test cases by calling the unified structure model, thereby alleviating the code redundancy problem of the test case and improving the development efficiency of testing and verification.

[0040] Figure 3 A flowchart of a test case generation method provided by at least one embodiment of the present disclosure is shown.

[0041] As shown in Figure 3 The test case generation method comprises steps S301-S303.

[0042] Step S301: Obtain a unified structure model of a design under test, the unified structure model comprising at least one to-be-configured parameter.

[0043] Step S302: Obtain parameter configuration information of the at least one to-be-configured parameter.

[0044] Step S303: Generate a test case based on the parameter configuration information and the unified structure model, so as to test the design under test by using the test case.

[0045] The test case generation method can automatically generate test cases by constructing a unified structure model in combination with parameter configuration information. When different random test scenarios need to be covered, only the parameter configuration information needs to be changed, and the test case generation method can be compatible with random scenarios and directional scenarios, has high flexibility, and has high test case development efficiency.

[0046] For step S301, the design under test is, for example, a hardware system that needs to be verified, for example, a chip. The unified structure model can describe the structure of the design under test and the functions of the components included in the design under test, for example, a digital representation or a language description of the design under test. The design under test can comprise multiple hardware parts or functional modules. When testing, only part of the functional modules or hardware parts can be tested, or the entire chip can be tested. In embodiments of the present disclosure, the hardware or functional module to be tested is referred to as a to-be-tested hardware. That is, the to-be-tested hardware can be a complete chip, or a functional module of the chip. If the design under test comprises multiple to-be-tested hardware, the cooperation between the multiple to-be-tested hardware can be functionally tested, or the functions of the multiple to-be-tested hardware can be functionally tested. For example, in a chip, there are multiple to-be-tested hardware, and the cooperation between the multiple to-be-tested hardware needs to be tested. In this case, the unified structure model can describe the cooperation between the multiple to-be-tested hardware. Figure 1In the illustrated example, the hardware to be tested can be one switch 102, two switches 102, or the entire set of switches 102 in the system.

[0047] For example, a unified structure model is obtained by numerically modeling the design under test. The unified structure model can be pre-established as part of a verification environment for verifying the design under test, and can be directly invoked when generating test cases.

[0048] In some embodiments of the present disclosure, step S301 includes obtaining attribute information of a plurality of test nodes included in the design under test; and generating the unified structure model based on the attribute information of each of the plurality of test nodes.

[0049] In some embodiments of the present disclosure, the design under test can be a hierarchical design obtained by using a hierarchical structure, for example, a tree structure. In the hierarchical design of a chip, a node is a basic unit constituting the hierarchical structure of the entire design, and nodes at different levels have different functions and roles. A node can be understood as an entity in the hierarchical structure of the chip design, which can represent a module, a sub-module, or a basic circuit unit. There is a hierarchical relationship between nodes, with parent nodes and child nodes, which together constitute a tree structure of the chip design.

[0050] The attribute information of the test node can include, for example, structural attributes and functional attributes. The structural attributes can include, for example, the hierarchical position of the node in the tree structure, the parent-child node relationship, the connection mode, and the like. The functional attributes can include, for example, the operation mode (e.g., sending data or receiving data) and the functional description (e.g., reading, writing, addition operation, and the like), and the like. The attribute information of the test node can include, for example, information such as whether the test node has child nodes, whether it participates in a task, what type of task it participates in, whether the test node has a parent node, and which node is the parent node, and the like.

[0051] In some embodiments of the present disclosure, the attribute information can be obtained through a callable function provided by the verification environment. For example, the number of test nodes can be obtained through the function get_all_sa_num(), and the number of dies can be obtained through the function get_die_num(), each die representing a hardware to be tested.

[0052] In some embodiments of the present disclosure, the Testbench described above can be established based on a Universal Verification Methodology (UVM). The UVM is a verification environment development framework based on a System Verilog class library, and verification engineers can use reusable components thereof to build a functional verification environment with a standardized hierarchy and interface. The UVM provides a set of standardized methods and class libraries. Structural information can be obtained by calling the methods or class libraries provided by the UVM. For example, the parent node of the test node can be obtained by calling get_parent() provided by the UVM.

[0053] After obtaining the structural information, a unified structural model of the design under test can be constructed according to the structural information. The structure described by the unified structural model is consistent with the hardware structure of the design under test.

[0054] In some embodiments of the present disclosure, the unified structural model is generated based on the attribute information of each of the plurality of test nodes, including: performing digital modeling on the design under test based on the attribute information of each of the plurality of test nodes to obtain the unified structural model.

[0055] In some embodiments of the present disclosure, the digital modeling refers to, for example, quantifying the attribute information into variables or setting values. For example, the attribute information includes non-configurable attributes and configurable attributes. A setting value is set for the non-configurable attributes, and an attribute variable is set for the configurable attributes, and the attribute is determined by assigning a value to the attribute variable. For example, the parent node of the test node is determined by the design under test itself, and belongs to the non-configurable attributes, so the parent node of the test node can be quantified into a certain setting value (e.g., the encoding of the parent node). For another example, whether the test node participates in the task can be changed according to the test requirements, and belongs to the configurable attributes, so the attribute of whether the test node participates in the task can be quantified into a variable x, and the value of x can be determined according to the parameter configuration information obtained in step S302. For example, if the parameter configuration information indicates that the test node participates in the task, the variable x is assigned a value of 1 when generating the test case; if the parameter configuration information indicates that the test node does not participate in the task, the variable x is assigned a value of 0 when generating the test case.

[0056] The unified structural model includes at least one to-be-configured parameter. The to-be-configured parameter is, for example, a variable that can be configured. For example, the attribute information quantified into a variable described above can be used as a to-be-configured parameter in the unified structural model.

[0057] For step S302, the parameter configuration information can be information input by a user through a parameter configuration interface provided by the verification environment. For another example, the parameter configuration information can also be information obtained by the verification environment from other devices through interaction with the other devices.

[0058] In some embodiments of the present disclosure, for example, the necessary to-be-configured parameter configuration in the unified structure model is set to a value according to the parameter configuration information, by setting the necessary to-be-configured parameter configuration, the test scenario can be customized, and the test case of the directional test scenario is automatically generated. For example, the design under test includes die 1, die 2 and die 3, if the number of to-be-tested hardware participating in the test this time is set to 2 in the parameter configuration information, then 2 can be selected from die 1, die 2 and die 3 as to-be-tested hardware to generate a test case. For another example, if the number of die participating in the test this time is indicated in the parameter configuration information, then the die corresponding to the number is used as to-be-tested hardware. The parameter configuration information can also indicate the number or number of test nodes participating in the test this time, etc.

[0059] In some embodiments of the present disclosure, the parameter configuration information can configure at least part of the at least one to-be-configured parameter as a random mode. For example, the design under test includes die 1, die 2 and die 3, if the number of to-be-tested hardware participating in the test this time is set to random in the parameter configuration information, then the verification environment can select any one or more of die 1, die 2 and die 3 to generate a test case. Similarly, the parameter configuration information can also set the number or number of test nodes participating in the test this time to be random, etc.

[0060] For step S303, for example, the parameter configuration information is configured to the at least one to-be-configured parameter, so as to generate a test case.

[0061] In some embodiments of the present disclosure, step S303 includes inputting the parameter configuration information to the unified structure model to configure the at least one to-be-configured parameter to obtain a system morphology model; and generating a test case based on the system morphology model.

[0062] For example, according to the parameter configuration information, the at least one to-be-configured parameter in the unified structure model is set to a value or set to random to obtain a system morphology model.

[0063] In some embodiments of the present disclosure, the system morphology model can be a random morphology model or a directional morphology model. The random morphology model is used to implement a random test scenario, and the directional morphology model is used to implement a directional test scenario. If the parameter configuration information configures each to-be-configured parameter in the unified structure model to be random, then a random morphology model is obtained, which is applied to a random test scenario; if the parameter configuration information sets at least one to-be-configured parameter in the unified structure model to a value, then a directional morphology model is obtained, which is applied to a directional test scenario.

[0064] In some embodiments of the present disclosure, the parameter configuration information is input into the unified structure model to configure the at least one to-be-configured parameter to obtain the system configuration model, including: in response to the parameter configuration information indicating that the value of the at least one to-be-configured parameter is random, randomly setting the value of the at least one to-be-configured parameter.

[0065] For example, if the parameter configuration information indicates that the value of the at least one to-be-configured parameter is random, the value of the at least one to-be-configured parameter is randomly set by the verification environment to randomly test the design under test. For example, the design under test includes die 1, die 2 and die 3, if the number of hardware to be tested in the parameter configuration information is set to be random, the verification environment can select any one or more of die 1, die 2 and die 3 to generate a test case. Similarly, the parameter configuration information can also set the number or the number of test nodes participating in the test to be random, etc.

[0066] In some embodiments of the present disclosure, the parameter configuration information is input into the unified structure model to configure the at least one to-be-configured parameter to obtain the system configuration model, including: in response to the parameter configuration information including at least part of the set value of the at least one to-be-configured parameter, configuring the set value to the unified structure model to obtain the system configuration model.

[0067] For example, the parameter configuration information configures the necessary to-be-configured parameters to be set values, and the set values are configured to the unified structure model.

[0068] The parameter configuration information can include some set values and a random identifier (for example, random), so that some to-be-configured parameters of the at least one to-be-configured parameter are configured to be the set values, and the other to-be-configured parameters are randomly set. The parameter configuration information can also be configured to be set values for each of the at least one to-be-configured parameter, or indicate that each of the to-be-configured parameters is set to be random. The parameter configuration information is set by the user according to the test requirement, and the embodiments of the present disclosure do not limit the parameter configuration information. The user can configure the at least one to-be-configured parameter according to the requirement. In addition, the at least one to-be-configured parameter can also be set according to the actual requirement.

[0069] In some embodiments of the present disclosure, the design under test includes a plurality of hardware to be tested, and the parameter configuration information includes the number or the number of test nodes participating in the test of the hardware to be tested participating in the test.

[0070] For example, the design under test includes die 1, die 2 and die 3, which are examples of a plurality of hardware to be tested. If at least one configuration parameter includes the code of the hardware to be tested participating in the current test, and the parameter configuration information indicates that the number of hardware to be tested is 2, then the code of 2 selected from die 1, die 2 and die 3 is configured into the unified structure model, for example, the code of the hardware to be tested participating in the current test is configured as 1 and 2, indicating that die 1 and die 2 participate in the current test. After the configuration of at least one configuration parameter is completed, the system form model is obtained. Similarly, the parameter configuration information can also include the number of test nodes in each hardware to be tested participating in the current test, for example, the parameter indicating the number of test nodes in the parameter configuration information is 10, then the number of 10 nodes selected from the hardware to be tested can be configured into the unified structure model to obtain the system form model.

[0071] In some embodiments of the present disclosure, the design under test includes a plurality of test nodes, and based on the system form model, a test stimulus is generated, including: determining the task information of each of the plurality of test nodes based on the system form model; configuring the task information of each test node to the task register corresponding to the test node; and generating a test case based on the task register and the verification environment configured for the design under test.

[0072] For example, each test node in the system form model is traversed, and the task information of each test node is extracted, and the task register corresponding to the test node is configured according to the task information.

[0073] In some embodiments of the present disclosure, after the task register is configured, the verification environment automatically executes the SystemVerilog code to generate a test case. In some embodiments of the present disclosure, the verification environment can be based on the sequence / sequencer mechanism, that is, the verification environment generates a test case based on the sequence / sequencer mechanism. For example, the verification environment first defines one or more sequences, each sequence including a series of tasks and corresponding constraints. For example, a sequencer instance is created in the verification environment, and then one or more sequences can be started in the test case and mounted on the sequencer. After the sequencer receives the transaction request of the sequence, the requests are sorted and resources are allocated according to the scheduling algorithm, and then the transaction is sent to the driver in the verification environment. The driver converts the transaction into actual signal driving the hardware to be tested.

[0074] In some embodiments of the present disclosure, configuring the task information of each test node to the task register corresponding to the test node comprises: creating data objects for each test node to obtain a plurality of data objects, wherein the value of a variable in each of the plurality of data objects is the task information of each test node; and assigning values to the task register based on the task information.

[0075] For example, after obtaining the system mode model, each test node of the system mode model is traversed, and an independent item (i.e., data object) is created for each stage, and the task information contained in the test node is quantified as a variable inside the item. For example, the task register is assigned values according to a plurality of items.

[0076] In some embodiments of the present disclosure, configuring the task information of each test node to the task register corresponding to the test node further comprises: encapsulating the plurality of data objects into a wrapper; and extracting the task information of each test node from the wrapper.

[0077] For example, all items are automatically integrated into a wrapper (i.e., a wrapper), and then the system mode model can be converted into a wrapper structure based on the System Verilog language and stored. After that, the obtained wrapper is accessed and traversed, and the task information of each item node is extracted and converted into the value of the corresponding register; then, the task register of the matching node is automatically found and configured. By encapsulating the item in the wrapper, new functions or properties can be easily added to the item without modifying the original item code. For example, additional signals, variables, or methods can be added to the wrapper to meet specific verification requirements. In this way, the maintainability and scalability of the code can be improved, direct modification of the original item can be avoided, the risk of introducing errors can be reduced, and the reusability of the code can be improved, the repeated writing of the code can be reduced, and the verification efficiency can be improved.

[0078] Figure 4 An architecture diagram of a system for generating test cases provided by at least one embodiment of the present disclosure is shown.

[0079] As Figure 4As shown, the system includes a verification environment 401 and a design under test (DUT), which may include, for example, three hardware components to be tested: DUT A, DUT B, and DUT C. The verification environment 401 includes a stimulus generator 411 and a transaction generation unit 421. The verification environment 401 may include, for example, the Testbench described above. The stimulus generator 411 is configured to generate test stimuli, including directed test stimuli and random test stimuli. The transaction generation unit 421, for example, manages the test stimuli and generates test cases based on a sequence / sequencer mechanism, and drives the test cases to the hardware under test.

[0080] Targeted test stimuli are used to test specific functions or characteristics of the hardware under test, ensuring that certain critical paths or boundary conditions are covered, and verifying that predefined behaviors need to be tested through targeted test stimuli. Figure 4 In this system, based on the stimulus generator 411, only the configuration parameters for the targeted test scenario need to be input, and the parameters can be automatically and accurately parsed to generate the corresponding test stimuli. Random test stimuli are used to more comprehensively verify the hardware design under test and discover invisible design defects. In addition to targeted test cases, random tests containing a large number of random test cases can effectively explore the boundary conditions of the design. For example... Figure 4 As shown, based on the excitation generator 411, the design under test is pre-modeled and quantized. By controlling the random parameters, theoretically, the more times the random test scenario is tested, the more comprehensive the convergence will be.

[0081] like Figure 4 As shown, the stimulus generator 411 includes a parameter preprocessing unit 4111, a unified stochastic model unit 4112, a model quantization unit 4113, and a task configuration unit 4114.

[0082] The parameter preprocessing unit 4111 provides a parameter configuration interface. After receiving external parameters (e.g., directional or random parameter configuration), it combines the verification environment 401 to preprocess the external input parameters and internal parameters, parsing out the required parameters for the current verification scenario. Internal parameters refer to the parameters of the design under test itself, such as the number of hardware components and nodes in the device under test.

[0083] The unified random model unit 4112 is used to digitally model the design system under test to obtain a unified structural model. For example, each test node of the unified structural model is digitally quantized, and the randomness of the design under test is realized through digital randomness.

[0084] The model quantization unit 4113 generates a customized or randomized system architecture model after calling the unified structure model, quantizes parameters of each test node of the system architecture model, and integrates the system architecture model into a specific data structure, so as to quantitatively simulate the architecture of the system under test. The model quantization unit 4113 enables the unified structure model to be compatible with external customization at the same time, generates a fixed required architecture, and the model can be called multiple times and automatically integrated into a larger random model.

[0085] The task configuration unit 4114 converts the model parameters into task register parameters and configures the task register parameters into the hardware to be tested.

[0086] Figure 5 A flowchart of another test case generation method provided by at least one embodiment of the present disclosure is shown.

[0087] As shown in Figure 5 , the method includes steps S501-S509.

[0088] Step S501: Obtain parameter configuration information of a test case. The step S501 is similar to the step S302 in Figure 3 .

[0089] Step S502: Automatically identify the structure of a design under test, and complete parameter preprocessing to obtain requirement parameters. The steps S501 and S502 are similar to the step S302 in Figure 3 .

[0090] For example, the stimulus generator 411 obtains the parameter configuration information, automatically identifies the structure of the design under test, and analyzes and preprocesses the parameter configuration information and internal parameters of the design under test. For example, if the design under test is a multi-die structure including multiple DUTs (DUT A, DUT B, and DUT C), the internal parameters include the number of multi-dies (for example, 3). The stimulus generator 411 can automatically realize rationalization of the parameters according to the parameter configuration information and the internal parameters. For example, if the parameter configuration information indicates that the hardware to be tested includes 2, then 2 are automatically selected from the 3 dies as requirement parameters of a current verification scenario. For another example, if the verification environment is a single-die structure including only the DUT A, then there can be more than one structure that meets the parameter configuration information, and automatic selection and customization are also important parts of the generator. For example, the parameter configuration information indicates that 10 test nodes participate in the test, and then the parameter preprocessing unit of the stimulus generator can select 10 test nodes from multiple test nodes to participate in the test.

[0091] Step S503: Based on digital modeling of the design under test, quantize attribute information of the test nodes, and construct a unified structure model. The step S501 is similar to the step S301 in Figure 3 .

[0092] Step S504: calling the unified structure model to obtain the system configuration model in combination with the requirement parameters. The requirement parameters are taken as the input parameters of the unified structure model, and the unified structure model is called to obtain the required system configuration model.

[0093] Step S505: traversing each test node in the system configuration model, creating an independent item, and quantifying the attribute information of the test node into an internal variable of the item and storing. For example, after obtaining the system configuration model, each node of the model is traversed, an independent item is created for each node, and the task information contained in the node is quantified into an internal variable of the item.

[0094] Step S506: automatically integrating the item of the test node into the wrapper, quantifying the system configuration model into the wrapper structure and storing.

[0095] Step S507: traversing all the items stored in the wrapper, extracting the test node and task information, converting the register values, and automatically configuring the task register of the test node.

[0096] Step S508: creating a test case using the sequence / sequencer mechanism.

[0097] Step S509: starting the test case to perform the test task.

[0098] Steps S504-S509 are similar to steps S303 in the method. Figure 3

[0099] The method can automatically generate random test cases and customized test cases through the parameterized configuration mechanism, build a standardized and perfect verification platform, and realize efficient verification of the completeness and randomness of the SHARP system.

[0100] It should be noted that, although the test case generation method is applied to the SHARP system as an example for illustration, this does not limit the present disclosure, and the embodiments of the present disclosure are applicable to the verification of any isomorphic and morphologically variable chip. The embodiments of the present disclosure are also applicable to gate-level simulation or post-synthesis simulation.

[0101] At least some embodiments of the present disclosure also provide a test method, including: obtaining a test case, the test case being generated according to the test case generation method provided by any embodiment of the present disclosure; and executing the test case by using a verification environment.

[0102] ​In some embodiments of the present disclosure, the simulation run of the verification environment can be performed by using EDA tools such as Verilog Compiler Simulator (VCS), NC-Verilog, irun, etc. The method for generating test cases is described above.

[0103] In some embodiments of the present disclosure, a set of independent stimulus generators is developed based on System Verilog, which supports morphological variability under different structures of single die and multi-die, automatically generates corresponding configuration verification environment, and greatly improves the completeness of chip verification. In embodiments of the present disclosure, a unified standardized calling mode is used, and only part of the parameters need to be adjusted when creating different test cases, which solves the code redundancy problem when there are many test cases, and improves the development efficiency of testing and verification. Using intuitive customization strategy, intuitive parameter configuration mechanism is used for directional test scene, without the need to increase additional parameter interface, directional scene test is completed; using general random strategy, even in multi-die environment, additional modules do not need to be added, which increases the flexibility and expansibility of the verification strategy, and also increases the convenience of large-scale simulation of chip verification regression. In addition, the stimulus generator developed using System Verilog has good configurability, which can flexibly adjust the configuration mechanism or modify the parameters that need to be configured according to the needs of the verification scene; the stimulus generator developed using System Verilog has good scalability, which can be flexibly extended to the chip verification based on the UVM verification platform that needs independent verification strategy according to the verification needs.

[0104] Figure 6 A schematic diagram of a test case generator 600 provided by at least one embodiment of the present disclosure is shown.

[0105] As shown in Figure 6 , the test case generator 600 includes a unified random model unit 610, a parameter preprocessing unit 620, and a test case generation unit 630.

[0106] The unified random model unit 610 is configured to obtain a unified structure model of a design under test, the design under test including hardware to be tested, and the unified structure model including at least one configuration parameter. For example, the unified random model unit 610 performs step S301 in Figure 3 .

[0107] The parameter preprocessing unit 620 is configured to obtain parameter configuration information of the at least one configuration parameter. The parameter preprocessing unit 620, for example, performs step S302 in Figure 3 .

[0108] The test case generation unit 630 is configured to generate the test case based on the parameter configuration information and the unified structure model, so as to test the hardware to be tested by using the test case. For example, the test case generation unit 630 performs the following steps. Figure 3 In step S303.

[0109] For example, the unified random model unit 610, the parameter preprocessing unit 620 and the test case generation unit 630 can be hardware, software, firmware and any feasible combination thereof. For example, the unified random model unit 610, the parameter preprocessing unit 620 and the test case generation unit 630 can be a special-purpose or general-purpose circuit, chip or device, or a combination of a processor and a memory. The embodiments of the present disclosure do not limit the specific implementation forms of the above-mentioned units.

[0110] It should be noted that in the embodiments of the present disclosure, the units of the test case generator 600 correspond to the steps of the test case generation method described above, and the specific functions of the test case generator 600 can be referred to the related description of the test case generation method, which will not be repeated here. Figure 6 The components and structures of the test case generator 600 shown are only exemplary and are not limiting, and the test case generator 600 can also include other components and structures as needed.

[0111] At least one embodiment of the present disclosure also provides an electronic device including a processor and a memory including one or more computer program modules. The one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the above. The electronic device can improve the efficiency of test case generation and improve the completeness and efficiency of chip verification.

[0112] Figure 7 An electronic device provided by some embodiments of the present disclosure is shown in a schematic block diagram. As Figure 7 The electronic device 700 includes a processor 710 and a memory 720. The memory 720 is used to store non-transitory computer readable instructions (for example, one or more computer program modules). The processor 710 is used to run the non-transitory computer readable instructions, and when the non-transitory computer readable instructions are run by the processor 710, one or more steps of the test case generation method described above can be performed. The memory 720 and the processor 710 can be interconnected by a bus system and / or other forms of connection mechanism (not shown).

[0113] For example, the processor 710 can be a central processing unit (CPU), a graphics processing unit (GPU), or other form of processing unit having data processing and / or program executing capabilities. For example, the central processing unit (CPU) can be of X86 or ARM architecture, etc. The processor 710 can be a general purpose processor or a special purpose processor, and can control other components in the electronic device 700 to perform desired functions.

[0114] For example, the memory 720 can include any combination of one or more computer program products. The computer program product can include various forms of computer-readable storage media, for example, volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), compact disc read-only memory (CD-ROM), USB memory, flash memory, and / or the like. One or more computer program modules can be stored on the computer-readable storage media, and the processor 710 can run the one or more computer program modules to implement various functions of the electronic device 700. Various application programs and various data used and / or generated by the application programs, etc. can also be stored in the computer-readable storage media.

[0115] It should be noted that, in the embodiments of the present disclosure, the specific functions and technical effects of the electronic device 700 can refer to the description of the test case generation method in the foregoing description, and will not be described herein.

[0116] Figure 8 Another schematic block diagram of an electronic device is provided for some embodiments of the present disclosure. The electronic device 800 is suitable for implementing the test case generation method provided by the embodiments of the present disclosure, for example. The electronic device 800 can be a terminal device, etc. It should be noted that, Figure 8 The electronic device 800 shown is merely an example, which does not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0117] As Figure 8 shown, the electronic device 800 can include a processing device (such as a central processing unit, a graphics processor, etc.) 810, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 820 or loaded from a storage device 880 to a random access memory (RAM) 830. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 830. The processing device 810, the ROM 820, and the RAM 830 are connected to each other through a bus 840. An input / output (I / O) interface 850 is also connected to the bus 840.

[0118] Typically, the following devices can be connected to I / O interface 850: input devices 860 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 870 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 880 including, for example, magnetic tapes, hard disks, etc.; and communication devices 890. Communication device 890 allows electronic device 800 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 8 An electronic device 800 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 800 may alternatively implement or have more or fewer devices.

[0119] For example, according to embodiments of this disclosure, the test case generation method described above can be implemented as a computer software program. For instance, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program including program code for executing the test case generation method described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 890, or installed from a storage device 880, or installed from a ROM 820. When the computer program is executed by the processing device 810, it can implement the functions defined in the test case generation method provided by embodiments of this disclosure.

[0120] At least some embodiments of this disclosure also provide a non-transitory storage medium. Figure 9 The illustration schematically depicts a computer-readable storage medium provided in an embodiment of the present disclosure. For example, such as Figure 9 As shown, the storage medium 900 stores non-transitory computer-readable instructions 901, which, when executed by a computer (including a processor), can perform the methods provided in any embodiment of this disclosure. This non-transitory storage medium can improve the efficiency of test case generation and enhance the completeness and efficiency of chip verification.

[0121] For example, one or more computer instructions may be stored on the storage medium 900. Some of the computer instructions stored on the storage medium 900 may be, for example, instructions for implementing one or more steps in the methods described above.

[0122] For example, the storage medium may include the storage component of a tablet computer, the hard disk of a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), optical disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0123] The technical effects of the storage medium provided by the embodiments of the present disclosure can be referred to the corresponding description of the method in the above embodiments, which will not be repeated here.

[0124] For the present disclosure, the following points need to be explained:

[0125] (1) In the drawings of the embodiments of the present disclosure, only the structures related to the embodiments of the present disclosure are involved, and other structures can be referred to the general design.

[0126] (2) In the case of no conflict, the features in the same and different embodiments of the present disclosure can be combined with each other.

[0127] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present disclosure, which should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A test case generation method, comprising: Obtain a unified structural model of the design under test, wherein the unified structural model includes at least one parameter to be configured; Obtain the parameter configuration information of the at least one parameter to be configured; and Based on the parameter configuration information and the unified structure model, the test cases are generated to test the design under test.

2. The method according to claim 1, wherein, Based on the parameter configuration information and the unified structure model, the test cases are generated, including: The parameter configuration information is input into the unified structural model to configure the at least one parameter to be configured, thereby obtaining a system morphological model; and The test cases are generated based on the system morphology model.

3. The method according to claim 2, wherein, The design under test includes multiple test nodes. Based on the system morphology model, the test cases are generated, including: Based on the system morphology model, the task information of each of the multiple test nodes is determined; Configure the task information of each test node into the task register corresponding to that test node; and The test cases are generated based on the task register and the verification environment configured for the design under test.

4. The method according to claim 3, wherein, Configure the task information of each test node into the task register corresponding to the test node, including: Multiple data objects are created for each test node, where the values ​​of variables in each data object represent the task information of each test node; and Based on the task information, assign a value to the task register.

5. The method according to claim 4, wherein, Configuring the task information of each test node into the task register corresponding to the test node also includes: Encapsulate the plurality of data objects into a wrapper; and Extract the task information for each test node from the wrapper.

6. The method according to claim 2, wherein, Inputting the parameter configuration information into the unified structural model to configure the at least one parameter to be configured to obtain the system morphological model includes: In response to the parameter configuration information indicating that the value of the at least one configurable parameter is random, the value of the at least one configurable parameter is randomly set; or In response to the parameter configuration information including at least a portion of the set values ​​of the at least one parameter to be configured, the set values ​​are configured into the unified structural model to obtain the system morphology model.

7. The method according to claim 6, wherein, The design under test includes multiple hardware components to be tested, and the parameter configuration information includes the number or serial number of the hardware components participating in this test; or The number or number of test nodes in the hardware under test participating in this test.

8. The method according to claim 1, wherein, Obtain a unified structural model of the design under test, including: Obtain attribute information of multiple test nodes included in the design under test; and The unified structural model is generated based on the attribute information of each of the multiple test nodes.

9. The method according to claim 8, wherein, Based on the attribute information of each of the multiple test nodes, the unified structural model is generated, including: Based on the attribute information of each of the multiple test nodes, the design under test is digitally modeled to obtain the unified structural model.

10. A testing method, comprising: Obtain test cases, wherein the test cases are generated by the test case generation method according to any one of claims 1-9; and Execute the test cases using the verification environment.

11. A test case generator, comprising: A unified stochastic model unit is configured to acquire a unified structural model of the design under test, wherein the unified structural model includes at least one parameter to be configured. The parameter preprocessing unit is configured to obtain parameter configuration information for the at least one parameter to be configured; and The test case generation unit is configured to generate test cases based on the parameter configuration information and the unified structure model, so as to use the test cases to test the design under test.

12. An electronic device, comprising: processor; as well as Memory, which includes one or more computer program instructions; The one or more computer program instructions are executed by the processor according to any one of claims 1 to 10.

13. A computer-readable storage medium for non-transitory storage of computer-readable instructions, wherein, The method of any one of claims 1 to 10 is implemented when the computer-readable instructions are executed by a processor.

Images