Verification method and device, electronic equipment, storage medium, program product and chip

Abstract

The invention provides a verification method and device, electronic equipment, a storage medium, a program product and a chip, and relates to the technical field of chip verification. Comprising the steps of obtaining an initial use case set and a first check item set corresponding to a tested object; determining a second check item set corresponding to the initial use case set; determining the inspection items which belong to the first inspection item set and do not belong to the second inspection item set as target inspection items; based on the target check item, updating the initial case set to obtain a target case set; and based on the target use case set, verifying the tested object. Therefore, before the use case is utilized to verify the tested object, the target check item which is not covered by the current initial use case set is determined, then the initial use case set is updated based on the target check item so as to improve the coverage rate of the target use case set, and then the target use case set is utilized to verify the tested object, so that the test efficiency is improved. The verification efficiency of the tested object can be improved while the verification coverage rate is ensured.

Description

Technical Field

[0001] The present disclosure relates to the field of chip verification technology, and in particular to a verification method, device, electronic device, storage medium, program product and chip. Background Art

[0002] Chip verification refers to the process of verifying the chip design to ensure that the chip design meets the required specifications, releases risks, and discovers and corrects all defects.

[0003] In the related art, after the use cases corresponding to the test points are generated, the use cases are directly used to drive the chip and the reference model to obtain the operating results output by the chip and the reference results output by the reference model, and then the functional coverage is calculated based on the same operating results and reference results. If the functional coverage does not meet the requirements, the use cases are adjusted, and the adjusted use cases are used to drive the chip and the reference model again until the functional coverage meets the requirements and the chip verification is completed. The verification efficiency of this method is low.

[0004] Therefore, how to improve chip verification efficiency while ensuring verification coverage has become a key research direction. Summary of the invention

[0005] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0006] The first embodiment of the present disclosure provides a verification method, including:

[0007] Obtaining an initial use case set and a first check item set corresponding to the object under test;

[0008] Determine a second check item set corresponding to the initial use case set;

[0009] Determine the inspection items that belong to the first inspection item set and do not belong to the second inspection item set as target inspection items;

[0010] Based on the target inspection items, the initial use case set is updated to obtain a target use case set;

[0011] Based on the target use case set, the object under test is verified.

[0012] The second aspect of the present disclosure provides a verification device, including:

[0013] An acquisition module, used to acquire an initial use case set and a first inspection item set corresponding to the object under test;

[0014] A first determination module, used to determine a second check item set corresponding to the initial use case set;

[0015] A second determination module, configured to determine an inspection item that belongs to the first inspection item set and does not belong to the second inspection item set as a target inspection item;

[0016] An updating module, configured to update the initial use case set based on the target inspection item to obtain a target use case set;

[0017] A verification module is used to verify the object under test based on the target use case set.

[0018] The third aspect of the present disclosure provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the verification method provided in the first aspect of the present disclosure is implemented.

[0019] The fourth aspect embodiment of the present disclosure proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the verification method proposed in the first aspect embodiment of the present disclosure is implemented.

[0020] The fifth aspect of the present disclosure provides a computer program product, including a computer program, which implements the verification method proposed in the first aspect of the present disclosure when the computer program is executed by a processor.

[0021] The sixth embodiment of the present disclosure provides a chip, which is obtained after verification by the verification method proposed in the first embodiment of the present disclosure.

[0022] The verification method, device, electronic device, storage medium, program product and chip provided by the present disclosure have the following beneficial effects:

[0023] In the disclosed embodiment, the initial use case set and the first inspection item set corresponding to the object under test are first obtained, and the second inspection item set corresponding to the initial use case set is determined, and then the inspection items that belong to the first inspection item set and do not belong to the second inspection item set are determined as target inspection items, and then based on the target inspection items, the initial use case set is updated to obtain the target use case set, and finally the object under test is verified based on the target use case set. Therefore, before using the use case to verify the object under test, the target inspection items that are not covered by the current initial use case set are first determined, and then based on the target inspection items, the initial use case set is updated to improve the coverage of the target use case set, and then the target use case set is used to verify the object under test, which can improve the verification efficiency of the object under test while ensuring the verification coverage.

[0024] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0026] Figure 1 A flowchart of a verification method is shown;

[0027] Figure 2 A flowchart of a verification method provided by an embodiment of the present disclosure;

[0028] Figure 3 A flowchart of a verification method provided by another embodiment of the present disclosure;

[0029] Figure 4 A flow chart of a verification method provided by another embodiment of the present disclosure

[0030] Figure 5 A schematic diagram of the structure of a verification device provided by another embodiment of the present disclosure;

[0031] Figure 6 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0032] Some embodiments of the present disclosure will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but can be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, for clarity and brevity, descriptions of features known in the art may be omitted.

[0033] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0034] In the related technology, the basic process of chip verification includes: (1) Extracting test points: analyzing the specifications and requirements of the chip to identify the specific points that need to be verified; (2) Formulating a verification plan: determining the verification methods and strategies, including functional coverage, code coverage, and the formulation of test cases; (3) Use case execution: discovering potential defects and problems by executing test cases; (4) Completeness analysis: ensuring that all test points have been covered, including confirmation of the completeness of functions, codes, and interfaces.

[0035] Through the above steps, chip verification can minimize errors and problems after chip production and improve product quality and reliability.

[0036] Figure 1 FIG. 1 is a flow chart of a verification method. Figure 1 As shown, the following steps are included:

[0037] (1) Extract test points: According to the test requirements, based on the functions and interface definitions of the chip under test, extract all the test points that need to be tested, including: module functions, interface data bit width, timing logic relationship, etc.

[0038] (2) Test case construction: Test cases are constructed based on the test points. The test cases are used to drive the reference model and the chip under test.

[0039] (3) Coverage model construction: According to the test points and test requirements, the check items that need to be tested are generated into SV files for subsequent coverage result statistics. The test points here include single-function test points and combined function test points. Combined function check points are composed of several single-function test point conditions.

[0040] (4) Reference model driven: Input test cases to stimulate the reference model to output correct test vectors, including prototype data, print parameters, structure data, key information of intermediate variables, etc.

[0041] (5) Chip driver under test: Input test cases, run the designed chip code on simulation platforms such as VCS, and generate output results.

[0042] (6) Output result matching: Match the correct test vector output by the reference model with the output result of the chip code under test to determine whether the design logic of the chip code under test is consistent with the reference model and is correct.

[0043] (7) Coverage result statistics: Extract the check items described in the SV file from the output results that match correctly in step (6) and perform coverage statistics using tools such as Verdi. If all functional tests are completed, the test is complete. Otherwise, it is necessary to continue to add and modify test cases and retest.

[0044] In some embodiments, if it is found that there are check items that cannot be covered after traversing the test cases (these check items are often unreasonable and difficult to be found in the construction), it is necessary to modify the coverage model and then re-test.

[0045] In the test process scheme of the related technology, after the use case corresponding to the test point is generated, the use case will be directly used to drive the chip and the reference model to obtain the operating results output by the chip and the reference results output by the reference model, and then the functional coverage will be calculated based on the same operating results and reference results. If the functional coverage does not meet the requirements, the use case is adjusted, and the adjusted use case is used to drive the chip and the reference model again until the functional coverage meets the requirements and the chip verification is completed. It often requires multiple rounds of iterations and takes a long time. In particular, the driving of the reference model, the driving of the IC under test, and the comparison of the results require high-time consuming items that rely heavily on server resources, so it takes several weeks to complete the verification process.

[0046] The present invention mainly addresses the problem that the verification process of related technologies is time-consuming. Before using test cases to drive the chip under test and the reference model, the test cases are first adjusted so that the final test cases can cover all inspection items. Then the determined test cases are used to drive the chip under test and the reference model. In this way, the reference model and the chip under test driver, which are heavily dependent on server resources, and the result comparison steps can be separated from the iterative process, reducing the verification convergence time and improving the verification efficiency.

[0047] The following describes the verification method, device, electronic device and storage medium according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0048] Figure 2 A flowchart of a verification method provided by an embodiment of the present disclosure.

[0049] The embodiment of the present disclosure takes the verification method being configured in a verification device as an example. The verification device can be applied to any electronic device or chip so that the electronic device or chip can perform a verification function.

[0050] like Figure 2 As shown, the verification method may include the following steps:

[0051] Step 201, obtaining an initial use case set and a first inspection item set corresponding to the object under test.

[0052] The object to be tested may be hardware, such as a chip, a processor, a graphics card, a memory, etc. It may also be software, such as an application program, etc. This disclosure does not limit this.

[0053] The initial use case set includes one or more initial use cases, which may be use cases generated in advance according to the corresponding test points under test.

[0054] In some embodiments, the initial use case may be manually generated based on the test points, or may be automatically generated based on the test points, which is not limited in the present disclosure.

[0055] In some embodiments, the test points include single test points and combined test points. The test points are used to construct use cases, and the single test points are input variables, output variables, or intermediate variables corresponding to the object under test. The combined test points can be a combination of single test points.

[0056] The first inspection item set includes inspection items corresponding to one or more test points, and the inspection items are values ​​or value combinations corresponding to one or more test points.

[0057] For example, if the test points of interest to the object under test are single test point A, single test point B, and combined test points A and B, the value of single test point A is {1,2,3}, the value of single test point B is {4,5,6}, and the combined value of combined test points A and B is {[1,5],[3,4]}, then the first check item set includes 8 check items, namely A(1); A(2); A(3); B(4); B(5); B(6); A(1)B(5); A(3)B(4).

[0058] In some embodiments, all the check items in the first check item set may be saved in a data file format such as json, xls, etc., so as to facilitate comparison with the second check item set of the initial use case set.

[0059] Step 202: determine a second check item set corresponding to the initial use case set.

[0060] In some embodiments, a target inspection item subset corresponding to each initial use case in the initial use case set may be determined first, and then a second inspection item set may be determined based on the target inspection item subset corresponding to each initial use case.

[0061] In some embodiments, the number of inspection items included in a target inspection item subset may be one or more, which is not limited in the present disclosure.

[0062] In some embodiments, the control parameter values ​​corresponding to the input parameter values ​​in the initial use case are determined, wherein the initial use case is a use case in the initial use case set, and then the initial inspection item subset corresponding to the initial use case is determined based on the input parameter values ​​and control parameter values ​​corresponding to the initial use case, and the intersection between the initial inspection item subset corresponding to the initial use case and the first inspection item set is determined as the target inspection item subset corresponding to the initial use case, and finally the union of the target inspection item subsets corresponding to all the initial use cases is determined as the second inspection item set.

[0063] The initial check item subset includes at least one of the following: a check item consisting of an input parameter and a corresponding input parameter value, a check item consisting of a control parameter and a corresponding control parameter value, and a check item consisting of an input parameter, a corresponding input parameter value, a control parameter, and a corresponding control parameter value;

[0064] Among them, the control parameters include intermediate parameters and output parameters.

[0065] In some embodiments, the control parameter value corresponding to the input parameter value in the initial use case can be determined according to the mapping relationship between the input parameter corresponding to the object under test and the control parameter.

[0066] In some embodiments, the input parameter value corresponding to each initial use case may also be input into the pseudo reference model to obtain the control parameter value corresponding to each initial use case.

[0067] The pseudo reference model is generated according to the reference model, and the pseudo reference model includes a mapping relationship between input parameters and control parameters.

[0068] In some embodiments, the reference model may include a parameter mapping module and a calculation module. The parameter mapping module includes a mapping relationship between input parameters and control parameters, and the calculation module is used to perform calculations according to the control parameter values. The pseudo reference model only includes the parameter mapping module of the reference model. Thus, the target inspection item subset corresponding to each use case can be quickly determined.

[0069] The reference model is used to determine the reference result output after the measured object processes the input parameter value. The reference result can be understood as the expected result.

[0070] Step 203: determine the inspection items that belong to the first inspection item set and do not belong to the second inspection item set as target inspection items.

[0071] It should be noted that the check items that belong to the first check item set and do not belong to the second check item set are check items that are not covered by the test case.

[0072] In some embodiments, the target inspection item may be one or more, which is not limited in the present disclosure.

[0073] Step 204: based on the target check items, update the initial use case set to obtain the target use case set.

[0074] In some embodiments, a new use case that can cover the target check item can be generated for the target check item, and then the new use case is added to the initial use case set to obtain the target use case set.

[0075] Step 205: verify the object under test based on the target use case set.

[0076] In the disclosed embodiment, after obtaining the target use case set, the target use case set can be used to verify the object under test to obtain a verification result.

[0077] In some embodiments, the input parameter values ​​corresponding to each target use case in the target use case set are respectively input into the object under test and the reference model corresponding to the object under test to obtain the operation results output by the object under test and the reference results output by the reference model. Then, the verification results corresponding to the object under test are determined based on the matching results between the operation results and the reference results.

[0078] The reference model is used to determine the reference result output after the measured object processes the input parameter value.

[0079] In some embodiments, if the operation result is exactly the same as the reference result, the object under test is considered to have passed the verification. If the operation result corresponding to any target use case is different from the reference result, the object under test is considered to have passed the verification. Furthermore, the object under test can also be modified according to the difference between the operation result and the reference result.

[0080] In the disclosed embodiment, the initial use case set and the first inspection item set corresponding to the object under test are first obtained, and the second inspection item set corresponding to the initial use case set is determined, and then the inspection items that belong to the first inspection item set and do not belong to the second inspection item set are determined as target inspection items, and then based on the target inspection items, the initial use case set is updated to obtain the target use case set, and finally the object under test is verified based on the target use case set. Therefore, before using the use case to verify the object under test, the target inspection items that are not covered by the current initial use case set are first determined, and then based on the target inspection items, the initial use case set is updated to improve the coverage of the target use case set, and then the target use case set is used to verify the object under test, which can improve the verification efficiency of the object under test while ensuring the verification coverage.

[0081] Figure 3 A flow chart of a verification method provided by an embodiment of the present disclosure is shown as follows: Figure 3 As shown, the verification method may include the following steps:

[0082] Step 301, obtaining an initial use case set and a first inspection item set corresponding to the object under test.

[0083] Step 302: determine a second check item set corresponding to the initial use case set.

[0084] Step 303: determine the inspection items that belong to the first inspection item set and do not belong to the second inspection item set as target inspection items.

[0085] The specific implementation forms of steps 301 to 303 can refer to the detailed descriptions in other embodiments of the present disclosure and will not be repeated here.

[0086] Step 304: determine the target test point according to the target inspection item.

[0087] In some embodiments, the first test point corresponding to the target inspection item can be determined first, and when the first test point is a combined test point, the second test points constituting the first test point can be determined, and then the first test point and the second test points can be determined as target test points.

[0088] For example, if the target inspection item is A(a3)B(b2), the corresponding first test point is the combined test points A and B, then the second test point is the single test point A and the single test point B, and the target test points include the single test point A, the single test point B and the combined test points A and B.

[0089] It should be noted that a use case may cover the check items corresponding to the combined test point while also covering the check items corresponding to the single test points that make up the combined test point. For example, when a use case covers the check item A(a3)B(b2), it also covers the two check items A(a3) and B(b2). Therefore, if the check items corresponding to the combined test point are not covered, it may also be due to the fact that the use cases generated for the combined test point and the single test points that make up the combined test point are not good enough.

[0090] Therefore, in the embodiment of the present disclosure, when the first test point corresponding to the target inspection item is a combined test point, the first test point and the second test points constituting the first test point can be determined as target test points, and the use case can be regenerated.

[0091] In some embodiments, when the first test point is a single test point, the first test point is directly determined as the target test point.

[0092] Step 305: determine a first use case set corresponding to the target test point.

[0093] In some embodiments, use cases may be regenerated for target test points to obtain a first set of use cases.

[0094] In some embodiments, in order to ensure that the first use case set generated for the target test point can cover all the inspection items corresponding to the target test point as much as possible, a candidate use case set corresponding to the target test point can be generated first, and then the first number of all different inspection items corresponding to the candidate use case set and the second number of all inspection items corresponding to the target test point in the first inspection item set are determined, and the ratio of the first number to the second number is determined as the first coverage rate corresponding to the candidate use case set, and then when the first coverage rate is less than the first threshold, the operation of generating the candidate use case set is returned until the first coverage rate corresponding to the generated candidate use case set is greater than or equal to the first threshold, and finally the candidate use case set with the first coverage rate greater than or equal to the first threshold is determined as the first use case set.

[0095] The first threshold may be 100%, 99%, etc. This disclosure does not limit this.

[0096] In some embodiments, a set of candidate use cases corresponding to a target test point may be generated in combination with the target inspection items.

[0097] In some embodiments, after determining the candidate use case set, the check items corresponding to each candidate use case in the candidate use case set may be determined first, and then all different check items corresponding to the candidate use case set may be determined. For specific implementation methods, reference may be made to the detailed description of determining the second check item set corresponding to the initial use case set, which will not be described in detail here.

[0098] Step 306: Use the first use case set to update the second use case set corresponding to the target test point in the initial use case set to obtain the target use case set.

[0099] In some embodiments, the second use case set in the initial use case set may be replaced by the first use case set to obtain a target use case set.

[0100] Step 307: verify the object under test based on the target use case set.

[0101] In the disclosed embodiment, after determining the target inspection items that are not covered by the initial use case set, the target test point can be determined based on the target inspection, and then the first use case set corresponding to the target test point can be determined. The second use case set corresponding to the target test point in the initial use case set can be updated using the first use case set to obtain the target use case set, and finally the object under test can be verified based on the target use case set. Thus, the target test point can be determined based on the target inspection items that are not covered by the initial use case set, and then the use case can be regenerated for the target test point, so that the use case regenerated for the target test point can cover the target inspection item, so as to improve the coverage rate corresponding to the target test point, and then improve the coverage rate corresponding to the target use case set, and further improve the verification efficiency of the object under test while ensuring the verification coverage rate.

[0102] Figure 4 A flow chart of a verification method provided by an embodiment of the present disclosure is shown as follows: Figure 4 As shown, the verification method may include the following steps:

[0103] Step 401, obtaining an initial use case set and a first inspection item set corresponding to the object under test.

[0104] Step 402: determine a second check item set corresponding to the initial use case set.

[0105] Step 403: determine the inspection items that belong to the first inspection item set and do not belong to the second inspection item set as target inspection items.

[0106] The specific implementation forms of steps 401 to 403 may refer to the detailed descriptions in other embodiments of the present disclosure and will not be repeated here.

[0107] Step 404: determine a third quantity corresponding to all inspection items in the second inspection item set and a fourth quantity corresponding to all inspection items in the first inspection item set.

[0108] Step 405: determine the ratio of the third number to the fourth number as the second coverage rate corresponding to the initial use case set.

[0109] Step 406, when the second coverage is less than the second threshold, the inspection items that belong to the first inspection item set and do not belong to the second inspection item set are determined as target inspection items.

[0110] It should be noted that, when the second coverage is less than the second threshold, the coverage of the initial use case set is low. If the initial use case set is used directly to verify the object under test, the verification effect is poor. If the second coverage is greater than or equal to the second threshold, directly verifying the object under test can also achieve better results. Therefore, in the embodiment of the present disclosure, when the second coverage corresponding to the initial use case set is less than the second threshold, the target inspection items that are not covered by the use cases can be determined, and then the initial use case set can be updated to improve the coverage corresponding to the obtained target use case set.

[0111] In some embodiments, the second threshold value may be the same as or different from the first threshold value, which is not limited in the present disclosure.

[0112] Step 407: Based on the target check items, the initial use case set is updated to obtain the target use case set.

[0113] The specific implementation form of step 407 can refer to the detailed description in other embodiments of the present disclosure, and will not be described in detail here.

[0114] Step 408: Determine the fifth number of all different inspection items corresponding to the target use case set.

[0115] In some embodiments, after determining the target use case set, the check items corresponding to each target use case in the target use case set may be determined first, and then all different check items corresponding to the target use case set may be determined. For specific implementation methods, reference may be made to the detailed description of determining the second check item set corresponding to the initial use case set, which will not be described in detail here.

[0116] Step 409: determine the ratio of the fifth number to the fourth number as the third coverage rate corresponding to the target use case set.

[0117] Step 410, when the third coverage is less than the second threshold, the target use case set is used as the initial use case set, and the operation of obtaining the target use case set is returned to be executed until the third coverage corresponding to the target use case set is greater than or equal to the second threshold.

[0118] It should be noted that if the third coverage rate corresponding to the target use case set is less than the second threshold, it means that the coverage rate corresponding to the target use case set still does not meet the requirements. If the target use case set whose coverage rate does not meet the requirements is directly used to verify the object under test, it may lead to inaccurate verification results. Therefore, in order to accurately verify the object under test, when the third coverage rate corresponding to the target use case set is less than the second threshold, the target use case set can be further updated so that the third coverage rate corresponding to the updated target use case set is greater than or equal to the second threshold.

[0119] In some embodiments, when the third coverage rate is less than the second threshold, the target use case set is used as the initial use case set, and the second inspection item set corresponding to the initial use case set is determined, and the inspection items that belong to the first inspection item set and do not belong to the second inspection item set are determined as target inspection items. Based on the target inspection items, the initial use case set is updated to obtain the target use case set, until the third coverage rate corresponding to the target use case set is greater than or equal to the second threshold.

[0120] Step 411, verifying the object under test based on the target use case set.

[0121] In the disclosed embodiment, when the second coverage rate corresponding to the initial use case set is less than the second threshold, the target inspection items not covered by the initial use case set can be determined, and then based on the target inspection items, the initial use case set can be updated to obtain the target use case set, and when the third coverage rate corresponding to the target use case set is less than the second threshold, the target use case set can be used as the initial use case set to further determine the uncovered target inspection items, and then the initial use case set can be updated until the third coverage rate corresponding to the target use case set is greater than or equal to the second threshold, and then the object under test can be verified. Thus, when the second coverage rate corresponding to the initial use case set is less than the second threshold, the initial use case set is updated again, and the third coverage rate corresponding to the updated target use case set is made greater than or equal to the second threshold, further improving the verification efficiency of the object under test while ensuring the verification coverage.

[0122] In order to implement the above embodiment, the present disclosure also proposes a verification device.

[0123] Figure 5 A schematic diagram of the structure of a verification device provided in an embodiment of the present disclosure.

[0124] like Figure 5 As shown, the verification device 500 may include:

[0125] An acquisition module 501 is used to acquire an initial use case set and a first inspection item set corresponding to the object under test;

[0126] A first determination module 502, configured to determine a second set of inspection items corresponding to an initial set of use cases;

[0127] A second determination module 503, configured to determine the inspection items that belong to the first inspection item set and do not belong to the second inspection item set as target inspection items;

[0128] An updating module 504 is used to update the initial use case set based on the target check item to obtain a target use case set;

[0129] The verification module 505 is used to verify the object under test based on the target use case set.

[0130] In some embodiments, the update module 504 is used to:

[0131] Determine the target test points according to the target inspection items;

[0132] Determine a first use case set corresponding to the target test point;

[0133] The first use case set is used to update the second use case set corresponding to the target test point in the initial use case set to obtain the target use case set.

[0134] In some embodiments, the update module 504 is used to:

[0135] Determine a first test point corresponding to the target inspection item;

[0136] In the case where the first test point is a combined test point, determining each second test point constituting the first test point;

[0137] The first test point and each second test point are determined as target test points.

[0138] In some embodiments, the update module 504 is used to:

[0139] Generate a set of candidate use cases corresponding to the target test points;

[0140] Determine a first number of all different inspection items corresponding to the candidate use case set, and a second number of all inspection items in the first inspection item set corresponding to the target test point;

[0141] Determine the ratio of the first quantity to the second quantity as a first coverage rate corresponding to the candidate use case set;

[0142] When the first coverage is less than the first threshold, return to execute the operation of generating a candidate use case set until the first coverage corresponding to the generated candidate use case set is greater than or equal to the first threshold;

[0143] A set of candidate use cases whose first coverage is greater than or equal to a first threshold is determined as a first use case set.

[0144] In some embodiments, the second determining module 503 is configured to:

[0145] Determine a third quantity corresponding to all inspection items in the second inspection item set and a fourth quantity corresponding to all inspection items in the first inspection item set;

[0146] Determine the ratio of the third number to the fourth number as the second coverage rate corresponding to the initial use case set;

[0147] When the second coverage is less than the second threshold, the inspection items that belong to the first inspection item set and do not belong to the second inspection item set are determined as target inspection items.

[0148] In some embodiments, the system further includes a processing module for:

[0149] Determine the fifth number of all different check items corresponding to the target use case set;

[0150] Determine the ratio of the fifth number to the fourth number as the third coverage rate corresponding to the target use case set;

[0151] When the third coverage is less than the second threshold, the target use case set is used as the initial use case set, and the operation of obtaining the target use case set is returned until the third coverage corresponding to the target use case set is greater than or equal to the second threshold.

[0152] In some embodiments, the verification module 505 is used to:

[0153] Inputting the input parameter values ​​corresponding to each target use case in the target use case set into the object under test and the reference model corresponding to the object under test, respectively, to obtain the running results output by the object under test and the reference results output by the reference model, wherein the reference model is used to determine the reference results output by the object under test after processing the input parameter values;

[0154] According to the matching result between the running result and the reference result, the verification result corresponding to the object under test is determined.

[0155] In some embodiments, the first determining module 502 is configured to:

[0156] Determine a control parameter value corresponding to an input parameter value in an initial use case, wherein the initial use case is a use case in the initial use case set;

[0157] Determine an initial check item subset corresponding to the initial use case according to the input parameter values ​​and control parameter values ​​corresponding to the initial use case, wherein the initial check item subset includes at least one of the following: a check item consisting of an input parameter and a corresponding input parameter value, a check item consisting of a control parameter and a corresponding control parameter value, and a check item consisting of an input parameter, a corresponding input parameter value, a control parameter, and a corresponding control parameter value;

[0158] Determine the intersection between the initial inspection item subset corresponding to the initial use case and the first inspection item set as the target inspection item subset corresponding to the initial use case;

[0159] The union of the target inspection item subsets corresponding to all initial use cases is determined as the second inspection item set.

[0160] The functions and specific implementation principles of the above modules in the embodiments of the present disclosure can be referred to the above method embodiments, and will not be repeated here.

[0161] The verification device of the disclosed embodiment first obtains the initial use case set and the first inspection item set corresponding to the object under test, and determines the second inspection item set corresponding to the initial use case set, and then determines the inspection items that belong to the first inspection item set and do not belong to the second inspection item set as target inspection items, and then updates the initial use case set based on the target inspection items to obtain the target use case set, and finally verifies the object under test based on the target use case set. Therefore, before using the use case to verify the object under test, first determine the target inspection items that are not covered by the current initial use case set, and then update the initial use case set based on the target inspection items to improve the coverage of the target use case set, and then use the target use case set to verify the object under test, which can improve the verification efficiency of the object under test while ensuring the verification coverage.

[0162] In order to implement the above embodiments, the present disclosure further proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the verification method proposed in the above embodiments of the present disclosure is implemented.

[0163] Figure 6 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 6 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0164] like Figure 6 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0165] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.

[0166] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0167] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present disclosure.

[0168] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0169] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0170] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the methods mentioned in the above embodiments.

[0171] In order to implement the above embodiments, the present disclosure further proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the verification method proposed in the above embodiments of the present disclosure is implemented.

[0172] In order to implement the above embodiments, the present disclosure further proposes a computer program product, including a computer program, which implements the verification method proposed in the above embodiments of the present disclosure when the computer program is executed by a processor.

[0173] In order to implement the above embodiments, the present disclosure further proposes a chip, which is verified by the verification method proposed in the above embodiments of the present disclosure.

[0174] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this disclosure shall comply with the relevant laws and regulations and shall not violate public order and good morals.

[0175] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0176] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0177] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0178] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0179] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0180] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0181] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0182] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims

1. A verification method, characterized in that: The method comprises: Obtaining an initial use case set and a first check item set corresponding to the object under test; Determine a second check item set corresponding to the initial use case set; Determine the inspection items that belong to the first inspection item set and do not belong to the second inspection item set as target inspection items; Based on the target inspection items, the initial use case set is updated to obtain a target use case set; Based on the target use case set, the object under test is verified.

2. The method according to claim 1, characterized in that The check item set includes check items corresponding to one or more test points, the check items include values ​​or value combinations corresponding to one or more test points, the test points are used to construct use cases, and the initial use case set is updated based on the target check items to obtain a target use case set, including: Determine the target test points according to the target inspection items; Determine a first use case set corresponding to the target test point; The first use case set is used to update the second use case set corresponding to the target test point in the initial use case set to obtain the target use case set.

3. The method according to claim 2, characterized in that Determining the target test point according to the target inspection item includes: Determine a first test point corresponding to the target inspection item; In the case where the first test point is a combined test point, determining each second test point constituting the first test point; The first test point and each of the second test points are determined as the target test points.

4. The method according to claim 2, characterized in that: The determining a first use case set corresponding to the target test point includes: Generate a set of candidate test cases corresponding to the target test point; Determine a first number of all different inspection items corresponding to the candidate use case set, and a second number of all inspection items in the first inspection item set corresponding to the target test point; Determine a ratio of the first number to the second number as a first coverage rate corresponding to the candidate use case set; In the case where the first coverage is less than the first threshold, returning to execute the operation of generating a candidate use case set until the first coverage corresponding to the generated candidate use case set is greater than or equal to the first threshold; A set of candidate use cases whose first coverage is greater than or equal to the first threshold is determined as the first use case set.

5. The method according to claim 1, characterized in that The step of determining the inspection items that belong to the first inspection item set and do not belong to the second inspection item set as target inspection items includes: Determine a third number corresponding to all inspection items in the second inspection item set, and a fourth number corresponding to all inspection items in the first inspection item set; Determine a ratio of the third number to the fourth number as a second coverage rate corresponding to the initial use case set; When the second coverage rate is less than a second threshold, the inspection items that belong to the first inspection item set and do not belong to the second inspection item set are determined as the target inspection items.

6. The method according to claim 5, characterized in that After the initial use case set is updated based on the target inspection item to obtain the target use case set, the method further includes: Determining a fifth number of all different inspection items corresponding to the target use case set; Determine a ratio of the fifth number to the fourth number as a third coverage rate corresponding to the target use case set; When the third coverage ratio is less than the second threshold, the target use case set is used as the initial use case set, and the operation of obtaining the target use case set is returned to be executed until the third coverage ratio corresponding to the target use case set is greater than or equal to the second threshold.

7. The method according to any one of claims 1 to 6, characterized in that: The verifying the object under test based on the target use case set includes: Inputting the input parameter values ​​corresponding to each target use case in the target use case set into the object under test and the reference model corresponding to the object under test, respectively, to obtain the running result output by the object under test and the reference result output by the reference model, wherein the reference model is used to determine the reference result output by the object under test after processing the input parameter values; According to the matching result between the running result and the reference result, the verification result corresponding to the object under test is determined.

8. The method according to any one of claims 1 to 6, characterized in that: The determining of a second set of inspection items corresponding to the initial set of use cases includes: Determining a control parameter value corresponding to an input parameter value in an initial use case, wherein the initial use case is a use case in the initial use case set; Determine an initial check item subset corresponding to the initial use case according to the input parameter values ​​and control parameter values ​​corresponding to the initial use case, wherein the initial check item subset includes at least one of the following: a check item consisting of an input parameter and a corresponding input parameter value, a check item consisting of a control parameter and a corresponding control parameter value, and a check item consisting of an input parameter, a corresponding input parameter value, a control parameter, and a corresponding control parameter value; Determine the intersection between the initial inspection item subset corresponding to the initial use case and the first inspection item set as the target inspection item subset corresponding to the initial use case; The union of the target inspection item subsets corresponding to all the initial use cases is determined as the second inspection item set.

9. A verification device, characterized in that: The device includes: An acquisition module, used to acquire an initial use case set and a first inspection item set corresponding to the object under test; A first determination module, used to determine a second check item set corresponding to the initial use case set; A second determination module, configured to determine an inspection item that belongs to the first inspection item set and does not belong to the second inspection item set as a target inspection item; An updating module, configured to update the initial use case set based on the target inspection item to obtain a target use case set; A verification module is used to verify the object under test based on the target use case set.

10. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the verification method as claimed in any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the verification method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the verification method according to any one of claims 1 to 8 when being executed by a processor.

13. A chip, characterized in that: The chip is obtained after verification by any verification method as claimed in claims 1-8.

Images