Method, device, medium, product and equipment for testing reverse function points of test cases

Abstract

The application discloses a kind of test case anti-label function points method, device, medium, product and equipment, the method includes: after each test case of the digital circuit to be verified is executed, the function point number of the function point covered by each test case is sequentially assigned to the pre-defined global variable, each function point of the digital circuit to be verified is provided with function point number;In the process of sampling function coverage, the assignment result of the global variable is sampled and counted;According to the assignment result of the global variable, the anti-label result of the test case anti-label function point of the digital circuit to be verified is obtained, and the anti-label result is used to indicate whether each function point is marked by test case and whether test case is executed.The application can realize the automatic, efficient anti-label of test case to function point based on function coverage, and ensure the objectivity and verifiability of anti-label result, facilitate the actual completion of effective review and verification work.

Description

Technical Field

[0001] This invention relates to the field of testing technology, and in particular to a method, apparatus, computer-readable storage medium, computer program product, and terminal device for back-labeling functional points in test cases. Background Technology

[0002] When verification engineers use System Verilog (SV) to verify digital circuits, they need to ensure that all functional points in the digital circuit are covered by corresponding test cases to ensure that no functional points are missed. Therefore, using test cases to back-annotate functional points is a common way to ensure that no functional points are missed. Specifically, test cases can be developed for each functional point first, and after the test cases are developed, they can be back-annotated to the corresponding functional points to ensure that all functional points have corresponding test case coverage.

[0003] To achieve back-annotation of function points using test cases, the traditional method is to manually mark the corresponding test cases after each function point and then manually check whether all function points have correct test case coverage. However, this traditional method has two main drawbacks: (1) Manual back-annotation is inefficient, especially when there are a large number of function points and test cases. For example, if the total number of function points of a digital circuit exceeds 1,000 and the total number of test cases exceeds 100, manual back-annotation and visual inspection will be a very heavy and time-consuming task; (2) After obtaining the back-annotation results, although it can be seen that all function points have correct test case coverage, it is impossible to determine whether the test cases corresponding to each function point have actually been executed. This makes it impossible for project managers to review the actual completion of the verification work, resulting in a lack of objectivity and poor verifiability of the back-annotation results. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, computer-readable storage medium, computer program product, and terminal device for back-annotating functional points using test cases. This method enables automated and efficient back-annotation of functional points using test cases based on functional coverage, while ensuring the objectivity and verifiability of the back-annotation results, thus facilitating effective review of the actual completion of verification work.

[0005] To achieve the above objectives, embodiments of the present invention provide a method for back-annotating function points in test cases, comprising: After each test case of the digital circuit to be verified is executed, the predefined global variable is sequentially assigned the function point number of the function point covered by each test case; wherein, each function point of the digital circuit to be verified is set with a function point number. During the sampling coverage process, the assignment results of the global variables are sampled and statistically analyzed; The back-annotation result of the test case back-annotation function point of the digital circuit to be verified is obtained according to the assignment result of the global variable; wherein, the back-annotation result is used to indicate whether each function point is marked by the test case and whether the test case is executed.

[0006] Furthermore, the function point number is associated with the level of the function point, wherein the total bit width of the function point number, the number of segments after segmentation, and the bit width of each segment are determined by the level and level number of the corresponding function point.

[0007] Furthermore, the number of segments after the function point number is divided is equal to the level of the corresponding function point, and the i-th segment number in the function point number represents the i-th level number of the corresponding function point, where i is a positive integer.

[0008] Furthermore, the step of sequentially assigning the predefined global variables to the function point numbers of the function points covered by each test case specifically includes: When a test case covers only one function point, the predefined global variable is assigned the function point number of the function point covered by the test case; When the same test case covers more than one function point, the predefined global variable is assigned the function point number of each function point covered by the test case in sequence.

[0009] Furthermore, the time interval between two consecutive assignments of the global variable is greater than the sampling period of the functional coverage.

[0010] Further, obtaining the back-annotation results of the test case back-annotation function points of the digital circuit to be verified based on the assignment results of the global variables specifically includes: Based on the assignment results of the global variables, obtain the number of times the function point number of each function point of the digital circuit to be verified is assigned. When there is a function point number that has been assigned zero times, the back-annotation result of the test case back-annotation function point of the digital circuit to be verified is that there is a function point that has not been marked by the test case and the test case has not been executed; wherein, the function point that has not been marked by the test case and the test case has not been executed is the function point corresponding to the function point number that has been assigned zero times.

[0011] To achieve the above objectives, embodiments of the present invention also provide an apparatus for back-annotating function points in test cases, comprising: The global variable assignment module is used to assign the function point number of each function point covered by the test case to the predefined global variable after the execution of each test case of the digital circuit to be verified; wherein, each function point of the digital circuit to be verified is set with a function point number. The assignment result statistics module is used to sample and statistically analyze the assignment results of the global variables during the sampling function coverage process. The back-annotation result acquisition module is used to obtain the back-annotation result of the test case back-annotation function point of the digital circuit to be verified based on the assignment result of the global variable; wherein, the back-annotation result is used to indicate whether each function point is marked by the test case and whether the test case is executed.

[0012] This invention also provides a computer-readable storage medium including a stored computer program, which, when running, controls the device where the computer-readable storage medium is located to execute the method for back-annotating function points of test cases as described above.

[0013] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method for back-annotating test cases as described in any of the above embodiments.

[0014] This invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the method for back-annotating test cases as described above.

[0015] This invention provides a method, apparatus, computer-readable storage medium, computer program product, and terminal device for back-annotating function points using test cases. First, after each test case of the digital circuit to be verified is executed, a predefined global variable is sequentially assigned the function point number of each function point covered by the test case. Each function point of the digital circuit to be verified is assigned a function point number. Then, during the sampling of function coverage, the assignment results of the global variables are sampled and statistically analyzed. Finally, based on the assignment results of the global variables, the back-annotation results of the back-annotated function points of the digital circuit to be verified are obtained. The back-annotation results indicate whether each function point is marked by a test case and whether the test case is executed. This invention enables automated and efficient back-annotation of function points by test cases based on function coverage, ensuring the objectivity and verifiability of the back-annotation results, and facilitating effective review of the actual completion of verification work. Attached Figure Description

[0016] Figure 1 This is a flowchart of a method for back-annotating function points in test cases according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the back-annotation result of a test case back-annotation function point provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the back-annotation result of a test case back-annotation function point provided in another embodiment of the present invention; Figure 4 This is a structural block diagram of a device for back-annotating function points in test cases according to an embodiment of the present invention; Figure 5 This is a structural block diagram of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be noted that in digital circuit simulation, the function points of digital circuits are usually decomposed into levels, as shown in Table 1. Table 1 illustrates a common method for decomposing function points in digital circuits. Level 1 function points (function points 1 and 2 in Table 1) represent relatively macroscopic functions, while Level 2 function points (function points 1.1, 1.2, 1.3, 2.1, 2.2, and 2.3 in Table 1) and Level 3 function points (function points 1.1.1, 1.1.2, ..., 2.3.3 in Table 1) represent more detailed functions. Each Level 1 function point can be further divided into several Level 2 function points, and each Level 2 function point can be further divided into several Level 3 function points. Correspondingly, each Level 1 function point... The sum of all secondary function points under a given point is the corresponding primary function point. The sum of all tertiary function points under each secondary function point is the corresponding secondary function point. Based on this, if it is necessary to verify a certain level function point, it is only necessary to verify all the next-level function points (if any) of that level function point. There is no need to verify the level function point separately. For example, if the verification of function points 1.1.1, 1.1.2, and 1.1.3 shown in Table 1 is completed, it means that function point 1.1 is verified. There is no need to verify function point 1.1 separately. Similarly, if the verification of function points 1.1, 1.2, and 1.3 is completed, it means that function point 1 is verified. There is also no need to verify function point 1 separately.

[0019] Understandably, the number of levels after each function point is broken down, the number of function points at each level, and the name of each function point can all be set according to actual needs. The content shown in Table 1 is only an example.

[0020] Table 1 Example of Function Point Hierarchy For example, suppose a digital circuit supports three reset types and three clock frequencies. When extracting function points for this digital circuit, to facilitate function point classification, it can first be described using two primary function points, and then each primary function point can be further broken down into three secondary function points for detailed description, as follows: 1. Supports multiple types of reset 1.1 Supports static reset; 1.2 Supports dynamic reset; 1.3 Supports semi-static reset; 2. Supports multiple clock frequencies 2.1 Supports a 100MHz clock frequency; 2.2 Supports a clock frequency of 500MHz; 2.3 Supports a clock frequency of 1000MHz.

[0021] Based on the background technical content, it can be seen that verification engineers can develop test cases for each functional point. After the test cases are developed, they will be back-annotated to the corresponding functional points to ensure that all functional points have corresponding test case coverage. Taking the functional point hierarchy shown in Table 1 as an example, the corresponding test cases are back-annotated to the last level functional point (i.e., the third level functional point). The final back-annotation results are shown in Table 2.

[0022] Table 2 Example of test cases indicating function points Understandably, a test case can cover multiple functional points (as shown in Table 2, "Test Case 3" covers "Function Point 1.1.3" and "Function Point 1.2.1", and "Test Case 16" covers "Function Point 2.3.2" and "Function Point 2.3.3"). A functional point can also be covered by multiple test cases, meaning that this functional point has been verified multiple times (one test case corresponds to one verification). However, theoretically, a functional point must be fully tested within one test case and cannot be scattered across multiple test cases. If a functional point cannot be fully tested within one test case, then either the content of the test cases corresponding to that functional point should be further enriched to ensure that the functional point is covered by only one test case, or the functional point should be broken down into smaller parts to ensure that each sub-level functional point after the breakdown is covered by only one test case.

[0023] However, in order to obtain the back-annotation results shown in Table 2, if the traditional methods described in the background technology are used, there are problems such as low efficiency of manual back-annotation, lack of objectivity of back-annotation results, and poor verifiability. For example, although it can be seen by visually inspecting the contents of Table 2 that "Function Point 2.1.1" is back-annotated by "Use Case 9", the project manager cannot determine whether the verification engineer actually executed "Use Case 9", which makes it impossible for the project manager to review the work of the verification engineer.

[0024] To address the problems of traditional methods, this invention provides a method for back-annotating function points in test cases. (See attached document.) Figure 1 The diagram shown is a flowchart of a method for back-annotating function points in test cases according to an embodiment of the present invention. The method includes steps S11 to S13: Step S11: After each test case of the digital circuit to be verified is executed, the predefined global variables are sequentially assigned the function point number of the function point covered by each test case; wherein, each function point of the digital circuit to be verified is set with a function point number. Step S12: During the sampling function coverage process, sample and statistically analyze the assignment results of the global variables; Step S13: Obtain the back-annotation result of the test case back-annotation function point of the digital circuit to be verified according to the assignment result of the global variable; wherein, the back-annotation result is used to indicate whether each function point is marked by the test case and whether the test case is executed.

[0025] It should be noted that test cases for each functional point of the digital circuit to be verified have been developed. Since the test cases are developed based on the functional points, the correspondence between the test cases and the functional points is known after the test cases are developed.

[0026] Furthermore, in the traditional verification process of digital circuits, the function points of digital circuits only have text names or names containing level numbers (such as "Function Point 1.1.1" shown in Tables 1 and 2). These names are generally text descriptions written in tables or documents and cannot be recognized by simulation tools and code. In other words, function points do not have a unique ID (Identity) that can be recognized by a machine. This is the fundamental reason why traditional methods can only perform manual back-annotation. That is, although the correspondence between test cases and function points is known, the machine does not know which test case verifies which function point. Therefore, in the embodiments of the present invention, each function point of the digital circuit to be verified is pre-set with a unique function point number. This function point number is a machine-recognizable number, which allows the correspondence between test cases and function points to be recognized, sampled, and statistically analyzed by the machine. This is the basis for achieving automated back-annotation.

[0027] In specific implementation of this invention, firstly, after each test case of the digital circuit to be verified is executed, the predefined global variable F_Num can be sequentially assigned the function point number of the function point covered by each test case. The assignment principle of the global variable F_Num is that after the execution of a test case, the global variable F_Num is assigned the function point number of the function point covered by that test case. Then, during the sampling of functional coverage using coverage tools, the assignment result of the global variable F_Num can be sampled to obtain the assignment result of the global variable F_Num after the execution of each test case, and the... All assignment results of the global variable F_Num obtained from sampling are statistically summarized to match and count the assignment results of the global variable F_Num obtained from each sampling with the corresponding function point number (for example, if each assignment result is sampled once, the count value of the corresponding function point number is incremented by 1; if the same assignment result is sampled several times, the count value of the corresponding function point number is incremented by that number). Finally, the back-annotation results of the test cases for the digital circuit to be verified can be obtained based on the assignment results of the global variable F_Num obtained from the statistical summary. These back-annotation results can be used to indicate whether each function point of the digital circuit to be verified has been marked by a test case and whether the test case corresponding to each function point has been executed.

[0028] For example, if the assignment result of the global variable F_Num obtained from the statistical summary determines that the function point number of a certain function point has been sampled at least once (i.e., the corresponding count value is ≥1), it means that the function point has been marked by the test case and the test case corresponding to the function point has been executed. This is because: only after the test case corresponding to the function point is actually completed will the global variable F_Num be assigned the function point number of the function point. Correspondingly, the assignment result of the global variable F_Num being assigned the function point number of the function point will be sampled and counted.

[0029] It's important to note that the global nature of the global variable F_Num is that it can be accessed throughout the code of all test cases. This allows the global variable F_Num to be assigned the function point number of the function point covered by each test case after execution. For example, as shown in Table 2, if "Test Case 1" covers "Function Point 1.1.1", then after "Test Case 1" completes execution, the global variable F_Num can be assigned the function point number of "Function Point 1.1.1". If "Test Case 2" covers "Function Point 1.1.2", then after "Test Case 2" completes execution, ... The global variable F_Num can then be assigned the function point number "Function Point 1.1.2". Similarly, if all test cases have been executed, the global variable F_Num can be sequentially assigned the function point number of each function point covered by the test case. Therefore, by sampling and statistically analyzing the assignment results of the global variable F_Num, we can determine whether the function point number of each function point has been assigned to the global variable F_Num, thereby determining whether each function point has been marked by a test case and whether the test case corresponding to each function point has been executed.

[0030] It should be noted that the digital circuit to be verified in the embodiments of the present invention can be a digital circuit inside the chip that needs to be verified, and the functional points of the digital circuit to be verified and reverse-annotated can be the functions of the internal modules of the chip, the logical behaviors to be implemented by the chip, etc. Furthermore, the verification environment, language system, simulation tools, etc. used in the embodiments of the present invention are all applicable to chip verification.

[0031] The present invention provides a method for back-annotating functional points using test cases. By using functional coverage rate to represent the test case coverage of each functional point of the digital circuit to be verified, the method enables automated and efficient back-annotation of functional points using test cases based on functional coverage rate. Furthermore, the back-annotation results obtained can intuitively indicate whether each functional point of the digital circuit to be verified has been marked by test cases and whether the test cases corresponding to each functional point have been executed. This makes the back-annotation results objective and verifiable, facilitating project managers to effectively review the actual completion of verification work.

[0032] In one optional embodiment, the function point number is associated with the level of the function point, wherein the total bit width of the function point number, the number of segments after segmentation, and the bit width of each segment are determined by the level and level number of the corresponding function point.

[0033] Specifically, in conjunction with the above embodiments, since the functional points of the digital circuit are hierarchically split, when setting the functional point number of each functional point of the digital circuit to be verified, the functional point number can be associated with the level of the functional point. The functional point number of each functional point is set based on the level information of each functional point, and the functional point number can adopt a segmented design. The total bit width of the functional point number, the number of segments after the functional point number is divided, and the bit width of each segment can all be determined by the level and level number of the corresponding functional point.

[0034] It should be noted that, since the embodiments of the present invention require that the global variable F_Num be assigned the function point number of the function point covered by the test case after the test case is executed, the bit width of the global variable F_Num needs to be consistent with the total bit width of the function point number when the global variable F_Num is predefined.

[0035] It is understandable that, since only the last-level function point of the digital circuit to be verified needs to be verified, in the embodiments of the present invention, when setting the function point number of each function point of the digital circuit to be verified, only the function point number of the last-level function point needs to be set. For example, the last-level function points of function point 1 and function point 2 shown in Table 1 and Table 2 are both third-level function points. Therefore, only the function point number of the third-level function points shown in Table 1 and Table 2 needs to be set.

[0036] It should be noted that, compared to arbitrarily assigning machine-recognizable unique IDs to function points (e.g., setting multiple characters as unique IDs), this embodiment of the invention associates function point numbers with the function point levels. This facilitates searching, more intuitively indicates the correspondence between function points and function point numbers, and is highly operable. For example, assuming the total number of function points exceeds 1000, randomly setting more than 1000 IDs without a pattern is cumbersome in practice, and ensuring the uniqueness of more than 1000 IDs is also a problem. However, this embodiment of the invention uses the function point level information to set the function point's function point... Numbering is convenient in practice, and the level information of each function point is unique, making the function point number of each function point unique as well. Especially when adding function points, the level information of the new function point is also sequentially extended according to a certain rule. The unique function point number of the new function point can be set directly using the level information of the new function point, which has high scalability. For example, as shown in Table 2, if a function point needs to be added after "Function Point 1.1.3", the new function point can be directly named "Function Point 1.1.4" and its function point number can be set using its level information "1.1.4".

[0037] In one optional embodiment, the number of segments after the function point number is divided is equal to the level of the corresponding function point, and the i-th segment number in the function point number represents the i-th level number of the corresponding function point, where i is a positive integer.

[0038] Specifically, in conjunction with the above embodiments, the function point numbering can adopt a segmented design. When segmenting the function point number, the number of segments after segmentation is consistent with the level of the function point. For example, for the three-level function points shown in Tables 1 and 2, the corresponding function point number can be divided into three segments (if the last level function point is a two-level function point, the corresponding function point number can be divided into two segments; if the last level function point is a four-level function point, the corresponding function point number can be divided into four segments, and so on). After segmentation, the i-th segment number in the function point number can be used to represent the i-th level number in the corresponding level of the function point. For example, for the three-level function points shown in Tables 1 and 2, the first segment number in the function point number represents the first level number of the three-level function point, the second segment number represents the second level number of the three-level function point, and the third segment number represents the third level number of the three-level function point.

[0039] For example, for the three-level function points shown in Tables 1 and 2, the corresponding function point number can be named Number[31:0]. The total bit width of this number is 32 bits, which can be divided into three segments: Number[31:24], Number[23:16], and Number[15:0]. The bit width of each segment is 8 bits, 8 bits, and 16 bits, respectively. (The total bit width of Number and the bit width of each segment can be determined by the level and level number of the corresponding function point. For example, if there are many function points and the level and level number are large, the bit width will be larger, and vice versa.) Accordingly, Number[31:24] represents the first-level number of the three-level function point. Number[23:16] represents the second-level number of the third-level function point, and Number[15:0] represents the third-level number of the third-level function point. For example, the third-level function point "Function Point 2.1.3" shown in Tables 1 and 2 has a first-level number of "2", a second-level number of "1", and a third-level number of "3". Then Number[31:24] = "2", Number[23:16] = "1", and Number[15:0] = "3". Furthermore, if the data is represented in hexadecimal format, then Number[31:0] = 0x02010003. Other data formats can also be used, and this embodiment of the invention does not impose specific limitations.

[0040] Based on the above function point numbering method, the function point numbers of the 18 third-level function points shown in Tables 1 and 2 are as follows: 0x01010001, 0x01010002, 0x01010003, 0x01020001, 0x01020002, 0x01020003, 0x01030001, 0x01030002, 0x01030003, 0x02010001, 0x02010002, 0x02010003, 0x02020001, 0x02020002, 0x02020003, 0x02030001, 0x02030002, 0x02030003.

[0041] In one optional embodiment, the step of sequentially assigning the predefined global variable to the function point number of each function point covered by the test case specifically includes: When a test case covers only one function point, the predefined global variable is assigned the function point number of the function point covered by the test case; When the same test case covers more than one function point, the predefined global variable is assigned the function point number of each function point covered by the test case in sequence.

[0042] Specifically, in conjunction with the above embodiments, after each test case of the digital circuit to be verified is executed, when assigning the function point number of the function point covered by each test case to the predefined global variable F_Num in sequence, for any given test case, it can be first determined whether the same test case covers only one function point. If the same test case covers only one function point, the predefined global variable F_Num can be assigned the function point number of the function point covered by the test case. If the same test case covers more than one function point, the predefined global variable F_Num can be assigned the function point number of each function point covered by the test case in sequence (the global variable F_Num is assigned the same number of times as the number of function points covered).

[0043] For example, for the three-level function points shown in Table 2, a global variable F_Num can be pre-declared in the verification environment. Its bit width is the same as the total bit width of the function point number Number[31:0] of the three-level function points, and its default value is 0 (the default value of F_Num only needs to not coincide with the value of any function point number Number[31:0]). After "Test Case 1" is executed, since "Test Case 1" only covers one function point "Function Point 1.1.1", and the function point number Number[31:0] of "Function Point 1.1.1" is 0x01010001, the global variable F_Num can be assigned the value 0x01010001. After "Test Case 2" is executed, since "Test Case 2" only covers one function point "Function Point 1 Since "Function Point 1.1.2" has a function point number [31:0] = 0x01010002, the global variable F_Num can be assigned the value 0x01010002. After "Use Case 3" is executed, since "Use Case 3" covers two function points, namely "Function Point 1.1.3" and "Function Point 1.2.1", and the function point number [31:0] = 0x01010003 for "Function Point 1.1.3" and the function point number [31:0] = 0x01020001 for "Function Point 1.2.1", the global variable F_Num can be assigned the value 0x01010003 first, then the global variable F_Num can be assigned the value 0x01020001, and so on.

[0044] In one alternative embodiment, the time interval between two consecutive assignments of the global variable is greater than the sampling period of the functional coverage.

[0045] Specifically, in conjunction with the above embodiments, since the assignment result of the global variable F_Num is obtained based on the sampling of functional coverage, and only one value can be sampled at a time point when sampling the functional coverage values, it is necessary to ensure that the time interval between two adjacent assignments of the global variable F_Num is greater than the sampling period of the functional coverage. Especially for cases where the same test case covers more than one functional point, the global variable F_Num needs to be assigned more than once, resulting in a small time interval between two adjacent assignments (without the execution time consumed by executing a test case as a buffer). It is even more important to ensure that the time interval between two adjacent assignments is greater than the sampling period of the functional coverage to ensure that each assignment of the global variable F_Num is sampled and there is no omission.

[0046] For example, assuming that functional coverage is collected in each clock cycle, after the execution of "Use Case 16" shown in Table 2 is completed, the global variable F_Num can be assigned the value 0x02030002 first. After several clock cycles (the time interval only needs to be greater than the sampling period of functional coverage), the global variable F_Num can be assigned the value 0x02030003. The feature of "Use Case 16" is that it directly covers two level 3 functional points. Therefore, the global variable F_Num needs to be assigned twice. There should be a time interval between the two assignments, and the time interval should be long enough to ensure that the values ​​of the global variable F_Num are sampled in both assignments.

[0047] For example, a global variable F_Num is defined in the verification testbench (verification platform / verification environment). The pseudocode based on the SV language is as follows: Module tb; bit [31:0] F_Num; Dut dut(); Endmodule For example, "Use Case 1" covers "Function Point 1.1.1". After "Use Case 1" is executed, the global variable F_Num is assigned the value 0x01010001. The pseudocode based on the SV language is as follows: Class case 1; Step 1; / / Execution steps for example 1 Step 2; / / Execution steps for example 1 F_Num = 0x01010001; endclass For example, "Use Case 16" covers two function points, namely "Function Point 2.3.2" and "Function Point 2.3.3". After "Use Case 16" is executed, the global variable F_Num is assigned the values ​​0x02030002 and 0x02030003 respectively. The pseudocode based on SV language is as follows: Class case 16; Step 1; / / Execution steps for test case 16 Step 2; / / Execution steps for test case 16 F_Num = 0x02030002; #100ns; F_Num = 0x02030002; endclass In one optional embodiment, obtaining the back-annotation result of the test case back-annotation function point of the digital circuit to be verified based on the assignment result of the global variable specifically includes: Based on the assignment results of the global variables, obtain the number of times the function point number of each function point of the digital circuit to be verified is assigned. When there is a function point number that has been assigned zero times, the back-annotation result of the test case back-annotation function point of the digital circuit to be verified is that there is a function point that has not been marked by the test case and the test case has not been executed; wherein, the function point that has not been marked by the test case and the test case has not been executed is the function point corresponding to the function point number that has been assigned zero times.

[0048] Specifically, in conjunction with the above embodiments, when obtaining the back-annotation results of the test case back-annotation function points of the digital circuit to be verified based on the assignment results of the global variable F_Num obtained from the statistical summary, the number of assignments (the number of times sampled) of the function point number of each function point of the digital circuit to be verified can be obtained from the assignment results of the global variable F_Num obtained from the statistical summary. Based on the number of assignments, it can be determined whether the function point number of each function point has been assigned to the global variable F_Num, thereby determining whether each function point has been marked by the test case and whether the test case corresponding to each function point has been executed. If there is a function point number with a number of assignments of zero, it means that there is a function point in the back-annotation results of the test case back-annotation function points of the digital circuit to be verified that has not been marked by the test case and the test case has not been executed. This function point is the function point corresponding to the function point number with a number of assignments of zero.

[0049] Understandably, if there is no function point number that has been assigned zero times, it means that there are no function points in the back-annotation results of the test cases for the digital circuit to be verified that have not been marked by test cases and have not been executed by test cases. That is, all function points of the digital circuit to be verified have been marked by test cases and all test cases corresponding to all function points have been executed.

[0050] For example, see Figure 2 The image shown is a schematic diagram illustrating the back-annotation result of a test case back-annotation function point according to an embodiment of the present invention. Figure 2 As shown in the "Hit Count", the function point number of each function point was successfully collected by the sampling tool. There is no function point number with a value of zero. In other words, each function point was back-annotated by the test case and the test case was executed (each function point has a corresponding test case coverage), and the back-annotation rate is 100%.

[0051] For example, see Figure 3 The diagram shown is a schematic representation of the back-annotation result of a test case back-annotation function point provided in another embodiment of the present invention. Figure 3 As shown in the "Hit Count", there are function point numbers that were assigned zero times, namely function point number "Function Point 2.3.2" and function point number "Function Point 2.3.3". In other words, "Function Point 2.3.2" and "Function Point 2.3.3" were not back-annotated by test cases and the test cases were not executed (they were not covered by test cases), with a back-annotation rate of 88.89%.

[0052] Understandably, since all test cases must be actually executed to achieve a 100% back-annotation rate, the back-annotation results are highly objective and verifiable. Using the back-annotation results, project managers can very intuitively review the actual completion of the verification engineers' work.

[0053] It should be noted that existing software testing solutions also include the practice of "marking the state of verified functional points as covered during the execution of test cases." However, this software testing solution cannot be directly applied to the working scenario of this invention. The working scenario of this invention is a digital circuit simulation scenario. In digital circuit simulation, the industry commonly uses VCS (compiled Verilog simulator). VCS cannot achieve the "marking the state of verified functional points as covered during the execution of test cases" described in existing software testing solutions. This is because: digital circuit simulation is a clock-cycle-based hardware behavior simulation, while software testing is a sequential program execution. The two have completely different operating mechanisms. Software testing, based on sequential program execution, can directly modify the coverage state of functional points during execution, and the tool naturally supports logical marking. Digital circuit simulation, on the other hand, is a clock-driven hardware behavior simulation, which can only identify hardware signals and must be sampled according to the clock cycle. It cannot arbitrarily write the state at any time, and the functional coverage mechanism can only sample values ​​and cannot directly mark them. Therefore, hardware simulators like VCS do not support dynamically marking the functional point coverage state.

[0054] Furthermore, existing software testing solutions are typically based on a unified verification platform / environment developed using web pages. This allows all test cases to be executed together in the same verification environment, and then the functional coverage can be calculated together. However, in digital circuit simulation, different verification engineers may collaborate to complete the simulation of the entire digital circuit. For example, it is common for engineer A to develop test cases for the first half of a digital circuit and execute them using verification environment A, while engineer B develops test cases for the second half and executes them using verification environment B. However, the test cases in verification environments A and B cannot be simulated together; each can only execute its own test cases. In this case, if existing software testing solutions are used, it is clear that it is impossible to execute all test cases together in the same verification environment, and therefore, it is impossible to obtain a global functional coverage. Thus, existing software testing solutions are not suitable for digital circuit simulation scenarios.

[0055] For digital circuit simulation scenarios, this embodiment of the invention defines a global variable F_Num in the testbench that can be accessed by all test cases. After the test cases are executed, the global variable F_Num is assigned the function point number of the function points covered by the test cases. The back-annotation results of the back-annotated function points of the test cases are obtained by sampling the function coverage. This method of function point numbering + global variable assignment + coverage sampling can be implemented using the industry-standard VCS software. Even if the test cases in verification environment A and verification environment B cannot be simulated together, a coverage file A can be generated after executing the corresponding test cases in verification environment A, and a coverage file B can be generated after executing the corresponding test cases in verification environment B. Then, the VCS software is used to integrate coverage files A and B to obtain a global coverage file, thereby obtaining a global back-annotation result.

[0056] This invention also provides an apparatus for back-annotating test cases, used to implement the method for back-annotating test cases as described in any of the above embodiments. See [link to apparatus]. Figure 4 The diagram shown is a structural block diagram of a device for back-annotating function points in test cases according to an embodiment of the present invention. The device includes: The global variable assignment module 11 is used to assign the function point number of the function point covered by each test case of the digital circuit to be verified to the predefined global variable in sequence after the execution of each test case of the digital circuit to be verified; wherein, each function point of the digital circuit to be verified is set with a function point number. The assignment result statistics module 12 is used to sample and statistically analyze the assignment results of the global variables during the sampling function coverage process. The back-annotation result acquisition module 13 is used to acquire the back-annotation result of the test case back-annotation function point of the digital circuit to be verified based on the assignment result of the global variable; wherein, the back-annotation result is used to indicate whether each function point is marked by the test case and whether the test case is executed.

[0057] Preferably, the function point number is associated with the level of the function point, wherein the total bit width, the number of segments after segmentation, and the bit width of each segment of the function point number are determined by the level and level number of the corresponding function point.

[0058] Preferably, the number of segments after the function point number is divided is equal to the level of the corresponding function point, and the i-th segment number in the function point number represents the i-th level number of the corresponding function point, where i is a positive integer.

[0059] Preferably, the global variable assignment module 11 assigns predefined global variables sequentially to the function point numbers of the function points covered by each test case, specifically including: When a test case covers only one function point, the predefined global variable is assigned the function point number of the function point covered by the test case; When the same test case covers more than one function point, the predefined global variable is assigned the function point number of each function point covered by the test case in sequence.

[0060] Preferably, the time interval between two consecutive assignments of the global variable is greater than the sampling period of the functional coverage.

[0061] Preferably, the back-annotation result acquisition module 13 specifically includes: The assignment count acquisition unit is used to obtain the assignment count of the function point number of each function point of the digital circuit to be verified based on the assignment result of the global variable. The back-annotation result acquisition unit is used to obtain the back-annotation result of the test case back-annotation function point of the digital circuit to be verified when there is a function point number with zero assignment count. The result is that there is a function point that has not been marked by the test case and the test case has not been executed. The function point that has not been marked by the test case and the test case has not been executed is the function point corresponding to the function point number with zero assignment count.

[0062] It should be noted that the apparatus for back-annotating test cases provided in this embodiment of the invention can implement all the processes of the method for back-annotating test cases described in any of the above embodiments. The functions and technical effects of each module and unit in the apparatus are the same as the functions and technical effects of the method for back-annotating test cases described in the above embodiments, and will not be repeated here.

[0063] This invention also provides a computer-readable storage medium including a stored computer program, which, when running, controls the device where the computer-readable storage medium is located to execute the method for back-annotating function points of test cases as described in any of the above embodiments.

[0064] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method for back-annotating test cases as described in any of the above embodiments.

[0065] This invention also provides a terminal device, see [link to relevant documentation]. Figure 5 The diagram shown is a structural block diagram of a terminal device according to an embodiment of the present invention. The terminal device includes a processor 10, a memory 20, and a computer program stored in the memory 20 and configured to be executed by the processor 10. When the processor 10 executes the computer program, it implements the method for back-labeling test cases as described in any of the above embodiments.

[0066] Preferably, the computer program can be divided into one or more modules / units (such as computer program 1, computer program 2, ...), and the one or more modules / units are stored in the memory 20 and executed by the processor 10 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0067] The processor 10 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor 10 may be any conventional processor. The processor 10 is the control center of the terminal device, connecting various parts of the terminal device through various interfaces and lines.

[0068] The memory 20 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc., while the data storage area can store related data, etc. Furthermore, the memory 20 can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard drive, a smart media card (SMC), a secure digital card (SD), and a flash card, or other volatile solid-state storage devices.

[0069] It should be noted that the aforementioned terminal devices may include, but are not limited to, processors and memory, as will be understood by those skilled in the art. Figure 5 The structural block diagram shown is merely a structural example of the terminal device described above and does not constitute a limitation on the structure of the terminal device. The terminal device may include more or fewer components than shown, or combine certain components, or use different components.

[0070] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for back-annotating functional points in test cases, characterized in that, include: After each test case of the digital circuit to be verified is executed, the predefined global variable is sequentially assigned the function point number of the function point covered by each test case; wherein, each function point of the digital circuit to be verified is set with a function point number. During the sampling coverage process, the assignment results of the global variables are sampled and statistically analyzed; The back-annotation result of the test case back-annotation function point of the digital circuit to be verified is obtained according to the assignment result of the global variable; wherein, the back-annotation result is used to indicate whether each function point is marked by the test case and whether the test case is executed.

2. The method for back-annotating function points in test cases as described in claim 1, characterized in that, The function point number is associated with the level of the function point, wherein the total bit width of the function point number, the number of segments after segmentation, and the bit width of each segment are determined by the level and level number of the corresponding function point.

3. The method for back-annotating function points in test cases as described in claim 2, characterized in that, The number of segments after the function point number is divided is equal to the level of the corresponding function point, and the i-th segment number in the function point number represents the i-th level number of the corresponding function point, where i is a positive integer.

4. The method for back-annotating function points in test cases as described in claim 1, characterized in that, The step of sequentially assigning predefined global variables to the function point numbers of the function points covered by each test case specifically includes: When a test case covers only one function point, the predefined global variable is assigned the function point number of the function point covered by the test case; When the same test case covers more than one function point, the predefined global variable is assigned the function point number of each function point covered by the test case in sequence.

5. The method for back-annotating function points in test cases as described in claim 4, characterized in that, The time interval between two consecutive assignments of the global variable is greater than the sampling period of the functional coverage.

6. The method for back-annotating function points in test cases as described in claim 1, characterized in that, The step of obtaining the back-annotation result of the test case back-annotation function point of the digital circuit to be verified based on the assignment result of the global variable specifically includes: Based on the assignment results of the global variables, obtain the number of times the function point number of each function point of the digital circuit to be verified is assigned. When there is a function point number that has been assigned zero times, the back-annotation result of the test case back-annotation function point of the digital circuit to be verified is that there is a function point that has not been marked by the test case and the test case has not been executed; wherein, the function point that has not been marked by the test case and the test case has not been executed is the function point corresponding to the function point number that has been assigned zero times.

7. A device for back-annotating function points in test cases, characterized in that, include: The global variable assignment module is used to assign the function point number of each function point covered by the test case to the predefined global variable after the execution of each test case of the digital circuit to be verified; wherein, each function point of the digital circuit to be verified is set with a function point number. The assignment result statistics module is used to sample and statistically analyze the assignment results of the global variables during the sampling function coverage process. The back-annotation result acquisition module is used to obtain the back-annotation result of the test case back-annotation function point of the digital circuit to be verified based on the assignment result of the global variable; wherein, the back-annotation result is used to indicate whether each function point is marked by the test case and whether the test case is executed.

8. A computer-readable storage medium, characterized in that, The system includes a stored computer program that, when run, controls the device containing the computer-readable storage medium to perform a method for back-labeling function points of test cases as described in any one of claims 1 to 6.

9. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method for back-annotating function points of test cases as described in any one of claims 1 to 6.

10. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the method for back-labeling function points of test cases as described in any one of claims 1 to 6.

Images