Method of marking a coverage exclusion point

Abstract

The application relates to a method for marking a coverage exemption point, comprising the following steps: S1, generating a coverage database based on preset test cases, wherein the coverage database comprises coverage points of all source codes, at least one piece of coverage exemption condition information is generated according to a preset data structure, the preset data structure comprises a condition id data segment, an application object data segment, a selection condition data segment and a verification code data segment, and the initial value of the verification code data segment is empty; S2, determining a coverage exemption point based on the source code, the coverage database and all coverage exemption condition information; and S3, generating a corresponding verification code based on upper-layer code corresponding to each coverage exemption point, updating the verification code into corresponding coverage exemption condition information, and generating coverage exemption information corresponding to the coverage exemption point. The application improves the efficiency of marking and maintaining the coverage exemption point.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method for marking overriding exemption points. Background Technology

[0002] In software testing and chip verification, coverage is typically used to measure whether testing and verification are sufficient. However, software and chip designs often contain test points that cannot be tested or verified, or that do not require testing or verification; these test points are called coverage exemptions. To achieve 100% coverage, developers need to review the source code and identify these coverage exemptions.

[0003] In software and chip design, as designs become larger, the number of coverage exemption points that need to be marked increases. Developers have to spend a lot of time analyzing and finding each coverage exemption point, reviewing these exemption points, and spending even more time reviewing and changing them when the design changes. The efficiency of marking and maintaining coverage exemption points is low. Therefore, how to improve the efficiency of marking and maintaining coverage exemption points has become an urgent technical problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a method for marking covered exemption points, which improves the efficiency of marking and maintaining covered exemption points.

[0005] This invention provides a method for marking covered exemption points, comprising:

[0006] Step S1: Based on preset test cases, generate a coverage database from the source code. The coverage database includes all coverage points of the source code. Generate at least one coverage exemption condition according to a preset data structure.

[0007] The preset data structure includes a condition ID data segment, an application object data segment, a selection condition data segment, and a verification code data segment, with the initial value of the verification code data segment being empty;

[0008] Step S2: Determine the coverage exemption points based on the source code, the coverage database, and all coverage exemption condition information;

[0009] Step S3: Generate a corresponding verification code based on the upper-level code corresponding to each coverage exemption point. The upper-level code is all the code that can affect the coverage exemption point. Update the verification code to the corresponding coverage exemption condition information and generate the coverage exemption information corresponding to the coverage exemption point.

[0010] Compared with existing technologies, this invention has significant advantages and beneficial effects. Through the above technical solution, the method for marking exemption points provided by this invention achieves considerable technological advancement and practicality, and has broad industrial application value. It possesses at least the following advantages:

[0011] This invention generates coverage exemption condition information through a preset data structure. Based on a single coverage exemption condition information, coverage exemption points with common characteristics can be batch-marked, facilitating batch marking and modification, thus improving the efficiency of marking and maintaining coverage exemption points. Furthermore, after source code updates, multiple coverage exemption points referenced in the original code can be compared based on a simple CAPTCHA, further improving the efficiency of marking and maintaining coverage exemption points.

[0012] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0013] Figure 1 A flowchart illustrating the method for marking coverage exemption points provided in this embodiment of the invention;

[0014] Figure 2 This is a schematic diagram of the design code before the update provided in an embodiment of the present invention;

[0015] Figure 3 This is a schematic diagram of the updated design code provided for an embodiment of the present invention. Detailed Implementation

[0016] To further illustrate the technical means and effects of the present invention in achieving the intended purpose, the following detailed description, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation and effects of a marking method for covering exemption points proposed according to the present invention.

[0017] This invention provides a method for marking covered exemption points, such as... Figure 1 As shown, it includes:

[0018] Step S1: Based on the preset test cases, the source code is executed to generate a coverage database. The coverage database includes all coverage points of the source code. At least one coverage exemption condition information is generated according to the preset data structure. The preset data structure includes a rule ID data segment, an applying targets data segment, selection conditions (expressions) data segment, and a checksum data segment. The initial value of the checksum data segment is empty.

[0019] The preset data structure is represented as follows:

[0020] {Rule ID}:{{{expressions}{applying targets}

[0021] {checksum…}}}

[0022] It should be noted that the method described in this invention is applicable to application scenarios such as software testing and chip verification. When performing software testing and chip verification, a large number of test cases are typically written to execute the source code, obtain all coverage information from the source code, and generate a corresponding coverage database. It is understood that generating a coverage database based on pre-defined test cases can be directly implemented using existing technologies, and will not be described in detail here. The condition ID data segment is used to identify coverage exemption condition information; the application object data segment and the selection condition data segment are used to select coverage exemption points based on the source code and coverage database; and the verification code data segment is used to store verification codes, which are used to reuse the coverage exemption points corresponding to the source code before the update after subsequent source code updates.

[0023] Step S2: Determine the coverage exemption points based on the source code, the coverage database, and all coverage exemption condition information;

[0024] Step S3: Generate a corresponding verification code based on the upper-level code corresponding to each coverage exemption point. The upper-level code is all the code that can affect the coverage exemption point. Update the verification code to the corresponding coverage exemption condition information and generate the coverage exemption information corresponding to the coverage exemption point.

[0025] It should be noted that once the exemption point is determined, an exemption point information file can be generated, or the corresponding exemption point location can be marked, etc. This invention does not limit this.

[0026] The coverage of testing or verification is calculated directly based on existing technologies, which will not be elaborated here.

[0027] As one embodiment, the application object data segment includes an application scope data segment. This application scope data segment is generated based on one of the following: design hierarchy tree ID, basic module ID, module instance ID, and source code file ID, or a combination of at least two of them. It should be noted that the ID mentioned in this invention refers to uniquely identifiable information, not limited to ID numbers, but also including uniquely identifiable information such as names. In step S2, a first candidate coverage point range is determined from all coverage points based on the application scope data segment.

[0028] As one embodiment, the selection condition data segment is generated based on one or a combination of at least two of logical expressions, regular expressions, and wildcards, such as "*". In step S2, a second candidate coverage point range is determined from the first candidate coverage point range based on the selection condition data segment. It is understood that the second candidate coverage point range is a subset of the first candidate coverage point range.

[0029] As one embodiment, the preset data structure further includes an action type data segment, which is set to either an exclude flag or an include flag. However, it is understood that the action type data segment may not be set, or it may be set to exclude by default.

[0030] When an action data segment is included, the default data structure is as follows:

[0031] {RuleID}:{{{action}{expressions}{applying targets}

[0032] {checksum…}}}

[0033] As one embodiment, the preset data structure further includes a coverage type data segment. This coverage type data segment is set separately or included in the application object data segment or the selection condition data segment. The application object data segment also includes the coverage type data segment, which may be empty or include at least one target exemption type. Taking a chip verification application scenario as an example, target exemption types (Metrics) include lines of code, conditions, toggle transitions, branches, and finite state machines (FSMs). Taking a software testing scenario as an example, target exemption types mainly include lines of code, but may also include branches and finite state machines (FSMs).

[0034] Based on the combination of different data segments in the data structure, different strategies for selecting and covering exemption points can be set. The following examples illustrate this:

[0035] Example 1

[0036] When the coverage type data segment is empty, in step S2, all coverage points corresponding to all exemption types within the range of the second candidate coverage point are determined as coverage exemption points. That is, when the coverage type data segment is empty, all exemption types are taken as target exemption types by default, without the need to list them one by one.

[0037] In Embodiment 1, the preset data structure may further include an action type data segment, which is set to remove the identifier. However, it is understood that the action type data segment may not be included.

[0038] Example 2

[0039] When the coverage type data segment includes at least one target exemption type, in step S2, all coverage points corresponding to all target exemption types within the range of the second candidate coverage points are determined as coverage exemption points. That is, the user specifies the target exemption type through the coverage type data segment.

[0040] In Embodiment 2, the preset data structure may further include an action type data segment, which is set to remove the identifier. However, it is understood that the action type data segment may not be included.

[0041] Example 3

[0042] The data structure includes a condition ID data segment, a first data level, a second data level, and a verification code data segment. The first data level includes a first action type data segment, a first application object data segment, and a first selection condition data segment. The second data level includes a second action type data segment, a second application object data segment, and a second selection condition data segment. The first action type data segment is set as a removal identifier, and the second action type data segment is set as a add-back identifier.

[0043] When the overlay type data segment is empty:

[0044] In step S2: all coverage points corresponding to all exemption types within the range of the second candidate coverage points determined based on the first application object data segment and the first selection condition data segment are determined as candidate coverage exemption points; all coverage points corresponding to all exemption types within the range of the second candidate coverage points determined based on the second application object data segment and the second selection condition data segment are determined as added-back coverage points; and the coverage points after removing the added-back coverage points from the candidate coverage exemption points are determined as coverage exemption points.

[0045] Example 3 is applicable to scenarios that require setting coverage exemption points over a large area continuously. By setting two levels of data, all coverage points over a large area are first identified as candidate coverage exemption points, and then non-coverage exemption points are removed from them, which improves the efficiency of obtaining coverage exemption points.

[0046] Example 4

[0047] The data structure includes a condition ID data segment, a first data level, a second data level, and a verification code data segment. The first data level includes a first action type data segment, a first application object data segment, and a first selection condition data segment. The second data level includes a second action type data segment, a second application object data segment, and a second selection condition data segment. The first action type data segment is set as a removal identifier, and the second action type data segment is set as a add-back identifier.

[0048] When the coverage type data segment includes at least one target exemption type:

[0049] Step S2 includes: determining all coverage points corresponding to all target exemption types within the range of the second candidate coverage points determined based on the first application object data segment and the first selection condition data segment as coverage exemption points; determining all coverage points corresponding to all target exemption types within the range of the second candidate coverage points determined based on the second application object data segment and the second selection condition data segment as added-back coverage points; and determining the coverage points after removing the added-back coverage points from the candidate coverage exemption points as coverage exemption points.

[0050] Example 4 is also applicable to scenarios that require setting coverage exemption points over a large area continuously. By setting two levels of data, all coverage points over a large area are first identified as candidate coverage exemption points, and then non-coverage exemption points are removed from them, which improves the efficiency of obtaining coverage exemption points.

[0051] The following simplified example illustrates how the removal flag at the first data level and the addition flag at the second data level are used together to indicate exemption coverage points:

[0052] Suppose we need to exempt the entire foo module, leaving only lines 100 to 110. We can remove the entire foo module through the first data level (exclude) and add back lines 100 to 110 through the second data level (include). This achieves the goal of covering the exemption condition information and avoids marking all lines in the foo module that need to be exempted.

[0053] {Rule1}:{{{exclude}{module foo}}

[0054] {{include}{line>=100&&line<110}{module foo}}

[0055] To further illustrate the method for identifying coverage exemption points, a specific example is provided below. Suppose we need to identify lines 874-887 in the design hierarchy tree: top.sub1.*, corresponding to the metrics (toggle, line, FSM, branch). The generated coverage exemption condition information is as follows:

[0056] {Rule2}: {{exclude}{line>=874&&line<887}{{metrics:line,condition,FSM,branch}{instances:top.sub1.*}}{checksum…}} Based on the above coverage exemption condition information, if it is necessary to identify the range of coverage exemption points from lines 874 to 887, assuming this code has 20 coverage exemption points (assuming it includes 12 Line type coverage exemption points, 3 Branch type coverage exemption points, 2 Condition type coverage exemption points, and 3 FSM type coverage exemption points), assuming there are 100 instances of this module in the top.sub1.* tree, and a total of 20*100 coverage points, it would take 2000 lines to identify the coverage exemption points using the traditional method. This embodiment of the invention identifies and manages these 2000 scattered coverage points through this one coverage exemption condition information, which greatly simplifies the cost of coverage exemption point identification and maintenance.

[0057] In applications such as software design and chip verification, source code is typically adjusted based on various factors. However, in most cases, these adjustments are small-scale. Existing technologies require re-representing the overridden exemption points after each source code adjustment, resulting in extremely low efficiency. Therefore, this invention further proposes that when the source code is updated, the method includes the following steps:

[0058] Step S10: Obtain the original overwrite exemption information corresponding to each source code before the update;

[0059] Step S20: Obtain the updated upper-level code corresponding to each original coverage exemption point in the updated source code, and update the verification code based on the updated upper-level code;

[0060] Step S30: Compare the original verification code and the updated verification code corresponding to each original coverage exemption. If they match, the coverage point corresponding to the original coverage exemption point in the updated source code is determined as the updated coverage exemption point. If they do not match, a prompt message is generated based on the original coverage exemption point.

[0061] It should be noted that the verification code can be directly calculated using a message digest algorithm (such as MD5). As those skilled in the art can understand, the updating and comparison of the verification code can be completed quickly. Therefore, through steps S10-S30, most of the corresponding coverage exemption information in the updated source code can be quickly determined based on the original coverage exemption information, which greatly improves the efficiency of identifying the coverage exemption points in the updated source code.

[0062] The following specific embodiment further illustrates steps S10-S30, such as... Figure 2 The original design code shown is shown below. Figure 3 The updated design code is shown, with line 102 marked as the coverage exemption point. The corresponding CAPTCHA for line 102 is recorded in the corresponding coverage exemption condition information. When this coverage exemption condition information is applied to the new design code, the CAPTCHA from the original line 102 is compared with the CAPTCHA from line 102 in the new design. Due to the change in line 100, the CAPTCHA does not match, indicating that the change in the new design has affected the accuracy of the coverage exemption condition information. Therefore, it cannot be applied directly and needs to be re-examined. In this embodiment, the original line 102 coverage exemption point needs to be changed to the corresponding line 106 coverage exemption point.

[0063] This invention generates coverage exemption condition information through a preset data structure. Based on a single coverage exemption condition information, coverage exemption points with common characteristics can be batch-marked, facilitating batch marking and modification, thus improving the efficiency of marking and maintaining coverage exemption points. Furthermore, after source code updates, multiple coverage exemption points referenced in the original code can be compared based on a simple CAPTCHA, further improving the efficiency of marking and maintaining coverage exemption points.

[0064] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0065] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0066] This invention also provides a computer-readable storage medium, wherein the computer instructions are used to execute the methods described in this invention.

[0067] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for marking covered exemption points, characterized in that, include: Step S1: Based on preset test cases, generate a coverage database from the source code. The coverage database includes all coverage points of the source code. Generate at least one coverage exemption condition according to a preset data structure. The preset data structure includes a condition ID data segment, an application object data segment, a selection condition data segment, and a verification code data segment, with the initial value of the verification code data segment being empty; Step S2: Determine the coverage exemption points based on the source code, the coverage database, and all coverage exemption condition information; Step S3: Generate a corresponding verification code based on the upper-level code corresponding to each coverage exemption point. The upper-level code is all the code that can affect the coverage exemption point. Update the verification code to the corresponding coverage exemption condition information and generate the coverage exemption information corresponding to the coverage exemption point. When the source code is updated, the method includes the following steps: Step S10: Obtain the original overwrite exemption information corresponding to each source code before the update; Step S20: Obtain the updated upper-level code corresponding to each original coverage exemption point in the updated source code, and update the verification code based on the updated upper-level code; Step S30: Compare the original verification code and the updated verification code corresponding to each original coverage exemption point. If they match, the coverage point corresponding to the original coverage exemption point in the updated source code is determined as the updated coverage exemption point. If they do not match, a prompt message is generated based on the original coverage exemption point.

2. The method according to claim 1, characterized in that, The application object data segment includes an application scope data segment, which is generated based on one of the design hierarchy tree ID, basic module ID, module instance ID, and source code file ID, or a combination of at least two of them. In step S2, the first candidate coverage point range is determined from all coverage points based on the application scope data segment.

3. The method according to claim 2, characterized in that, The selection condition data segment is generated based on one of logical expressions, regular expressions, and wildcards, or a combination of at least two of them. In step S2, a second candidate coverage point range is determined from the first candidate coverage point range based on the selection condition data segment.

4. The method according to claim 3, characterized in that, The preset data structure also includes an overlay type data segment, which is set separately or in the application object data segment or the selection condition data segment. The application object data segment also includes an overlay type data segment, which is either empty or includes at least one target exemption type.

5. The method according to claim 4, characterized in that, When the coverage type data segment is empty, in step S2, all coverage points corresponding to all exemption types within the range of the second candidate coverage point are determined as coverage exemption points.

6. The method according to claim 4, characterized in that, When the coverage type data segment includes at least one target exemption type, in step S2, all coverage points corresponding to all target exemption types within the range of the second candidate coverage point are determined as coverage exemption points.

7. The method according to claim 4, characterized in that, The data structure includes a condition ID data segment, a first data level, a second data level, and a verification code data segment. The first data level includes a first action type data segment, a first application object data segment, and a first selection condition data segment. The second data level includes a second action type data segment, a second application object data segment, and a second selection condition data segment. The first action type data segment is set as a removal identifier, and the second action type data segment is set as a add-back identifier.

8. The method according to claim 7, characterized in that, When the overlay type data segment is empty: In step S2: all coverage points corresponding to all exemption types within the range of the second candidate coverage points determined based on the first application object data segment and the first selection condition data segment are determined as candidate coverage exemption points; all coverage points corresponding to all exemption types within the range of the second candidate coverage points determined based on the second application object data segment and the second selection condition data segment are determined as added-back coverage points; and the coverage points after removing the added-back coverage points from the candidate coverage exemption points are determined as coverage exemption points.

9. The method according to claim 7, characterized in that, When the coverage type data segment includes at least one target exemption type: Step S2 includes: determining all coverage points corresponding to all target exemption types within the range of the second candidate coverage points determined based on the first application object data segment and the first selection condition data segment as coverage exemption points; determining all coverage points corresponding to all target exemption types within the range of the second candidate coverage points determined based on the second application object data segment and the second selection condition data segment as added-back coverage points; and determining the coverage points after removing the added-back coverage points from the candidate coverage exemption points as coverage exemption points.

Images