A non-intrusive coverage collection system

Abstract

The application relates to the chip verification technical field, in particular to a non-intrusive coverage collection system, compared with the adding mark code mode in the prior art, the system collects the coverage of firmware code execution in a non-intrusive mode, does not modify the firmware code, guarantees the verification efficiency of the firmware code, various types of code segments in the firmware code each correspond to at least one program pointer, through the mode of recording the program pointer, the execution of all the code segments in the firmware code can be guaranteed to be completely covered, and the verification completeness of the firmware code is improved.

Description

Technical Field

[0001] This invention relates to the field of chip verification technology, and in particular to a non-invasive coverage collection system. Background Technology

[0002] Currently, repairing firmware programs in chips is quite difficult. When firmware programs are stored in the flash memory of the chip, repairing the firmware programs requires re-burning the flash memory. When firmware programs are solidified in the chip, it may even be necessary to re-fabricate the chip. Therefore, verification of firmware programs in chips has become particularly important.

[0003] In the prior art, an intrusive coverage collection method is proposed. Specifically, a marker code corresponding to a code segment is added to the firmware program. The marker code is used to write the corresponding code line number into the memory, so that after a certain code segment is executed, the marker code corresponding to that code segment will be automatically executed. The recording results of the marker code are used to determine whether each code segment has been executed, thereby completing the coverage collection of the firmware program code.

[0004] However, this method requires modification of the firmware program code. The marker code is essentially redundant code that the firmware program does not need. The intrusive approach will reduce the efficiency of firmware program code verification. Moreover, in some scenarios, the marker code cannot completely represent the execution of firmware code segments. For example, when a firmware code segment is executed when any one of multiple trigger conditions is met, the marker code can record that the firmware code segment is executed, but it cannot guarantee that it covers the scenario where each trigger condition is met individually, resulting in poor verification completeness of the trigger code of the firmware code segment.

[0005] Therefore, improving verification efficiency and completeness has become an urgent problem to be solved. Summary of the Invention

[0006] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0007] A non-intrusive coverage collection system includes a processor and a memory storing a computer program, which, when executed by the processor, performs the following steps:

[0008] S101 executes the set of test cases corresponding to the firmware in the chip. By recording the program pointer during the execution process, it obtains several executed assembly instruction identifiers.

[0009] S102, sort and deduplicate all executed assembly instruction identifiers to obtain the target assembly instruction identifier sequence.

[0010] S103, determine the initial coverage rate based on the target assembly instruction identifier sequence and the preset reference assembly instruction identifier sequence corresponding to the firmware.

[0011] S104, when the initial coverage does not meet the first preset condition, determine a number of unexecuted reference assembly instruction identifiers based on the target assembly instruction identifier sequence and the reference assembly instruction identifier sequence.

[0012] S105, determine several unexecuted firmware code segments from the code corresponding to the firmware based on the identifiers of each unexecuted reference assembly instruction.

[0013] S106, After analyzing each unexecuted firmware code segment, the user sets the masking status of each unexecuted firmware code segment, wherein the masking status includes a masked status and an unmasked status.

[0014] S107, determine the target coverage rate based on the target assembly instruction identifier sequence, the reference assembly instruction identifier sequence, and the reference assembly instruction identifiers corresponding to each unexecuted firmware code segment in the masked state.

[0015] Compared with the prior art, the present invention has significant advantages. Through the above technical solution, the non-invasive coverage collection system provided by the present invention achieves considerable technological progress and practicality, and has broad industrial application value. It has at least the following advantages:

[0016] Compared with the existing technology of adding marked code, the method of collecting firmware code execution coverage in a non-intrusive manner does not modify the firmware code, thus ensuring the verification efficiency of the firmware code. Each type of code segment in the firmware code corresponds to at least one program pointer. By recording the program pointer, it can be ensured that the execution of all code segments in the firmware code is completely covered, thereby improving the verification completeness of the firmware code. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the execution of a computer program by a processor in a non-intrusive coverage collection system provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This embodiment provides a non-invasive coverage collection system, see [link to documentation]. Figure 1 This is a flowchart illustrating the execution of a computer program by a processor in a non-intrusive coverage collection system according to an embodiment of the present invention. The system includes a processor and a memory storing the computer program. When the computer program is executed by the processor, the following steps are implemented:

[0021] S101 executes the set of test cases corresponding to the firmware in the chip, and obtains several executed assembly instruction identifiers by recording the program pointer during the execution process;

[0022] S102, sort and deduplicate all executed assembly instruction identifiers to obtain the target assembly instruction identifier sequence;

[0023] S103, determine the initial coverage rate based on the target assembly instruction identifier sequence and the preset reference assembly instruction identifier sequence corresponding to the firmware;

[0024] S104, when the initial coverage does not meet the first preset condition, determine a number of unexecuted reference assembly instruction identifiers based on the target assembly instruction identifier sequence and the reference assembly instruction identifier sequence;

[0025] S105, determine several unexecuted firmware code segments from the code corresponding to the firmware based on each unexecuted reference assembly instruction identifier;

[0026] S106, After the user analyzes each unexecuted firmware code segment, the user sets the blocking status of each unexecuted firmware code segment, wherein the blocking status includes a blocked status and an unblocked status.

[0027] S107, determine the target coverage rate based on the target assembly instruction identifier sequence, the reference assembly instruction identifier sequence, and the reference assembly instruction identifiers corresponding to each unexecuted firmware code segment in the masked state.

[0028] The firmware in the chip can be either embedded in the chip as bootcode or stored in the chip's hardware as firmware, typically in flash memory.

[0029] Firmware is typically written in C and compiled into machine code by a compiler. The machine code contains multiple instructions corresponding to the firmware, and each instruction corresponds to a program pointer. By recording the program pointer, the executed instructions can be recorded.

[0030] The test case set can include several test cases for testing the firmware program. When a test case is executed, certain code segments in the firmware program will be executed. Each code segment in the firmware program corresponds to at least one program pointer, and a single program pointer corresponds to only one code segment.

[0031] Initial coverage can characterize the verification coverage before user processing, while target coverage can characterize the verification coverage after user masking.

[0032] Specifically, sorting all executed assembly instruction identifiers can mean sorting the executed assembly instruction identifiers in ascending order to obtain an initial sorting result, and then deduplicating the initial sorting result to obtain the target assembly instruction identifier sequence.

[0033] In one specific implementation, each code segment in the firmware corresponds to at least one program pointer, and the program pointers corresponding to different code segments are different.

[0034] In this embodiment, a single code segment can correspond to multiple program pointers, but a single program pointer corresponds to only one code segment. Accordingly, when all the program pointers corresponding to a single code segment are recorded, the code segment can be considered to have been fully executed. When only a portion of the program pointers corresponding to a single code segment are recorded, the code segment can be considered to have been partially executed. When none of the program pointers corresponding to a single code segment are recorded, the code segment can be considered to have not been executed. Since the complete execution of the code segment can be determined based on the recording of program pointers, this embodiment can ensure the completeness of firmware code verification.

[0035] In one specific implementation, the test case set corresponding to the firmware in the execution chip obtains several executed assembly instruction identifiers by recording the program pointer during execution, including:

[0036] For any test case in the test case set, execute the test case, record the program pointers during the execution of the test case, and form a record subfile;

[0037] The record subfiles corresponding to each test case are merged to obtain the record file;

[0038] Each program pointer contained in the log file is used as an identifier for the executed assembly instruction.

[0039] Each test case corresponds to a record sub-file. After merging all the record sub-files, a record file is obtained.

[0040] In one specific implementation, determining the initial coverage based on the target assembly instruction identifier sequence and the preset reference assembly instruction identifier sequence corresponding to the firmware includes:

[0041] Count the number of first instruction identifiers corresponding to the target assembly instruction identifier sequence and the number of second instruction identifiers corresponding to the reference assembly instruction identifier sequence;

[0042] The initial coverage rate is the ratio of the number of the first instruction identifiers to the number of the second instruction identifiers.

[0043] The first instruction identifier count may refer to the number of executed assembly instruction identifiers contained in the target assembly instruction identifier sequence, and the second instruction identifier count may refer to the number of reference assembly instruction identifiers contained in the reference assembly instruction identifier sequence.

[0044] The reference assembly instruction identifier sequence can be obtained directly from the source assembly file of the firmware code and used as prior information to compare with the target assembly instruction identifier sequence.

[0045] In one specific implementation, the first preset condition is that the initial coverage rate is greater than or equal to a preset coverage rate threshold.

[0046] The preset coverage threshold can be set to 1, which means that when the firmware code is verified to be complete, the extraction of unexecuted firmware code segments and user analysis processing will not be performed. Otherwise, user analysis processing is required because firmware code repair is difficult, so the completeness of the verification should be ensured during the verification process.

[0047] In one specific implementation, determining a plurality of unexecuted reference assembly instruction identifiers based on the target assembly instruction identifier sequence and the reference assembly instruction identifier sequence includes:

[0048] The intersection of each executed assembly instruction identifier in the target assembly instruction identifier sequence and each reference assembly instruction identifier in the reference assembly instruction identifier sequence is calculated to obtain the intersection calculation result.

[0049] Based on the intersection calculation result and each reference assembly instruction identifier in the reference assembly instruction identifier sequence, several unexecuted reference assembly instruction identifiers are determined.

[0050] The assembly instruction identifiers included in the intersection calculation result exist in both the target assembly instruction identifier sequence and the reference assembly instruction identifier sequence.

[0051] Specifically, the assembly instruction identifiers contained in the intersection calculation result are removed from the reference assembly instruction identifier sequence, and the remaining reference assembly instruction identifiers in the reference assembly instruction identifier sequence are the unexecuted reference assembly instruction identifiers.

[0052] In one specific implementation, determining a plurality of unexecuted firmware code segments from the code corresponding to the firmware based on each unexecuted reference assembly instruction identifier includes:

[0053] Based on the identifiers of each unexecuted reference assembly instruction, several temporary code segments are determined from the code corresponding to the firmware;

[0054] Each temporary code segment is deduplicated to obtain each of the unexecuted firmware code segments.

[0055] Since a single code segment can correspond to multiple assembly instructions, when a firmware code segment is not executed or is not fully executed, it may correspond to multiple unexecuted reference assembly instruction identifiers. Therefore, it is necessary to perform deduplication processing on temporary code segments.

[0056] Specifically, this embodiment identifies each unexecuted firmware code segment based on the identifier of each unexecuted reference assembly instruction, and leverages the readability of the firmware code to enable subsequent users to analyze and process the unexecuted firmware code segments.

[0057] In one specific implementation, determining the target coverage based on the target assembly instruction identifier sequence, the reference assembly instruction identifier sequence, and the reference assembly instruction identifiers corresponding to each unexecuted firmware code segment in a masked state includes:

[0058] Count the number of first instruction identifiers corresponding to the target assembly instruction identifier sequence and the number of second instruction identifiers corresponding to the reference assembly instruction identifier sequence;

[0059] The number of reference assembly instruction identifiers corresponding to each unexecuted firmware code segment in the masked state is counted to obtain the number of third instruction identifiers;

[0060] Add the number of the first instruction identifiers and the number of the third instruction identifiers to obtain the sum;

[0061] The target coverage rate is the ratio of the summation result to the number of the second instruction identifiers.

[0062] Users can set the masking status of each unexecuted firmware code segment according to the actual situation, and add up the number of reference assembly instruction identifiers corresponding to each unexecuted firmware code segment in the masked state to obtain the number of third instruction identifiers.

[0063] Specifically, when the target coverage does not meet the second preset condition, the user can be prompted to construct test cases to ensure the completeness of firmware code verification. The second preset condition can be that the target coverage is greater than or equal to a preset coverage threshold.

[0064] In this embodiment, compared with the existing technology of adding marked code, the firmware code execution coverage is collected in a non-intrusive manner without modifying the firmware code, which ensures the verification efficiency of the firmware code. Each type of code segment in the firmware code corresponds to at least one program pointer. By recording the program pointer, it can be ensured that the execution of all code segments in the firmware code is completely covered, thereby improving the verification completeness of the firmware code.

[0065] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of this invention is defined by the appended claims.

Claims

1. A non-invasive coverage collection system, characterized in that, The system includes a processor and a memory storing a computer program, which, when executed by the processor, performs the following steps: S101 executes the set of test cases corresponding to the firmware in the chip, and obtains several executed assembly instruction identifiers by recording the program pointer during the execution process; S102, sort and deduplicate all executed assembly instruction identifiers to obtain the target assembly instruction identifier sequence; S103, determine the initial coverage rate based on the target assembly instruction identifier sequence and the preset reference assembly instruction identifier sequence corresponding to the firmware; S104, when the initial coverage does not meet the first preset condition, determine a number of unexecuted reference assembly instruction identifiers based on the target assembly instruction identifier sequence and the reference assembly instruction identifier sequence; S105, determine several unexecuted firmware code segments from the code corresponding to the firmware based on each unexecuted reference assembly instruction identifier; S106, After the user analyzes each unexecuted firmware code segment, the user sets the blocking status of each unexecuted firmware code segment, wherein the blocking status includes a blocked status and an unblocked status. S107, determine the target coverage rate based on the target assembly instruction identifier sequence, the reference assembly instruction identifier sequence, and the reference assembly instruction identifiers corresponding to each unexecuted firmware code segment in the masked state.

2. The non-invasive coverage collection system according to claim 1, characterized in that, The firmware code contains at least one program pointer for each code segment, and different code segments have different program pointers.

3. The non-invasive coverage collection system according to claim 1, characterized in that, The test case set corresponding to the firmware in the execution chip, by recording the program pointer during execution, yields several executed assembly instruction identifiers, including: For any test case in the test case set, execute the test case, record the program pointers during the execution of the test case, and form a record subfile; The record subfiles corresponding to each test case are merged to obtain the record file; Each program pointer contained in the log file is used as an identifier for the executed assembly instruction.

4. The non-invasive coverage collection system according to claim 1, characterized in that, The step of determining the initial coverage based on the target assembly instruction identifier sequence and the preset reference assembly instruction identifier sequence corresponding to the firmware includes: Count the number of first instruction identifiers corresponding to the target assembly instruction identifier sequence and the number of second instruction identifiers corresponding to the reference assembly instruction identifier sequence; The initial coverage rate is the ratio of the number of the first instruction identifiers to the number of the second instruction identifiers.

5. The non-invasive coverage collection system according to claim 1, characterized in that, The first preset condition is that the initial coverage rate is greater than or equal to a preset coverage rate threshold.

6. The non-invasive coverage collection system according to claim 1, characterized in that, The step of determining several unexecuted reference assembly instruction identifiers based on the target assembly instruction identifier sequence and the reference assembly instruction identifier sequence includes: The intersection of each executed assembly instruction identifier in the target assembly instruction identifier sequence and each reference assembly instruction identifier in the reference assembly instruction identifier sequence is calculated to obtain the intersection calculation result. Based on the intersection calculation result and each reference assembly instruction identifier in the reference assembly instruction identifier sequence, several unexecuted reference assembly instruction identifiers are determined.

7. The non-invasive coverage collection system according to claim 1, characterized in that, The step of determining several unexecuted firmware code segments from the corresponding code of the firmware based on each unexecuted reference assembly instruction identifier includes: Based on the identifiers of each unexecuted reference assembly instruction, several temporary code segments are determined from the code corresponding to the firmware; Each temporary code segment is deduplicated to obtain each of the unexecuted firmware code segments.

8. The non-invasive coverage collection system according to claim 1, characterized in that, The step of determining the target coverage based on the target assembly instruction identifier sequence, the reference assembly instruction identifier sequence, and the reference assembly instruction identifiers corresponding to each unexecuted firmware code segment in a masked state includes: Count the number of first instruction identifiers corresponding to the target assembly instruction identifier sequence and the number of second instruction identifiers corresponding to the reference assembly instruction identifier sequence; The number of reference assembly instruction identifiers corresponding to each unexecuted firmware code segment in the masked state is counted to obtain the number of third instruction identifiers; Add the number of the first instruction identifiers and the number of the third instruction identifiers to obtain the sum; The target coverage rate is the ratio of the summation result to the number of the second instruction identifiers.

Images