Test case optimization method and device for kernel configuration item modification

Abstract

The invention discloses a test case optimization method and device for kernel configuration item modification. The method comprises the following steps: extracting kernel source code information associated with a target configuration item; obtaining an address of a to-be-sorted test case execution kernel function, and converting the address into a corresponding source code path and a line number; and optimizing the test case set to be sorted based on the kernel source code information, the source code path and the line number. According to the method, the overhead of incremental testing can be reduced, and meanwhile, the coverage rate of the associated code blocks which is the same as (or similar to) that of executing the original test case set can still be achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer software, and relates to the scenario of modifying Linux kernel configuration items, and proposes a method and device for optimizing test cases for modifying kernel configuration items. Background Art

[0002] Today, Linux has been widely used in many scenarios from embedded devices to supercomputers. This universality has generated corresponding adaptability requirements. To meet this high adaptability, Linux provides users with a set of kernel configuration items to facilitate users to modify the functions and features of the Linux kernel to adapt to different scenario requirements. Due to the open-source nature of the Linux kernel, developers and contributors in the community actively participate in this project, continuously promoting the iterative update of the kernel. However, with the iteration of the kernel version, to meet the ever-expanding hardware support, application scenarios, and functional requirements, a large amount of source code and configuration items have been added to the kernel, which makes the architecture of the kernel increasingly large, and the dependency relationships between various modules and subsystems of the kernel become very complex.

[0003] In order to fully test the complex-structured Linux kernel to ensure the usability and stability of the Linux kernel and discover and fix potential defects and vulnerabilities, so far, researchers and developers have proposed many kernel automation test tools, but the existing test tools do not specifically target the customized scenarios of the Linux kernel. On the one hand, the kernel community will release the kernel configured by the official after testing, and users often reconfigure the official kernel according to their actual needs, such as the providers of Linux distributions such as Ubuntu, Debian, CentOS, Fedora, etc., by turning on or off configuration items or modifying specific values, and then redistribute it after adding other integrated tools. On the other hand, device manufacturers will also modify the configuration file to make the Linux kernel support various special devices. After modifying the configuration item, when compiling the kernel, the compilation tool may additionally compile some specific files or cancel the compilation of some files; at the code granularity, modifying the configuration item may add or delete some kernel functions, or change the execution logic inside the function, and these changes may contain potential errors. Therefore, after modifying the configuration file, in order to ensure the stability and usability of the kernel, it is necessary to conduct targeted tests on the code segments affected by the configured items to test the Linux kernel functions more efficiently with less overhead. However, currently, there is no relevant work on testing the changed configuration items, resulting in unnecessary overhead for users when performing incremental testing on the modified kernel. Summary of the Invention

[0004] To solve the problems of weak purpose and excessive overhead in Linux incremental testing, the purpose of the present invention is to provide a method and device for optimizing test cases for kernel configuration item modification, which extracts code blocks associated with target configuration items from Linux kernel files, thereby determining the relationship between them and the kernel functions executed by test cases, realizing the sorting and optimization of test cases, and while reducing the kernel test overhead, still achieving the same (or similar) coverage of associated code blocks as that of executing the original test case set.

[0005] A method for optimizing test cases for kernel configuration item modification, the method comprising:

[0006] Extracting kernel source code information associated with target configuration items;

[0007] Obtaining the addresses of the kernel functions executed by the test cases to be sorted, and converting the addresses into corresponding source code paths and line numbers;

[0008] Optimizing the test case set to be sorted based on the kernel source code information and the source code paths and line numbers.

[0009] Further, the extracting of the kernel source code information associated with target configuration items includes:

[0010] Analyzing the Kconfig file to obtain a list of kernel configuration items to confirm the values of target configuration items;

[0011] Parsing the kernel source code and matching the parsing result of the kernel source code with the values of the target configuration items to obtain the kernel source code that may be associated with the target configuration items;

[0012] For the kernel source code that may be associated with the target configuration items, determine whether the kernel source code is the kernel source code associated with the target configuration items according to whether the state of the target configuration item has changed in the current kernel and the correlation between the target configuration item and the kernel source code;

[0013] Parsing the Makefile file and obtaining the compilation information of the source code files associated with the target configuration items based on the parsing result of the Makefile file;

[0014] Combining the kernel source code associated with the target configuration items and the compilation information of the source code files associated with the target configuration items to obtain the kernel source code information associated with the target configuration items.

[0015] Further, the determining whether the kernel source code is the kernel source code associated with the target configuration items according to whether the state of the target configuration item has changed in the current kernel and the correlation between the target configuration item and the kernel source code includes:

[0016] If the kernel source code that the target configuration item may be associated with activates any kernel file, function, or modifies the execution logic inside the function, then the kernel source code that the target configuration item may be associated with is the kernel source code associated with the target configuration item;

[0017] If the kernel source code that the target configuration item may be associated with cannot activate any kernel file, function, or cannot modify the execution logic inside the function, and the kernel source code that the target configuration item may be associated with contains the definition of the entire function, then the kernel source code that the target configuration item may be associated with is not the kernel source code associated with the target configuration item;

[0018] If the kernel source code that the target configuration item may be associated with cannot activate any kernel file, function, or cannot modify the execution logic inside the function, and the kernel source code that the target configuration item may be associated with only affects part of the execution logic in a certain function, then the kernel source code that the target configuration item may be associated with is the kernel source code associated with the target configuration item.

[0019] Further, the obtaining the addresses of the kernel functions for the test cases to be sorted and converting the addresses into corresponding source code paths and line numbers includes:

[0020] Install the kernel in the qemu emulator and execute the test cases to be sorted;

[0021] Through the recording function of the qemu emulator, track the situation of the test cases to be sorted calling kernel functions to obtain the execution addresses of different kernel functions;

[0022] Based on the addr2line tool, parse the execution addresses of different kernel functions to obtain the source code paths and line numbers corresponding to the test cases to be sorted.

[0023] Further, the optimizing the test case set to be sorted based on the kernel source code information and the source code paths and line numbers includes:

[0024] According to the kernel source code information and the source code paths and line numbers, construct the relationship between the test case set to be sorted and the target kernel configuration item;

[0025] Based on the relationship between the test case set to be sorted and the target kernel configuration item, calculate the sufficiency of the test case set to be sorted calling code blocks where i represents the serial number of the test case to be sorted, and j represents the serial number of the code block;

[0026] Calculate the execution time related variables for each test case to be sorted

[0027] According to the sufficiency and the execution time related variables Calculate the score R of the test case to be sorted i ;

[0028] Sort the test cases to be sorted in descending order according to the score R i ;

[0029] Take out the test cases to be sorted after sorting in turn, and when the test case to be sorted can improve the total code block coverage rate, while putting the test case to be sorted into the selected test case set A, update the total code block coverage rate

[0030] Until the total code block coverage rate is 1 or all the test cases to be sorted are tested, obtain the optimization result of the test cases to be sorted based on the test case set A

[0031] Further, the degree of sufficiency where x j represents the number of times that the test case i to be sorted calls the code block j, and α(j) represents the execution times threshold of the code block j

[0032] Further, the execution time related variable where t i is the execution time of the test case i to be sorted

[0033] Further, the score where the code block related score of the test case i to be sorted m(j) represents the weight of the code block j, and k represents the total number of code blocks

[0034] Further, the weight N represents the total number of test cases to be sorted, and N(j) represents how many test cases call the code block j

[0035] A test case optimization device for kernel configuration item modification, the device includes:

[0036] An associated code extraction module, configured to extract kernel source code information associated with a target configuration item

[0037] A test case execution record module, configured to obtain the address of the kernel function executed by the test case to be sorted, and convert the address into a corresponding source code path and line number

[0038] A sorting module, configured to optimize the test case set to be sorted based on the kernel source code information and the source code path and line number

[0039] Compared with the prior art, the positive effects of the present invention are:

[0040] (1) The present invention can establish the relationship between kernel test cases and kernel configuration items.

[0041] (2) Given the modified kernel configuration items, the present invention can optimize the test cases based on these configuration items, thereby reducing the overhead brought by incremental testing. Description of the Drawings

[0042] Figure 1 Overall architecture diagram of the test case optimization device for kernel configuration item modification.

[0043] Figure 2 Relationship diagram between kernel configuration items and kernel source code.

[0044] Figure 3 Flowchart of the test case optimization method for kernel configuration item modification. Detailed Implementation Manner

[0045] The present invention will be further described below by way of examples, but it is not limited to the scope of the present invention in any way.

[0046] The test case optimization device for kernel configuration item modification of the present invention has the following technical solutions.

[0047] The technical solution is as Figure 1 shown. Specifically, first, after inputting the set of target kernel configuration items, the associated code extraction module will match the code blocks associated with the target configuration items in all files of the Linux kernel and extract them; second, after inputting the set of test cases to be sorted, the test case execution record module will call them, record the addresses of the kernel functions executed by the test cases and resolve them into corresponding paths and line numbers; finally, the sorting module will take the outputs of the above two modules as inputs, call the sorting algorithm to sort the set of test cases to be sorted, and obtain the final optimization result.

[0048] 1) Associated code extraction module.

[0049] The configuration item-related code block PC that the association code extraction module needs to extract includes two parts: First, the source code block information related to kernel configuration items; second, the relevant information in the Makefile file. Therefore, it is necessary to match and analyze the Linux kernel files by category. The extraction module will analyze the Linux kernel (Kbuild) configuration and return the relationship between kernel configuration items and code blocks. The basic architecture of this module consists of two parts: First, in order to obtain the source code block information related to kernel configuration items, the present invention analyzes the Kconfig file, obtains the list of kernel configuration items, and confirms the values of these configuration items. Subsequently, the kernel source code is parsed to return the matching results; second, in order to obtain the Makefile information related to kernel configuration items, this module also parses the Makefile file to obtain the compilation information of the source code files related to the target configuration items in the Makefile.

[0050] Since the update of the kernel configuration item status determines whether new source code participates in the kernel compilation process. Therefore, during the extraction process, the extraction module will decide whether to retain the code block based on whether the status of the target configuration item has changed in the current kernel and the relevance between the configuration item and the current code block. If a configuration item activates a certain kernel file, a certain function, or modifies the execution logic inside the function, then the extraction module will retain the code block; conversely, if a configuration item deactivates an entire file or code block, then it needs to be divided into multiple situations, such as Figure 2 shown. In the kernel source code, if the associated code block contains the definition of an entire function (such as Figure 2 ① in it, the value of CONFIG_A comments out the definition of the entire function foo), then since this code will not be executed by the kernel, this code block will not be retained; conversely, if a configuration item only affects part of the execution logic in a certain function (such as Figure 2 ② in it, the execution logic of bar is changed due to the value of CONFIG_B), then this code block will still be retained. In this way, the extraction module further filters the matching results and writes the final results into the database.

[0051] 2) Test case execution record module.

[0052] The test case execution record module first obtains the execution addresses of the kernel functions for test case execution and converts these addresses into corresponding source code paths and line numbers. Only in this way can it be matched with the code blocks extracted by the association code extraction module, and finally calculate the relevant data used by the sorting module, that is, the number of times the code block is called by each test case and the number of test cases that call this code block.

[0053] In order to accurately trace the instructions executed by the system when executing test cases, the present invention installs a kernel in the qemu emulator and executes the test cases. By using the recording function of qemu, the situation of the test cases calling kernel functions is traced to obtain the execution addresses of different kernel functions. When parsing this address, the present invention uses the addr2line tool to convert the function address into the source code path and line number.

[0054] 3) Sorting module.

[0055] The present invention believes that when judging the priority of a test case, the following three criteria should be considered and the following strategy should be adopted:

[0056] First, the test case should cover as many code blocks associated with the target configuration item as possible. Therefore, the present invention sets a score for each code block, and the test case can only obtain this score when it calls this code block. Finally, the scores of all code blocks are summed to obtain the total score R. For test case i, its corresponding score R i is set as follows:

[0057]

[0058] Among them, represents the code block related score of this test case, while

[0059] represents the time related score of this test case, and this score is obtained from the execution time of each test case. The higher the total score, the more functions the test case calls. The calculation formula of the code block related score

[0060]

[0061] is as follows: Among them, m(j) represents the weight of code block j,

[0062]

[0063] represents the sufficiency degree of test case i calling code block j. When calculating m(j), in order to make the code block with a smaller test case call rate obtain a higher weight, the setting of m(j) adopts the form of inverse document frequency index (IDF), and the form is as follows.

[0064] Second, the number of times the code block associated with the test case execution target configuration item should be executed as many times as possible. In this paper, the sigmoid function is used to calculate the sufficiency of a certain test case calling a code block When the number of calls to the code block increases below a certain threshold, the rate of increase of this value accelerates; when it exceeds this threshold, the rate of increase decreases to prevent the score obtained by this test case on the code block from affecting other code blocks. The definition formula is as follows:

[0065]

[0066] Among them, x j represents the number of times test case i calls code block j, and α(j) represents the execution times threshold of this code block. In the present invention, it is set to the average value of the number of times all test cases execute this code block.

[0067] Third, the execution time of the test case should be as little as possible. In this paper, a time-related variable R is added time . The lower the execution time of the test case, the smaller the value of this variable, making the final score of this test case larger. The present invention sets also in the form of the sigmoid function, where t i is the execution time of the test case, and β is the average value of the execution times of all test cases:

[0068]

[0069] Based on the above indicators, the process of the algorithm designed by the present invention is as follows: Given a set of test case sets S = {s0, s1,..., s n} to be sorted, the test time sets T = {t0, t1,..., t n} of this set of test cases and the configuration item sets C = {c0, c1,..., c m} that need to be incrementally tested. The algorithm will output a set of sorted test case sets A = {s i , s j ,..., s k} (the sizes of A and S are not necessarily equal). The algorithm first calculates the score of each test case t i and then sorts the test cases in descending order according to the score R total . Subsequently, the algorithm traverses the sorted test case set S and calculates whether the total coverage rate c total can be improved after selecting this test case. If so, the algorithm puts this test case into the selected test case set A and updates the total code block coverage rate c total, it is stated that it should not be selected and it is removed from the to-be-selected queue R. During the traversal process, if the situation of c total = 1 occurs, this indicates that the current test case set has covered all associated code blocks, and the algorithm will terminate immediately; or when there are no test cases available for selection in R, the algorithm will also terminate.

[0070] In summary, the present invention can achieve the sorting of all test cases, remove the test cases that cannot improve the associated code coverage rate, retain the relevant test cases, and select the test case set that can complete the test in the shortest time and achieve the same code coverage rate as the original test cases.

[0071] As Figure 3 shown, the present invention also provides a test case optimization method for kernel configuration item modification, including:

[0072] Step 1: Extract the kernel source code information associated with the target configuration item;

[0073] Step 2: Obtain the addresses of the kernel functions executed by the test cases to be sorted, and convert the addresses into corresponding source code paths and line numbers;

[0074] Step 3: Optimize the test case set to be sorted based on the kernel source code information and the source code path and line numbers.

[0075] In one embodiment, assuming a given target configuration item set C = {c1, c2,... c n}, a to-be-selected test case set B = {b1, b2,..., b m}, and a test kernel, then the implementation steps of the test case optimization method of the present invention are as follows:

[0076] (1) Based on the target configuration item set C, obtain the relationship between the configuration item and the kernel source code, that is, the code block PC associated with the configuration item;

[0077] (2) Based on the test case set B to be sorted, obtain the addresses of the kernel functions executed by the test cases during execution and resolve them into corresponding paths and line numbers, so as to establish the relationship between the target configuration item c and the test case b;

[0078] (3) Call the test case optimization algorithm to obtain the optimized test case set A.

Claims

1. A test case optimization method for kernel configuration item modification, characterized in that The method includes: Extracting kernel source code information associated with the target configuration item; Obtaining the execution addresses of the kernel functions of the test cases to be sorted, and converting the addresses into corresponding source code paths and line numbers; Optimizing the test case set to be sorted based on the kernel source code information and the source code paths and line numbers.

2. The method according to claim 1, wherein The extracting of the kernel source code information associated with the target configuration item includes: Analyzing the Kconfig file to obtain a list of kernel configuration items to confirm the value of the target configuration item; Parsing the kernel source code and matching the parsing result of the kernel source code with the value of the target configuration item to obtain the kernel source code that the target configuration item may be associated with; For the kernel source code that the target configuration item may be associated with, determining whether the kernel source code is the kernel source code associated with the target configuration item according to whether the state of the target configuration item in the current kernel has changed and the correlation between the target configuration item and the kernel source code; Parsing the Makefile file and obtaining the source file compilation information associated with the target configuration item based on the parsing result of the Makefile file; Combining the kernel source code associated with the target configuration item and the source file compilation information associated with the target configuration item to obtain the kernel source code information associated with the target configuration item.

3. The method according to claim 2, characterized in that, The determining, for the kernel source code that the target configuration item may be associated with, whether the kernel source code is the kernel source code associated with the target configuration item according to whether the state of the target configuration item in the current kernel has changed and the correlation between the target configuration item and the kernel source code includes: If any kernel file, function, or the execution logic inside the function is activated by the kernel source code that the target configuration item may be associated with, then the kernel source code that the target configuration item may be associated with is the kernel source code associated with the target configuration item; If the kernel source code that the target configuration item may be associated with cannot activate any kernel file, function, or cannot modify the execution logic inside the function, and the kernel source code that the target configuration item may be associated with contains the definition of the entire function, then the kernel source code that the target configuration item may be associated with is not the kernel source code associated with the target configuration item; If the kernel source code that the target configuration item may be associated with cannot activate any kernel file, function, or cannot modify the execution logic inside the function, and the kernel source code that the target configuration item may be associated with only affects part of the execution logic in a certain function, then the kernel source code that the target configuration item may be associated with is the kernel source code associated with the target configuration item.

4. The method according to claim 1, characterized in that The obtaining of the execution addresses of the kernel functions of the test cases to be sorted, and converting the addresses into corresponding source code paths and line numbers includes: Installing the kernel in the qemu emulator and executing the test cases to be sorted; Tracking the situation of the test cases to be sorted calling kernel functions through the recording function of the qemu emulator to obtain the execution addresses of different kernel functions; Parsing the execution addresses of different kernel functions based on the addr2line tool to obtain the source code paths and line numbers corresponding to the test cases to be sorted.

5. The method according to claim 1, characterized in that The optimizing of the test case set to be sorted based on the kernel source code information and the source code paths and line numbers includes: Construct the relationship between the test case set to be sorted and the target kernel configuration item according to the kernel source code information, the source code path, and the line number; Calculate the sufficiency of the test case call code block to be sorted based on the relationship between the test case set to be sorted and the target kernel configuration item where i represents the serial number of the test case to be sorted, and j represents the serial number of the code block; Calculate the execution time-related variables of each test case to be sorted Based on the sufficiency degree and the execution time related variable calculate the score R of the test case to be sorted i ; According to the score R i Sort the test cases to be sorted in descending order; Successively take out the test cases to be sorted after sorting, and when the test case to be sorted can improve the total code block coverage rate, put the test case to be sorted into the selected test case set A and update the total code block coverage rate at the same time; Until the total code block coverage rate is 1 or all the test cases to be sorted are tested, obtain the optimization result of the test cases to be sorted based on the test case set A.

6. The method according to claim 5, characterized in that The degree of sufficiency where x j represents the number of times the test case i to be sorted calls the code block j, and α(j) represents the execution times threshold of the code block j.

7. The method according to claim 5, characterized in that The execution time-related variable where t i is the execution time of the test case i to be sorted.

8. The method according to claim 5, wherein The fraction Among them, the code block related fraction of the test case i to be sorted m(j) represents the weight of code block j, and k represents the total number of code blocks.

9. The method according to claim 8, wherein The weight mentioned above N represents the total number of test cases to be sorted, and N(j) represents the number of test cases that call code block j.

10. A test case optimization device for kernel configuration item modification, characterized in that The device includes: An associated code extraction module, configured to extract kernel source code information associated with a target configuration item; A test case execution record module, configured to obtain the address of the kernel function executed by the test case to be sorted, and convert the address into the corresponding source code path and line number; A sorting module, configured to optimize the test case set to be sorted based on the kernel source code information, the source code path, and the line number.

Images