A software defect detection method and device based on error injection and a storage medium
By calculating the similarity of instruction sequence hash values at the call point and conducting dynamic testing, software defects are automatically detected, solving the problems of low efficiency and time consumption in existing technologies, and achieving efficient and accurate defect detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAZHONG UNIV OF SCI & TECH
- Filing Date
- 2023-09-15
- Publication Date
- 2026-07-31
AI Technical Summary
Existing software testing techniques are inefficient at finding defects in the error handling of occasional errors. Manually marking error locations is time-consuming and unsuitable for large-scale testing. In dynamic testing, hash value calculation is time-consuming and unnecessary.
Suspicious call points are identified by calculating the similarity of the instruction sequence hash values at the call point of the erroneous function. After instrumentation, an error injection method is used for dynamic testing. The error injection is controlled by a control block, and combined with fuzz testing tools and timing control, defect points are automatically detected.
It improves the efficiency and accuracy of software defect detection, is suitable for large-scale testing, reduces manual workload and hash value calculation, and quickly and accurately identifies defects.
Smart Images
Figure CN117407286B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of software defect detection, and more specifically, relates to a software defect detection method, apparatus and storage medium based on error injection. Background Technology
[0002] During software operation, various errors may occur, and a robust program needs to handle these errors correctly. These errors are generally divided into two categories: input-related errors and sporadic errors. Common software testing techniques, such as fuzzing, can theoretically detect defects in the error handling of the first type of error. However, for the second type of sporadic errors, due to their low probability of occurrence and low triggering rate during testing, it is difficult to detect defects in their error handling. Therefore, researchers have proposed Software Fault Injection (SFI) technology, which injects errors at locations where they may occur to control the stable occurrence of errors, thereby testing the corresponding error handling and enhancing the robustness of the software.
[0003] However, conventional SFI (Site Injection Fault) techniques require manual annotation of injected errors, which demands a thorough understanding of the software and a significant amount of time, making it unsuitable for large-scale testing of different software. Furthermore, a single erroneous function may have multiple corresponding call points, and not every call point is defective. Performing defect detection on each call point impacts the efficiency of software testing. Additionally, in dynamic testing, since the call points corresponding to erroneous functions are called multiple times, current techniques require calculating the function call stack hash in each test. However, the specific hash value is not needed when controlling whether an error occurs, making the testing process time-consuming. Summary of the Invention
[0004] In view of the shortcomings of existing technologies and the need for improvement, this invention provides a software defect detection method, device and storage medium based on error injection, which aims to improve testing efficiency.
[0005] To achieve the above objectives, according to a first aspect of the present invention, a software defect detection method based on error injection is provided, comprising:
[0006] S1. Obtain the list of erroneous functions in the program under test;
[0007] S2. Traverse the error handling part of each error function in the error function list, and calculate the hash value sequence of the instruction sequence at the call point with the error handling part; wherein, the error handling part of the error function is composed of the corresponding instruction sequence;
[0008] S3. For the same error function, calculate the similarity between the hash value sequence of the instruction sequence at each call point and the hash value sequence of the instruction sequence at other call points, and take the average of the obtained similarities as the similarity at the corresponding call point; if the similarity at the corresponding call point is lower than a preset first threshold, then the call point is a suspected call point to be tested; the suspected call points to be tested corresponding to each error function in the error function list constitute the call point list to be tested;
[0009] S4. Insert instrumentation at the call points in the list of call points to be tested, and use error injection to dynamically test the program under test to obtain the location of the defect points.
[0010] Furthermore, in S4, when instrumenting the call points in the list of call points to be tested, the method further includes: adding a control block to each call point, the control block being used to control whether an injection error is injected at the injection point, wherein the instrumented call point is the injection point; the control block controlling whether an injection error is injected at the injection point includes:
[0011] Set a callback function, which is used to query whether an injection point should inject an error in each dynamic test;
[0012] If an error needs to be injected at the injection point, the program under test will set the return value of the error function corresponding to the injection point to -1 or a null pointer, causing an error to occur at the injection point.
[0013] If the injection point does not require error injection, the program under test directly calls the error function corresponding to the injection point.
[0014] Furthermore, the callback function includes:
[0015] A static counter is set at each call point to count the number of times the error function corresponding to each injection point is called in each dynamic test, so as to generate a timing sequence of the error function being triggered. The injection point numbers corresponding to the timing sequence are recorded as 1-N.
[0016] The first shared memory area is used to store the injection points where the control block needs to inject errors in each test. If the value of the static counter at the current call point is equal to the number of an injection point in the timing sequence in which the error function is triggered, it indicates that an error needs to be injected at that injection point; otherwise, no error needs to be injected at that injection point.
[0017] The second shared memory area is used to store the timing sequence of all error functions triggered in the test path.
[0018] Furthermore, in S4, an error injection method is used to dynamically test the program under test and obtain the location of the defect points, including:
[0019] S41. Use a fuzz testing tool to generate the current test cases, input the test cases into the program under test, and run the program under test;
[0020] S42. Obtain the injection point in the current path, wherein the injection point in the current path is the injection point corresponding to the timing sequence of the current error function being triggered obtained from the second shared memory region; fill the injection point number corresponding to the timing sequence into the first shared memory region in sequence; if the filled injection point number is equal to the value of the static counter at the current call point, then the control block injects an error at the injection point.
[0021] S43. If a new injection point is collected on the test path, the new injection point is stored in the second shared memory area, and the timing sequence of the current error function being triggered is updated.
[0022] S44. Repeat S42-S43 until the preset first test time is reached or the user terminates the test.
[0023] S45. Repeat S41-S44 until the preset second test time is reached or the user terminates the test, and obtain the location of the defect point during the test.
[0024] Furthermore, after the preset test time or preset test rounds are reached, in S41, when the fuzzing tool is used to generate the current test cases, the probability P of selecting the fuzzing seed is... i for:
[0025]
[0026] Where, N i N represents the number of injection points on the path corresponding to seed i in the i-th test case. sum This represents the number of injection points collected across all test cases in the current test.
[0027] Furthermore, in S45, the location of the defect point during the test is obtained, including:
[0028] Whenever an injection point fails, the program under test is run again to obtain the crash log output.
[0029] Analyze the program call stack at the injection point location and the program call stack at the crash location in the crash log;
[0030] The similarity between the program call stack at the injection point location and the program call stack at the crash location is compared using an edit distance algorithm. If the similarity does not exceed a preset third threshold, the injection point location is considered to be the actual defect location.
[0031] Further, in S2, the hash value sequence of the instruction sequence at the call point with the error handling section is calculated, including:
[0032] For function call instructions in the instruction sequence, calculate the corresponding hash value using the instruction type and the name of the called function;
[0033] For each control flow instruction in the instruction sequence, calculate the corresponding hash value using the instruction type, the source position of the control flow jump, and the destination position of the jump;
[0034] For other types of instructions in the instruction sequence, calculate the corresponding hash value based on the instruction type.
[0035] Furthermore, in S1, a list of erroneous functions in the program under test is obtained, including:
[0036] S11. Obtain the list of candidate error functions Func1, Func2, ..., Func in the program under test. n ;
[0037] S12. Traverse the program to be tested and count the number of times each candidate error function in the candidate error function list is called at the corresponding call point, C1, C2, ..., C. n And the number of times UC1, UC2, ..., UC were not called. n ;
[0038] S13. Calculate the ratio P of each of the candidate error functions being called. i =C i ÷(C i +UC i ), where i = 1, ..., n, if the ratio P being called i If the error exceeds a preset second threshold, then the candidate error function is an error function in the error function list.
[0039] Furthermore, S11 includes:
[0040] Compile the program to be tested, and then iterate through the compiled program to identify functions with the following characteristics:
[0041] The function returns a pointer or an integer.
[0042] The function returns the parameter of the branch control statement, and the branch control statement is immediately followed by at least two subsequent branches;
[0043] Functions that satisfy the above characteristics are selected as candidate error functions, thus obtaining the candidate error function list.
[0044] According to another aspect of the present invention, a software defect detection device based on error injection is provided, including a computer-readable storage medium and a processor;
[0045] The computer-readable storage medium is used to store executable instructions;
[0046] The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method described in any one of the first aspects;
[0047] Or / and, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method as described in any of the first aspects.
[0048] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:
[0049] (1) The software defect detection method based on error injection of the present invention takes into account that there may be multiple corresponding call points for an error function, and not every call point has a defect. By traversing the error handling part of each error function in the error function list, the hash value sequence of the instruction sequence at the call point with the error handling part is calculated, and the similarity at the call point is calculated. If the similarity is lower than the preset threshold, it is considered that the error handling part at the call point is different from the error handling parts at other call points of the error function. Using similarity as one of the measurement standards, the suspicious call points to be tested are identified, the test space is reduced, and the efficiency of subsequent testing is improved.
[0050] (2) Furthermore, in the dynamic testing process, the control block designed in this invention uses timing control to determine whether an error needs to be injected at a location in different tests. The timing is used instead of the specific hash value, so that the hash value does not need to be calculated in each test, which reduces the amount of code and computation at the injection point and further improves the efficiency of the test.
[0051] (3) Furthermore, during the dynamic testing process, the output samples of the fuzz testing tool are used for testing. Errors are injected into the injection points that meet the requirements in sequence through the control block on the running path. New injection points are collected on the new path where the running logic is changed after the injection of errors. This automated general error handling code method of the present invention can quickly and accurately test the defect points on the path, avoids manually marking the location of the injection errors, and is suitable for large-scale testing of different software.
[0052] (4) Further, after the preset test time is reached, the present invention no longer selects the seed with high code coverage, but instead considers that the selection probability of each seed is affected by the number of injection points on its running path, and sets the selection probability of the fuzz test seed to P. iIt focuses on selecting seeds with more erroneous function call points (or injection points) on the mutation path, thereby obtaining more new injection points and covering as many erroneous function call points as possible, further improving the accuracy of testing, and enabling testing to reach the test location in a short time.
[0053] (5) Furthermore, in this invention, it was observed that the main defects in software testing are caused by the absence or misuse of two special instructions: function call instructions and control flow related instructions. Therefore, when performing hash encoding, the function call instructions and control flow related instructions are specially processed accordingly. The design is relatively simple and can improve the efficiency and accuracy of identifying suspicious call points to be tested, thereby improving the efficiency and accuracy of software testing.
[0054] (6) Furthermore, by statistically analyzing the characteristics of the function forms that return errors in the program under test, and using the functions with the corresponding characteristics as candidate error functions, the testing accuracy can be improved. Attached Figure Description
[0055] Figure 1 This is a schematic diagram of the software defect detection method based on error injection according to the present invention.
[0056] Figure 2 This is a schematic diagram illustrating new injection points collected during the testing of the program under test using the error injection method in an embodiment of the present invention.
[0057] Figure 3 This is a flowchart illustrating how a fuzz tester is used to inject errors and complete the test in an embodiment of the present invention. Detailed Implementation
[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0059] In this invention, the terms "first," "second," etc., used in the invention and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0060] like Figure 1 As shown, the software defect detection method based on error injection of the present invention mainly includes:
[0061] S1. Obtain the list of erroneous functions in the program under test;
[0062] S2. Iterate through the error handling section of each error function in the error function list, and calculate the hash value sequence H1, H2, ..., H of the instruction sequence at the call point with the error handling section. m The error handling part of each error function, which is also the basic block of the successor function of each error function, consists of the corresponding instruction sequence; m represents the length of the instruction sequence at the call point where the error handling part is located.
[0063] S3. For the same erroneous function, calculate the similarity between the hash value sequence of the instruction sequence at each call point and the hash value sequence of the instruction sequence at other call points, and take the average of the obtained similarities as the similarity at that call point; calculate the similarity at each call point in turn. If the calculated similarity at a call point is lower than a preset first threshold, it is considered that the error handling part at that call point is different from the error handling parts at other call points of the erroneous function, and is recorded as a suspected call point to be tested, that is, the error handling part at the suspected call point to be tested may be defective; here, a call point refers to a call point with an error handling part.
[0064] Using the same method, calculate the suspicious call points corresponding to each erroneous function in the list of erroneous functions to obtain the list of call points to be tested;
[0065] S4. Recompile the program under test and insert instrumentation at the call points in the call point list to be tested. Use error injection to dynamically test the program under test and obtain the location of the defect points. In this embodiment of the invention, the error injection method used is the SFI method.
[0066] Specifically, in S1, a list of erroneous functions in the program under test is obtained, including:
[0067] S11. Obtain a list of candidate error functions in the program under test, denoted as: Func1, Func2, ..., Func n The candidate error function list is a list of potential functions in the program under test that may return errors.
[0068] S12. Traverse the program under test and count the number of times each candidate error function in the candidate error function list is called at the corresponding call point: C1, C2, ..., C n And the number of times UC1, UC2, ..., UC were not called. n ;
[0069] In S12, for each candidate error function, the sum of the number of times it is called and the number of times it is not called at the corresponding call point equals the total number of call points corresponding to that candidate error function. For the candidate error function FunC in the candidate error function list... i The number of times it is called at the corresponding call point is C.i And the number of times it was not called is UC i .
[0070] S13. For each candidate erroneous function, calculate the ratio P of its calls. i =C i ÷(C i +UC i ), where i = (1, ..., n), if the ratio P being called i If the value exceeds a preset second threshold, then the candidate error function becomes an error function in the error function list, where the ratio P of the called functions is... i The higher the value, the higher the probability that the corresponding candidate error function returns an error. In this embodiment of the invention, in order to reduce the false positive rate, the preset second threshold is 0.7.
[0071] Specifically, in S11 of this embodiment of the invention, obtaining a list of candidate erroneous functions in the program to be tested includes:
[0072] Compile the program to be tested, and iterate through the compiled program to identify functions with the following characteristics as candidate error functions:
[0073] (a) The function returns a pointer or an integer;
[0074] (b) The function returns the parameter of the branch control statement, and the branch control statement is immediately followed by at least two subsequent branches;
[0075] Functions that satisfy the above characteristics are selected as candidate error functions, resulting in a list of candidate error functions.
[0076] In this embodiment of the invention, the program under test is compiled using the LLVM framework to obtain the intermediate representation bytecode (LLVM IR Bitcode), denoted as IR1, IR2, ..., IR3. n It then iterates through the intermediate representation bytecode and identifies functions with the aforementioned characteristics in the intermediate representation bytecode as candidate error functions.
[0077] In the program under test, functions that return errors generally satisfy the above characteristics. By statistically analyzing functions with the above characteristics as candidate error functions, the accuracy of testing can be improved.
[0078] Correspondingly, as a preferred embodiment of the present invention, in S12, the number of times each candidate error function is called by the branch control statement at the corresponding call point and the number of times it is not called are counted.
[0079] Specifically, in S2, the hash value sequence H1, H2, ..., H of the instruction sequence at the call point with error handling is calculated. m,include:
[0080] For function call instructions in the instruction sequence, calculate the corresponding hash value using the type of the instruction and the name of the function being called;
[0081] For control flow-related instructions in the instruction sequence, calculate the corresponding hash value using the instruction type, the source position of the control flow jump, and the destination position of the jump;
[0082] For other types of instructions in the instruction sequence, the corresponding hash value is calculated directly using the instruction type; in this embodiment of the invention, instructions that use LLVM built-in functions are ignored.
[0083] In this invention, it was observed that the main defects in software testing are caused by the absence or misuse of two special types of instructions: function call instructions and control flow related instructions. Therefore, when performing hash encoding, function call instructions and control flow related instructions are specially processed accordingly. The design is relatively simple and can improve the efficiency and accuracy of identifying suspicious call points to be tested, thereby improving the efficiency and accuracy of software testing.
[0084] Specifically, in embodiment S3 of the present invention, the edit distance algorithm is used to calculate the hash value sequence similarity of instruction sequences at two different call points. The calculation formula is as follows:
[0085] Sim i,j =1-ED i，j ÷Max i，j
[0086] Where i and j represent the hash value sequences of the instruction sequences of the same erroneous function at two different call locations, the ED function is used to calculate the edit distance between the hash value sequences i and j, and the Max function is used to take the longer sequence length between the lengths of the hash value sequences i and j.
[0087] Correspondingly, the similarity at the call point is:
[0088]
[0089] in, This represents the similarity of the erroneous function Func at call point i, where call point i corresponds to hash value sequence i; m represents the total number of call points for the erroneous function Func, i.e., the number of hash value sequences; and the erroneous function Func is an element in the list of erroneous functions.
[0090] For the same erroneous function, the lower the similarity at each call point, the more significantly the error handling at that call point differs from the error handling at other call points of the same erroneous function. In particular, if the erroneous function has no error handling at that call point, the corresponding similarity is 0, and the corresponding call point is also considered a suspected call point to be tested and will participate in subsequent tests.
[0091] In S4, the call point after instrumentation is called the injection point. Instrumenting call points in the list of call points to be tested also includes adding a control block to each call point. This control block is used to control whether the corresponding error function returns an error during dynamic testing of the program under test; that is, how to control whether an error is injected at the injection point, and simultaneously collect new injection points along the test path. Specifically, the process of whether the control block injects an error at the injection point includes:
[0092] Set a callback function, which is used to query whether an error should be injected at the injection point in each dynamic test; the callback function is the corresponding runtime component linked during the program linking phase.
[0093] If an error needs to be injected at the injection point, the program under test will not call the error function corresponding to the injection point, but will directly set the return value of the error function to -1 or a null pointer, causing an error to occur at the injection point;
[0094] If the injection point does not require error injection, the program under test will directly call the error function corresponding to the injection point.
[0095] Specifically, the callback function includes: a static counter, a first shared memory region, and a second shared memory region;
[0096] A static counter is set at each call point to count the number of times the erroneous function corresponding to each injection point is called in each dynamic test, generating a timing sequence of erroneous functions being triggered, and the injection point numbers corresponding to this timing sequence are recorded as 1-N; the counter is incremented by 1 each time the erroneous function is called;
[0097] The first shared memory area is used to store the injection points where the fuzz tester needs to control the block injection of errors in each test. If the value of the static counter at the current call point is equal to the number of an injection point in the timing sequence in which the error function is triggered, the callback function returns a true value, indicating that an error needs to be injected at that position; otherwise, it returns a negative value, indicating that an error does not need to be injected at that moment.
[0098] The second shared memory area is used to store the timing sequence of all error functions triggered in the test path during dynamic testing, that is, the injection point numbers of all possible error injection points.
[0099] In S4, such as Figures 2-3 As shown, the error injection method is used to test the program under test and obtain the location of the defect points, including:
[0100] S41. Use a fuzz testing tool to generate the current test cases, input the test cases into the program under test, and run the program under test; in this embodiment of the invention, the fuzz testing tool used is AFL;
[0101] S42. Obtain the injection point in the current path (i.e., Figure 2 The feedback injection point in the current path is the injection point corresponding to the timing sequence of the current error function being triggered obtained from the second shared memory area. The injection point number corresponding to the timing sequence is sequentially filled into the first shared memory area. When the filled injection point number is equal to the value of the static counter at the current call point, the control block injects an error at the injection point; otherwise, it is not necessary to inject an error at the injection point.
[0102] S43. If a new injection point is collected on the test path, store it in the second shared memory region and update the timing sequence of the current fault function being triggered; where, Figure 2 The global injection point is all the injection points collected in one round of testing (corresponding to the same test case). It is used to determine whether a new injection point has been collected. That is, the new injection point is the difference between the global injection point and the feedback injection point.
[0103] S44. Repeat S42-S43 until the preset first test time is reached or the user terminates the test.
[0104] S45. Repeat S41-S44 until the preset second test time is reached or the user terminates the test, and obtain the location of the defect point during the test.
[0105] Preferably, when using fuzzing tools to generate test cases, a heuristic algorithm is used in the seed selection phase of fuzzing. After a preset test time or test rounds (each round corresponds to generating one test case, and the loop process of S42-S43 is considered one round of testing, corresponding to the same test cases), to improve test accuracy, the seed selection probability P of fuzzing is increased. i Set to:
[0106]
[0107] Where, N i N represents the number of injection points on the path corresponding to seed i in the i-th test case. sum This represents the number of injection points collected across all test cases in the current test.
[0108] In this embodiment of the invention, seed selection is performed using the original method of the fuzz tester half an hour before the test; after half an hour of testing, the seed selection probability is P. i .
[0109] After reaching a preset number of test rounds or time, this invention no longer selects seeds with high code coverage. Instead, considering that the selection probability of each seed is affected by the number of injection points on its execution path, the selection probability of the fuzzing seed is set to P. i It focuses on selecting seeds with more erroneous function call points (or injection points) on the mutation path, thereby obtaining more new injection points and covering as many erroneous function call points as possible, further improving the accuracy of testing, and enabling testing to reach the test location in a short time.
[0110] Specifically, in S45, the location of the defect point during the test is obtained, including:
[0111] Whenever an injection point fails, the program under test is run again to obtain the output crash logs. During the test, third-party detection tools, such as AddressSanitizer and MemorySanitizer, can be used to generate crash logs during the test.
[0112] The crash logs generated during the testing process are filtered and output to determine the location of the defect, specifically including:
[0113] Analyze the program call stack at the injection point location and the program call stack at the crash location in the crash log;
[0114] The edit distance algorithm is used to compare the similarity of two call stacks. If the similarity does not exceed a preset third threshold, the injection point is considered a true defect location. That is, the higher the similarity, the greater the overlap between the injection point and the crash location, and the higher the false positive rate. In this embodiment of the invention, the preset third threshold is 0.9. When the similarity is higher than 0.9, the crash is ignored; that is, the injection point is not a true defect location.
[0115] The software defect detection method based on error injection of the present invention takes into account that there may be multiple corresponding call points for a single error function, and not every call point has a defect. By traversing the error handling part of each error function in the error function list, the hash value sequence of the instruction sequence at the call point with the error handling part is calculated, and the similarity at the call point is calculated. If the similarity is lower than a preset threshold, it is considered that the error handling part at that call point is different from the error handling parts at other call points of the same error function. Using similarity as one of the measurement criteria, suspicious call points to be tested are identified, thus narrowing the test space and improving the efficiency of subsequent testing.
[0116] During dynamic testing, the control block designed in this invention uses timing control to determine whether an error needs to be injected at a location in different tests. By using timing instead of a specific hash value, the hash value does not need to be calculated in every test, reducing the amount of code and computation at the injection point and further improving the efficiency of testing.
[0117] During dynamic testing, the output samples of a fuzz testing tool are used for testing. Errors are injected sequentially into the currently qualified injection points along the running path through control blocks. New injection points are collected on the new path where the running logic has changed after the injection of errors. This automated and general error handling code method of the present invention can quickly and accurately test the defect points on the path, avoiding the need for manual annotation of the injection error location, and is suitable for large-scale testing of different software.
[0118] The software defect detection method based on error injection of this invention comprises two parts. The static analysis process involves calculating the list of call points to be tested (S1-S3), while the dynamic analysis process involves determining the location of defect points (S4). The static analysis process uses pattern matching to identify candidate error functions and then uses statistical methods to filter these functions, ultimately obtaining a list of call points to be tested that can be instrumented. This method reduces manual workload and provides more accurate results. In the dynamic analysis process, instrumentation is performed at pre-selected locations during compilation, and runtime components (control blocks) controlling error injection are linked. Then, the output samples of a fuzzing tool are used for testing. Errors are injected sequentially at all error injection points along the runtime path, and the number of error points guides the seed selection process of the fuzzing tool, iteratively generating more test cases. Finally, the error logs (crash logs) generated during testing are filtered and output to obtain the location of the defect points. This combination of static and dynamic analysis in this invention can quickly and accurately detect defects in program error handling code.
[0119] The present invention also provides a software defect detection device based on error injection, including a computer-readable storage medium and a processor;
[0120] Computer-readable storage media are used to store executable instructions;
[0121] The processor is used to read executable instructions stored in a computer-readable storage medium and execute the software defect detection method based on error injection in the above embodiments;
[0122] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the software defect detection method based on error injection in the above embodiments.
[0123] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for software defect detection based on error injection, the method comprising: include: S1. Obtain the list of erroneous functions in the program under test; S2. Traverse the error handling part of each error function in the error function list, and calculate the hash value sequence of the instruction sequence at the call point with the error handling part; wherein, the error handling part of the error function is composed of the corresponding instruction sequence; S3. For the same error function, calculate the similarity between the hash value sequence of the instruction sequence at each call point and the hash value sequence of the instruction sequence at other call points, and take the average of the obtained similarities as the similarity at the corresponding call point; if the similarity at the corresponding call point is lower than a preset first threshold, then the call point is a suspected call point to be tested; the suspected call points to be tested corresponding to each error function in the error function list constitute the call point list to be tested; S4. Insert instrumentation at the call points in the list of call points to be tested, and use error injection to dynamically test the program under test to obtain the location of the defect points; In S4, when instrumenting the call points in the list of call points to be tested, the method further includes: adding a control block to each call point, the control block being used to control whether an error is injected at the injection point, wherein the instrumented call point is the injection point; the control block controlling whether an error is injected at the injection point includes: Set a callback function, which is used to query whether an injection point should inject an error in each dynamic test; If an error needs to be injected at the injection point, the program under test will set the return value of the error function corresponding to the injection point to -1 or a null pointer, causing an error to occur at the injection point. If the injection point does not require error injection, the program under test directly calls the error function corresponding to the injection point; The callback function includes: A static counter is set at each call point to count the number of times the error function corresponding to each injection point is called in each dynamic test, so as to generate a timing sequence of the error function being triggered. The injection point numbers corresponding to the timing sequence are recorded as 1-N. The first shared memory area is used to store the injection points where the control block needs to inject errors in each test. If the value of the static counter at the current call point is equal to the number of an injection point in the timing sequence in which the error function is triggered, it indicates that an error needs to be injected at that injection point; otherwise, no error needs to be injected at that injection point. The second shared memory area is used to store the timing sequence of all error functions triggered in the test path.
2. The method of claim 1, wherein, In S4, an error injection method is used to dynamically test the program under test and obtain the location of defect points, including: S41. Use a fuzz testing tool to generate the current test cases, input the test cases into the program under test, and run the program under test; S42. Obtain the injection point in the current path, wherein the injection point in the current path is the injection point corresponding to the timing sequence of the current error function being triggered obtained from the second shared memory region; fill the injection point number corresponding to the timing sequence into the first shared memory region in sequence; if the filled injection point number is equal to the value of the static counter at the current call point, then the control block injects an error at the injection point. S43. If a new injection point is collected on the test path, the new injection point is stored in the second shared memory area, and the timing sequence of the current error function being triggered is updated. S44. Repeat S42-S43 until the preset first test time is reached or the user terminates the test. S45. Repeat S41-S44 until the preset second test time is reached or the user terminates the test, and obtain the location of the defect point during the test.
3. The method of claim 2, wherein, When the preset test time or preset test rounds are reached, in step S41, when generating the current test case using a fuzzing tool, the probability of selecting the fuzzing seed is... for: wherein, is the number of injection points on the path corresponding to seed i in the ith test case, is the number of injection points collected from all test cases in the current test.
4. The method according to claim 2 or 3, characterized in that, In S45, the location of the defect point during the test is obtained, including: Whenever an injection point fails, the program under test is run again to obtain the crash log output. Analyze the program call stack at the injection point location and the program call stack at the crash location in the crash log; The similarity between the program call stack at the injection point location and the program call stack at the crash location is compared using an edit distance algorithm. If the similarity does not exceed a preset third threshold, the injection point location is considered to be the actual defect location.
5. The method of claim 1, wherein, In S2, the hash value sequence of the instruction sequence at the call point with error handling is calculated, including: For function call instructions in the instruction sequence, calculate the corresponding hash value using the instruction type and the name of the called function; For each control flow instruction in the instruction sequence, calculate the corresponding hash value using the instruction type, the source position of the control flow jump, and the destination position of the jump; For other types of instructions in the instruction sequence, calculate the corresponding hash value based on the instruction type.
6. The method of claim 1, wherein, In S1, obtain the list of erroneous functions in the program under test, including: S11、acquiring a candidate error function list in the program to be tested ; S12. Traverse the program to be tested and count the number of times each candidate error function in the candidate error function list is called at the corresponding call point. and the number of times it was not called ; S13. Calculate the ratio of each of the candidate error functions being called. ,in, If the ratio being called If the error exceeds a preset second threshold, then the candidate error function is an error function in the error function list.
7. The method of claim 6, wherein, S11 includes: Compile the program to be tested, and then iterate through the compiled program to identify functions with the following characteristics: The function returns a pointer or an integer. The function returns the parameter of the branch control statement, and the branch control statement is immediately followed by at least two subsequent branches; Functions that satisfy the above characteristics are selected as candidate error functions, thus obtaining the candidate error function list.
8. A software defect detection device based on error injection, characterized in that, Includes computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method according to any one of claims 1-7; Or / and, a computer-readable storage medium having a computer program stored thereon, characterized in that, when the program is executed by a processor, it implements the method as described in any one of claims 1-7.