A fuzzing method for continuous integration processes

Abstract

The application discloses a kind of fuzzy testing methods for continuous integration process, it is related to software collaborative development field.The application includes the following steps: in the project of applying continuous integration, difference analysis is carried out to two adjacent submissions, the difference information of both is obtained, and change point is obtained according to difference information, and is stored in change point set;In the process of building project, data flow analysis is carried out according to change point information, and data flow analysis result is obtained;Program is inserted into using data flow analysis result, and the measured program that has been inserted is obtained;The measured program is carried out fuzzy testing, the fitness of seed test case is calculated in the testing process, and corresponding test resource is allocated to it according to fitness.The application more specifically tests the place where change is generated in continuous integration process, and reduces the overhead brought by manual construction test case.

Description

Technical Field

[0001] This invention relates to the field of collaborative software development, and more specifically to a fuzz testing method for continuous integration processes. Background Technology

[0002] Software evolution refers to the dynamic behavior of a software system as it is maintained and enhanced throughout its lifecycle, and it is one of the important processes in the software lifecycle. Software systems continuously evolve from development to post-delivery to adapt to changing business needs and extend their lifespan. With the popularization of software development methodologies such as Agile and Extreme Programming, the frequency of software iterations is increasing. Therefore, an efficient collaborative work method is needed for software development. Continuous Integration (CI) is a software development practice that can detect integration defects early and improve software quality. First proposed by Grady Booch, it effectively solves various problems encountered during system integration. In CI, team members typically perform multiple integrations daily, with each integration involving automated builds and tests to detect integration errors as quickly as possible. This approach significantly reduces integration problems during development, thereby improving the quality of the developed software.

[0003] Regression testing is performed during continuous integration to detect integration errors as early as possible. However, the frequent integration processes in continuous integration necessitate frequent regression testing. Before regression testing begins, it is often necessary to manually select and modify existing test cases and create a large number of new test cases to adequately test the changes that occur during continuous integration. Therefore, frequent regression testing incurs significant manpower costs. Fuzz testing, a technique that automatically generates and executes a large number of test cases, offers advantages such as high efficiency and automation, and has received widespread attention from researchers in recent years. Fuzz testing can, to some extent, help users reduce the overhead of frequently constructing a large number of test cases in frequent regression testing. However, existing fuzz testing methods do not consider the information generated during continuous integration. Directly applying fuzz testing to continuous integration will make it difficult to adequately test the changes that occur during continuous integration, resulting in unnecessary overhead. Summary of the Invention

[0004] In view of this, the present invention provides a fuzzing method for continuous integration processes. First, continuous integration difference analysis is performed on a single commit of the project under test to identify the change points generated by that commit. Second, during the build process, these change points are used as taint sources for taint analysis, and the distances between each basic block and the taint sources are obtained. Third, program instrumentation is performed based on the distances between the basic blocks and the taint sources. Fourth, fuzzing is performed, and during the fuzzing process, the fitness of the seed test cases is calculated based on their execution information, and corresponding test resources are allocated to them to more effectively test the change points generated by continuous integration.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A fuzz testing method for continuous integration processes includes the following steps:

[0007] Step 1: In a project using continuous integration, perform a difference analysis on two adjacent commits to obtain the difference information between them, and then identify the change points based on the difference information and store them in a change point set.

[0008] Step 2: During the project construction process, perform data flow analysis based on change point information and obtain the data flow analysis results;

[0009] Step 3: Use the data flow analysis results to perform program instrumentation to obtain the instrumented program under test;

[0010] Step 4: Perform fuzz testing on the program under test. During the testing process, calculate the fitness of the seed test cases and allocate corresponding test resources to them based on the fitness.

[0011] Optionally, step 1 includes:

[0012] Step 1-1: Set the initial state and input the item to be tested, P;

[0013] Steps 1-2: Perform a difference analysis on two adjacent submissions in the tested project P to obtain the set of change points C = {c1, c2, ..., c...} n};

[0014] Steps 1-3: Output the set of change points C.

[0015] Optionally, step 2 includes:

[0016] Step 2-1: Set the initial state and input the item to be tested, P;

[0017] Step 2-2: Determine whether the statements in the tested item P have been traversed. If the traversal is complete, jump to step 2-6; if the traversal is not complete, jump to step 2-3.

[0018] Steps 2-3: Access the next statement in the tested item P;

[0019] Step 2-4: Determine whether the current statement Stat is related to a change point. If it is related to a change point, proceed to step 2-5; otherwise, proceed to step 2-3.

[0020] Step 2-5: Perform definition-use relationship analysis on the current statement Stat; if Stat reads from memory, it is considered that Stat has performed a "use" operation on a certain variable; if Stat modifies memory, it is considered that Stat has performed a "definition" operation on a certain variable; after Stat analysis is completed, jump to step 2-2;

[0021] Steps 2-6: Output all definition and usage relationship information D related to change points in the tested item P;

[0022] Steps 2-7: Input the set of change points C obtained in Step 1;

[0023] Step 2-8: Calculate the distance between each basic block and the taint source; take each change point in the change point set C as the taint source, and combine it with the definition obtained in Step 2-6 - use the relation information D to perform taint analysis to obtain the taint status of variables in each basic block. If a variable in a basic block is tainted, then the basic block is considered tainted; if a variable in a basic block is not tainted, then the basic block is considered not tainted.

[0024] Optionally, the distance between each basic block and the taint source is calculated, and the specific steps are as follows:

[0025] Retrieve the basic block containing each taint source and store it in a queue TS = {ts1, ts2, ..., ts n}, where ts i This represents the basic block containing the i-th taint source;

[0026] The distance between each basic block and the taint source is calculated using the following formula:

[0027]

[0028] In the formula, bb represents the basic block whose distance is to be calculated, and ts i Shortest(bb, ts) represents the basic block containing the i-th taint source. i ) represents the basic block and the basic block ts where the taint source is located. i The shortest distance between them. If bb is ts i If contamination occurs, calculate bb and ts. i The shortest distance between bb and ts plus 1, is used as the distance between bb and ts. iThe distance between them d(bb, ts) i If the basic block is not ts i Pollution, by default bb and ts i The distance between them d(bb, ts) i () is infinity;

[0029] The final distance of a basic block is calculated based on the distance between the basic block and the basic block containing each taint source. There are three strategies, as shown in formulas (1)-(4):

[0030]

[0031] dist max (bb, TS) = max(d(bb, ts) i )), 1≤i≤n&d(bb,T i )<∞ (2)

[0032] dist min (bb, TS) = min(d(bb, ts) i )), 1≤i≤n (3)

[0033] dist(bb, TS) = dist avg (bb, TS)||dist max (bb, TS)||dist min (bb, TS) (4)

[0034] In the formula, dist(bb, TS) represents the distance between the basic block bb and the taint source queue TS.

[0035] Optionally, step 3 includes:

[0036] Step 3-1: Input the distance of each basic block and start the program instrumentation;

[0037] Step 3-2: Determine whether the basic blocks have been traversed. If they have been traversed, jump to step 3-5; if they have not been traversed, jump to step 3-3.

[0038] Step 3-3: Obtain the distance to the currently visited basic block and insert the distance into the current basic block;

[0039] Step 3-4: Obtain the next basic block to be accessed, and then jump to step 3-2;

[0040] Steps 3-5: Output the program under test that has been instrumented.

[0041] Optionally, step 4 includes:

[0042] Step 4-1: Set the initial state and input the initial seed test cases;

[0043] Step 4-2: Determine whether the fuzz test stopping condition has been met. If it has, proceed to step 4-8; otherwise, proceed to step 4-3.

[0044] Step 4-3: Select a seed test case. Choose a seed test case from the seed test case queue and use it as the parent to generate child test cases.

[0045] Step 4-4: Calculate the fitness of the seed test case. The seed test case needs to be allocated corresponding test resources according to its fitness. The fitness of the seed test case s is calculated as shown in formula (6):

[0046]

[0047] In the formula, s is the seed test case, p FUZZ ξ(s) is the fitness of s evaluated by traditional fuzz testing tools such as AFL, and ξ(s) is the execution path of s. It is the sum of the reciprocals of the distances between each basic block in the execution path of s, where |ξ(s)| represents the number of basic blocks in the execution path of s;

[0048] Step 4-5: Generation of child test cases. Based on the fitness obtained in step 4-4, allocate corresponding test resources to the seed to generate child test cases s';

[0049] Steps 4-6: Test case execution. Send the child test cases s' generated in step 4-5 to the instrumented program under test and monitor the execution of the test cases.

[0050] Step 4-7: Feedback Result Analysis. Analyze the running results of the child test case s' in the program under test. Based on the running results, decide whether to keep or discard the child test case. After the analysis is completed, jump to step 4-2.

[0051] Steps 4-8: The fuzzing test has met the specified stopping conditions; stop the fuzzing test.

[0052] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a fuzz testing method for continuous integration processes, which has the following beneficial effects:

[0053] 1. It can effectively utilize testing resources to perform fuzz testing on change points in continuous integration projects, thereby improving the efficiency of applying fuzz testing to continuous integration;

[0054] 2. By performing data flow analysis on the project under test and taking the change points as taint sources, taint analysis is performed to calculate the distance between each basic block and the taint source, so as to effectively guide fuzz testing to automatically generate a large number of test cases to test the change points generated by continuous integration.

[0055] 3. It can effectively reduce the overhead of manually building a large number of test cases during continuous integration, thereby reducing development and maintenance costs. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0057] Figure 1 This is a technical roadmap of the present invention;

[0058] Figure 2 This is a flowchart of the continuous integration difference analysis process of the present invention;

[0059] Figure 3 This is a flowchart of the data flow analysis of the present invention;

[0060] Figure 4 This is a flowchart of the program instrumentation process of the present invention;

[0061] Figure 5 This is a flowchart of the continuous integration information-guided fuzzy testing process of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] In continuous integration, multiple integrations may occur daily. Each integration involves automated builds and tests to detect integration errors as quickly as possible. To fully test changes generated during continuous integration, numerous test cases need to be manually built before testing begins, resulting in unnecessary manpower overhead and increased maintenance and development costs. Therefore, automatically generating test cases for changes generated during continuous integration, and then thoroughly testing these changes, is particularly necessary.

[0064] This invention discloses a fuzz testing method for continuous integration, such as... Figure 1 As shown, it includes:

[0065] Step 1: In a project using continuous integration, perform a difference analysis on two adjacent commits to obtain the difference information between them, and then identify the change points based on the difference information and store them in a change point set.

[0066] Step 2: During the project construction process, perform data flow analysis based on the change point information and obtain the data flow analysis results.

[0067] Step 3: Use the data flow analysis results to perform program instrumentation and obtain the instrumented program under test.

[0068] Step 4: Perform fuzz testing on the project under test, calculate the fitness of seed test cases during the testing process, and allocate corresponding test resources to them based on the fitness.

[0069] like Figure 2 As shown, step 1 includes:

[0070] Step 1-1: Initial state, input the item to be tested P;

[0071] Steps 1-2: Perform a difference analysis on two adjacent submissions in the tested project P to obtain the set of change points C = {c1, c2, ..., c...} n};

[0072] Steps 1-3: Output the set of change points C.

[0073] like Figure 3 As shown, step 2 includes:

[0074] Step 2-1: Initial state, input the item to be tested P;

[0075] Step 2-2: Determine whether the statements in the tested item P have been traversed. If the traversal is complete, jump to step 2-6; if the traversal is not complete, jump to step 2-3.

[0076] Steps 2-3: Access the next statement in the tested item P;

[0077] Step 2-4: Determine whether the current statement Stat is related to a change point. If it is related to a change point, proceed to step 2-5; otherwise, proceed to step 2-3.

[0078] Step 2-5: Perform definition-usage relationship analysis on the current statement Stat. If Stat reads from memory, it is considered that Stat has performed a "usage" operation on a variable; if Stat modifies memory, it is considered that Stat has performed a "definition" operation on a variable. After the analysis of Stat is complete, proceed to step 2-2.

[0079] Steps 2-6: Output all definitions related to change points in the tested item P - using relationship information D;

[0080] Steps 2-7: Input the set of change points C obtained in Step 1;

[0081] Step 2-8: Calculate the distance between each basic block and the taint source. Treat each change point in the change point set C as a taint source, and perform taint analysis using the definition obtained in Step 2-6 – using relational information D – to obtain the taint status of variables in each basic block. If a variable in a basic block is tainted, then the basic block is considered tainted; if a variable in a basic block is not tainted, then the basic block is considered untainted. The specific steps for calculating the distance between each basic block are as follows:

[0082] First, obtain the basic block containing each taint source and store it in the queue TS = {ts1, ts2, ..., ts}. n}, where ts i This represents the basic block containing the i-th taint source.

[0083] Then, the distance between each basic block and the taint source is calculated using the following formula:

[0084]

[0085] In the formula, bb represents the basic block whose distance is to be calculated, and ts i Shortest(bb, ts) represents the basic block containing the i-th taint source. i ) represents the basic block and the basic block ts where the taint source is located. i The shortest distance between them. If bb is ts i If contamination occurs, calculate bb and ts. i The shortest distance between them plus 1 (to avoid a distance of 0), is used as the distance between bb and ts. i The distance between them d(bb, ts) i If the basic block is not ts i Pollution, by default bb and ts i The distance between them d(bb, ts) i () is infinity.

[0086] Then, based on the distance between the basic block and the basic block containing each taint source, the final distance of the basic block is calculated. There are three strategies, as shown in formulas (1)-(4):

[0087]

[0088] dist max (bb, TS) = max(d(bb, ts) i )), 1≤i≤n&d(bb,T i )<∞ (2)

[0089] dist min (bb, TS) = min(d(bb, ts) i )), 1≤i≤n (3)

[0090] dist(bb, TS) = dist avg (bb, TS)||dist max (bb, TS)||dist min (bb, TS)(4)

[0091] In the formula, dist(bb, TS) represents the distance between the basic block bb and the taint source queue TS. There are three strategies for calculating the distance between the basic block and the taint source queue:

[0092] First, for strategy (1), the denominator cnt represents the number of taint sources between the basic block bb and all taint sources whose distance is not infinite; the numerator represents the sum of the distances between the basic block bb and all taint sources whose distance is not infinite.

[0093] Second, for strategy (2), the final result is the maximum value of the distances between the basic block bb and all taint sources, and it is not infinite. If the distances between the basic block bb and all taint sources are infinite, then the value of dist(bb,TS) is considered to be infinite.

[0094] Third, for strategy (3), the final result is the minimum distance between the basic block bb and all taint sources.

[0095] like Figure 4 As shown, step 3 includes:

[0096] Step 3-1: Input the distance of each basic block and start the program instrumentation;

[0097] Step 3-2: Determine whether the basic blocks have been traversed. If they have been traversed, jump to step 3-5; if they have not been traversed, jump to step 3-3.

[0098] Step 3-3: Obtain the distance to the currently visited basic block and insert the distance into the current basic block;

[0099] Step 3-4: Obtain the next basic block to be accessed, and then jump to step 3-2;

[0100] Steps 3-5: Output the program under test that has been instrumented.

[0101] like Figure 5 As shown, step 4 includes:

[0102] Step 4-1: Initial state, input the initial seed test cases;

[0103] Step 4-2: Determine whether the fuzz test stopping condition has been met. If it has, proceed to step 4-8; otherwise, proceed to step 4-3.

[0104] Step 4-3: Select a seed test case. Select a seed test case from the seed test case queue and use it as the parent to generate child test cases.

[0105] Step 4-4: Seed fitness calculation. Seed test cases need to be allocated corresponding test resources according to their fitness. The fitness calculation of seed test case s is shown in formula (6):

[0106]

[0107] In the formula, s is the seed test case, p FUZZ ξ(s) is the fitness of s evaluated by traditional fuzz testing tools such as AFL, and ξ(s) is the execution path of s. It is the sum of the reciprocals of the distances between each basic block in the execution path of s, and |ξ(s)| represents the number of basic blocks in the execution path of s.

[0108] Step 4-5: Generation of child test cases. Based on the fitness obtained in Step 4-4, allocate corresponding test resources to the seed to generate child test cases s'.

[0109] Steps 4-6: Test Case Execution. Send the child test cases s' generated in Step 4-5 to the instrumented program under test and monitor the execution of the test cases.

[0110] Step 4-7: Feedback Result Analysis. Analyze the execution results of the child test case s' in the program under test to decide whether to keep or discard the child test case. After the analysis is complete, proceed to step 4-2.

[0111] Steps 4-8: The fuzzing test has met the specified stopping conditions; stop the fuzzing test.

[0112] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0113] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fuzz testing method for continuous integration processes, characterized in that, Includes the following steps: Step 1: In a project using continuous integration, perform a difference analysis on two adjacent commits to obtain the difference information between them, and then identify the change points based on the difference information and store them in a change point set. Step 1 includes: Step 1-1: Set the initial state and input the item to be tested. ; Steps 1-2: Test the item Perform a difference analysis on two adjacent submissions to obtain a set of change points. ; Steps 1-3: Output the set of change points ; Step 2: During the project construction process, perform data flow analysis based on change point information and obtain the data flow analysis results; Step 3: Use the data flow analysis results to perform program instrumentation to obtain the instrumented program under test; Step 3 includes: Step 3-1: Input the distance of each basic block and start the program instrumentation; Step 3-2: Determine whether the basic blocks have been traversed. If they have been traversed, jump to step 3-5; if they have not been traversed, jump to step 3-3. Step 3-3: Obtain the distance to the currently visited basic block and insert the distance into the current basic block; Step 3-4: Obtain the next basic block to be accessed, and then jump to step 3-2; Steps 3-5: Output the tested program after instrumentation is complete; Step 4: Perform fuzz testing on the program under test. During the testing process, calculate the fitness of the seed test cases and allocate corresponding test resources to them based on the fitness. Step 4 includes: Step 4-1: Set the initial state and input the initial seed test cases; Step 4-2: Determine whether the fuzz test stopping condition has been met. If it has, proceed to step 4-8; otherwise, proceed to step 4-3. Step 4-3: Select a seed test case. Select a seed test case from the seed test case queue and use it as the parent to generate child test cases. Step 4-4: Calculate the fitness of the seed test cases. Seed test cases need to be allocated corresponding test resources based on their fitness. The fitness is calculated as shown in formula (6): In the formula, These are seed test cases. Evaluation of AFL traditional fuzz testing tools fitness yes Execution path, yes The sum of the reciprocals of the distances between each basic block in the execution path. express The number of basic blocks in the execution path. The basic block representing the distance to be calculated. Represents the taint source queue; Steps 4-5: Generating child test cases; Based on the fitness obtained in step 4-4, allocate corresponding test resources to the seed, thereby generating offspring test cases. ; Steps 4-6: Test case execution, executing the child test cases generated in Step 4-5. Send the test cases to the instrumented program under test and monitor their execution. Steps 4-7: Feedback Result Analysis, Analyzing Sub-generation Test Cases Based on the running results in the program under test, decide whether to keep or discard the child test cases. After the analysis is completed, jump to step 4-2. Steps 4-8: The fuzzing test has met the specified stopping conditions; stop the fuzzing test.

2. The fuzz testing method for continuous integration processes according to claim 1, characterized in that, Step 2 includes: Step 2-1: Set the initial state and input the item to be tested. ; Step 2-2: Determine the item being tested If the statements in the query have been traversed, jump to step 2-6; otherwise, jump to step 2-3. Steps 2-3: Access the project under test The next statement in; Steps 2-4: Determine the current statement Is it related to the change point? If it is related to the change point, proceed to step 2-5; if it is not related to the change point, proceed to step 2-3. Steps 2-5: For the current statement Define - use relation analysis; if If memory has been read, then it is considered... A "use" operation was performed on a variable; if If memory has been modified, then it is considered... A "definition" operation was performed on a certain variable; After the analysis is complete, proceed to step 2-2; Steps 2-6: Output the tested items All definitions related to change points use relational information. ; Steps 2-7: Input the set of change points obtained in Step 1 ; Steps 2-8: Calculate the distance between each basic block and the taint source; set the change points. Each change point in the data is treated as a taint source, and combined with the definition-use relationship information obtained in steps 2-6. Perform taint analysis to obtain the taint status of variables in each basic block. If a variable in a basic block is tainted, the basic block is considered tainted; if a variable in a basic block is not tainted, the basic block is considered not tainted.

3. A fuzz testing method for continuous integration processes according to claim 2, characterized in that, The specific steps for calculating the distance between each basic block and the taint source are as follows: Retrieve the basic block containing each taint source and store it in a queue. ,in, Indicates the first The basic block where the source of the blemish is located; The distance between each basic block and the taint source is calculated using the following formula: In the formula, The basic block representing the distance to be calculated. Indicates the first The basic block where the blemish source is located. This indicates the basic block and the basic block where the taint source is located. The shortest distance between; if quilt If it is contaminated, then calculate. and The shortest distance between them plus 1, as and Distance between If the basic block is not... Pollution, by default and Distance between It is infinite; The final distance of a basic block is calculated based on the distance between the basic block and the basic block containing each taint source. There are three strategies, as shown in formulas (1)-(4): In the formula, Representing basic blocks With taint source queue The distance between them.

Images