Complex software system testing method based on fault propagation path coverage

Abstract

The invention discloses a complex software system test method based on fault propagation path coverage, which realizes accurate simulation and capture of a dynamic propagation process of a defect between modules by introducing a fault propagation path model and combining static slicing and dynamic slicing technologies, comprehensively covers propagation paths possibly existing in a complex software system, and improves the test efficiency of the complex software system. And the limitation of the traditional method in the aspect of dynamic defect detection is overcome. Meanwhile, through a seed defect injection strategy, potential defects can be excited at key propagation nodes, associated defects between modules are further mined, and the detection problem caused by module interactivity is solved. In addition, according to the method, test case expansion and fault propagation path analysis are achieved through an automatic process, manual intervention is remarkably reduced, the time cost and resource consumption in the test process are reduced, the test coverage rate and defect detection capacity of a complex software system are comprehensively improved, and an important guarantee is provided for reliability and safety of the system.

Description

Technical Field

[0001] This application relates to the technical field of software system testing, and particularly to a testing method for complex software systems based on fault propagation path coverage. Background Art

[0002] With the increasing complexity of modern software systems, software development and testing have become the core links to ensure software quality. However, traditional software testing methods mainly focus on defect detection in single modules or components, and cannot meet the growing testing requirements of complex software systems. Especially in complex software systems with high interaction between modules, defects may exist in the form of dynamic propagation and gradually spread through multiple modules or components, forming systematic faults or failures. This fault propagation characteristic poses great challenges to traditional software testing methods, and there is an urgent need for a testing method that can effectively cover the fault propagation path to more comprehensively discover potential associated defects and improve the reliability and security of software.

[0003] Currently, a variety of testing methods have been proposed in the field of software testing, such as boundary testing, mutation testing, and defect pattern-based testing. These methods have certain advantages in detecting single-module defects. However, most of these traditional methods assume that defects exist independently and do not fully consider the characteristic of dynamic propagation of defects through module interaction in complex systems. Therefore, they show obvious limitations in dealing with associated defects between modules. For example, although existing path coverage-based testing methods can cover the logical paths of programs, it is difficult to identify dynamic associated defects in complex systems because the concept of fault propagation paths is not introduced. In addition, to improve testing efficiency, intelligent algorithms (such as genetic algorithms and simulated annealing algorithms) have also been introduced into test case generation, but these algorithms often lead to a decrease in coverage rate and efficiency due to getting stuck in local optimal solutions when facing the huge input space of complex systems.

[0004] Therefore, in the testing of complex software systems, the significant dynamic propagation characteristic of defects, the complex association between modules, and the low efficiency of test case generation have become problems that need to be solved urgently. Summary of the Invention

[0005] This application provides a testing method for complex software systems based on fault propagation path coverage, aiming to solve the problems of significant dynamic propagation characteristic of defects, complex association between modules, and low efficiency of test case generation in the testing of existing complex software systems.

[0006] A testing method for complex software systems based on fault propagation path coverage, the method includes:

[0007] Based on program slicing technology, establish a fault propagation path model for the target program to describe the dynamic propagation path of defects between modules;

[0008] Inject seed defects into the target program to trigger potential defects through the seed defects; based on the fault propagation path model, design an initial test case set and run the target program to detect whether the seed defects are covered;

[0009] According to the dual criteria of defect coverage rate and statement coverage rate, expand the test case set to generate test cases that cover the fault propagation path;

[0010] Based on the expanded test case set, conduct a comprehensive test on the fault propagation path to identify and locate potential associated defects;

[0011] The test sufficiency discrimination based on the fault propagation path includes defect coverage rate, statement coverage rate, and algorithm termination conditions.

[0012] In the above solution, optionally, the establishment of the fault propagation path model includes:

[0013] Analyze the static dependency relationship of the program through static slicing technology;

[0014] Analyze the dynamic running path of the program under specific input conditions through dynamic slicing technology;

[0015] Generate a model of the fault propagation path according to the results of static slicing and dynamic slicing.

[0016] In the above solution, optionally, the injected seed defects include:

[0017] Design seed defects based on the historical defect data of the target program;

[0018] Inject seed defects at the start node and key nodes of the fault propagation path;

[0019] The injected seed defects include types such as logical errors, undefined variables, and conditional branch errors.

[0020] In the above solution, optionally, the process of designing the initial test case set includes:

[0021] Based on the functional requirements of the target program, perform equivalent class partitioning of the input domain;

[0022] Randomly select test inputs from the equivalent classes as the initial test cases;

[0023] Based on the fault propagation path model, verify the coverage of the initial test cases for the critical path.

[0024] In the above solution, optionally, the process of expanding the test case set adopts an improved intelligent algorithm, and the algorithm includes the following steps:

[0025] Based on the initial test case set, dynamically generate new test cases;

[0026] Calculate the defect coverage rate and statement coverage rate of the generated test case set;

[0027] Screen the test cases through a two-layer state receiving function, where the outer-layer state receiving function is based on the defect coverage rate, and the inner-layer state receiving function is based on the statement coverage rate;

[0028] Iteratively optimize the screened test cases to generate a test case set that covers more fault propagation paths.

[0029] In the above solution, optionally, the intelligent algorithm adopts a parameter optimization strategy based on dynamic adjustment, including:

[0030] Set the initial parameter value to T0, and dynamically adjust the parameter value based on the adjustment coefficient k;

[0031] When the parameter value drops to the set threshold, stop generating new test cases;

[0032] In each adjustment process, dynamically generate test cases and verify the defect coverage rate and statement coverage rate of the test cases.

[0033] In the above solution, optionally, the test of the fault propagation path includes:

[0034] Combine the test cases with the seeded defects to generate a dynamic slice of the fault propagation path;

[0035] Analyze the dynamic slice to identify potential defects on the path;

[0036] Based on the propagation characteristics of the potential defects, expand the test case set to cover new fault propagation paths.

[0037] In the above solution, optionally, the injected seeded defects are generated by program mutation technology, and the mutation technology includes:

[0038] Introduce logical mutation, arithmetic mutation or structural mutation into the source code of the target program;

[0039] Test the mutated target program to verify the coverage of the injected defects;

[0040] Adjust the mutation parameters according to the coverage rate of the seeded defects to generate more representative seeded defects.

[0041] In the above solution, optionally, the test sufficiency discrimination includes:

[0042] Whether the defect coverage rate of the test cases reaches the preset coverage threshold;

[0043] Whether the statement coverage rate of the test cases reaches the preset coverage threshold;

[0044] Whether the test termination conditions are met, including algorithm convergence and maximum running time.

[0045] In the above solution, optionally, the final test case set of the method covers all injected seeded defects and their associated fault propagation paths, and has the ability to identify potential associated defects in the target program.

[0046] Compared with the prior art, the present application has at least the following beneficial effects:

[0047] Based on further analysis and research of the problems of the prior art, the present application recognizes the problems of significant dynamic propagation characteristics of defects, complex inter-module correlation, and low test case generation efficiency in the testing of existing complex software systems. By introducing a fault propagation path model and combining static slicing and dynamic slicing techniques, it realizes the accurate simulation and capture of the dynamic propagation process of defects between modules, comprehensively covers the possible propagation paths in complex software systems, and overcomes the limitations of traditional methods in dynamic defect detection. At the same time, through the seeded defect injection strategy, potential defects can be excited at key propagation nodes, further mining the associated defects between modules, and solving the detection problems caused by module interactivity. Combined with an improved intelligent algorithm, test cases are dynamically generated and screened based on the dual criteria of defect coverage rate and statement coverage rate, avoiding the problem that traditional methods are prone to falling into local optimal solutions and significantly improving the test case generation efficiency. In addition, this method realizes the expansion of test cases and the analysis of fault propagation paths through an automated process, significantly reducing manual intervention, reducing the time cost and resource consumption in the testing process, comprehensively improving the test coverage rate and defect detection ability of complex software systems, and providing an important guarantee for the reliability and security of the system. Description of the Drawings

[0048] Figure 1 It is a schematic flowchart of a method for testing a complex software system based on fault propagation path coverage provided by an embodiment of the present application;

[0049] Figure 2 It is a schematic diagram of an example program of a method for testing a complex software system based on fault propagation path coverage provided by an embodiment of the present application;

[0050] Figure 3 It is one of the schematic diagrams of slicing examples of a method for testing a complex software system based on fault propagation path coverage provided by an embodiment of the present application;

[0051] Figure 4 The second schematic diagram of a slice example of a complex software system testing method based on fault propagation path coverage provided by an embodiment of the present application;

[0052] Figure 5 The schematic diagram of the improved intelligent algorithm process of a complex software system testing method based on fault propagation path coverage provided by an embodiment of the present application;

[0053] Figure 6 The schematic diagram of the testing process of a complex software system testing method based on fault propagation path coverage provided by an embodiment of the present application;

[0054] Figure 7 The schematic diagram of test case generation of a complex software system testing method based on fault propagation path coverage provided by an embodiment of the present application;

[0055] Figure 8 The flowchart of the testing method of a complex software system testing method based on fault propagation path coverage provided by an embodiment of the present application;

[0056] Figure 9 The schematic diagram of the research process of a complex software system testing method based on fault propagation path coverage provided by an embodiment of the present application;

[0057] Figure 10 The schematic diagram of the defect transfer object relationship of a complex software system testing method based on fault propagation path coverage provided by an embodiment of the present application. Detailed implementation manners

[0058] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] With the increasing complexity of modern software systems, software development and testing have become the core links to ensure software quality. However, traditional software testing methods mainly focus on defect detection of single modules or components and cannot meet the growing testing requirements of complex software systems. Especially in complex software systems with high interaction between modules, defects may exist in the form of dynamic propagation and gradually spread through multiple modules or components, forming systematic faults or failures. This fault propagation characteristic poses great challenges to traditional software testing methods, and there is an urgent need for a testing method that can effectively cover the fault propagation path to more comprehensively discover potential associated defects and improve the reliability and security of software.

[0060] At present, a variety of testing methods have been proposed in the field of software testing, such as boundary testing, mutation testing, and defect pattern-based testing. These methods have certain advantages in detecting single-module defects. However, most of these traditional methods assume that defects exist independently and do not fully consider the characteristics of dynamic propagation of defects through module interactions in complex systems. Therefore, they show obvious limitations in dealing with inter-module related defects. For example, although existing path coverage-based testing methods can cover the logical paths of programs, due to the failure to introduce the concept of fault propagation paths, it is difficult to identify dynamic related defects in complex systems. In addition, to improve testing efficiency, intelligent algorithms (such as genetic algorithms, simulated annealing algorithms) have also been introduced into test case generation. However, when facing the huge input space of complex systems, these algorithms often fall into local optimal solutions, resulting in a decrease in coverage rate and efficiency. To sum up, the existing technologies still have the following problems in dealing with fault propagation path coverage and dynamic related defect detection in complex software systems:

[0061] Traditional methods are limited to single-module defect detection and cannot fully capture defects dynamically propagated in inter-module interactions;

[0062] Existing path coverage methods are difficult to handle potential error propagation paths in complex systems;

[0063] Intelligent algorithms have certain improvements in the efficiency and coverage rate of generating test cases, but lack precise support for dynamic fault propagation paths, resulting in insufficient defect detection capabilities.

[0064] In view of the above background, this application aims to solve the problem that existing software testing methods cannot effectively detect related defects on dynamic propagation paths in complex software systems, and proposes a testing method for complex software systems based on fault propagation path coverage. By introducing a fault propagation path model and an improved intelligent algorithm, this method can accurately cover potential defects on dynamic propagation paths. Specifically, the objectives of this application are:

[0065] Provide a testing method that can cover dynamic fault propagation paths in complex software systems to improve the coverage rate of test cases and defect detection capabilities;

[0066] Optimize the test case generation process through an improved intelligent algorithm to avoid falling into local optimal solutions and achieve more efficient test case generation;

[0067] Automate the testing process, reduce manual intervention and resource consumption, and improve the testing efficiency and reliability of complex software systems.

[0068] In one embodiment, as Figure 1 shown, a testing method for complex software systems based on fault propagation path coverage is provided, including the following steps:

[0069] Based on program slicing technology, establish a fault propagation path model for the target program to describe the dynamic propagation path of defects between modules;

[0070] Inject seed defects into the target program to trigger potential defects through the seed defects; based on the fault propagation path model, design an initial test case set and run the target program to detect whether the seed defects are covered;

[0071] According to the dual criteria of defect coverage rate and statement coverage rate, expand the test case set to generate test cases that cover the fault propagation path;

[0072] Based on the expanded test case set, comprehensively test the fault propagation path to identify and locate potential associated defects;

[0073] The test sufficiency discrimination based on the fault propagation path includes defect coverage rate, statement coverage rate, and algorithm termination conditions.

[0074] In this embodiment, a fault propagation path model is constructed through program slicing technology, which is one of the core steps of the whole method. Program slicing technology includes two types: static slicing and dynamic slicing:

[0075] Static slicing: Analyze the definition and dependency relationships of variables in the program to determine the structural defect propagation path of the program. Static slicing can discover potential propagation paths by analyzing the code syntax tree and control flow graph without running the program.

[0076] Dynamic slicing: According to the actual input conditions, track the running state of the program and capture the propagation behavior of defects in a dynamic environment. Dynamic slicing can simulate the propagation mode of defects in real scenarios and make up for the deficiencies of static slicing under actual running conditions.

[0077] Combine static slicing and dynamic slicing to generate a fault propagation path model for the target program to clearly describe the process of defects propagating from one module to other modules.

[0078] Inject seed defects into the key modules of the target program as an important means to trigger potential defects. The design of seed defects includes:

[0079] According to the historical defect types of the target program, select the defect types most likely to cause fault propagation (such as logical errors, conditional branch errors, etc.).

[0080] The injection location is selected at the starting node of the propagation path or the key interaction module to ensure that the propagation of defects can be fully triggered.

[0081] Use program mutation technology to mutate the target program, simulate real defect scenarios, and inject mutation operators to generate seed defects.

[0082] Based on the functional requirements of the target program, an initial test case set is designed using the equivalence class partitioning method. Specifically, it includes:

[0083] The input domain is divided into multiple equivalence classes, and each equivalence class represents a specific input condition.

[0084] Several test cases are randomly selected from each equivalence class to cover the main input range of the program as much as possible.

[0085] Verify the coverage of the initial test case set for the fault propagation path to ensure that it can trigger the propagation of seed defects.

[0086] Based on the initial test case set, an improved intelligent algorithm is used to dynamically expand the test case set. The implementation steps of the intelligent algorithm include:

[0087] Dynamically generate new test cases and calculate their defect coverage rate and statement coverage rate.

[0088] Use a two-layer state reception function for screening, where the outer function is based on the defect coverage rate and the inner function is based on the statement coverage rate to ensure that the newly generated test cases have higher effectiveness.

[0089] Continuously iterate and optimize the test case set until all fault propagation paths are covered.

[0090] Run the target program using the expanded test case set, analyze the fault propagation path through dynamic slicing technology, and identify potential defects. The specific operations include:

[0091] Generate a dynamic slice of the fault propagation path according to the test cases and seed defects, and trace the propagation process of the defect on the path.

[0092] Analyze each key node in the slice to locate hidden potential defects.

[0093] Further expand the test case set to cover new fault propagation paths.

[0094] After the test is completed, judge whether the test is sufficient according to the results of the defect coverage rate and the statement coverage rate. Specifically, it includes:

[0095] Judge whether the defect coverage rate reaches the preset threshold.

[0096] Judge whether the statement coverage rate reaches the expected standard.

[0097] Verify whether the termination conditions of the algorithm are met (such as temperature threshold, running time limit, etc.).

[0098] In complex software systems, defects often exist in the form of dynamic propagation and gradually spread through multiple modules. Traditional methods cannot effectively capture this dynamic propagation characteristic, while the present invention realizes the accurate simulation and capture of the dynamic propagation process of defects by introducing a fault propagation path model and combining dynamic slicing technology. This characteristic greatly improves the detection ability of dynamic defects. In complex software systems, the high interactivity between modules makes the detection of associated defects difficult. Traditional testing methods cannot cover the associated defects between modules, while the present invention can explicitly model the fault propagation path between modules through program slicing technology and seed defect injection strategy, and can discover the associated defects that are easily overlooked in traditional methods.

[0099] In this embodiment, an improved intelligent algorithm is used to dynamically generate test cases. Combining the dual criteria of defect coverage rate and statement coverage rate, the test cases are screened through a two-layer state reception function. Compared with the problem that traditional algorithms are prone to falling into local optimal solutions, this method can dynamically adjust input conditions and efficiently generate test cases with high coverage rate, thus significantly improving the test efficiency. By introducing the fault propagation path model, it is ensured that the test cases can cover the dynamic propagation paths and potential associated defects in the program. This high-coverage testing method effectively makes up for the deficiencies of traditional methods in terms of coverage range and can discover more hidden defects. By accurately detecting potential defects in complex software, especially the dynamic associated defects that are difficult to capture by traditional methods, this method significantly reduces the failure risk of the system during operation and provides guarantee for the stable operation of the software. The process of generating test cases and analyzing the fault propagation path is highly automated, reducing the need for manual intervention, significantly reducing the time and labor costs during the testing process, and improving the overall efficiency of testing complex software systems.

[0100] In summary, through the coverage of the fault propagation path and the improvement of the intelligent algorithm, this method can effectively solve the problems of significant dynamic propagation characteristics, complex inter-module correlation, and low efficiency of test case generation mentioned in the background technology, and provides a comprehensive solution for the efficient testing of complex software systems.

[0101] In this embodiment, establishing the fault propagation path model includes:

[0102] Analyzing the static dependency relationship of the program through static slicing technology;

[0103] Analyzing the dynamic operation path of the program under specific input conditions through dynamic slicing technology;

[0104] Generating a model of the fault propagation path according to the results of static slicing and dynamic slicing.

[0105] In this embodiment, the injected seed defects include:

[0106] Design seed defects based on the historical defect data of the target program;

[0107] Inject seed defects into the starting nodes and key nodes of the fault propagation path;

[0108] The injected seed defects include logical errors, undefined variables, and conditional branch error types.

[0109] In this embodiment, the process of designing the initial test case set includes:

[0110] Based on the functional requirements of the target program, perform equivalence class partitioning of the input domain;

[0111] Randomly select test inputs from the equivalence classes as initial test cases;

[0112] Based on the fault propagation path model, verify the coverage of the initial test cases for the critical path.

[0113] In this embodiment, the process of expanding the test case set adopts an improved intelligent algorithm, and the algorithm includes the following steps:

[0114] Based on the initial test case set, dynamically generate new test cases;

[0115] Calculate the defect coverage rate and statement coverage rate of the generated test case set;

[0116] Screen test cases through a double-layer state receiving function, where the outer-layer state receiving function is based on the defect coverage rate, and the inner-layer state receiving function is based on the statement coverage rate;

[0117] Iteratively optimize the screened test cases to generate a test case set that covers more fault propagation paths.

[0118] In this embodiment, the intelligent algorithm adopts a parameter optimization strategy based on dynamic adjustment, including:

[0119] Set the initial parameter value as T0, and dynamically adjust the parameter value based on the adjustment coefficient k;

[0120] When the parameter value drops to the set threshold, stop generating new test cases;

[0121] In each adjustment process, dynamically generate test cases and verify the defect coverage rate and statement coverage rate of the test cases.

[0122] In this embodiment, the testing of the fault propagation path includes:

[0123] Combine the test cases with the seed defects to generate a dynamic slice of the fault propagation path;

[0124] Analyze dynamic slices to identify potential defects in the path;

[0125] Based on the propagation characteristics of potential defects, expand the test case set to cover new fault propagation paths.

[0126] In this embodiment, the injected seeded defects are generated by program mutation techniques, and the mutation techniques include:

[0127] Introduce logical mutations, arithmetic mutations, or structural mutations into the source code of the target program;

[0128] Test the mutated target program to verify the coverage of the injected defects;

[0129] Adjust the mutation parameters according to the coverage rate of the seeded defects to generate more representative seeded defects.

[0130] In this embodiment, the test sufficiency discrimination includes:

[0131] Whether the defect coverage rate of the test cases reaches a preset coverage threshold;

[0132] Whether the statement coverage rate of the test cases reaches a preset coverage threshold;

[0133] Whether the test termination conditions are met, including algorithm convergence and maximum running time.

[0134] In this embodiment, the final test case set of the method covers all the injected seeded defects and their associated fault propagation paths, and has the ability to identify potential associated defects in the target program.

[0135] In one embodiment, a testing method for complex software systems based on fault propagation path coverage is provided. By covering the fault propagation paths in complex software systems, potential associated defects are detected. A fault propagation path refers to the process in which defects gradually propagate to form faults during the interaction between modules, between software, and between software and hardware. This method is applicable to complex systems with dynamic propagation and uncertainty, especially those associated defects that are difficult to detect using traditional testing strategies.

[0136] This technology proposes a testing method for complex software systems based on fault propagation path coverage, aiming to detect potential associated defects by covering the fault propagation paths in complex software systems. A fault propagation path refers to the process in which defects gradually propagate to form faults during the interaction between modules, between software, and between software and hardware. This method is particularly applicable to complex systems with dynamic propagation and uncertainty, especially those associated defects that are difficult to detect using traditional testing strategies.

[0137] The software testing method involved in this embodiment is based on path coverage and focuses on the coverage of fault propagation paths. During the testing process, through the path identification of the control flow, it is ensured that the test cases traverse specific paths to capture potential defects. This method is usually applied to complex software systems. Due to the large number of paths and their mutual dependencies, it is difficult for traditional testing methods to ensure the coverage rate. The fault propagation path coverage method discovers more potential defects by dynamically generating test cases, relying on the program structure and the characteristics of fault propagation.

[0138] Compared with traditional defect detection methods, methods such as boundary testing based on boundary assumptions and testing based on defect patterns mainly focus on discovering defects in a single module, often ignoring the defect correlation and dynamic propagation process between modules in a complex system. This makes them limited in effectively discovering interrelated and dynamically propagating defects and unable to fully solve the fault propagation problem in complex software systems. To improve the efficiency of path coverage technology, we propose a dynamic path generation method based on a new intelligent algorithm. This improved method shows higher efficiency in path search and defect detection. By dynamically generating test cases during the path coverage process, it can significantly improve the test coverage rate.

[0139] The core of this technical solution lies in automatically generating test cases with high defect detection capabilities through the coverage of fault propagation paths. This method can not only discover obvious defects but also capture deeper potential defects, filling the deficiencies in the coverage scope and defect detection capabilities of traditional testing methods.

[0140] The Chinese patent "A Method and System for Generating Test Cases with Multi-Path Coverage" proposes a method and system for generating test cases with multi-path coverage. By clustering the initial test case set, generating a genetic population and inputting it into a fitness prediction model for prediction, the test cases are screened according to the fitness values. For the cases that meet the requirements, the system will perform program instrumentation to obtain the covered paths. If the coverage condition is not met, case mutation is carried out to generate a new population. Through iterative loops and fitness prediction, excellent cases that cover multi-objective paths are optimized, improving the test case generation efficiency and reducing the number of instrumentation times.

[0141] The Chinese patent "Unit Testing Method for Automatic Generation of Test Cases Based on Path Coverage" introduces a multi-path coverage test case generation system that uses genetic algorithms and fitness prediction models to improve the efficiency of test case generation. First, a genetic population of initial test cases is generated through a clustering method, and its fitness is calculated. Eligible test cases will be instrumented to obtain covered paths. If the coverage requirements of the target paths are not met, the test cases will be extracted and mutated to form a new mutant population. Through iterative loops, this method continuously optimizes the test cases and finally outputs excellent test cases that cover all target paths, significantly improving the test efficiency and reducing the time cost.

[0142] This embodiment proposes a new software testing method to overcome the deficiencies of the prior art, especially for optimizing the fault propagation characteristics in complex software systems. The specific objectives include:

[0143] Solving the limitations of traditional testing methods: Existing testing methods often fail to effectively identify and detect associated defects, especially in complex environments with multi-module interactions. The present invention designs a software testing method that can cover these paths by introducing the concept of "fault propagation paths" to discover potential defects that are easily overlooked in traditional testing.

[0144] Addressing the challenges of dynamic propagation defects: The fault propagation in complex software systems is dynamic and uncertain. By establishing a fault propagation path model, the present invention hopes to more accurately simulate the propagation process of defects in the system, thereby achieving effective detection of associated defects and enhancing the reliability and security of the software.

[0145] Improving test efficiency and coverage: The present invention also aims to automatically generate test cases using an improved intelligent algorithm that combines the dual criteria of defect coverage and statement coverage. This can ensure more potential defects are covered while maintaining test efficiency, thus enhancing the comprehensiveness and effectiveness of the test.

[0146] Reducing manual intervention and resource consumption: By automating the test case generation process, the time and labor costs required for manual testing are significantly reduced, making the software testing process more efficient and economical and capable of meeting the growing test requirements of complex software systems.

[0147] This embodiment provides a testing method for complex software systems based on fault propagation path coverage, which includes the following main concepts and operations:

[0148] Modeling method of fault propagation paths based on program slicing:

[0149] In view of the characteristics of associated defects in complex software, seed defects are designed, and a fault propagation path model and a modeling method for fault propagation paths based on slicing are proposed on the basis of the original fault propagation model based on data flow and control flow, providing a platform for the generation of test cases.

[0150] During the software testing process, we often encounter such problems: after a large-scale test of a software, a small modification is made to the software, which is bound to affect other parts of the program. Do we still need to conduct a large-scale test on the program again? The use of program slicing solves this problem well. First, find the differences between the old and new versions of the program, compare their slices and dependency graphs, and do not consider those nodes with the same slices. Mark the nodes that appear in the new version dependency graph but not in the old version dependency graph as "impact points", calculate its static slice at the program input statement and the forward slice at the modified statement, and take the intersection of the two. In this way, only this intersection needs to be tested.

[0151] Similarly, when testing for fault propagation in a program, what we need to do is to discover these interrelated errors and find all the statements related to error propagation. Then, using error propagation slices, this can be easily achieved. For example, using error propagation slices, a slice with far fewer statements than the source program can be obtained based on the test case where an error is found, enabling testers to clearly obtain the path of error propagation in the program.

[0152] Definition 1: Error propagation slice: For variables i and j, when the input sequence is X, at the n and m states where associated errors occur in the program, the slice defined relative to the dynamic slice standard (X, n, m, i, j) is the statements and assertions that affect the transfer of the error from the variable i defined or controlled at the n state to the variable j defined or controlled at the m state when the input is x and at the n and m states.

[0153] Suppose a program is as Figure 2 shown. Change the second line of the program to if(a>b), and set test cases for the program. That is, when the input sequence X = {2, 2, 1, 1}, an undefined variable e will be output on the twelfth line of the program. Then we say that when the input sequence is X, the error in the program propagates from the value defined by if(a>b) on the second line to the variable e defined on the twelfth line. Therefore, we can make a slice (X, 2, 12, (a>=b), e), as Figure 3 shown;

[0154] Therefore, the fault propagation path mentioned in the above embodiment can be described in the form of Figure 8 That is to say, Figure 3The error propagation slice in this indicates the propagation path of errors from the second line to the twelfth line in this program.

[0155] In this way, after we obtain the error propagation path, when an error occurs in the program, we can locate the error to the path where it occurs and propagates, which helps testers quickly track errors and study the mechanism of error propagation in the program.

[0156] However, in the test method using the fault propagation path mentioned in the previous section, we mentioned using the injection of known seed defects to try to activate potential defects latent in the program that are not easily directly discovered by test cases. In this method, we only know the location where the error occurs, that is Figure 2 in the second line of the code in, if(a>=b) is wrongly written as if(a>b). And the related error existing in the twelfth line is unknown to us in advance. At this time, the information we know is only the input sequence X and the variable i controlled by the n state. Then we cannot construct an error propagation slice. At this time, we can use a static slice to describe the fault propagation path.

[0157] The program under test is still Figure 2 the code in. Still change the second line to if(a>b), then when the input sequence is X={2,2,1,1}, we can also use another form close to the static slice to represent all the branches that this error may cover under the current input, narrowing the scope of searching for potential errors to a static slice. As Figure 4 shown, that is to say, when the data we obtain is only the input sequence and the seed defect, we can use this static slice to describe an extension of the fault propagation path, and we expect to find potential defects associated with the seed defect on this extended path.

[0158] Test case generation based on fault propagation path coverage:

[0159] This step improves the Metropolis criterion in the algorithm and introduces a two-layer state acceptance function. In the application of existing intelligent algorithms in the generation of software test cases, the handling of inferior solutions is simply accepting them according to probability, which leads to the process of accepting inferior solutions being only for formal consideration, or blindly searching for global optimal solutions outside the local optimal solutions. However, in practical applications, due to efficiency and time limitations, global optimal solutions outside the optimal solutions rarely appear. To avoid this blindness in software testing, this paper introduces the idea of double verification combining defect coverage rate and statement coverage rate, that is, nesting another layer of state acceptance function in the original outer layer state acceptance function.

[0160] The improved algorithm flow is asFigure 5 As shown in Figure 5 the process shown in Figure 5 , the outer state receiving function f1 represents the number of known defects that have not been discovered. Let the function of the i-th solution be f1(i), and the function of the (i + 1)-th solution be f1(i + 1). When f1(i + 1) < f1(i), the new solution is received. When f1(i + 1) > f1(i), in addition to satisfying r > ξ, ξ ∈ [0, 1), another condition f2(i + 1) > f2(i) needs to be satisfied, where f2 is the statement coverage rate of the objective function under the current solution.

[0161] In this algorithm, we set the initial temperature as T = 100 °C, the temperature reduction coefficient as k, and the temperature reduction amplitude each time as kT. Therefore, T2 = T1 - kT. Randomly generate an initial test case set, and after running the program, obtain the results, that is, the number of covered paths and the number of newly discovered defects. So far, the first iteration ends. Then generate a new set of inputs from the input domain and run the program. Obtain the new results. Determine whether more paths are covered. If so, accept this new set of inputs and determine whether the termination condition of the algorithm is satisfied. If it is satisfied, the algorithm terminates. If it is not satisfied, reduce the temperature, and then search for a new test case set in the neighborhood of the current input. Then perform the next iteration. If the new input does not cover more paths, then we need to determine again whether more new defects are discovered. If so, accept this new set of inputs according to the probability. If not, discard the current input, search for a new test case set in the neighborhood of the original input, and start the next iteration. Until the algorithm ends.

[0162] Because the higher the statement coverage rate when the test case runs the program, the more defects may be discovered. After program mutation, the number and patterns of known defects are limited after all. Considering only the coverage of known defects during the iteration process cannot achieve the discovery of as many unknown defects as possible. Therefore, we also use the statement coverage rate as the iteration criterion. Under this criterion, accept the inferior solution according to the probability, which can avoid blindly accepting the inferior solution, and the new solution obtained will be more valuable.

[0163] Next, a test case generation method based on an improved intelligent algorithm will be proposed. Its core is to generate a test case set that can cover software defects. The improved intelligent algorithm mentioned above is used in the iteration process of generating the test case set, which improves the automation degree of this process, that is, the efficiency of software testing is improved. Its flowchart is as Figure 6 shown:

[0164] Program mutation testing:

[0165] Program mutation testing is a type of fault-based testing technique. It is a means of evaluating a test suite by injecting errors into the program code. The original intention of program mutation testing is to generate effective test data, and this technique has a strong ability to discover errors. In fact, all errors are represented by a model composed of a set of mutation operators. To generate a mutant program, only need to apply the relevant operators to the original program P. The mutated program is called P'. In P', the mutants are very similar to real defects. If the mutation operators are derived from existing fault models, then test cases that can discover known defects are reasonably believed to be able to discover real defects. Therefore, through the technique of program mutation, we ultimately aim to discover more unknown defects.

[0166] Initial test case set design:

[0167] Generate an initial test case set based on software requirements. When generating the initial test cases, we can perform equivalent class partitioning on the solution space based on the functions of the software, and randomly select a set of solutions from each equivalent class to form the initial solution set, that is, the initial test case set.

[0168] Generate a test case set based on intelligent algorithms:

[0169] This part improves the Metropolis criterion of the intelligent algorithm and introduces a double-layer state acceptance function.

[0170] In the application of the existing SA algorithm in software test case generation, the handling of inferior solutions is simply accepting them according to probability. This leads to the acceptance of inferior solutions only for formal considerations, or blindly searching for the global optimal solution outside the local optimal solution. In practical applications, due to efficiency and time limitations, the global optimal solution outside the local optimal solution rarely appears. To avoid this blindness in software testing, this paper introduces the idea of double verification combining defect coverage and statement coverage, that is, nesting another layer of state acceptance function in the original outer layer state acceptance function.

[0171] Because the higher the statement coverage rate when the test case runs the program, the more defects may be discovered. After program mutation, the number and patterns of known defects are limited after all. In the iterative process, only considering the coverage of known defects cannot achieve the discovery of as many unknown defects as possible. Therefore, we also use the statement coverage rate as the iteration criterion, and accept inferior solutions according to probability under this criterion. This will avoid blindly accepting inferior solutions, and the new solutions obtained will be more valuable.

[0172] Test sufficiency discrimination:

[0173] Cover known defects and determine whether to accept the new solution. Use the initial test case set as the input of the target program P', run the program, and use existing defect localization methods to discover the known defects injected in the target program P'. Determine whether to accept the current solution set by comparing the defect coverage rate and statement coverage rate obtained from this run. If accepted, expand based on the current solution set; if not, expand based on the old solution.

[0174] Test end criterion. The test will terminate under the following circumstances: the defect coverage rate reaches the test requirement; the temperature T in the algorithm cools down to 0°C. Take the test case set at the end of the algorithm as the final test result.

[0175] Search for the fault propagation path based on the improved intelligent algorithm:

[0176] In the actual software system testing, software testers will encounter this situation: even if a program is not too complex, the combination of its paths may be an extremely large number. Therefore, covering all paths in a program under test is a task with a huge workload. To solve this problem, the number of paths has to be compressed to a certain limit, such as only executing the loop body in the program once. This method generates test cases by finding a set of representative and reasonable path sets and generating test cases for each path. The test cases generated in this way can ensure that as many independent paths as possible are included while covering the basic path set.

[0177] However, the path coverage method is only a test method for covering the program logic structure, and the paths covered by the generated tests are all correct paths without errors. However, in a complex software system, even using the basic path coverage method, the derivation of the basic path set is also a task with a huge workload. Moreover, after all modules and components are tested and interact with each other to form a complex software system, among all the paths, the number of paths with errors must be much less than the number of correct paths. Furthermore, the methods mentioned above are difficult to discover potential defects that are difficult to detect in complex software. Therefore, if we can cover the paths with errors and artificially inject some potential defects that may be associated with them and exist on these paths at the starting stage of these paths, and finally design test cases with the injected known defects as the target. Construct a slice of the fault propagation path based on the test cases and the injected known defects, and we expect to discover potential defects on the constructed slice.

[0178] Therefore, this embodiment proposes a test case generation method for the fault propagation path. This method includes the design of the initial test case set, the cultivation and screening of seed defects, the automated generation of test cases using an improved intelligent algorithm, the establishment of a slice of the fault propagation path, and the discovery of potential defects that we believe are activated by the seed defects in the slice. After the test is completed, what we obtain is a test case set that can cover all seed defects. The process of obtaining the final test case set is as Figure 6 shown.

[0179] The establishment of the fault propagation model is based on two basic premises: 1. During the test process, it is easier to trigger other defects when a defect is activated and evolves into a failure, that is, defects have dynamic clustering. 2. The discovery of associated defects usually requires first activating a defect to trigger the associated defect. Based on the above two premises, the method principle of test case generation for the fault propagation path will be elaborated in detail below, as Figure 7 shown, specifically:

[0180] Cultivate seed defects:

[0181] Associated defects are also a manifestation of cascading faults at the code level. Therefore, when conducting software testing proposed in this paper, especially in white-box testing, we take associated defects as the focus of testing. One manifestation of the association relationship between associated defects is that when one defect does not exist, it is difficult to discover the other defect. In response to this situation, we introduce the concept of seed defects, that is, inject some known defects into the software, hoping that these defects are the defects associated with the undetected defects in the program. And design test cases to cover the seed defects we implanted to make them manifest.

[0182] Therefore, the design and cultivation of seeded defects become an important part in discovering potential defects. Seeded defects need to be carefully screened by us so that more associated defects can be discovered during the testing process. First of all, we can design seeded defects based on the experience of predecessors and the types of associated defects summarized during the previous associated defect testing process. This way of selecting seeded defects requires us to refer to as many existing literatures and materials as possible and collect the types of associated defects. This directly determines the number of potential defects discovered by the testing method oriented to the fault propagation path. Secondly, the sources of seeded defects during the testing process of complex software systems can be the defects discovered and eliminated during the testing of each autonomous software or each component before integration. Because strict testing must have been carried out before the integration of each software system, and a large number of defects have been discovered. However, during the testing process, testers may modify or eliminate a defect immediately after discovering it. For such defects, we can make two inferences: 1) After eliminating this defect, some other potential defects in this autonomous software or component may not be discovered; 2) If these defects are not eliminated, then after the integration of each component system, some other defects will be triggered. Therefore, based on the above two inferences, we have reason to believe that such discovered defects have high value for detecting potential defects in complex software systems.

[0183] To sum up, the process of selecting seeded defects is actually the method of guessing wrong in test case design. Guessing wrong means assuming what types of defects exist in the system and which types have strong ability to propagate and induce new defects. The stronger the ability of the designed seeded defects to induce new defects, the more problems will be discovered during the testing process, and the more sufficient the testing will be.

[0184] Generation of test cases:

[0185] The above embodiments mention implanting seeded defects in the software under test. Since the test object is complex software, in order to discover as many problems as possible, we must implant a large number of seeded defects, and then we must design more test cases to cover these defects. Experiments have proved that the improved intelligent algorithm has a powerful search function, so it can be applied to the automatic generation of test cases to search for test cases that can discover the seeded defects we implanted among a large number of test cases.

[0186] Detection of associated defects:

[0187] After automatically generating test cases that cover all seeded defects using intelligent algorithms, we have actually covered the fault propagation paths we established. That is, we believe that the seeded defects for injection have been activated. The next step is to verify whether these seeded defects induce associated potential defects along the propagation paths. Therefore, we can perform program slicing on the obtained test case set and the corresponding seeded defects, and then test on each fault propagation path represented by the slice to search for possible defects associated with the seeded defects.

[0188] This embodiment has significant innovation and research achievements in the following aspects:

[0189] Introduction of the fault propagation path model: Innovatively proposed the concept of fault propagation paths, emphasizing that in complex software systems, the propagation of defects occurs dynamically through interactions between modules, providing a theoretical basis for understanding and detecting associated defects in complex software.

[0190] Testing strategy for associated defects: Proposed a new testing method that focuses on covering fault propagation paths to identify and cover associated defects that are difficult to discover in traditional testing. By injecting "seeded defects", this method can activate potential defects and trace their propagation paths, thereby enhancing the comprehensiveness and effectiveness of defect detection.

[0191] Application and improvement of intelligent algorithms: Adopted an improved intelligent algorithm, combined with the dual criteria of defect coverage rate and statement coverage rate, to optimize the generation of test cases. This algorithm significantly improves the efficiency and coverage rate of test case generation, enabling more effective test cases to be generated in a shorter time to meet the requirements of complex systems.

[0192] Automated and intelligent testing process: Achieved the automation of test case generation, reduced manual intervention, and lowered testing costs and time. The automated process design makes this method highly flexible and adaptable in practical applications, and can efficiently handle the testing requirements of different complex software systems.

[0193] Fault propagation modeling: Proposed the concept of error propagation slicing, which is used to describe the paths and moments when defect A causes defect B to occur, as well as the corresponding test cases.

[0194] Method for generating test cases based on fault propagation path coverage: Elaborated in detail the generation of path coverage and basic path coverage test cases, emphasizing that in complex software systems, covering error paths will significantly reduce the resources required to generate the test case set.

[0195] As Figure 8 - Figure 10 shown, this embodiment provides a flowchart of the testing method for this solution, a schematic diagram of the research process, and a schematic diagram of the relationship of defect transfer objects.

[0196] This embodiment proposes a software testing method based on fault propagation path coverage. This method can discover associated defects that exist on the fault propagation path and can only be discovered in dynamic path coverage testing, and these defects are difficult to be discovered in testing methods based on state diagrams, branch structures, etc. In addition, covering the fault propagation path can solve the problem of missed or false reports of defects in the scenario where component failures may not necessarily cause system failures in complex software systems, that is, some potential defects. The ISAA intelligent algorithm adopted by this method has great value in generating a large number of test cases and search paths based on seed defects. This not only saves a large amount of manual work but also saves the time for generating software test cases.

[0197] The specific effects can be elaborated from the following aspects:

[0198] Improve defect detection rate: By introducing the "fault propagation path" model, the present invention can effectively identify and detect associated defects in complex software systems. These defects are often difficult to be discovered in traditional testing methods based on state diagrams or branch structures. In the experiment, an improved intelligent algorithm was used to generate test cases. Compared with traditional testing methods, the defect detection rate was significantly improved. Experimental data shows that the number of potential defects detected by using the method of the present invention is about 70% more than that of traditional methods, proving its effectiveness in complex scenarios.

[0199] Optimize the efficiency of test case generation: The present invention uses an intelligent algorithm to optimize the test case generation process. Under the same number of iterations, the average number of effective test cases generated is increased by more than 50% compared with traditional methods. This improvement in efficiency not only shortens the testing time but also enhances the coverage of the overall testing of complex software systems. In a specific experiment, the test case generation time was shortened from several hours to 30 minutes, greatly improving the testing efficiency.

[0200] Reduce manual intervention and resource consumption: Through automated test case generation, the present invention significantly reduces the need for manual intervention and reduces the resources and time required for testing. In practical applications, traditional methods require a large amount of manpower for manual testing, while the automated design of the present invention reduces the work intensity of testers by about 30%. In addition, the test case generation time is also reduced from several hours to dozens of minutes, further improving the economy and efficiency of testing.

[0201] Improving software reliability and security: By comprehensively covering the fault propagation paths, the present invention can not only discover more potential defects but also identify critical issues that may lead to system failures at an early stage, significantly enhancing the overall reliability and security of the software. Experimental results show that after adopting the present invention, the failure rate of the software system has been reduced by 40%, effectively reducing the operation risk of the system and enhancing user trust.

[0202] Flexibly adapting to complex environments: The test method of the present invention is designed flexibly and can adapt to the requirements of different complex software systems. This adaptability makes this method widely applicable in practical applications and enables effective testing of complex software systems in different industries and fields. For example, in industries such as finance and healthcare where extremely high software stability is required, this embodiment provides a reliable solution.

[0203] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

Claims

1. A complex software system testing method based on fault propagation path coverage, characterized in that: The method comprises: Based on program slicing technology, a fault propagation path model of the target program is established to describe the dynamic propagation path of defects between modules; Inject seed defects into the target program to stimulate potential defects through seed defects; design the initial test case set based on the fault propagation path model, and run the target program to detect whether the seed defects are covered; Based on the dual standards of defect coverage and statement coverage, the test case set is expanded to generate test cases that cover the fault propagation path; Based on the expanded test case set, the fault propagation path is fully tested to identify and locate potential related defects; Test adequacy judgment based on fault propagation paths, including defect coverage, statement coverage, and algorithm termination conditions.

2. The complex software system testing method based on fault propagation path coverage according to claim 1 is characterized in that: The establishing of the fault propagation path model comprises: Analyze the static dependencies of the program through static slicing technology; Analyze the dynamic running path of the program under specific input conditions through dynamic slicing technology; A model of the fault propagation path is generated based on the static slicing and dynamic slicing results.

3. The complex software system testing method based on fault propagation path coverage according to claim 1 is characterized in that: The injected seed defects include: Design seed defects based on historical defect data of the target program; Inject seed defects into the starting nodes and key nodes of the fault propagation path; The injected seed defects include logic errors, variable undefined, and conditional branch error types.

4. The complex software system testing method based on fault propagation path coverage according to claim 1 is characterized in that: The process of designing the initial test case set includes: Based on the functional requirements of the target program, the input domain is divided into equivalence classes; Randomly select test inputs from the equivalence class as initial test cases; Based on the fault propagation path model, verify the coverage of the critical path by the initial test cases.

5. The complex software system testing method based on fault propagation path coverage according to claim 1 is characterized in that: The process of expanding the test case set adopts an improved intelligent algorithm, which includes the following steps: Dynamically generate new test cases based on the initial test case set; Calculate the defect coverage and statement coverage of the generated test case set; Filter test cases through double-layer state receiving functions, where the outer state receiving function is based on defect coverage and the inner state receiving function is based on statement coverage; Iteratively optimize the screened test cases to generate a test case set that covers more fault propagation paths.

6. The complex software system testing method based on fault propagation path coverage according to claim 5 is characterized in that: The intelligent algorithm adopts a parameter optimization strategy based on dynamic adjustment, including: Set the initial parameter value to T0, and dynamically adjust the parameter value based on the adjustment coefficient k; When the parameter value drops to the set threshold, stop generating new test cases; During each adjustment process, test cases are dynamically generated and their defect coverage and statement coverage are verified.

7. The complex software system testing method based on fault propagation path coverage according to claim 1 is characterized in that: The test of the fault propagation path includes: Combine test cases with seed defects to generate dynamic slices of fault propagation paths; Analyze dynamic slices to identify potential defects on the path; Based on the propagation characteristics of potential defects, the test case set is expanded to cover new fault propagation paths.

8. The complex software system testing method based on fault propagation path coverage according to claim 1 is characterized in that: The injected seed defects are generated by a program mutation technique, wherein the mutation technique includes: Introducing logical mutation, arithmetic mutation or structural mutation into the source code of the target program; Test the mutated target program to verify the coverage of injected defects; The mutation parameters are adjusted according to the coverage of seed defects to generate more representative seed defects.

9. The complex software system testing method based on fault propagation path coverage according to claim 1 is characterized in that: The test adequacy determination includes: Whether the defect coverage of the test case reaches the preset coverage threshold; Whether the statement coverage of the test case reaches the preset coverage threshold; Whether the test termination conditions are met, including algorithm convergence and maximum running time.

10. The complex software system testing method based on fault propagation path coverage according to claim 1, characterized in that: The final test case set of the method covers all injected seed defects and their associated fault propagation paths, and has the ability to identify potential associated defects in the target program.

Images